# Sprint 15.2: Exception Handling Hardening

> **Size**: M | **Owner**: Backend | **Priority**: HIGH
> **Phase**: 4.5 - Stabilization
> **Prerequisites**: Sprint 15.1 complete
> **Status**: ✅ COMPLETE

---

## Objective

Replace 48 broad `except Exception` handlers with specific exception types to:
- Surface masked errors that could indicate bugs
- Improve debuggability with precise stack traces
- Enable proper error recovery strategies
- Fix the single `type: ignore` violation

---

## Issue Summary

| Issue | Count | Severity | Location Pattern |
|-------|-------|----------|------------------|
| Broad `except Exception` handlers | 48 | HIGH | Throughout codebase |
| `type: ignore` violation | 1 | MEDIUM | `session.py:158` |
| Missing audio writer open exception | 1 | MEDIUM | `streaming.py` |
| No diarization job timeout | 1 | MEDIUM | `diarization.py` |

---

## Task 1: Audit and Categorize Exception Handlers

### Exception Handler Inventory

Based on codebase search, here are all 48 `except Exception` locations grouped by module:

#### Infrastructure Layer (26 handlers)

| File | Line | Current Usage | Recommended Fix |
|------|------|---------------|-----------------|
| `audio/writer.py` | 156 | Flush on close | Catch `OSError`, `IOError` |
| `audio/playback.py` | 333 | Cleanup on error | Catch `sounddevice.PortAudioError` |
| `asr/engine.py` | 96 | Model loading | Catch `RuntimeError`, `FileNotFoundError` |
| `persistence/database.py` | 278 | Connection pool | Catch `SQLAlchemyError`, `asyncpg.PostgresError` |
| `persistence/database.py` | 404 | Migrations | Catch `SQLAlchemyError` |
| `metrics/collector.py` | 147 | System metrics | Catch `psutil.Error`, `OSError` |
| `security/crypto.py` | 101 | Encryption | Catch `ValueError`, `cryptography.exceptions.*` |
| `security/crypto.py` | 146 | Decryption | Catch `ValueError`, `cryptography.exceptions.*` |
| `diarization/engine.py` | 144, 180, 254 | Model inference | Catch `RuntimeError`, `torch.cuda.OutOfMemoryError` |
| `triggers/foreground_app.py` | 80, 122 | OS APIs | Catch `OSError`, platform-specific |
| `triggers/app_audio.py` | 95, 106, 162, 178, 190, 273 | Audio monitoring | Catch `sounddevice.PortAudioError`, `OSError` |
| `webhooks/executor.py` | 56, 327 | HTTP delivery | Catch `httpx.HTTPError`, `asyncio.TimeoutError` |
| `summarization/ollama_provider.py` | 44, 182 | LLM calls | Catch `httpx.HTTPError`, `json.JSONDecodeError` |
| `summarization/cloud_provider.py` | 47, 262, 301 | API calls | Catch `anthropic.APIError`, `openai.APIError` |
| `logging/log_buffer.py` | 158, 176 | Buffer operations | Catch `ValueError`, `IndexError` |

#### gRPC Layer (17 handlers)

| File | Line | Current Usage | Recommended Fix |
|------|------|---------------|-----------------|
| `service.py` | 362 | Recovery fallback | Catch `SQLAlchemyError`, `grpc.RpcError` |
| `server.py` | 219, 233 | Shutdown cleanup | Catch `asyncio.CancelledError`, `grpc.RpcError` |
| `_streaming_session.py` | 241, 254 | Session cleanup | Catch `RuntimeError`, `ValueError` |
| `_mixins/sync.py` | 132 | Sync operation | Catch specific sync errors |
| `_mixins/streaming.py` | 111, 196, 334 | Audio processing | Catch `RuntimeError`, `ValueError` |
| `_mixins/meeting.py` | 102, 107 | Webhook firing | Catch `httpx.HTTPError` (already fire-and-forget) |
| `_mixins/diarization.py` | 185, 217, 385 | Job processing | Catch `RuntimeError`, `asyncio.TimeoutError` |
| `_mixins/diarization_job.py` | 37 | Job lookup | Remove handler entirely |
| `_mixins/summarization.py` | 83 | Webhook firing | Catch `httpx.HTTPError` |
| `_client_mixins/streaming.py` | 185, 198 | Client-side | Catch `grpc.RpcError` |

#### Application Layer (5 handlers)

| File | Line | Current Usage | Recommended Fix |
|------|------|---------------|-----------------|
| `services/retention_service.py` | 133 | Cleanup errors | Catch `SQLAlchemyError`, `OSError` |
| `services/webhook_service.py` | 235 | Delivery errors | Catch `httpx.HTTPError` |

---

## Task 2: Fix `type: ignore` Violation

### Location

`src/noteflow/infrastructure/diarization/session.py:158`

### Current Code

```python
def close(self) -> None:
    """Close the session and release resources."""
    if self._closed:
        return

    self._closed = True
    self._turns.clear()
    # Explicitly release pipeline reference to allow GC and GPU memory release
    self._pipeline = None  # type: ignore[assignment]
    logger.debug("Session %s closed", self.meeting_id)
```

### Issue

The `_pipeline` attribute is typed as `Pipeline` (non-optional), but we're assigning `None` to it.

### Fix Required

Change the type annotation to `Pipeline | None`:

```python
from __future__ import annotations

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from diart.pipelines import Pipeline

class StreamingDiarizationSession:
    def __init__(self, ...) -> None:
        self._pipeline: Pipeline | None = pipeline  # Allow None
        # ...

    def close(self) -> None:
        if self._closed:
            return

        self._closed = True
        self._turns.clear()
        self._pipeline = None  # No type: ignore needed
        logger.debug("Session %s closed", self.meeting_id)
```

---

## Task 3: Add Audio Writer Open Exception Handling

### Location

`src/noteflow/grpc/_mixins/streaming.py` (in `_init_stream_session` or similar)

### Issue

Audio writer initialization can fail (permissions, disk full, invalid path) but exceptions may not be properly caught and converted to gRPC errors.

### Fix Required

```python
async def _init_stream_session(self, meeting_id: str) -> _StreamSessionInit:
    try:
        # Create audio writer
        writer = MeetingAudioWriter(
            meeting_id=meeting_id,
            base_path=self._settings.meetings_dir,
            sample_rate=self._settings.sample_rate,
        )
        self._audio_writers[meeting_id] = writer
    except (OSError, IOError, PermissionError) as e:
        logger.error("Failed to create audio writer for %s: %s", meeting_id, e)
        return _StreamSessionInit(
            success=False,
            error_code=grpc.StatusCode.INTERNAL,
            error_message=f"Failed to initialize audio storage: {e}",
        )
```

---

## Task 4: Add Diarization Job Timeout

### Location

`src/noteflow/grpc/_mixins/diarization.py`

### Issue

Diarization jobs can run indefinitely if the inference hangs, consuming resources without bound.

### Fix Required

```python
import asyncio
from contextlib import asynccontextmanager

DIARIZATION_TIMEOUT_SECONDS = 300  # 5 minutes default

async def _run_diarization_job(
    self: ServicerHost,
    job_id: str,
    num_speakers: int | None,
) -> None:
    """Run background diarization job with timeout."""
    try:
        async with asyncio.timeout(DIARIZATION_TIMEOUT_SECONDS):
            updated_count = await self.refine_speaker_diarization(
                meeting_id=meeting_id,
                num_speakers=num_speakers,
            )
    except asyncio.TimeoutError:
        logger.error("Diarization job %s timed out after %ds", job_id, DIARIZATION_TIMEOUT_SECONDS)
        # Update status to FAILED with timeout message
        async with self._create_repository_provider() as repo:
            if repo.supports_diarization_jobs:
                await repo.diarization_jobs.update_status(
                    job_id,
                    noteflow_pb2.JOB_STATUS_FAILED,
                    error_message=f"Job timed out after {DIARIZATION_TIMEOUT_SECONDS} seconds",
                )
                await repo.commit()
        return
```

---

## Task 5: Priority Exception Handler Replacements

Focus on the highest-impact handlers first:

### Priority 1: Database Operations

```python
# persistence/database.py:278
from sqlalchemy.exc import SQLAlchemyError
import asyncpg

try:
    # database operation
except SQLAlchemyError as e:
    logger.exception("Database error during %s", operation)
    raise DatabaseError(f"Database operation failed: {e}") from e
except asyncpg.PostgresError as e:
    logger.exception("PostgreSQL error during %s", operation)
    raise DatabaseError(f"PostgreSQL error: {e}") from e
```

### Priority 2: gRPC Streaming

```python
# _mixins/streaming.py:111
try:
    self._audio_writers[current_meeting_id].flush()
except (OSError, IOError) as e:
    logger.warning("Failed to flush audio for %s: %s", current_meeting_id, e)
except ValueError as e:
    logger.warning("Invalid audio buffer state for %s: %s", current_meeting_id, e)
```

### Priority 3: Webhook Delivery

```python
# webhooks/executor.py:327
import httpx

try:
    response = await self._client.post(url, json=payload, headers=headers)
except httpx.TimeoutException as e:
    logger.warning("Webhook delivery timed out: %s", url)
    return WebhookDeliveryResult(success=False, error="Timeout")
except httpx.HTTPStatusError as e:
    logger.warning("Webhook delivery HTTP error: %s %s", e.response.status_code, url)
    return WebhookDeliveryResult(success=False, error=f"HTTP {e.response.status_code}")
except httpx.RequestError as e:
    logger.warning("Webhook delivery request error: %s", e)
    return WebhookDeliveryResult(success=False, error=str(e))
```

### Priority 4: Summarization Providers

```python
# summarization/cloud_provider.py:262
import anthropic
import openai

try:
    response = await self._anthropic.messages.create(...)
except anthropic.APIConnectionError as e:
    raise SummarizationError(f"Failed to connect to Anthropic API: {e}") from e
except anthropic.RateLimitError as e:
    raise SummarizationError(f"Anthropic rate limit exceeded: {e}") from e
except anthropic.APIStatusError as e:
    raise SummarizationError(f"Anthropic API error {e.status_code}: {e.message}") from e
```

---

## Tasks Checklist

### Phase A: Critical Path (Day 1-2)
- [x] Fix `type: ignore` in `session.py:158`
- [x] Add audio writer open exception handling
- [x] Add diarization job timeout
- [x] Replace exception handlers in `persistence/database.py`
- [x] Replace exception handlers in `_mixins/streaming.py`

### Phase B: Infrastructure (Day 3-4)
- [x] Replace exception handlers in `webhooks/executor.py`
- [x] Replace exception handlers in `summarization/cloud_provider.py`
- [x] Replace exception handlers in `summarization/ollama_provider.py`
- [x] Replace exception handlers in `security/crypto.py`
- [x] Replace exception handlers in `diarization/engine.py`

### Phase C: gRPC Layer (Day 5-6)
- [x] Replace exception handlers in `_mixins/meeting.py`
- [x] Replace exception handlers in `_mixins/diarization.py`
- [x] Replace exception handlers in `_mixins/summarization.py`
- [x] Replace exception handlers in `service.py` and `server.py`

### Phase D: Remaining (Day 7)
- [x] Replace exception handlers in `audio/` modules
- [x] Replace exception handlers in `triggers/` modules
- [x] Replace exception handlers in `metrics/collector.py`
- [x] Replace exception handlers in `logging/log_buffer.py`

---

## Acceptance Criteria

### Functional

- [x] No `except Exception` handlers remain in codebase (except documented intentional ones)
- [x] No `type: ignore` violations in codebase
- [x] Audio writer failures return proper gRPC errors
- [x] Diarization jobs timeout after configured duration (300s)

### Quality Gates

- [x] `ruff check src/noteflow` passes
- [x] `basedpyright` passes with no errors (0 errors)
- [x] All remaining `except Exception` handlers documented with INTENTIONAL comments (25 handlers)
- [x] `grep -r "type: ignore" src/noteflow | wc -l` returns 0

---

## Exception Handler Documentation Template

For any intentionally broad handler that must remain, add this comment:

```python
# INTENTIONAL BROAD HANDLER: [reason]
# - Specific exceptions caught: [list]
# - Why broad handler needed: [explanation]
# - Recovery strategy: [what happens on exception]
except Exception as e:  # noqa: BLE001
    logger.exception("...")
```

---

## Files Modified

| File | Change Type | Handler Count |
|------|-------------|---------------|
| `infrastructure/diarization/session.py` | Modify | 1 (type fix) |
| `infrastructure/persistence/database.py` | Modify | 2 |
| `infrastructure/webhooks/executor.py` | Modify | 2 |
| `infrastructure/summarization/cloud_provider.py` | Modify | 3 |
| `infrastructure/summarization/ollama_provider.py` | Modify | 2 |
| `infrastructure/security/crypto.py` | Modify | 2 |
| `infrastructure/diarization/engine.py` | Modify | 3 |
| `infrastructure/audio/writer.py` | Modify | 1 |
| `infrastructure/audio/playback.py` | Modify | 1 |
| `infrastructure/triggers/app_audio.py` | Modify | 6 |
| `infrastructure/triggers/foreground_app.py` | Modify | 2 |
| `infrastructure/metrics/collector.py` | Modify | 1 |
| `infrastructure/logging/log_buffer.py` | Modify | 2 |
| `grpc/service.py` | Modify | 1 |
| `grpc/server.py` | Modify | 2 |
| `grpc/_streaming_session.py` | Modify | 2 |
| `grpc/_mixins/streaming.py` | Modify | 3 |
| `grpc/_mixins/meeting.py` | Modify | 2 |
| `grpc/_mixins/diarization.py` | Modify | 3 |
| `grpc/_mixins/summarization.py` | Modify | 1 |
| `grpc/_mixins/sync.py` | Modify | 1 |
| `grpc/_client_mixins/streaming.py` | Modify | 2 |
| `application/services/retention_service.py` | Modify | 1 |
| `application/services/webhook_service.py` | Modify | 1 |

---

## Rollback Plan

1. Each file can be reverted independently
2. If specific exception causes issues, temporarily widen to parent class
3. Maintain list of problematic handlers for post-sprint analysis

---

## Post-Sprint

- Add custom exception hierarchy for domain errors
- Add exception telemetry to track error patterns
- Consider adding retry decorators for transient failures
