# Spec Validation Fixes - Agent-Driven Execution Plan

> **Sprint Reference**: `docs/sprints/sprint_spec_validation_fixes.md`
> **Source Evidence**: `docs/spec.md`
> **Quality Gates**: `docs/sprints/QUALITY_STANDARDS.md`

---

## Executive Summary

This plan orchestrates fixes for 12 validated spec findings using specialized agents for discovery, validation, and implementation. Each phase is designed for parallel execution where possible, with mandatory quality gates after every file modification.

---

## CRITICAL: Quality Enforcement Rules

### ABSOLUTE PROHIBITIONS

1. **NEVER modify quality test thresholds** - If violations exceed thresholds, FIX THE CODE, not the tests
2. **NEVER claim errors are "preexisting"** without baseline proof - Capture baselines BEFORE any changes
3. **NEVER batch quality checks** - Run `make quality-py` after EVERY file modification
4. **NEVER skip quality gates** to "fix later" - All code must pass before proceeding
5. **NEVER add `# type: ignore`** without explicit user approval
6. **NEVER use `Any` type** - Use specific types always

### MANDATORY QUALITY CHECKPOINTS

**After EVERY code change** (not "at the end of a phase"):

```bash
# Run after EACH file edit - no exceptions
make quality-py
```

This runs:
- `ruff check .` - Linting (ALL code)
- `basedpyright` - Type checking (ALL code)
- `pytest tests/quality/ -q` - Code smell detection (ALL code)

### BASELINE CAPTURE (Required Before Phase 2)

```bash
# Capture baseline BEFORE any implementation work
make quality-py 2>&1 | tee /tmp/quality_baseline_spec_$(date +%Y%m%d_%H%M%S).log

# Record current threshold violations
pytest tests/quality/ -v --tb=no | grep -E "(PASSED|FAILED|violations)" > /tmp/threshold_baseline_spec.txt
```

Any NEW violations introduced during implementation are **agent responsibility** and must be fixed immediately.

### CODE QUALITY STANDARDS (Apply to ALL Code)

| Rule | Applies To | Enforcement |
|------|-----------|-------------|
| No `# type: ignore` | ALL Python code | `basedpyright` |
| No `Any` types | ALL Python code | `basedpyright` |
| Union syntax `str \| None` | ALL Python code | `ruff UP` |
| Module < 500 lines (soft) | ALL modules | `tests/quality/test_code_smells.py` |
| Module < 750 lines (hard) | ALL modules | `tests/quality/test_code_smells.py` |
| Function < 75 lines | ALL functions | `tests/quality/test_code_smells.py` |
| Complexity < 15 | ALL functions | `tests/quality/test_code_smells.py` |
| Parameters <= 7 | ALL functions | `tests/quality/test_code_smells.py` |
| No magic numbers > 100 | ALL code | `tests/quality/test_magic_values.py` |
| No hardcoded paths | ALL code | `tests/quality/test_magic_values.py` |

---

## Phase 0: Pre-Flight Validation

### Agent Task: Verify All Findings Still Exist

| Agent | Purpose | Query | Deliverable |
|-------|---------|-------|-------------|
| `Explore` | Confirm timestamp inconsistencies | "Find all timestamp fields in noteflow.proto with their types" | Type inventory |
| `Explore` | Confirm UpdateAnnotation sentinel logic | "Find sentinel value checks in annotation.py handler" | Pattern list |
| `Explore` | Confirm sync task tracking | "Find all asyncio.create_task calls in _mixins/" | Task registry map |
| `Explore` | Confirm segmenter pop(0) usage | "Find list.pop(0) calls in asr/segmenter.py" | Line numbers |

**Commands to validate evidence:**
```bash
# Verify proto file exists and has expected patterns
grep -n "double created_at\|int64 start_time\|string started_at" src/noteflow/grpc/proto/noteflow.proto

# Verify annotation handler logic
grep -n "request.text:\|request.start_time > 0" src/noteflow/grpc/_mixins/annotation.py

# Verify sync task is untracked
grep -n "asyncio.create_task" src/noteflow/grpc/_mixins/sync.py

# Verify segmenter pop(0)
grep -n "pop(0)" src/noteflow/infrastructure/asr/segmenter.py
```

---

## Phase 1: Discovery & Target Mapping

### 1.1 Agent: Timestamp Field Inventory

**Agent Type**: `Explore` (very thorough)

**Objective**: Complete inventory of all timestamp representations in proto and handlers.

**Queries**:
1. "Find all fields with 'timestamp', 'time', 'at' suffix in noteflow.proto"
2. "Find all timestamp conversion functions in grpc/_mixins/converters.py"
3. "Map which handlers use which timestamp formats"

**Expected Output**:
```yaml
timestamp_inventory:
  double_fields:
    - created_at (line 217)
    - server_timestamp (line 149)
    - start_time (annotation, line 504)
    - end_time (annotation, line 505)
  int64_fields:
    - start_time (calendar, line 745)
  string_fields:
    - started_at (sync, line 1203)
  conversions_needed:
    - annotation handler: double -> domain
    - sync handler: string -> datetime
```

### 1.2 Agent: Sentinel Default Pattern Analysis

**Agent Type**: `feature-dev:code-explorer`

**Objective**: Map all uses of sentinel value checks in update handlers.

**Tasks**:
1. Find all `if request.field:` patterns in _mixins/
2. Find all `if request.field > 0:` patterns
3. Find all `if request.field != ENUM_UNSPECIFIED:` patterns
4. Identify which fields need `optional` treatment

**Expected Output**:
```yaml
sentinel_patterns:
  annotation.py:
    - line 127: annotation_type != UNSPECIFIED
    - line 128: text (truthy check)
    - line 129: start_time > 0
    - line 130: end_time > 0
    - line 131: segment_ids (truthy check)
  webhooks.py:
    - line XX: uses optional fields correctly (reference pattern)
```

### 1.3 Agent: Background Task Registry Analysis

**Agent Type**: `feature-dev:code-explorer`

**Objective**: Map all background task creation and cancellation patterns.

**Tasks**:
1. Find all `asyncio.create_task()` calls in grpc/
2. Identify which tasks are stored for cancellation
3. Find the shutdown cleanup code in service.py
4. Map the diarization task pattern for reuse

**Expected Output**:
```yaml
task_tracking:
  tracked:
    - _diarization_tasks (dict[str, Task])
    - location: _mixins/diarization/_jobs.py:130
    - cleanup: service.py:445
  untracked:
    - sync_task (_mixins/sync.py:109)
    - no storage reference
    - done_callback is noop
```

### 1.4 Agent: HTTP Client Configuration Audit

**Agent Type**: `Explore` (thorough)

**Objective**: Audit all httpx.AsyncClient instantiations for timeouts and limits.

**Queries**:
1. "Find all httpx.AsyncClient instantiations in infrastructure/"
2. "Check which have timeout= configured"
3. "Check which have limits= configured"

**Expected Output**:
```yaml
http_clients:
  outlook_adapter.py:81:
    timeout: None (missing)
    limits: None (missing)
  webhook_executor.py:103:
    timeout: self._timeout (configured)
    limits: None (missing)
```

### 1.5 Agent: Crypto Bounds Check Analysis

**Agent Type**: `feature-dev:code-explorer`

**Objective**: Trace ChunkedAssetReader for all boundary conditions.

**Tasks**:
1. Find chunk framing logic in crypto.py
2. Identify all size assumptions
3. Map what happens with truncated/malformed input
4. Find NONCE_SIZE and TAG_SIZE constants

**Expected Output**:
```yaml
crypto_bounds:
  constants:
    NONCE_SIZE: 12  # or wherever defined
    TAG_SIZE: 16    # or wherever defined
  missing_checks:
    - chunk_length minimum not validated (line 279)
    - version header read not checked (line XX)
  existing_checks:
    - len(chunk_data) < chunk_length raises ValueError
```

---

## Phase 2: Implementation - P0 Critical Fixes

### 2.1 Fix: ChunkedAssetReader Bounds Checks

**File**: `src/noteflow/infrastructure/security/crypto.py`

**Agent Type**: `feature-dev:code-architect`

**Design Constraints**:
- No `Any` types
- No `# type: ignore`
- All functions must have return type annotations
- Docstrings written imperatively

**Implementation Spec**:

```python
"""Add read_exact helper and validate chunk bounds."""

# Constants (verify existing or add)
NONCE_SIZE: Final[int] = 12
TAG_SIZE: Final[int] = 16
MIN_CHUNK_LENGTH: Final[int] = NONCE_SIZE + TAG_SIZE


def _read_exact(handle: BinaryIO, size: int, description: str) -> bytes:
    """Read exactly size bytes or raise ValueError.

    Args:
        handle: File handle to read from.
        size: Number of bytes to read.
        description: Description for error message.

    Returns:
        Exactly size bytes.

    Raises:
        ValueError: If fewer than size bytes available.
    """
    data = handle.read(size)
    if len(data) < size:
        raise ValueError(f"Truncated {description}: expected {size}, got {len(data)}")
    return data


# In ChunkedAssetReader.__next__ or equivalent:
def _read_next_chunk(self) -> bytes:
    """Read and decrypt the next chunk.

    Returns:
        Decrypted chunk data.

    Raises:
        ValueError: If chunk is truncated or malformed.
        StopIteration: If end of file reached cleanly.
    """
    # Read length header
    length_bytes = self._handle.read(4)
    if len(length_bytes) == 0:
        raise StopIteration  # Clean EOF
    if len(length_bytes) < 4:
        raise ValueError("Truncated chunk length header")

    chunk_length = struct.unpack(">I", length_bytes)[0]

    # Validate minimum chunk size
    if chunk_length < MIN_CHUNK_LENGTH:
        raise ValueError(
            f"Invalid chunk length {chunk_length}: "
            f"minimum is {MIN_CHUNK_LENGTH} (nonce + tag)"
        )

    # Read chunk data
    chunk_data = _read_exact(self._handle, chunk_length, "chunk data")

    # Extract components with validated bounds
    nonce = chunk_data[:NONCE_SIZE]
    ciphertext = chunk_data[NONCE_SIZE:-TAG_SIZE]
    tag = chunk_data[-TAG_SIZE:]

    # Decrypt...
```

**Quality Gate**:
```bash
# After editing crypto.py
make quality-py
# MUST PASS before proceeding
```

### 2.2 Fix: OTEL Insecure Flag

**File**: `src/noteflow/infrastructure/observability/otel.py`

**Agent Type**: `agent-python-executor`

**Implementation Spec**:

```python
# In settings or config, add:
NOTEFLOW_OTEL_INSECURE: bool = Field(
    default=False,
    description="Use insecure (non-TLS) connection to OTLP endpoint"
)

# In otel.py, update exporter creation:
def _create_otlp_exporter(endpoint: str, insecure: bool | None = None) -> OTLPSpanExporter:
    """Create OTLP exporter with appropriate security settings.

    Args:
        endpoint: OTLP endpoint URL.
        insecure: Override security setting. If None, infer from endpoint scheme.

    Returns:
        Configured OTLP span exporter.
    """
    if insecure is None:
        # Infer from scheme - http:// is insecure, https:// is secure
        insecure = endpoint.startswith("http://")

    return OTLPSpanExporter(endpoint=endpoint, insecure=insecure)
```

**Quality Gate**:
```bash
make quality-py
```

---

## Phase 3: Implementation - P1 API Contract Fixes

### 3.1 Fix: UpdateAnnotation Presence Tracking

**Files**:
- `src/noteflow/grpc/proto/noteflow.proto`
- `src/noteflow/grpc/_mixins/annotation.py`

**Agent Type**: `feature-dev:code-architect`

**Proto Changes** (backward compatible):

```protobuf
message UpdateAnnotationRequest {
  string annotation_id = 1;

  // DEPRECATED: Use optional fields below
  AnnotationType annotation_type = 2;
  string text = 3;
  double start_time = 4;
  double end_time = 5;
  repeated int32 segment_ids = 6;

  // NEW: Presence-tracked fields (proto3 optional)
  optional AnnotationType new_annotation_type = 7;
  optional string new_text = 8;
  optional double new_start_time = 9;
  optional double new_end_time = 10;

  // Clear flags for backward compatibility
  bool clear_segment_ids = 11;
}
```

**Handler Changes**:

```python
async def UpdateAnnotation(
    self,
    request: noteflow_pb2.UpdateAnnotationRequest,
    context: grpc.aio.ServicerContext,
) -> noteflow_pb2.Annotation:
    """Update an existing annotation with presence tracking."""
    # ... validation ...

    # Use new optional fields if present, fall back to legacy
    if request.HasField("new_annotation_type"):
        annotation.annotation_type = proto_to_annotation_type(request.new_annotation_type)
    elif request.annotation_type != noteflow_pb2.ANNOTATION_TYPE_UNSPECIFIED:
        # Legacy path
        annotation.annotation_type = proto_to_annotation_type(request.annotation_type)

    if request.HasField("new_text"):
        annotation.text = request.new_text  # Can be empty string
    elif request.text:
        # Legacy path (cannot set empty)
        annotation.text = request.text

    if request.HasField("new_start_time"):
        annotation.start_time = request.new_start_time  # Can be 0.0
    elif request.start_time > 0:
        # Legacy path
        annotation.start_time = request.start_time

    if request.HasField("new_end_time"):
        annotation.end_time = request.new_end_time  # Can be 0.0
    elif request.end_time > 0:
        # Legacy path
        annotation.end_time = request.end_time

    if request.clear_segment_ids:
        annotation.segment_ids = []
    elif request.segment_ids:
        annotation.segment_ids = list(request.segment_ids)
```

**Quality Gate**:
```bash
# After proto change
python -m grpc_tools.protoc -I src/noteflow/grpc/proto \
  --python_out=src/noteflow/grpc/proto \
  --grpc_python_out=src/noteflow/grpc/proto \
  src/noteflow/grpc/proto/noteflow.proto

# After handler change
make quality-py
```

### 3.2 Fix: Add Status Enums

**File**: `src/noteflow/grpc/proto/noteflow.proto`

**Agent Type**: `feature-dev:code-architect`

**Proto Additions**:

```protobuf
enum SyncRunStatus {
  SYNC_RUN_STATUS_UNSPECIFIED = 0;
  SYNC_RUN_STATUS_RUNNING = 1;
  SYNC_RUN_STATUS_SUCCESS = 2;
  SYNC_RUN_STATUS_ERROR = 3;
}

enum OAuthConnectionStatus {
  OAUTH_CONNECTION_STATUS_UNSPECIFIED = 0;
  OAUTH_CONNECTION_STATUS_DISCONNECTED = 1;
  OAUTH_CONNECTION_STATUS_CONNECTED = 2;
  OAUTH_CONNECTION_STATUS_ERROR = 3;
}

// In SyncRun message:
message SyncRun {
  // ...existing fields...
  string status = 3;  // DEPRECATED
  SyncRunStatus status_enum = 10;  // NEW
}

// In OAuthConnection or similar:
message IntegrationStatus {
  // ...existing fields...
  string status = 2;  // DEPRECATED
  OAuthConnectionStatus status_enum = 10;  // NEW
}
```

**Quality Gate**:
```bash
# After proto regeneration
make quality-py
```

### 3.3 Design: TranscriptUpdateV2 (Future Sprint)

**Agent Type**: `Plan`

**Objective**: Design v2 message without implementing (breaking change requires coordination).

**Output**: Design document with:
1. New `TranscriptUpdateV2` message with `oneof`
2. New `StreamTranscriptionV2` RPC
3. Migration strategy for clients
4. Deprecation timeline for v1

---

## Phase 4: Implementation - P2 Reliability Fixes

### 4.1 Fix: Background Task Registry

**Files**:
- `src/noteflow/grpc/service.py` (add registry)
- `src/noteflow/grpc/_mixins/sync.py` (use registry)

**Agent Type**: `feature-dev:code-architect`

**Implementation Spec**:

```python
# In service.py, add to NoteFlowServicer.__init__:
from asyncio import Task
from typing import Final

class NoteFlowServicer:
    """NoteFlow gRPC servicer with background task tracking."""

    def __init__(self, ...) -> None:
        # ... existing init ...

        # Unified background task registry
        self._background_tasks: dict[str, Task[None]] = {}

    def register_background_task(self, task_id: str, task: Task[None]) -> None:
        """Register a background task for shutdown cleanup.

        Args:
            task_id: Unique identifier for the task.
            task: The asyncio task to track.
        """
        self._background_tasks[task_id] = task
        task.add_done_callback(
            lambda t: self._on_task_done(task_id, t)
        )

    def _on_task_done(self, task_id: str, task: Task[None]) -> None:
        """Handle task completion, logging any exceptions.

        Args:
            task_id: The task identifier.
            task: The completed task.
        """
        self._background_tasks.pop(task_id, None)
        if not task.cancelled():
            exc = task.exception()
            if exc is not None:
                logger.exception(
                    "Background task failed",
                    task_id=task_id,
                    exc_info=exc,
                )

    async def _shutdown_background_tasks(self) -> None:
        """Cancel and await all background tasks."""
        for task_id, task in list(self._background_tasks.items()):
            if not task.done():
                task.cancel()
                with contextlib.suppress(asyncio.CancelledError):
                    await task
                logger.info("Cancelled background task", task_id=task_id)


# In sync.py, update to use registry:
async def _start_sync(self, integration_id: str, sync_run_id: str, provider: str) -> None:
    """Start a background sync task."""
    sync_task = asyncio.create_task(
        self._perform_sync(integration_id, sync_run_id, provider),
        name=f"sync-{sync_run_id}",
    )
    # Register for tracking and cleanup
    self.register_background_task(f"sync-{sync_run_id}", sync_task)
```

**Quality Gate**:
```bash
make quality-py
```

### 4.2 Fix: Segmenter Deque

**File**: `src/noteflow/infrastructure/asr/segmenter.py`

**Agent Type**: `agent-python-executor`

**Implementation Spec**:

```python
from collections import deque

# Change class attribute:
# Before: self._leading_buffer: list[np.ndarray] = []
# After:
self._leading_buffer: deque[np.ndarray] = deque()

# Change trim logic:
# Before:
# while total_duration > self.config.leading_buffer and self._leading_buffer:
#     removed = self._leading_buffer.pop(0)
#     self._leading_buffer_samples -= len(removed)

# After:
while total_duration > self.config.leading_buffer and self._leading_buffer:
    removed = self._leading_buffer.popleft()  # O(1) instead of O(n)
    self._leading_buffer_samples -= len(removed)
```

**Quality Gate**:
```bash
make quality-py
```

### 4.3 Fix: Centralize gRPC Size Limits

**Files**:
- `src/noteflow/config/settings.py`
- `src/noteflow/grpc/server.py`
- `src/noteflow/grpc/service.py`

**Agent Type**: `agent-python-executor`

**Implementation Spec**:

```python
# In settings.py, add to Settings class:
grpc_max_message_size: int = Field(
    default=100 * 1024 * 1024,  # 100MB
    description="Maximum gRPC message size in bytes",
)
grpc_max_chunk_size: int = Field(
    default=1024 * 1024,  # 1MB
    description="Maximum streaming chunk size in bytes",
)

# In server.py, use settings:
settings = get_settings()
self._server = grpc.aio.server(
    options=[
        ("grpc.max_send_message_length", settings.grpc_max_message_size),
        ("grpc.max_receive_message_length", settings.grpc_max_message_size),
    ],
)

# In service.py, use settings:
# Remove: MAX_CHUNK_SIZE: Final[int] = 1024 * 1024
# Use: get_settings().grpc_max_chunk_size

# In constants.py:
# Remove or deprecate MAX_GRPC_MESSAGE_SIZE
```

**Quality Gate**:
```bash
make quality-py
```

### 4.4 Fix: Outlook Adapter Timeouts & Pagination

**File**: `src/noteflow/infrastructure/calendar/outlook_adapter.py`

**Agent Type**: `feature-dev:code-architect`

**Implementation Spec**:

```python
from typing import Final

# Constants
GRAPH_API_TIMEOUT: Final[float] = 30.0  # seconds
MAX_CONNECTIONS: Final[int] = 10
MAX_ERROR_BODY_LENGTH: Final[int] = 500


async def _fetch_events_page(
    self,
    url: str,
    headers: dict[str, str],
    params: dict[str, str] | None = None,
) -> tuple[list[dict[str, Any]], str | None]:
    """Fetch a single page of events from Graph API.

    Args:
        url: API endpoint URL.
        headers: Request headers.
        params: Query parameters (only for first page).

    Returns:
        Tuple of (events list, next page URL or None).

    Raises:
        OutlookCalendarError: If API request fails.
    """
    async with httpx.AsyncClient(
        timeout=httpx.Timeout(GRAPH_API_TIMEOUT),
        limits=httpx.Limits(max_connections=MAX_CONNECTIONS),
    ) as client:
        response = await client.get(url, params=params, headers=headers)

    if response.status_code != HTTP_STATUS_OK:
        error_body = response.text[:MAX_ERROR_BODY_LENGTH]
        if len(response.text) > MAX_ERROR_BODY_LENGTH:
            error_body += "... (truncated)"
        logger.error("Microsoft Graph API error: %s", error_body)
        raise OutlookCalendarError(f"{ERR_API_PREFIX}{error_body}")

    data = response.json()
    events = data.get("value", [])
    next_link = data.get("@odata.nextLink")

    return events, next_link


async def get_events(
    self,
    start: datetime,
    end: datetime,
    limit: int | None = None,
) -> list[CalendarEvent]:
    """Fetch calendar events with pagination support.

    Args:
        start: Start of time range.
        end: End of time range.
        limit: Maximum number of events to return.

    Returns:
        List of calendar events within the time range.
    """
    url = f"{GRAPH_API_BASE}/me/calendarview"
    headers = self._get_headers()
    params = {
        "startDateTime": start.isoformat(),
        "endDateTime": end.isoformat(),
        "$top": str(min(limit or 100, 100)),  # Graph API max is 100
    }

    all_events: list[CalendarEvent] = []

    while url:
        events_data, next_link = await self._fetch_events_page(
            url, headers, params if not all_events else None
        )

        for event_data in events_data:
            all_events.append(self._parse_event(event_data))
            if limit and len(all_events) >= limit:
                return all_events

        url = next_link

    return all_events
```

**Quality Gate**:
```bash
make quality-py
```

### 4.5 Fix: Webhook Delivery ID Correlation

**File**: `src/noteflow/infrastructure/webhooks/executor.py`

**Agent Type**: `agent-python-executor`

**Implementation Spec**:

```python
# In deliver method, ensure delivery_id is reused:
async def deliver(
    self,
    config: WebhookConfig,
    event_type: WebhookEventType,
    payload: dict[str, Any],
) -> WebhookDelivery:
    """Deliver webhook payload with retry logic.

    Args:
        config: Webhook configuration.
        event_type: Type of event being delivered.
        payload: Event payload data.

    Returns:
        WebhookDelivery record with delivery details.
    """
    delivery_id = uuid4()  # Generate once
    timestamp = datetime.now(UTC).isoformat()

    headers = {
        HTTP_HEADER_WEBHOOK_DELIVERY: str(delivery_id),
        HTTP_HEADER_WEBHOOK_TIMESTAMP: timestamp,
        # ... other headers
    }

    # ... retry logic ...

    # Use same delivery_id in record
    return WebhookDelivery(
        id=delivery_id,  # SAME as sent in header
        webhook_id=config.id,
        event_type=event_type,
        # ...
    )


# Add client limits at initialization:
def __init__(
    self,
    timeout: float = DEFAULT_TIMEOUT,
    max_retries: int = DEFAULT_MAX_RETRIES,
    max_connections: int = 20,
    max_keepalive_connections: int = 10,
) -> None:
    """Initialize webhook executor with connection limits.

    Args:
        timeout: Request timeout in seconds.
        max_retries: Maximum retry attempts.
        max_connections: Maximum concurrent connections.
        max_keepalive_connections: Maximum keepalive connections.
    """
    self._timeout = httpx.Timeout(timeout)
    self._max_retries = max_retries
    self._client = httpx.AsyncClient(
        timeout=self._timeout,
        limits=httpx.Limits(
            max_connections=max_connections,
            max_keepalive_connections=max_keepalive_connections,
        ),
    )
```

**Quality Gate**:
```bash
make quality-py
```

---

## Phase 5: Test Coverage

### 5.1 Agent: Create Annotation Patch Test

**Agent Type**: `agent-testing-architect`

**File**: `tests/grpc/test_annotation_patch.py`

**Test Requirements** (per QUALITY_STANDARDS.md):
- No loops in tests
- No conditionals in tests
- Use `pytest.mark.parametrize`
- Use `pytest.param` with descriptive IDs
- All fixtures must have type hints
- `pytest.raises` with `match=`

**Test Spec**:

```python
"""Test UpdateAnnotation patch semantics with presence tracking."""

import pytest
from unittest.mock import AsyncMock

from noteflow.grpc.proto import noteflow_pb2


class TestUpdateAnnotationPresence:
    """Test presence-tracked field updates."""

    @pytest.mark.parametrize(
        ("field_name", "set_value", "expected_value"),
        [
            pytest.param("new_text", "", "", id="clear-text-to-empty"),
            pytest.param("new_text", "updated", "updated", id="set-text"),
            pytest.param("new_start_time", 0.0, 0.0, id="set-start-to-zero"),
            pytest.param("new_end_time", 0.0, 0.0, id="set-end-to-zero"),
        ],
    )
    async def test_optional_field_updates(
        self,
        grpc_server: "NoteFlowServicer",
        field_name: str,
        set_value: str | float,
        expected_value: str | float,
    ) -> None:
        """Optional fields can be set to zero/empty values."""
        # Arrange
        request = noteflow_pb2.UpdateAnnotationRequest(
            annotation_id="test-id",
        )
        setattr(request, field_name, set_value)

        # Act
        result = await grpc_server.UpdateAnnotation(request, AsyncMock())

        # Assert
        actual = getattr(result, field_name.replace("new_", ""))
        assert actual == expected_value, f"Expected {field_name}={expected_value}"

    async def test_clear_segment_ids(
        self,
        grpc_server: "NoteFlowServicer",
    ) -> None:
        """Clear flag removes all segment IDs."""
        # Arrange
        request = noteflow_pb2.UpdateAnnotationRequest(
            annotation_id="test-id",
            clear_segment_ids=True,
        )

        # Act
        result = await grpc_server.UpdateAnnotation(request, AsyncMock())

        # Assert
        assert list(result.segment_ids) == [], "segment_ids should be empty"
```

### 5.2 Agent: Create Sync Task Lifecycle Test

**Agent Type**: `agent-testing-architect`

**File**: `tests/grpc/test_sync_task_lifecycle.py`

**Test Spec**:

```python
"""Test sync task registration and shutdown cancellation."""

import asyncio
import pytest
from unittest.mock import AsyncMock, MagicMock


class TestBackgroundTaskRegistry:
    """Test background task tracking and cleanup."""

    async def test_task_registered_on_sync_start(
        self,
        grpc_server: "NoteFlowServicer",
    ) -> None:
        """Starting sync registers task in background registry."""
        # Arrange
        integration_id = "test-integration"

        # Act
        await grpc_server.StartSync(
            noteflow_pb2.StartSyncRequest(integration_id=integration_id),
            AsyncMock(),
        )

        # Assert
        assert any(
            "sync-" in task_id
            for task_id in grpc_server._background_tasks
        ), "Sync task should be registered"

    async def test_tasks_cancelled_on_shutdown(
        self,
        grpc_server: "NoteFlowServicer",
    ) -> None:
        """Shutdown cancels all registered background tasks."""
        # Arrange
        mock_task = AsyncMock()
        mock_task.done.return_value = False
        grpc_server._background_tasks["test-task"] = mock_task

        # Act
        await grpc_server._shutdown_background_tasks()

        # Assert
        mock_task.cancel.assert_called_once()

    async def test_task_exception_logged(
        self,
        grpc_server: "NoteFlowServicer",
        caplog: pytest.LogCaptureFixture,
    ) -> None:
        """Task exceptions are logged when task completes."""
        # Arrange
        async def failing_task() -> None:
            raise ValueError("Test error")

        task = asyncio.create_task(failing_task())
        grpc_server.register_background_task("failing-task", task)

        # Act - wait for task to complete
        with pytest.raises(ValueError, match="Test error"):
            await task

        # Assert
        assert "Background task failed" in caplog.text
```

### 5.3 Agent: Create Outlook Pagination Test

**Agent Type**: `agent-testing-architect`

**File**: `tests/infrastructure/calendar/test_outlook_pagination.py`

**Test Spec**:

```python
"""Test Outlook adapter pagination handling."""

import pytest
from unittest.mock import AsyncMock, patch

from noteflow.infrastructure.calendar.outlook_adapter import OutlookCalendarAdapter


class TestOutlookPagination:
    """Test @odata.nextLink pagination."""

    async def test_follows_next_link(
        self,
        outlook_adapter: OutlookCalendarAdapter,
    ) -> None:
        """Adapter follows @odata.nextLink for all pages."""
        # Arrange
        page1 = {
            "value": [{"id": "event1"}],
            "@odata.nextLink": "https://graph.microsoft.com/v1.0/next",
        }
        page2 = {
            "value": [{"id": "event2"}],
        }

        mock_responses = [
            AsyncMock(status_code=200, json=lambda: page1),
            AsyncMock(status_code=200, json=lambda: page2),
        ]

        with patch.object(
            outlook_adapter._client, "get", side_effect=mock_responses
        ):
            # Act
            events = await outlook_adapter.get_events(
                start=datetime(2025, 1, 1),
                end=datetime(2025, 1, 31),
            )

        # Assert
        assert len(events) == 2, "Should return events from both pages"

    async def test_respects_limit_across_pages(
        self,
        outlook_adapter: OutlookCalendarAdapter,
    ) -> None:
        """Limit is respected even when spanning pages."""
        # Arrange - 50 events per page, limit of 75
        page1 = {"value": [{"id": f"event{i}"} for i in range(50)]}
        page1["@odata.nextLink"] = "https://graph.microsoft.com/v1.0/next"
        page2 = {"value": [{"id": f"event{i}"} for i in range(50, 100)]}

        # ... mock setup ...

        # Act
        events = await outlook_adapter.get_events(
            start=datetime(2025, 1, 1),
            end=datetime(2025, 1, 31),
            limit=75,
        )

        # Assert
        assert len(events) == 75, "Should stop at limit"
```

**Quality Gate**:
```bash
# After each test file
make quality-py
pytest tests/grpc/test_annotation_patch.py -v
pytest tests/grpc/test_sync_task_lifecycle.py -v
pytest tests/infrastructure/calendar/test_outlook_pagination.py -v
```

---

## Phase 6: Timestamp Converter Helpers

### 6.1 Add Converter Functions

**File**: `src/noteflow/grpc/_mixins/converters.py`

**Agent Type**: `agent-python-executor`

**Implementation Spec**:

```python
"""Add timestamp conversion helpers to reduce ad-hoc conversions."""

from datetime import datetime, UTC
from google.protobuf.timestamp_pb2 import Timestamp


def datetime_to_proto_timestamp(dt: datetime) -> Timestamp:
    """Convert datetime to protobuf Timestamp.

    Args:
        dt: Datetime to convert (must be timezone-aware).

    Returns:
        Protobuf Timestamp message.
    """
    ts = Timestamp()
    ts.FromDatetime(dt)
    return ts


def proto_timestamp_to_datetime(ts: Timestamp) -> datetime:
    """Convert protobuf Timestamp to datetime.

    Args:
        ts: Protobuf Timestamp message.

    Returns:
        Timezone-aware datetime (UTC).
    """
    return ts.ToDatetime().replace(tzinfo=UTC)


def epoch_seconds_to_datetime(seconds: float) -> datetime:
    """Convert Unix epoch seconds to datetime.

    Args:
        seconds: Unix epoch seconds (float for sub-second precision).

    Returns:
        Timezone-aware datetime (UTC).
    """
    return datetime.fromtimestamp(seconds, tz=UTC)


def datetime_to_epoch_seconds(dt: datetime) -> float:
    """Convert datetime to Unix epoch seconds.

    Args:
        dt: Datetime to convert.

    Returns:
        Unix epoch seconds as float.
    """
    return dt.timestamp()


def iso_string_to_datetime(iso_str: str) -> datetime:
    """Parse ISO 8601 string to datetime.

    Args:
        iso_str: ISO 8601 formatted string.

    Returns:
        Timezone-aware datetime (UTC if no timezone in string).
    """
    dt = datetime.fromisoformat(iso_str.replace("Z", "+00:00"))
    if dt.tzinfo is None:
        dt = dt.replace(tzinfo=UTC)
    return dt


def datetime_to_iso_string(dt: datetime) -> str:
    """Format datetime as ISO 8601 string.

    Args:
        dt: Datetime to format.

    Returns:
        ISO 8601 formatted string with timezone.
    """
    return dt.isoformat()
```

**Quality Gate**:
```bash
make quality-py
```

---

## Execution Order & Dependencies

```mermaid
graph TD
    P0[Phase 0: Pre-Flight] --> P1[Phase 1: Discovery]
    P1 --> P2[Phase 2: P0 Critical]
    P2 --> P3[Phase 3: P1 API Contract]
    P3 --> P4[Phase 4: P2 Reliability]
    P4 --> P5[Phase 5: Tests]
    P5 --> P6[Phase 6: Converters]

    subgraph "Phase 2 - Parallel"
        P2A[2.1 Crypto Bounds]
        P2B[2.2 OTEL Insecure]
    end

    subgraph "Phase 4 - Parallel"
        P4A[4.1 Task Registry]
        P4B[4.2 Segmenter Deque]
        P4C[4.3 Size Limits]
        P4D[4.4 Outlook Timeouts]
        P4E[4.5 Webhook ID]
    end
```

**Parallelization Opportunities**:
- Phase 2 fixes (crypto + OTEL) can run in parallel
- Phase 4 fixes (task registry, segmenter, size limits, outlook, webhook) can run in parallel
- Test writing can happen in parallel with implementation

---

## Agent Orchestration Protocol

### MANDATORY: Quality Gate After Every Edit

**Every agent MUST run this after each file modification:**

```bash
# IMMEDIATE - after every file edit
make quality-py

# If ANY failure:
# 1. FIX THE VIOLATION IMMEDIATELY
# 2. Do NOT proceed to next file
# 3. Do NOT claim "preexisting" without baseline proof
```

### Agent Workflow Pattern

```
┌─────────────────────────────────────────────────────────────┐
│  FOR EACH FILE MODIFICATION:                                │
│                                                             │
│  1. Read current file state (use find_symbol/get_overview)  │
│  2. Make edit (use replace_symbol_body or edit tools)       │
│  3. Run: make quality-py                                    │
│  4. IF FAIL → Fix immediately, go to step 3                 │
│  5. IF PASS → Proceed to next edit                          │
│                                                             │
│  NEVER: Skip step 3-4                                       │
│  NEVER: Batch multiple edits before quality check           │
│  NEVER: Change threshold values in tests/quality/           │
└─────────────────────────────────────────────────────────────┘
```

### Discovery Phase Agent Assignment

```yaml
discovery_agents:
  - agent: Explore
    mode: very_thorough
    queries:
      - "Find all timestamp fields in noteflow.proto"
      - "Find all asyncio.create_task calls in grpc/"
      - "Find all httpx.AsyncClient instantiations"

  - agent: feature-dev:code-explorer
    tasks:
      - "Map sentinel value checks in update handlers"
      - "Trace ChunkedAssetReader boundary conditions"
      - "Analyze background task lifecycle"
```

### Implementation Phase Agent Assignment

```yaml
implementation_agents:
  p0_critical:
    - agent: feature-dev:code-architect
      files: [crypto.py]
      task: "Add bounds checks with read_exact helper"

    - agent: agent-python-executor
      files: [otel.py, settings.py]
      task: "Add configurable insecure flag"

  p1_api:
    - agent: feature-dev:code-architect
      files: [noteflow.proto, annotation.py]
      task: "Add optional fields with presence tracking"

  p2_reliability:
    - agent: agent-python-executor
      files: [service.py, sync.py]
      task: "Add background task registry"

    - agent: agent-python-executor
      files: [segmenter.py]
      task: "Replace list with deque"
```

### Test Phase Agent Assignment

```yaml
test_agents:
  - agent: agent-testing-architect
    files:
      - tests/grpc/test_annotation_patch.py
      - tests/grpc/test_sync_task_lifecycle.py
      - tests/infrastructure/calendar/test_outlook_pagination.py
    requirements:
      - No loops in tests
      - No conditionals in tests
      - pytest.mark.parametrize for multiple cases
      - pytest.raises with match=
```

---

## Migration Abort Conditions

**STOP IMMEDIATELY if any of these occur:**

1. **Threshold modification detected** - Any change to `tests/quality/*.py` threshold values
2. **Cumulative violations > 5** - Too many unfixed violations accumulating
3. **Type errors without fix** - `basedpyright` errors not immediately addressed
4. **Baseline not captured** - Starting implementation without `/tmp/quality_baseline_spec.log`
5. **Proto regeneration fails** - Generated stubs have errors

**Recovery**: Revert all changes since last known-good state, re-capture baseline, restart

---

## Success Criteria

- [ ] #6 ChunkedAssetReader bounds checks implemented
- [ ] #10 OTEL insecure flag configurable
- [ ] #2 UpdateAnnotation has presence tracking
- [ ] #11 Status enums added to proto
- [ ] #4 Background task registry implemented
- [ ] #5 Segmenter uses deque
- [ ] #7 gRPC size limits centralized
- [ ] #8 Outlook adapter has timeouts and pagination
- [ ] #9 Webhook delivery ID correlation fixed
- [ ] #6 Timestamp converters added
- [ ] All tests pass
- [ ] All quality checks pass (`make quality-py`)
- [ ] All type checks pass (`basedpyright`)
- [ ] Proto regenerated successfully
- [ ] Documentation updated

---

## Appendix A: Quality Compliance Checklist

### PROHIBITED ACTIONS (Violation = Immediate Rollback)

- [ ] **NEVER** modify threshold values in `tests/quality/*.py`
- [ ] **NEVER** add `# type: ignore` without explicit user approval
- [ ] **NEVER** use `Any` type
- [ ] **NEVER** skip `make quality-py` after a file edit
- [ ] **NEVER** blame "preexisting issues" without baseline comparison

### Per-Edit Verification Workflow

```bash
# After EVERY edit (not batched):
make quality-py

# Expected output for clean code:
# === Ruff (Python Lint) ===
# All checks passed!
# === Basedpyright ===
# 0 errors, 0 warnings, 0 informations
# === Python Test Quality ===
# XX passed in X.XXs

# If ANY failure: FIX IMMEDIATELY before next edit
```

---

## Appendix B: Threshold Values (READ-ONLY Reference)

**These values are READ-ONLY. Agents MUST NOT modify them.**

From `tests/quality/test_code_smells.py`:
```python
MODULE_SOFT_LIMIT = 500
MODULE_HARD_LIMIT = 750
FUNCTION_LINE_LIMIT = 75
COMPLEXITY_LIMIT = 15
PARAMETER_LIMIT = 7
```

From `tests/quality/test_magic_values.py`:
```python
MAX_MAGIC_NUMBERS = 10
MAX_REPEATED_STRINGS = 30
MAX_HARDCODED_PATHS = 0
```

If your code exceeds these limits, **refactor the code**, not the thresholds.

---

## Appendix C: Proto Regeneration Commands

After any `.proto` file changes:

```bash
# Regenerate Python stubs
python -m grpc_tools.protoc -I src/noteflow/grpc/proto \
  --python_out=src/noteflow/grpc/proto \
  --grpc_python_out=src/noteflow/grpc/proto \
  src/noteflow/grpc/proto/noteflow.proto

# Verify imports work
python -c "from noteflow.grpc.proto import noteflow_pb2, noteflow_pb2_grpc"

# Run quality checks
make quality-py
```

**Note**: Rust/TS client stubs need separate regeneration - coordinate with client team.
