# Sprint 0 Completion Plan

> **Current Status**: 100% complete
> **Target**: All verification gates passing
> **Completed**: December 2024

---

## Executive Summary

Sprint 0 established the proto schema, database migrations, dependencies, and feature flags foundation. However, several verification gates, Docker integration, and TypeScript sync remain incomplete. This plan addresses each gap while following established patterns and quality standards.

---

## Task 1: Proto Regeneration Script

**Gap**: No `scripts/regenerate_proto.sh` exists (spec requirement)

**Pattern Source**:
- Python: `CLAUDE.md` documents `grpc_tools.protoc` command
- Rust: `client/src-tauri/build.rs` auto-regenerates via `tonic_build`

**Implementation**:

```bash
# scripts/regenerate_proto.sh
#!/usr/bin/env bash
set -euo pipefail

PROTO_DIR="src/noteflow/grpc/proto"
PROTO_FILE="$PROTO_DIR/noteflow.proto"

echo "=== NoteFlow Proto Regeneration ==="
echo ""

# Python stubs
echo "[1/3] Regenerating Python stubs..."
python -m grpc_tools.protoc \
    -I "$PROTO_DIR" \
    --python_out="$PROTO_DIR" \
    --grpc_python_out="$PROTO_DIR" \
    --pyi_out="$PROTO_DIR" \
    "$PROTO_FILE"

# Fix imports for Python 3.12+ (relative imports in grpc stub)
echo "[2/3] Fixing Python imports..."
if [[ "$OSTYPE" == "darwin"* ]]; then
    sed -i '' 's/^import noteflow_pb2/from . import noteflow_pb2/' "$PROTO_DIR/noteflow_pb2_grpc.py"
else
    sed -i 's/^import noteflow_pb2/from . import noteflow_pb2/' "$PROTO_DIR/noteflow_pb2_grpc.py"
fi

# Rust stubs (via cargo build)
echo "[3/3] Regenerating Rust stubs..."
(cd client && cargo build --package noteflow-tauri 2>/dev/null || echo "Rust build skipped (run 'cd client && cargo build' manually)")

echo ""
echo "=== Proto regeneration complete ==="
echo ""
echo "Files updated:"
echo "  - $PROTO_DIR/noteflow_pb2.py"
echo "  - $PROTO_DIR/noteflow_pb2_grpc.py"
echo "  - $PROTO_DIR/noteflow_pb2.pyi"
echo "  - client/src-tauri/src/grpc/noteflow.rs (if Rust toolchain available)"
echo ""
echo "Next steps:"
echo "  1. Update TypeScript types in client/src/api/types.ts"
echo "  2. Run tests: pytest tests/grpc/"
echo "  3. Commit all generated files together"
```

**Verification**: `bash scripts/regenerate_proto.sh` runs without errors

---

## Task 2: Add updated_at Triggers to Alembic Migrations

**Gap**: Migrations don't create triggers; they rely on `schema.sql` only

**Pattern Source**: `schema.sql` lines 243-245, 273-274 show trigger pattern

**Implementation**: Add trigger creation to both migrations

**File**: `migrations/versions/h2c3d4e5f6g7_add_named_entities_table.py`

Add after `op.create_index`:
```python
# Create updated_at trigger
op.execute("""
    CREATE TRIGGER trg_named_entities_updated_at
    BEFORE UPDATE ON noteflow.named_entities
    FOR EACH ROW EXECUTE FUNCTION noteflow.set_updated_at();
""")
```

Add to `downgrade()` before `op.drop_index`:
```python
op.execute("DROP TRIGGER IF EXISTS trg_named_entities_updated_at ON noteflow.named_entities;")
```

**File**: `migrations/versions/i3d4e5f6g7h8_add_webhook_tables.py`

Add after `op.create_index` for webhook_configs:
```python
# Create updated_at trigger for webhook_configs
op.execute("""
    CREATE TRIGGER trg_webhook_configs_updated_at
    BEFORE UPDATE ON noteflow.webhook_configs
    FOR EACH ROW EXECUTE FUNCTION noteflow.set_updated_at();
""")
```

Add to `downgrade()`:
```python
op.execute("DROP TRIGGER IF EXISTS trg_webhook_configs_updated_at ON noteflow.webhook_configs;")
```

**Verification**: `alembic upgrade head && alembic downgrade -2 && alembic upgrade head`

---

## Task 3: Create Migration Verification Tests

**Gap**: No `tests/infrastructure/persistence/test_migrations.py`

**Pattern Source**:
- Use `support/db_utils.py` utilities
- Follow `tests/integration/conftest.py` session_factory pattern
- Quality standards require type hints, AAA pattern

**Implementation**:

```python
# tests/infrastructure/persistence/test_migrations.py
"""Migration verification tests.

Tests that Alembic migrations can be applied and rolled back correctly.
"""

import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession

from support.db_utils import PgTestContainer


@pytest.fixture(scope="module")
def alembic_config(tmp_path_factory: pytest.TempPathFactory) -> Config:
    """Create Alembic config pointing to test database."""
    config = Config("alembic.ini")
    return config


@pytest.mark.integration
class TestMigrationUpgradeDowngrade:
    """Verify migrations can upgrade and downgrade cleanly."""

    async def test_upgrade_to_head(
        self, session: AsyncSession, alembic_config: Config
    ) -> None:
        """All migrations apply successfully to head."""
        # Arrange: Config already set up

        # Act: Upgrade to head
        command.upgrade(alembic_config, "head")

        # Assert: Tables exist
        result = await session.execute(
            text("SELECT table_name FROM information_schema.tables WHERE table_schema = 'noteflow'")
        )
        tables = {row[0] for row in result.fetchall()}
        assert "named_entities" in tables, "named_entities table should exist"
        assert "webhook_configs" in tables, "webhook_configs table should exist"
        assert "webhook_deliveries" in tables, "webhook_deliveries table should exist"

    async def test_downgrade_named_entities(
        self, session: AsyncSession, alembic_config: Config
    ) -> None:
        """Named entities migration can downgrade."""
        # Arrange: Upgrade to head first
        command.upgrade(alembic_config, "head")

        # Act: Downgrade one step
        command.downgrade(alembic_config, "-1")

        # Assert: webhook tables still exist, named_entities gone
        result = await session.execute(
            text("SELECT table_name FROM information_schema.tables WHERE table_schema = 'noteflow'")
        )
        tables = {row[0] for row in result.fetchall()}
        assert "named_entities" not in tables, "named_entities should be dropped"

    async def test_triggers_exist_after_upgrade(
        self, session: AsyncSession, alembic_config: Config
    ) -> None:
        """Verify updated_at triggers are created."""
        # Arrange: Upgrade to head
        command.upgrade(alembic_config, "head")

        # Act: Check triggers
        result = await session.execute(
            text("""
                SELECT trigger_name FROM information_schema.triggers
                WHERE trigger_schema = 'noteflow'
            """)
        )
        triggers = {row[0] for row in result.fetchall()}

        # Assert: Both triggers exist
        assert "trg_named_entities_updated_at" in triggers
        assert "trg_webhook_configs_updated_at" in triggers
```

**Verification**: `pytest tests/infrastructure/persistence/test_migrations.py -v`

---

## Task 4: Create Proto Compilation Tests

**Gap**: No `tests/grpc/test_proto_compilation.py`

**Pattern Source**: Quality standards require specific assertions with messages

**Implementation**:

```python
# tests/grpc/test_proto_compilation.py
"""Proto compilation and import verification tests.

Ensures proto stubs compile correctly and new Sprint 0 messages exist.
"""

import pytest


class TestProtoImports:
    """Verify proto stubs import without errors."""

    def test_proto_pb2_imports(self) -> None:
        """Proto message module imports cleanly."""
        # Act & Assert: Import should not raise
        from noteflow.grpc.proto import noteflow_pb2

        assert noteflow_pb2 is not None, "noteflow_pb2 module should import"

    def test_proto_pb2_grpc_imports(self) -> None:
        """Proto gRPC module imports cleanly."""
        from noteflow.grpc.proto import noteflow_pb2_grpc

        assert noteflow_pb2_grpc is not None, "noteflow_pb2_grpc module should import"


class TestSprint0Messages:
    """Verify Sprint 0 proto additions exist."""

    def test_summarization_options_exists(self) -> None:
        """SummarizationOptions message exists (Sprint 1)."""
        from noteflow.grpc.proto import noteflow_pb2

        assert hasattr(noteflow_pb2, "SummarizationOptions"), (
            "SummarizationOptions message missing"
        )
        opts = noteflow_pb2.SummarizationOptions()
        assert opts.tone == "", "tone should default to empty string"
        assert opts.format == "", "format should default to empty string"
        assert opts.verbosity == "", "verbosity should default to empty string"

    def test_extract_entities_messages_exist(self) -> None:
        """NER messages exist (Sprint 4)."""
        from noteflow.grpc.proto import noteflow_pb2

        assert hasattr(noteflow_pb2, "ExtractEntitiesRequest")
        assert hasattr(noteflow_pb2, "ExtractEntitiesResponse")
        assert hasattr(noteflow_pb2, "ExtractedEntity")

        entity = noteflow_pb2.ExtractedEntity()
        assert entity.confidence == 0.0, "confidence should default to 0.0"
        assert entity.is_pinned is False, "is_pinned should default to False"

    def test_calendar_messages_exist(self) -> None:
        """Calendar messages exist (Sprint 5)."""
        from noteflow.grpc.proto import noteflow_pb2

        assert hasattr(noteflow_pb2, "CalendarEvent")
        assert hasattr(noteflow_pb2, "CalendarProvider")
        assert hasattr(noteflow_pb2, "ListCalendarEventsRequest")
        assert hasattr(noteflow_pb2, "ListCalendarEventsResponse")
        assert hasattr(noteflow_pb2, "InitiateCalendarAuthRequest")
        assert hasattr(noteflow_pb2, "CompleteCalendarAuthResponse")

    def test_export_format_pdf_exists(self) -> None:
        """EXPORT_FORMAT_PDF enum value exists (Sprint 3)."""
        from noteflow.grpc.proto import noteflow_pb2

        assert noteflow_pb2.EXPORT_FORMAT_PDF == 3, (
            "EXPORT_FORMAT_PDF should equal 3"
        )


class TestSprint0RPCs:
    """Verify Sprint 0 RPC stubs exist."""

    def test_servicer_has_extract_entities(self) -> None:
        """NoteFlowServicer has ExtractEntities method."""
        from noteflow.grpc.proto import noteflow_pb2_grpc

        servicer = noteflow_pb2_grpc.NoteFlowServiceServicer
        assert hasattr(servicer, "ExtractEntities"), "ExtractEntities RPC missing"

    def test_servicer_has_calendar_rpcs(self) -> None:
        """NoteFlowServicer has calendar RPCs."""
        from noteflow.grpc.proto import noteflow_pb2_grpc

        servicer = noteflow_pb2_grpc.NoteFlowServiceServicer
        assert hasattr(servicer, "ListCalendarEvents")
        assert hasattr(servicer, "GetCalendarProviders")
        assert hasattr(servicer, "InitiateCalendarAuth")
        assert hasattr(servicer, "CompleteCalendarAuth")
```

**Verification**: `pytest tests/grpc/test_proto_compilation.py -v`

---

## Task 5: Create Feature Flag Tests

**Gap**: No tests verifying feature flag behavior

**Pattern Source**:
- `tests/integration/test_trigger_settings.py` shows monkeypatch pattern
- Use `cache_clear()` before tests

**Implementation**:

```python
# tests/config/test_feature_flags.py
"""Feature flag configuration tests.

Verifies FeatureFlags class defaults and environment variable parsing.
"""

import pytest

from noteflow.config.settings import FeatureFlags, get_feature_flags


@pytest.fixture(autouse=True)
def _clear_feature_flags_cache() -> None:
    """Clear cached feature flags before each test."""
    get_feature_flags.cache_clear()


class TestFeatureFlagDefaults:
    """Verify default values for feature flags."""

    def test_templates_enabled_by_default(self) -> None:
        """Templates feature enabled by default."""
        flags = FeatureFlags()
        assert flags.templates_enabled is True

    def test_pdf_export_enabled_by_default(self) -> None:
        """PDF export feature enabled by default."""
        flags = FeatureFlags()
        assert flags.pdf_export_enabled is True

    def test_ner_disabled_by_default(self) -> None:
        """NER feature disabled by default (requires model download)."""
        flags = FeatureFlags()
        assert flags.ner_enabled is False

    def test_calendar_disabled_by_default(self) -> None:
        """Calendar feature disabled by default (requires OAuth setup)."""
        flags = FeatureFlags()
        assert flags.calendar_enabled is False

    def test_webhooks_enabled_by_default(self) -> None:
        """Webhooks feature enabled by default."""
        flags = FeatureFlags()
        assert flags.webhooks_enabled is True


class TestFeatureFlagEnvironment:
    """Verify environment variable parsing."""

    @pytest.mark.parametrize(
        ("env_var", "attr", "value", "expected"),
        [
            pytest.param(
                "NOTEFLOW_FEATURE_NER_ENABLED", "ner_enabled", "true", True,
                id="ner-enabled-true"
            ),
            pytest.param(
                "NOTEFLOW_FEATURE_NER_ENABLED", "ner_enabled", "false", False,
                id="ner-enabled-false"
            ),
            pytest.param(
                "NOTEFLOW_FEATURE_CALENDAR_ENABLED", "calendar_enabled", "true", True,
                id="calendar-enabled-true"
            ),
            pytest.param(
                "NOTEFLOW_FEATURE_WEBHOOKS_ENABLED", "webhooks_enabled", "false", False,
                id="webhooks-disabled"
            ),
        ],
    )
    def test_env_var_parsing(
        self,
        monkeypatch: pytest.MonkeyPatch,
        env_var: str,
        attr: str,
        value: str,
        expected: bool,
    ) -> None:
        """Feature flags parse from environment variables."""
        # Arrange
        monkeypatch.setenv(env_var, value)

        # Act
        flags = get_feature_flags()

        # Assert
        assert getattr(flags, attr) == expected


class TestFeatureFlagCaching:
    """Verify feature flags are cached."""

    def test_get_feature_flags_cached(self) -> None:
        """get_feature_flags returns same instance."""
        # Act
        flags1 = get_feature_flags()
        flags2 = get_feature_flags()

        # Assert
        assert flags1 is flags2, "Should return cached instance"
```

**Verification**: `pytest tests/config/test_feature_flags.py -v`

---

## Task 6: Update TypeScript Types for Sprint 0

**Gap**: Missing NER types, Calendar auth types, and PDF in ExportFormat

**Pattern Source**: `client/src/api/types.ts` existing patterns

**Implementation**:

### 6a. Update ExportFormat enum

**File**: `client/src/api/types.ts` (line ~83)

```typescript
// Before
export type ExportFormat = 'markdown' | 'html';

// After
export type ExportFormat = 'markdown' | 'html' | 'pdf';
```

### 6b. Update grpcEnums mapping

**File**: `client/src/api/tauri-adapter.ts` (line ~432)

```typescript
// Before
format: { markdown: 1, html: 2 },

// After
format: { markdown: 1, html: 2, pdf: 3 },
```

### 6c. Add NER types

**File**: `client/src/api/types.ts` (add after existing types)

```typescript
// =============================================================================
// Named Entity Extraction Types (Sprint 4)
// =============================================================================

/** Entity category from NER extraction */
export type EntityCategory =
  | 'person'
  | 'company'
  | 'product'
  | 'technical'
  | 'acronym'
  | 'location'
  | 'date'
  | 'other';

/** Extracted entity from meeting transcript */
export interface ExtractedEntity {
  id: string;
  text: string;
  category: EntityCategory;
  segment_ids: number[];
  confidence: number;
  is_pinned: boolean;
}

/** Request to extract entities from a meeting */
export interface ExtractEntitiesRequest {
  meeting_id: string;
  force_refresh?: boolean;
}

/** Response from entity extraction */
export interface ExtractEntitiesResponse {
  entities: ExtractedEntity[];
  total_count: number;
  cached: boolean;
}
```

### 6d. Add Calendar auth types

**File**: `client/src/api/types.ts` (add after calendar types)

```typescript
// =============================================================================
// Calendar Authentication Types (Sprint 5)
// =============================================================================

/** Calendar provider information */
export interface CalendarProvider {
  name: string;
  is_authenticated: boolean;
  display_name: string;
}

/** Request to list calendar events */
export interface ListCalendarEventsRequest {
  hours_ahead?: number;
  limit?: number;
  provider?: string;
}

/** Response from calendar events list */
export interface ListCalendarEventsResponse {
  events: CalendarEvent[];
  total_count: number;
}

/** Request to get available calendar providers */
export interface GetCalendarProvidersRequest {}

/** Response with available calendar providers */
export interface GetCalendarProvidersResponse {
  providers: CalendarProvider[];
}

/** Request to initiate OAuth flow */
export interface InitiateCalendarAuthRequest {
  provider: string;
  redirect_uri: string;
}

/** Response with OAuth authorization URL */
export interface InitiateCalendarAuthResponse {
  auth_url: string;
  state: string;
}

/** Request to complete OAuth flow */
export interface CompleteCalendarAuthRequest {
  provider: string;
  code: string;
  state: string;
}

/** Response from OAuth completion */
export interface CompleteCalendarAuthResponse {
  success: boolean;
  error_message?: string;
  provider_email?: string;
}
```

### 6e. Update entity.ts to match proto

**File**: `client/src/types/entity.ts` (line ~3)

```typescript
// Before
export type EntityCategory = 'person' | 'company' | 'product' | 'technical' | 'acronym' | 'other';

// After (matches proto definition)
export type EntityCategory =
  | 'person'
  | 'company'
  | 'product'
  | 'technical'
  | 'acronym'
  | 'location'
  | 'date'
  | 'other';
```

**Verification**: `cd client && npm run lint && npm run type-check`

---

## Task 7: Add httpx Dependency

**Gap**: Webhooks will need HTTP client; httpx is the modern async choice

**Pattern Source**: No existing HTTP client in codebase (uses SDK clients only)

**Implementation**:

**File**: `pyproject.toml` (line ~31, after existing dependencies)

```toml
dependencies = [
    # ... existing deps ...

    # HTTP client for webhooks and integrations (Sprint 6)
    "httpx>=0.27",
]
```

**Verification**: `pip install -e . && python -c "import httpx; print(httpx.__version__)"`

---

## Task 8: Create Docker NER Stage

**Gap**: No Dockerfile stage for NER model download

**Pattern Source**: `docker/server.Dockerfile` existing patterns

**Implementation**:

**File**: `docker/server.Dockerfile` (add after existing content)

```dockerfile
# =============================================================================
# Base stage (existing)
# =============================================================================
FROM python:3.12-bookworm AS base

ENV PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    pkg-config \
    portaudio19-dev \
    libsndfile1 \
    && rm -rf /var/lib/apt/lists/*

COPY . /workspace
WORKDIR /workspace

RUN python -m pip install --upgrade pip && \
    python -m pip install -e ".[dev]" watchfiles

EXPOSE 50051

# =============================================================================
# NER models stage (Sprint 4)
# =============================================================================
FROM base AS with-ner

# Install NER dependencies and download spaCy model
RUN python -m pip install -e ".[ner]" && \
    python -m spacy download en_core_web_sm

# Verify model is available
RUN python -c "import spacy; nlp = spacy.load('en_core_web_sm'); print('NER model loaded')"

# =============================================================================
# Development target (default)
# =============================================================================
FROM base AS dev

CMD ["python", "scripts/dev_watch_server.py"]

# =============================================================================
# Production with NER target
# =============================================================================
FROM with-ner AS production-ner

CMD ["python", "-m", "noteflow.grpc.server"]
```

**Verification**: `docker build --target with-ner -t noteflow:ner .`

---

## Task 9: Update compose.yaml with Feature Flags

**Gap**: No feature flag environment variables, db service commented out

**Implementation**:

**File**: `compose.yaml`

```yaml
services:
  server:
    build:
      context: .
      dockerfile: docker/server.Dockerfile
      target: dev  # Use 'with-ner' for NER support
    ports:
      - "50051:50051"
    env_file:
      - .env
    environment:
      # Feature flags (can override .env)
      NOTEFLOW_FEATURE_TEMPLATES_ENABLED: "true"
      NOTEFLOW_FEATURE_PDF_EXPORT_ENABLED: "true"
      NOTEFLOW_FEATURE_NER_ENABLED: "false"
      NOTEFLOW_FEATURE_CALENDAR_ENABLED: "false"
      NOTEFLOW_FEATURE_WEBHOOKS_ENABLED: "true"
    command: python scripts/dev_watch_server.py
    volumes:
      - .:/workspace
    depends_on:
      db:
        condition: service_healthy

  frontend:
    image: node:20-alpine
    working_dir: /app
    ports:
      - "5173:5173"
    volumes:
      - ./client:/app
    command: sh -c "npm install -g npm@latest && npm install && npm update caniuse-lite browserslist || true && npm run dev"
    environment:
      - NODE_ENV=development

  db:
    image: pgvector/pgvector:pg15
    ports:
      - "5432:5432"
    environment:
      POSTGRES_DB: noteflow
      POSTGRES_USER: noteflow
      POSTGRES_PASSWORD: noteflow
    volumes:
      - noteflow_pg_data:/var/lib/postgresql/data
      - ./docker/db/schema.sql:/docker-entrypoint-initdb.d/01-schema.sql:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U noteflow -d noteflow"]
      interval: 5s
      timeout: 5s
      retries: 10

volumes:
  noteflow_pg_data:
```

**Verification**: `docker compose up -d db && docker compose ps`

---

## Task 10: Run Verification Gates

**Verification Checklist**:

```bash
# 1. Proto compilation
python -c "from noteflow.grpc.proto import noteflow_pb2; print('Proto OK')"

# 2. Feature flags
python -c "from noteflow.config.settings import get_feature_flags; print(get_feature_flags())"

# 3. Quality suite
pytest tests/quality/ -v

# 4. New tests
pytest tests/grpc/test_proto_compilation.py -v
pytest tests/config/test_feature_flags.py -v
pytest tests/infrastructure/persistence/test_migrations.py -v --ignore-glob="*test_repositories*"

# 5. Type checking
ruff check src/noteflow
mypy src/noteflow
basedpyright

# 6. Client checks
cd client && npm run lint && npm run type-check

# 7. Docker build
docker build --target with-ner -t noteflow:ner-test .

# 8. Proto regeneration script
bash scripts/regenerate_proto.sh
```

---

## Definition of Done Checklist

- [ ] `scripts/regenerate_proto.sh` exists and runs cleanly
- [ ] Migrations create `updated_at` triggers
- [ ] `tests/infrastructure/persistence/test_migrations.py` passes
- [ ] `tests/grpc/test_proto_compilation.py` passes
- [ ] `tests/config/test_feature_flags.py` passes
- [ ] TypeScript types include PDF, NER, Calendar auth
- [ ] `httpx>=0.27` in `pyproject.toml`
- [ ] Docker `with-ner` target builds successfully
- [ ] `compose.yaml` has feature flag env vars and active db service
- [ ] `pytest tests/quality/` passes without new violations
- [ ] `ruff check` and `mypy` clean
- [ ] `npm run lint` and `npm run type-check` clean in client

---

## Dependency Order

```
Task 1 (proto script) ──┐
Task 2 (triggers)    ───┼──> Task 3 (migration tests)
Task 7 (httpx)       ───┘

Task 4 (proto tests) ──> independent
Task 5 (flag tests)  ──> independent
Task 6 (TS types)    ──> independent
Task 8 (Docker NER)  ──> Task 9 (compose)

All tasks ──> Task 10 (verification)
```

**Parallelization**: Tasks 1, 4, 5, 6, 7, 8 can run in parallel. Tasks 2, 3 are sequential. Task 10 runs last.
