# Sprint 14: Preferences Sync

> **Priority**: 8 | **Owner**: Backend + Client | **Complexity**: Medium
> **Prerequisite**: Sprint 12, Sprint 13

---

## ✅ IMPLEMENTATION COMPLETE (2025-12-29)

All components implemented and tested. Preferences sync with ETag-based conflict detection.

| Component | Location | Status |
|-----------|----------|--------|
| Backend preferences repo | `repositories/preferences_repo.py` | ✅ Full CRUD + bulk operations (get_all, set_bulk, get_all_with_metadata) |
| Preferences table | `models/identity/settings.py:83-102` | ✅ UserPreferencesModel with JSONB |
| Client preferences | `client/src/lib/preferences.ts` (499 lines) | ✅ Complete with Tauri + localStorage |
| **gRPC endpoints** | `grpc/_mixins/preferences.py` | ✅ **IMPLEMENTED** - GetPreferences/SetPreferences RPCs |
| **Proto messages** | `grpc/proto/noteflow.proto` | ✅ **IMPLEMENTED** - Full sync protocol with ETag |
| **Preferences sync module** | `client/src/lib/preferences-sync.ts` | ✅ **IMPLEMENTED** - hydrate/push/conflict resolution |
| **Sync status component** | `client/src/components/preferences-sync-status.tsx` | ✅ **IMPLEMENTED** - Full UI with conflict dialog |
| **React hook** | `client/src/hooks/use-preferences-sync.ts` | ✅ **IMPLEMENTED** - Reactive state management |
| PreferencesSyncMeta type | `client/src/lib/preferences-sync.ts` | ✅ **IMPLEMENTED** - etag, serverUpdatedAt, syncState |
| Backend tests | `tests/grpc/test_preferences_mixin.py` | ✅ 13 tests passing |
| Integration tests | `tests/integration/test_preferences_repository.py` | ✅ 8 bulk operation tests |
| Client tests | `client/src/lib/preferences-sync.test.ts`, `client/src/components/preferences-sync-status.test.tsx` | ✅ Core sync + status UI covered |

**Note**: Sprint 12 (connection-state) is implemented; preferences sync uses connection state to pause sync operations when offline.

---

## Objective

Persist user preferences in the backend database so settings survive reinstall and can sync across devices, with **conflict-safe updates** and **read-only behavior when offline**.

---

## Current State Analysis

### Backend Infrastructure

**PreferencesRepository** (`repositories/preferences_repo.py`):
- `get(key)`, `set(key, value)`, `delete(key)` for KV-style settings
- Currently used for cloud consent and other global settings

**Database Schema** (`models/identity/settings.py:82-100`):
```python
class UserPreferencesModel(Base):
    __tablename__ = "user_preferences"

    key: Mapped[str] = mapped_column(String(64), primary_key=True)
    value: Mapped[dict[str, object]] = mapped_column(JSONB, nullable=False, default=dict)
    updated_at: Mapped[datetime] = mapped_column(onupdate=utc_now)
```

**Gap**: No gRPC endpoints to read/write preferences.

### Client Preferences

**Preferences Store** (`client/src/api/types/requests.ts:290`, `client/src/lib/preferences.ts`):
- `UserPreferences` uses snake_case fields (e.g., `simulate_transcription`, `default_export_format`, `ai_config`, etc.)

**Storage**:
- Browser: `localStorage`
- Desktop: Tauri file storage (`preferences.json`)
- No backend sync

---

## Scope

| Task | Effort | Risk |
|------|--------|------|
| Add preferences gRPC endpoints | Medium | Low |
| Add startup hydration | Low | Low |
| Add save-time sync | Low | Low |
| Handle conflict resolution (etag + timestamp) | Medium | Medium |
| Add sync status indicator | Low | Low |
| Enforce cached read-only behavior when offline | Low | Low |
| **Phase 5 prep: Add user_id column** | Low | Low |

---

## Phase 5 Preparation (Added 2025-12-29)

> Sprint 16 (Identity Foundation) introduces user/workspace context. Preparing now avoids a migration later.

### Task 0: Extend Schema for User Scoping

**File**: `src/noteflow/infrastructure/persistence/models/identity/settings.py`

Update `UserPreferencesModel` to include optional `user_id`:

```python
from uuid import UUID
from sqlalchemy import ForeignKey

class UserPreferencesModel(Base):
    """Store key-value user preferences for persistence across server restarts."""

    __tablename__ = "user_preferences"
    __table_args__: ClassVar[dict[str, str]] = {"schema": "noteflow"}

    # Composite key: (user_id, key) for multi-user support
    # user_id is nullable for local-first mode (Sprint 16 will populate it)
    user_id: Mapped[UUID | None] = mapped_column(
        ForeignKey("noteflow.users.id", ondelete="CASCADE"),
        primary_key=True,
        nullable=True,  # NULL = local/default user (Phase 4)
        default=None,
    )
    key: Mapped[str] = mapped_column(String(64), primary_key=True)
    value: Mapped[dict[str, object]] = mapped_column(JSONB, nullable=False, default=dict)
    updated_at: Mapped[datetime] = mapped_column(
        DateTime(timezone=True),
        nullable=False,
        default=utc_now,
        onupdate=utc_now,
    )
```

**Migration**: `alembic revision --autogenerate -m "add user_id to preferences"`

```python
def upgrade() -> None:
    op.add_column(
        'user_preferences',
        sa.Column('user_id', postgresql.UUID(as_uuid=True), nullable=True),
        schema='noteflow'
    )
    op.create_foreign_key(
        'fk_user_preferences_user_id',
        'user_preferences', 'users',
        ['user_id'], ['id'],
        source_schema='noteflow', referent_schema='noteflow',
        ondelete='CASCADE'
    )
    # Drop old primary key, create composite
    op.drop_constraint('user_preferences_pkey', 'user_preferences', schema='noteflow')
    op.create_primary_key(
        'user_preferences_pkey',
        'user_preferences',
        ['user_id', 'key'],
        schema='noteflow'
    )

def downgrade() -> None:
    op.drop_constraint('user_preferences_pkey', 'user_preferences', schema='noteflow')
    op.create_primary_key(
        'user_preferences_pkey',
        'user_preferences',
        ['key'],
        schema='noteflow'
    )
    op.drop_constraint('fk_user_preferences_user_id', 'user_preferences', schema='noteflow')
    op.drop_column('user_preferences', 'user_id', schema='noteflow')
```

**Repository Update**: `repositories/preferences_repo.py`

```python
class SqlAlchemyPreferencesRepository(PreferencesRepository):
    def __init__(self, session: AsyncSession, user_id: UUID | None = None) -> None:
        self._session = session
        self._user_id = user_id  # None = local/default user

    async def get(self, key: str) -> dict[str, object] | None:
        query = select(UserPreferencesModel).where(
            UserPreferencesModel.key == key,
            UserPreferencesModel.user_id == self._user_id,  # None matches NULL
        )
        result = await self._session.execute(query)
        model = result.scalar_one_or_none()
        return model.value if model else None
```

**Impact**:
- Existing preferences with `user_id = NULL` continue to work
- Sprint 16 sets `user_id` once identity is established
- No breaking changes to current functionality

---

## Implementation Plan

### Task 1: Add Preferences gRPC Endpoints

**Proto Additions** (`noteflow.proto`):
```protobuf
import "google/protobuf/struct.proto";
import "google/protobuf/timestamp.proto";

message GetPreferencesRequest {}
message GetPreferencesResponse {
  google.protobuf.Struct preferences = 1;
  google.protobuf.Timestamp updated_at = 2;
  string etag = 3;
}

message SetPreferencesRequest {
  google.protobuf.Struct preferences = 1;
  string if_match = 2;  // ETag for optimistic concurrency
  google.protobuf.Timestamp client_updated_at = 3;
}
message SetPreferencesResponse {
  bool success = 1;
  bool conflict = 2;
  google.protobuf.Struct server_preferences = 3;  // If conflict
  google.protobuf.Timestamp server_updated_at = 4;
  string etag = 5;
}
```

**gRPC Mixin** (`preferences.py`):
> Use shared conversion helpers (planned in `src/noteflow/grpc/_mixins/converters.py`) for Struct ↔ dict and Timestamp conversion to stay compliant with existing proto conversion patterns.
```python
"""Preferences sync gRPC mixin."""

from __future__ import annotations

import json
from datetime import datetime

import grpc

from noteflow.grpc import noteflow_pb2
from noteflow.grpc._mixins.errors import abort_invalid_argument
from noteflow.grpc._mixins.protocols import ServicerHost
from noteflow.infrastructure.persistence.repositories.preferences_repo import (
    PreferencesRepository,
)


class PreferencesMixin:
    """Mixin providing preference sync endpoints."""

    async def GetPreferences(
        self: ServicerHost,
        request: noteflow_pb2.GetPreferencesRequest,
        context: grpc.aio.ServicerContext,
    ) -> noteflow_pb2.GetPreferencesResponse:
        """Return user preferences from server."""
        async with self._create_repository_provider() as uow:
            raw = await uow.preferences.get("preferences")
            updated_at = await uow.preferences.get_updated_at("preferences")
            etag = await uow.preferences.get_etag("preferences")

        return noteflow_pb2.GetPreferencesResponse(
            preferences=struct_from_dict(raw or {}),
            updated_at=to_timestamp(updated_at),
            etag=etag,
        )

    async def SetPreferences(
        self: ServicerHost,
        request: noteflow_pb2.SetPreferencesRequest,
        context: grpc.aio.ServicerContext,
    ) -> noteflow_pb2.SetPreferencesResponse:
        """Persist user preferences on server with conflict detection."""
        new_prefs = struct_to_dict(request.preferences)

        async with self._create_repository_provider() as uow:
            result = await uow.preferences.set_with_etag(
                "preferences",
                new_prefs,
                if_match=request.if_match,
            )
            await uow.commit()

        return noteflow_pb2.SetPreferencesResponse(
            success=result.success,
            conflict=result.conflict,
            server_preferences=struct_from_dict(result.current_value or {}),
            server_updated_at=to_timestamp(result.updated_at),
            etag=result.etag,
        )
```

### Task 2: Client Sync Infrastructure

**File**: `client/src/lib/preferences-sync.ts`

```typescript
import { getAPI } from '../api';
import { getConnectionState } from '../api/connection-state';  // From Sprint 12
import type { UserPreferences } from '../api/types';
import { preferences } from './preferences';

// NOTE: UserPreferences interface must be extended with sync metadata:
// interface UserPreferences {
//   ...existing fields...
//   _etag?: string;       // ETag from server
//   _updatedAt?: string;  // ISO timestamp for conflict detection
// }

export interface SyncState {
  status: 'idle' | 'syncing' | 'synced' | 'conflict' | 'error';
  lastSyncedAt: Date | null;
  error: string | null;
}

let syncState: SyncState = {
  status: 'idle',
  lastSyncedAt: null,
  error: null,
};

const listeners: Set<(state: SyncState) => void> = new Set();

function notifyListeners(): void {
  for (const listener of listeners) {
    listener(syncState);
  }
}

export function getSyncState(): SyncState {
  return { ...syncState };
}

export function subscribeSyncState(listener: (state: SyncState) => void): () => void {
  listeners.add(listener);
  return () => listeners.delete(listener);
}

/**
 * Pull preferences from server on startup.
 * Merges with local preferences, preferring newer values.
 */
export async function hydrateFromServer(): Promise<void> {
  const connection = getConnectionState();
  if (connection.mode !== 'connected') {
    return; // Skip if offline
  }

  syncState = { ...syncState, status: 'syncing' };
  notifyListeners();

  try {
    const api = getAPI();
    const response = await api.getPreferences();

    if (!response.preferences || Object.keys(response.preferences).length === 0) {
      // No server preferences, push local to server
      await pushToServer();
      return;
    }

    const serverPrefs = response.preferences as Partial<UserPreferences>;
    const serverUpdatedAt = response.updatedAt
      ? new Date(response.updatedAt)
      : null;

    const localPrefs = preferences.get();
    const localUpdatedAt = localPrefs._updatedAt
      ? new Date(localPrefs._updatedAt)
      : null;

    // Merge strategy: prefer more recent
    if (serverUpdatedAt && localUpdatedAt) {
      if (serverUpdatedAt > localUpdatedAt) {
        // Server is newer, use server
        preferences.replace({
          ...localPrefs,
          ...serverPrefs,
          _etag: response.etag,
          _updatedAt: response.updatedAt,
        });
      } else {
        // Local is newer, push to server
        await pushToServer();
      }
    } else if (serverUpdatedAt) {
      // Only server has timestamp, use server
      preferences.replace({
        ...localPrefs,
        ...serverPrefs,
        _etag: response.etag,
        _updatedAt: response.updatedAt,
      });
    }
    // If only local has timestamp or neither has, keep local

    syncState = {
      status: 'synced',
      lastSyncedAt: new Date(),
      error: null,
    };
    notifyListeners();
  } catch (error) {
    syncState = {
      status: 'error',
      lastSyncedAt: syncState.lastSyncedAt,
      error: error instanceof Error ? error.message : 'Sync failed',
    };
    notifyListeners();
  }
}

/**
 * Push local preferences to server.
 */
export async function pushToServer(): Promise<void> {
  const connection = getConnectionState();
  if (connection.mode !== 'connected') {
    return; // Skip if offline
  }

  syncState = { ...syncState, status: 'syncing' };
  notifyListeners();

  try {
    const api = getAPI();
    const prefs = preferences.get();
    const response = await api.setPreferences({
      preferences: prefs,
      ifMatch: prefs._etag || '',
      clientUpdatedAt: prefs._updatedAt || '',
    });

    if (response.conflict) {
      // Server has newer data
      syncState = {
        status: 'conflict',
        lastSyncedAt: syncState.lastSyncedAt,
        error: 'Server has newer preferences',
      };
      notifyListeners();

      // Option: auto-resolve by merging
      const serverPrefs = response.serverPreferences as Partial<UserPreferences>;
      preferences.replace({
        ...prefs,
        ...serverPrefs,
        _etag: response.etag,
        _updatedAt: response.serverUpdatedAt,
      });

      syncState = { status: 'synced', lastSyncedAt: new Date(), error: null };
      notifyListeners();
    } else {
      syncState = {
        status: 'synced',
        lastSyncedAt: new Date(),
        error: null,
      };
      notifyListeners();
    }
  } catch (error) {
    syncState = {
      status: 'error',
      lastSyncedAt: syncState.lastSyncedAt,
      error: error instanceof Error ? error.message : 'Push failed',
    };
    notifyListeners();
  }
}
```

### Task 3: Wire Startup Hydration

**File**: `client/src/App.tsx` (or startup file)

```typescript
import { useEffect } from 'react';
import { hydrateFromServer } from './lib/preferences-sync';
import { useConnection } from './contexts/connection-context';

export function App() {
  const { isOnline } = useConnection();

  useEffect(() => {
    if (isOnline) {
      hydrateFromServer();
    }
  }, [isOnline]);

  // ...
}
```

### Task 4: Wire Save-Time Sync ✅ Implemented

**Implementation**: `client/src/hooks/use-preferences-sync.ts`

Preferences changes are observed via `preferences.subscribe()`, debounced, and pushed to the
backend with `pushToServer()` when the connection is `connected`. This avoids circular
dependencies in `preferences.ts` while ensuring save-time sync.

### Task 5: Sync Status Indicator ✅ Implemented

**File**: `client/src/components/preferences-sync-status.tsx`

Renders a Preferences Sync card with status icon, last synced timestamp, error/conflict
details, and action buttons (pull/push or conflict resolution). Uses
`usePreferencesSync({ passive: true })` to avoid duplicate sync work.

---

## Behavioral Tests (Implemented)

**Backend**
- `tests/grpc/test_preferences_mixin.py` (13 tests; gRPC Get/Set preferences)
- `tests/integration/test_preferences_repository.py` (8 tests; bulk repo operations)

**Client**
- `client/src/lib/preferences-sync.test.ts` (hydrate/push/conflict paths)
- `client/src/components/preferences-sync-status.test.tsx` (status UI states)

---

## Acceptance Criteria

### Functional

- [x] Preferences sync from server on startup
- [x] Preferences push to server on save
- [x] Preferences survive app reinstall
- [x] Conflicts auto-resolve (prefer newer)
- [x] Sync status visible in UI

### Technical

- [x] Sync respects connection state (cached read-only when offline)
- [x] Invalid payload rejected with clear error
- [x] ETag + timestamp used for conflict detection
- [x] Background sync doesn't block UI

### Quality Gates

- [x] `pytest tests/grpc/test_preferences_mixin.py` passes
- [x] `pytest tests/integration/test_preferences_repository.py` passes
- [x] `npm run test` passes for frontend
- [x] Manual test: change prefs, reinstall, verify restore
- [x] Meets `docs/sprints/QUALITY_STANDARDS.md` (lint + test quality thresholds)

---

## Rollback Plan

1. **Sync issues**: Disable backend sync, use local only
2. **Conflict issues**: Remove conflict detection, last-write-wins
3. **Performance issues**: Debounce pushes, reduce sync frequency

---

## Post-Sprint

- Add manual "Sync Now" button
- Add preferences export/import
- Add multi-device sync notification
- Consider WebSocket for real-time sync
