Files

Travis Vasceannie 49354b80e9 feat: enhance project management and observability features

- Updated client submodule to the latest commit for improved integration.
- Added `types-psutil` as a dependency in `pyproject.toml` to support type checking for the `psutil` library.
- Enhanced observability by refining the `UsageEventSink` interface to include additional attributes for better event tracking.
- Improved project management documentation to reflect recent changes in project roles and settings.

All quality checks pass.

2025-12-31 04:51:39 +00:00

40 KiB

Raw Blame History

NoteFlow Feature Gap Analysis & Development Roadmap

Generated: 2025-12-23 | Updated: 2025-12-29 Focus: Platform evolution — auth, workspaces, artifacts, rules, AI orchestration

Executive Summary

This document identifies features not yet developed and provides a phased roadmap for NoteFlow's evolution from a single-user meeting tool to a multi-user, rule-driven knowledge platform with RAG capabilities.

Current State (Validation 2025-12-29)

Component	Status	Coverage
Transcription	✅ Complete	Real-time streaming, VAD, partial/final segments
Summarization	✅ Complete	AI generation with templates, citation verification
Diarization	✅ Complete	Streaming + offline refinement, speaker renaming
Export	✅ Complete	Markdown/HTML/PDF with styling
NER	✅ Complete	Entity extraction with segment tracking
Integrations	✅ Complete	Calendar sync, webhooks, observability
OAuth	✅ Complete	PKCE S256, deep links, encrypted token storage
Triggers	✅ Complete	Audio, foreground app, calendar (23 config fields)
Offline Mode	✅ Complete	Cached read-only mode + offline banner + guarded mutations
Simulation Toggle	✅ Complete	Toggle UI + MockTranscriptionStream + SimulationIndicator
Preferences Sync	✅ Complete	gRPC endpoints + ETag conflict detection + sync module

Existing Infrastructure (Verified 2025-12-29)

Asset	Location	Status	Implication
`WorkspaceModel`, `UserModel`, `WorkspaceMembershipModel`	`persistence/models/identity/identity.py`	✅	Multi-tenancy modeled
`MeetingModel.workspace_id`, `TaskModel.workspace_id`	ORM models	✅	Workspace scoping exists
`SegmentModel.embedding` (1536 dims)	`models/core/meeting.py:202-205`	✅	Vector infrastructure ready
`cosine_distance` similarity query	`repositories/segment_repo.py:148`	✅	RAG retrieval plumbing exists
Generic OAuth messages	`noteflow.proto` lines 54-58, 766-857	✅	OAuth extensible
`LogBuffer`	`infrastructure/logging/log_buffer.py`	✅	Observability foundation exists
`MetricsCollector`	`infrastructure/metrics/collector.py`	✅	Performance monitoring exists
`SummaryModel.tokens_used`, `latency_ms`	`models/core/summary.py`	✅	Usage tracking partial
`TriggerSettings` (23 fields)	`config/settings.py`	✅	Rules-like config exists
`parse_meeting_id_or_abort()`	`grpc/_mixins/converters.py`	✅	Error pattern exists

Phase 5 Implementation Status (2025-12-29)

Sprint Status Overview

Sprint	Name	Status	Blockers
15	Platform Hardening	✅ Complete	Central error taxonomy, OTel instrumentation, usage events, metadata persistence
16	Identity Foundation	✅ Ready	Prerequisites verified
17	Custom OAuth	✅ Ready	Prerequisites verified
18	Projects v1	❌ Not started	—
19	Artifacts v1	✅ Ready	Prerequisites verified
20	Artifacts v2	🚫 Blocked	Requires Sprint 19
21	MCP Configuration	🚫 Blocked	Requires Sprint 18 (Projects)
22	Rules v1	🚫 Blocked	Requires Sprint 21 (MCP)
23	Analytics	🚫 Blocked	Requires Sprint 15 (Usage) + Sprint 22 (Rules)
24	Graph + Explore	⚠️ Partial	Core ready, awaits Sprint 18/19
25+	LangGraph	🚫 Blocked	Requires Sprint 15 (Usage) + Sprint 21 (MCP)

Critical Path

Sprint 15 (OpenTelemetry + Usage Events + Metadata) ───────────┐
                                                               │
Sprint 18 (Projects) ─────┬───────────────────────────────────┐│
                          │                                   ││
                          ▼                                   ▼▼
                   Sprint 21 (MCP) ────► Sprint 22 (Rules) ────► Sprint 23 (Analytics)
                          │
                          └──────────────────────────────────────► Sprint 25 (LangGraph)

Unblock Order: Sprint 15 + Sprint 18 → Sprint 21 → Sprint 22 → Sprint 23/25

Parallel Execution Map

Wave 1: Foundation (Can Start Immediately)

Sprint	Why Parallelizable	Dependencies
15 Platform Hardening	Backend-only, no UI	Phase 4 complete
16 Identity Foundation	Prerequisites verified	Sprint 15 (can overlap)
17 Custom OAuth	Prerequisites verified	Sprint 16
18 Projects v1	Independent of 15/17	Sprint 16
19 Artifacts v1	Vector infra ready	Sprint 18 (can start core work early)

Recommended parallel groupings:

🔀 Sprint 15 + Sprint 16 (backend focus, different subsystems)
🔀 Sprint 17 + Sprint 18 (both depend on 16, don't overlap)
🔀 Sprint 19 + Sprint 21 (both depend on 18, different concerns)

Wave 2: Integration (After Wave 1)

Sprint	Why Sequential	Dependencies
20 Artifacts v2	Requires Artifacts v1 pipeline	Sprint 19
21 MCP Configuration	Requires Project scoping	Sprint 18
22 Rules v1	Requires MCP + Identity	Sprint 16, Sprint 21

Recommended parallel groupings:

🔀 Sprint 20 + Sprint 21 (different pipelines, both after 18/19)
🔀 Sprint 22 + Sprint 24 (Rules backend + Graph frontend, minimal overlap)

Wave 3: Intelligence (After Wave 2)

Sprint	Why Sequential	Dependencies
23 Analytics	Requires Rules + Usage events	Sprint 15, Sprint 22
24 Graph + Explore	Requires Projects + Artifacts	Sprint 18, Sprint 19
25+ LangGraph	Requires MCP + Usage tracking	Sprint 15, Sprint 21

Recommended parallel groupings:

🔀 Sprint 23 + Sprint 25 (both depend on 15+21/22, different AI subsystems)
🔀 Sprint 24 can run alongside 23/25 if 18+19 complete

Sprint Overview

Phase 4: Productization (Validation 2025-12-29)

Sprint	Name	Status	Finding
10	Integration Config + OAuth	✅ Complete	All 10 components verified with line numbers
11	Trigger System Wiring	✅ Complete	All 9 components verified (23 TriggerSettings fields confirmed)
12	Tauri Fallback & Offline	✅ Complete	Cached read-only offline mode + reconnect + banner
13	Simulated Transcription	✅ Complete	All 5 components verified: toggle UI, preference storage, MockTranscriptionStream, control logic (3 locations), SimulationIndicator
14	Preferences Sync	✅ Complete	gRPC endpoints + ETag conflict detection + client sync module + PreferencesSyncStatus component

Resolved: Sprint 12 implemented; dependencies for Sprint 13/14 now satisfied.

Phase 5: Platform Evolution

Sprint	Name	Size	Prerequisites	Status	Key Deliverable
15	Platform Hardening	M	Phase 4	✅ Complete	Central error taxonomy, OpenTelemetry instrumentation, usage events
16	Identity Foundation	L	Sprint 15	✅ Ready	User auth mechanism, workspace enforcement
17	Custom OAuth Providers	L	Sprint 16	✅ Ready	OIDC discovery, Authentik/Authelia presets
18	Projects v1	L	Sprint 16	❌ Not started	Project entity, ProjectRole, rule inheritance, UI
19	Artifacts v1	XL	Sprint 18	✅ Ready	Upload + chunking + embedding pipeline
20	Artifacts v2 + RAG	XL	Sprint 19	🚫 Blocked	External connectors, Qdrant migration, Q&A
21	MCP Configuration	L	Sprint 18	🚫 Blocked	Scoped MCP registry (workspace defaults + project overrides)
22	Rules v1	XL	Sprint 16, 21	🚫 Blocked	Rules schema, auto-record, templates, outputs
23	Analytics	M-L	Sprint 15, 22	🚫 Blocked	Conflict detection, usage/evaluations tabs
24	Graph + Explore	L-XL	Sprint 18, 19	⚠️ Partial	Graph persistence, Explore visualization
25+	LangGraph Migration	XL	Sprint 15, 21	🚫 Blocked	RAG/research/Q&A graphs, eval hooks

Phase 4: Productization (Validated 2025-12-29)

Validation complete. See individual README files for detailed implementation status.

Sprint 10: Integration Config + OAuth Types

Status: ✅ COMPLETE — README

All OAuth infrastructure verified and production-ready:

InitiateOAuth, CompleteOAuth, GetOAuthConnectionStatus, DisconnectOAuth RPCs (calendar.py:113-190)
PKCE S256 implementation (oauth_manager.py:312-420)
Deep link handler with CSRF protection (tauri.conf.json + use-oauth-flow.ts)
IntegrationSecretModel with encrypted storage (integration.py:108-137)
Behavioral tests: tests/grpc/test_oauth.py (19 tests), use-oauth-flow.test.ts (19 tests)

Sprint 11: Trigger System Wiring

Status: ✅ COMPLETE (100% VERIFIED) — README

All trigger infrastructure verified with 108 passing tests:

Backend: TriggerService (17 tests), CalendarProvider (36 tests), AudioActivityProvider (12 tests), ForegroundAppProvider (17 tests)
Domain: TriggerSignal, TriggerDecision, TriggerAction (18 tests)
Settings: TriggerSettings with exactly 23 fields (8 tests)
Rust: TriggerService, audio monitor with adaptive noise floor, foreground polling
UI: Toast-based trigger prompt (tauri-event-listener.tsx:36-67)
Quality: All 48 quality gate tests pass

Sprint 12: Tauri Fallback & Offline State

Status: ✅ COMPLETE — README

All offline/read-only infrastructure implemented:

✅ Connection state tracking + context provider
✅ Cached read-only adapter + meeting cache store
✅ Offline banner + compact indicator
✅ Guarded mutation hook for write blocking
✅ Reconnection backoff logic

Sprint 13: Simulated Transcription Toggle

Status: ✅ COMPLETE — README

All components verified and fully implemented (validated 2025-12-29):

✅ Settings toggle UI (developer-options-section.tsx:45-62)
✅ Preference storage (preferences.ts:78, preferences.ts:278-282)
✅ MockTranscriptionStream (mock-transcription-stream.ts:23-133)
✅ Control logic: Recording.tsx lines 165, 207, 268 check preferences.simulate_transcription
✅ SimulationIndicator component (simulation-indicator.tsx:27-84)

End-to-end flow: Toggle in Settings → persisted to localStorage/Tauri → checked at recording start → branches to Mock or real gRPC stream → indicator shown when simulating.

Sprint 14: Preferences Sync

Status: ✅ COMPLETE — README

All sync infrastructure implemented and tested (validated 2025-12-29):

✅ PreferencesRepository with CRUD + bulk operations (preferences_repo.py)
✅ UserPreferencesModel with JSONB (settings.py:83-102)
✅ Client preferences store (preferences.ts - 499 lines)
✅ gRPC endpoints (grpc/_mixins/preferences.py) — GetPreferences/SetPreferences RPCs
✅ Proto messages (noteflow.proto) — Full sync protocol with ETag conflict detection
✅ Sync module (client/src/lib/preferences-sync.ts) — hydrate/push with conflict resolution
✅ Sync status component (client/src/components/preferences-sync-status.tsx)
✅ React hook (client/src/hooks/use-preferences-sync.ts)
✅ Backend tests (13 tests in tests/grpc/test_preferences_mixin.py)
✅ Integration tests (8 tests in tests/integration/test_preferences_repository.py)
✅ Client tests (34 tests across hook and module)

End-to-end flow: App startup → hydrateFromServer() → ETag comparison → merge preferences → pushToServer() on save → conflict detection + resolution dialog.

Phase 5: Platform Evolution

Sprint 15: Platform Hardening

Size: M | Owner: Backend | Prerequisites: Phase 4 complete Status: ✅ COMPLETE (2025-12-30)

Objective: Make the system diagnosable and stable before adding surface area.

Implementation Status

Component	Status	Location
Central error taxonomy	✅ Implemented	`domain/errors.py` — ErrorCode enum + DomainError hierarchy
Structured logging	✅ Implemented	LogBuffer with trace_id/span_id, OTel context extraction
OpenTelemetry instrumentation	✅ Implemented	`infrastructure/observability/otel.py` with graceful degradation
Usage event stream	✅ Implemented	`infrastructure/observability/usage.py` — 3 sink implementations
Usage metadata persistence	✅ Implemented	`UsageEventModel` + repository + Alembic migration
Correlation ID propagation	✅ Implemented	LogBufferHandler extracts trace/span IDs from OTel context
LogBuffer	✅ Implemented	Ring buffer with 1000 capacity
MetricsCollector	✅ Implemented	CPU, memory, disk, network (history only grows when collected)

Dependencies unlocked: Sprint 23 (Analytics) and Sprint 25 (LangGraph) can now proceed.

Delivered Components

src/noteflow/domain/errors.py — 18 error codes with gRPC status mapping
src/noteflow/application/observability/ports.py — UsageEvent dataclass + UsageEventSink protocol
src/noteflow/infrastructure/observability/otel.py — OTel setup with no-op fallback
src/noteflow/infrastructure/observability/usage.py — Logging, OTel, and BufferedDatabase sinks
src/noteflow/infrastructure/persistence/models/observability/usage_event.py — ORM model
src/noteflow/infrastructure/persistence/repositories/usage_event_repo.py — Repository with aggregation
src/noteflow/infrastructure/persistence/migrations/versions/n8o9p0q1r2s3_add_usage_events_table.py
src/noteflow/application/services/summarization_service.py — Emits usage events, persists tokens/latency

Sprint 16: Identity Foundation

Size: L | Owner: Backend + Client | Prerequisites: Sprint 15 Status: ✅ PREREQUISITES VERIFIED

Objective: Every operation has a "user POV" and tenant boundaries are enforced.

Verified Assets

Asset	Location
UserModel	`persistence/models/identity/identity.py:87-125`
WorkspaceModel	`persistence/models/identity/identity.py:27-85`
WorkspaceMembershipModel	`persistence/models/identity/identity.py:127-159`
MeetingModel.workspace_id	FK with RESTRICT delete
Keystore	3 implementations in `infrastructure/security/keystore.py`

Deliverables

Auth mechanism implementation (local-first)
CurrentUser + ScopeContext propagation (workspace + project)
Repository query filters for scope enforcement
Workspace switcher component

Sprint 17: Custom OAuth Providers

Size: L | Owner: Backend + Client | Prerequisites: Sprint 16 Status: ✅ PREREQUISITES VERIFIED

Objective: Support Authentik, Authelia, and other OIDC providers.

Verified Assets

Asset	Location
Generic OAuth messages	`noteflow.proto` lines 54-58, 766-857
OAuthManager	`infrastructure/calendar/oauth_manager.py` (431 lines, PKCE)
IntegrationModel.config	JSONB field at `integration.py:62-66`
IntegrationSecretModel	Encrypted storage at `integration.py:108-137`

Deliverables

src/noteflow/infrastructure/auth/oidc_registry.py
OidcProviderConfig domain entity
Provider management in Settings UI

Sprint 18: Projects v1

Size: L | Owner: Backend + Client | Prerequisites: Sprint 16 Status: ✅ IMPLEMENTED

Objective: Introduce Projects as first-class container with roles, settings, and rule inheritance.

Key Decisions (2025-12-30)

Decision	Choice
Ownership	Single Workspace (projects scoped to one workspace)
Roles	Project-level (viewer/editor/admin separate from WorkspaceRole)
Rules	Merge/inherit (projects inherit workspace rules, can override)
Migration	Default project per workspace for unassigned meetings

Implemented Components

Component	Location
Project entity + ProjectSettings	`domain/entities/project.py`
ProjectRole enum	`domain/identity/roles.py`
ProjectMembership entity	`domain/identity/entities.py`
ProjectModel + ProjectMembershipModel	`persistence/models/identity/identity.py`
ProjectRepository + impl	`ports/repositories/identity.py` + `infrastructure/persistence/repositories/identity_repo.py`
ProjectService	`application/services/project_service.py`
Project RPCs	`grpc/proto/noteflow.proto` + `grpc/_mixins/project.py`
Project UI (sidebar, switcher, settings)	`client/src/components/projects/`

✅ Unblocks: Sprint 21 (MCP Config) and Sprint 22 (Rules) project scoping prerequisites.

Deliverables

src/noteflow/domain/entities/project.py — Project, ProjectSettings, ExportRules, TriggerRules
src/noteflow/domain/identity/roles.py — ProjectRole enum with permissions
src/noteflow/domain/identity/entities.py — ProjectMembership
src/noteflow/infrastructure/persistence/models/identity/identity.py — ORM models
src/noteflow/application/services/project_service.py — Lifecycle + rule merging
src/noteflow/grpc/_mixins/project.py — 8 RPCs (CRUD + membership)
client/src/components/projects/ProjectSidebar.tsx
client/src/components/projects/ProjectSwitcher.tsx
client/src/components/projects/ProjectSettingsPanel.tsx
Alembic migrations for projects, memberships, meeting.project_id

Sprint 19: Artifacts v1

Size: XL | Owner: Backend | Prerequisites: Sprint 18 Status: ✅ PREREQUISITES VERIFIED

Objective: Get any non-meeting corpus into embeddings and retrievable context.

Verified Assets

Asset	Location
SegmentModel.embedding	`models/core/meeting.py:202-205` with Vector(1536)
cosine_distance query	`repositories/segment_repo.py:148` in `search_semantic()`
Encrypted asset storage	`infrastructure/audio/writer.py` with AES-GCM
EMBEDDING_DIM = 1536	`models/_base.py:8`

Deliverables

src/noteflow/domain/entities/artifact.py
src/noteflow/infrastructure/artifacts/ — Storage, chunking, embedding
RetrieveContext RPC for RAG queries
Artifact upload UI

Sprint 20: Artifacts v2 + RAG Migration

Size: XL | Owner: Backend + Client | Prerequisites: Sprint 19 Status: 🚫 BLOCKED

Objective: External connectors + migrate from pgvector to Qdrant for RAG.

Key Decision (2025-12-30)

Decision	Choice	Rationale
RAG Backend	Qdrant	Replace pgvector for better scaling, hybrid search, collection-per-project isolation

Verified Assets

Asset	Location
StartIntegrationSync RPC	`noteflow.proto:73`, `grpc/_mixins/sync.py:47-105`
Google OAuth	`oauth_manager.py` with PKCE
CalendarEventModel	`models/integrations/integration.py:188-255`
SegmentModel.embedding	`models/core/meeting.py:202-205` (migration source)

Blocked by: Sprint 19 (Artifacts v1) must be implemented first.

Deliverables

src/noteflow/infrastructure/artifacts/sources/ — Directory, Drive, OneDrive
ArtifactSource domain entity with sync metadata
src/noteflow/domain/rag/ — Chunk, SearchResult, QAResponse, ports
src/noteflow/infrastructure/rag/ — Qdrant client, embedding providers, chunking, indexing
src/noteflow/application/services/qa_service.py — Q&A orchestration
SearchProject, AskQuestion RPCs
client/src/components/qa/QAPanel.tsx — Q&A interface with citations
scripts/migrate_to_qdrant.py — pgvector migration
Docker compose Qdrant service with rag profile

Sprint 21: MCP Server Configuration

Size: L | Owner: Backend + Client | Prerequisites: Sprint 18 Status: 🚫 BLOCKED

Objective: Centralize "where context/tools come from" before Rules + LangGraph.

Missing Prerequisites

Prerequisite	Status
Project scoping (Sprint 18)	✅ Implemented

Missing Components

Component	Required Location
MCP registry schema	`domain/entities/mcp_config.py`
MCP registry infrastructure	`infrastructure/mcp/registry.py`
MCP RPCs	proto messages needed
Scope precedence	Workspace defaults + project overrides + resource overrides
Credential boundary	Per-scope secrets (workspace vs project)

Action required: Sprint 18 prerequisite satisfied; proceed with MCP implementation.

⚠️ Blocker: Sprint 22 (Rules) and Sprint 25 (LangGraph) depend on MCP configuration.

Sprint 22: Rules v1

Size: XL | Owner: Backend + Client | Prerequisites: Sprint 16, Sprint 21 Status: 🚫 BLOCKED — NOT IMPLEMENTED

Objective: Automate recording, templates, and outputs without chaos.

Missing Prerequisites

Prerequisite	Status
Sprint 21 (MCP Config)	🚫 Blocked by Sprint 18

Missing Components

Component	Required Location
Rules schema	`domain/rules/schema.py`
Rules evaluator	`domain/rules/evaluator.py`
Rules service	`application/services/rules_service.py`
Rules RPCs	proto messages needed
Rules UI	`client/src/pages/Rules.tsx`
Domain event schema	`domain/events/schema.py` (inputs for rules + audit)

Verified Assets

Asset	Location
TriggerSettings (23 fields)	`config/settings.py`
trigger_meeting_apps	List of 13 app names
Summarization templates	4 tone × 4 format × 4 verbosity
Export infrastructure	`infrastructure/export/`

Sprint 23: Rules Auditor + Analytics

Size: M-L | Owner: Backend + Client | Prerequisites: Sprint 15, Sprint 22 Status: 🚫 BLOCKED

Objective: Governance and feedback loops.

Prerequisites Status

Prerequisite	Status	Impact
Usage events (Sprint 15)	✅ Complete	Usage aggregation available
Rules schema (Sprint 22)	🚫 Blocked	Cannot audit rule execution

Action required: Complete Sprint 22 (Rules Schema) before starting. Sprint 15 is complete.

Deliverables

src/noteflow/application/services/rules_auditor.py
src/noteflow/domain/entities/evaluation.py
Analytics pages in client

Sprint 24: Entities Graph + Explore

Size: L-XL | Owner: Backend + Client | Prerequisites: Sprint 18, Sprint 19 Status: ⚠️ PARTIAL PREREQUISITES

Objective: Transform entities into a navigable knowledge layer.

Verified Assets

Asset	Location
NamedEntity with segment_ids	`domain/entities/named_entity.py:69`
Entity extraction pipeline	`application/services/ner_service.py`, `infrastructure/ner/engine.py`
People.tsx page	`client/src/pages/People.tsx`

Missing Prerequisites

Prerequisite	Status
Sprint 18 (Projects)	✅ Implemented
Sprint 19 (Artifacts)	❌ Not implemented

Partial blocker: Core entity infrastructure exists. Graph schema can proceed, but artifact integration requires Sprint 19.

Deliverables

src/noteflow/infrastructure/graph/ — Schema, queries
client/src/pages/Explore.tsx

Sprint 25+: LangGraph Migration

Size: XL (multi-sprint) | Owner: Backend | Prerequisites: Sprint 15, Sprint 21 Status: 🚫 BLOCKED

Objective: Replace AI workflows with LangGraph after context sources exist.

Prerequisites Status

Prerequisite	Status	Impact
Usage events (Sprint 15)	✅ Complete	Run metadata emission available
Project scoping (Sprint 18)	✅ Implemented	No longer blocks Sprint 21
MCP configuration (Sprint 21)	🚫 Blocked	Cannot configure tool sources

Verified Assets

Asset	Location
Summarization service	`application/services/summarization_service.py`
RAG retrieval	`SegmentModel.embedding` + cosine_distance
Usage event infrastructure	`infrastructure/observability/usage.py`

Action required: Complete Sprint 21 (MCP Config) before starting. Sprint 15 and Sprint 18 are complete.

Dependency Graph (Updated 2025-12-29)

Phase 4 Complete
       │
       ▼
┌──────────────────────────────────────────────────────────────────────────┐
│                           WAVE 1: Foundation                              │
├──────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│  ┌──────────────┐                                                        │
│  │  Sprint 15   │ Platform Hardening                                     │
│  │ ✅ COMPLETE  │ OTel + usage events + persistence ─────────┐           │
│  └──────┬───────┘                                            │           │
│         │                                                    │           │
│         ▼                                                    │           │
│  ┌──────────────┐                                            │           │
│  │  Sprint 16   │ Identity Foundation                        │           │
│  │ ✅ READY     │                                            │           │
│  └──────┬───────┘                                            │           │
│         │                                                    │           │
│         ├─────────────────┬──────────────────┐               │           │
│         ▼                 ▼                  ▼               │           │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐        │           │
│  │  Sprint 17   │  │  Sprint 18   │  │              │        │           │
│  │ ✅ READY     │  │ ❌ NEEDED    │  │   (Sprint    │        │           │
│  │ Custom OAuth │  │  Projects    │  │    21 MCP    │        │           │
│  └──────────────┘  └──────┬───────┘  │   blocked)   │        │           │
│                           │          └──────────────┘        │           │
└───────────────────────────┼──────────────────────────────────┼───────────┘
                            │                                  │
┌───────────────────────────┼──────────────────────────────────┼───────────┐
│                           │      WAVE 2: Integration         │           │
├───────────────────────────┼──────────────────────────────────┼───────────┤
│                           │                                  │           │
│                           ▼                                  │           │
│  ┌───────────────────────────────────────┐                   │           │
│  │              Sprint 19                │                   │           │
│  │           ✅ Artifacts v1             │                   │           │
│  │        (Prerequisites ready)          │                   │           │
│  └──────────────────┬────────────────────┘                   │           │
│                     │                                        │           │
│         ┌───────────┴───────────┐                            │           │
│         ▼                       ▼                            │           │
│  ┌──────────────┐       ┌──────────────┐                     │           │
│  │  Sprint 20   │       │  Sprint 21   │                     │           │
│  │ Artifacts v2 │       │ 🚫 BLOCKED   │◄────────────────────┤           │
│  │ (needs S19)  │       │   MCP Config │  needs Sprint 18    │           │
│  └──────────────┘       └──────┬───────┘                     │           │
│                                │                             │           │
└────────────────────────────────┼─────────────────────────────┼───────────┘
                                 │                             │
┌────────────────────────────────┼─────────────────────────────┼───────────┐
│                                │      WAVE 3: Intelligence   │           │
├────────────────────────────────┼─────────────────────────────┼───────────┤
│                                │                             │           │
│                                ▼                             │           │
│                        ┌──────────────┐                      │           │
│                        │  Sprint 22   │                      │           │
│                        │ 🚫 BLOCKED   │                      │           │
│                        │   Rules v1   │                      │           │
│                        └──────┬───────┘                      │           │
│                               │                              │           │
│                    ┌──────────┴──────────┐                   │           │
│                    ▼                     ▼                   │           │
│             ┌──────────────┐      ┌──────────────┐           │           │
│             │  Sprint 23   │      │  Sprint 25+  │◄──────────┘           │
│             │ 🚫 BLOCKED   │      │ 🚫 BLOCKED   │  needs Usage events   │
│             │  Analytics   │      │  LangGraph   │                       │
│             └──────────────┘      └──────────────┘                       │
│                                                                          │
│             ┌──────────────┐                                             │
│             │  Sprint 24   │ (can start after S18+S19)                   │
│             │ ⚠️ PARTIAL   │                                             │
│             │Graph/Explore │                                             │
│             └──────────────┘                                             │
│                                                                          │
└──────────────────────────────────────────────────────────────────────────┘

LEGEND:
  ✅ READY      = Prerequisites verified, can start
  ⚠️ PARTIAL    = Some work done, gaps remain
  ❌ NEEDED     = Blocker for downstream sprints
  🚫 BLOCKED    = Cannot start until prerequisites complete

Recommended Execution Order

Sequential Constraints (Must Follow)

Sprint 15 ─► Sprint 16 ─► Sprint 17   (Identity chain)
Sprint 16 ─► Sprint 18 ─► Sprint 19   (Projects → Artifacts)
Sprint 18 ─► Sprint 21 ─► Sprint 22   (MCP → Rules)
Sprint 19 ─► Sprint 20                (Artifacts v1 → v2)
Sprint 22 ─► Sprint 23                (Rules → Analytics)

Parallel Opportunities

Parallel Group	Sprints	Shared Resource Conflicts
Group A	15 + 16	None (different subsystems)
Group B	17 + 18	Both need Sprint 16 complete first
Group C	19 + 21	Both need Sprint 18 complete first
Group D	20 + 22	Low overlap (artifacts vs rules)
Group E	23 + 24 + 25	All need different prereqs, can parallelize

Optimal Execution Timeline

Week 1-2:   Sprint 15 (OpenTelemetry + Usage Events) + Sprint 16 (Identity) ─ parallel
Week 2-3:   Sprint 17 (OAuth) + Sprint 18 (Projects) ─ parallel
Week 3-5:   Sprint 19 (Artifacts v1) + Sprint 21 (MCP) ─ parallel
Week 5-7:   Sprint 20 (Artifacts v2) + Sprint 22 (Rules) ─ parallel
Week 7-9:   Sprint 23 + Sprint 24 + Sprint 25 ─ all can parallelize

Effort Sizing Summary

Size	Sprints	Description
S	—	Single-focus, < 1 week
M	15, 23	Multi-file, 1-2 weeks
M-L	18	Significant but bounded
L	16, 17, 21, 24	Multi-component, 2-3 weeks
XL	19, 20, 22, 25+	Major subsystem, 3+ weeks

Architecture Decisions (Finalized 2025-12-29)

Decision	Sprint	Choice	Rationale
Auth Model	16	Local-first	Single user per install, design for multi-user. Reduces Sprint 16 from L → M.
OIDC Config	17	Discovery only	Require issuer URL with `.well-known`. Covers 90%+ of providers.
Project Ownership	18	Single Workspace	Projects scoped to one workspace; simpler ACL model.
Project Roles	18	Project-level	Separate ProjectRole (viewer/editor/admin) from WorkspaceRole.
Rule Inheritance	18	Merge/inherit	Projects inherit workspace rules, can add/modify with precedence.
Meeting Migration	18	Default project	Each workspace gets 'General' project for unassigned meetings.
Artifact Scope	19	Per-workspace	Simpler ACL, easier cross-project sharing.
RAG Backend	20	Qdrant	Replace pgvector with dedicated vector DB for scaling, hybrid search, project isolation.
MCP Storage	21	Backend DB	Enables team sharing, consistent behavior across devices.
Rules Execution	22	Backend-evaluated	Audit trails, single source of truth. Client handles triggers only.
Offline Mode	12	Cached read-only	Prevents silent divergence; no mock writes.
Observability	15	OpenTelemetry	Standardized traces/metrics/logs + correlation IDs.
Scope Lattice	16/18	Workspace → Project → Resource	Consistent scoping across repos, RPCs, and MCP.

Risk Assessment

Risk	Mitigation
Auth complexity explosion	Start local-first, defer multi-user
Artifact storage costs	Implement quotas, lazy embedding
Rules engine complexity	Start with simple precedence, iterate
LangGraph learning curve	Wrap existing code first, migrate incrementally
Breaking changes cascade	Strong proto versioning, feature flags
Sprint 18 blocking cascade	Prioritize Projects implementation to unblock 21/22/25
Usage events blocking analytics	Complete Sprint 15 OpenTelemetry + usage events early
Scope leakage across projects	Enforce `ScopeContext` filters in repos + gRPC interceptors from Sprint 16/18

Appendix: Scope Lattice Enforcement (Planned)

Goal: Ensure every request is scoped consistently: workspace → project → resource.

Scope Rules (Best‑in‑Class Defaults)

Workspace is mandatory for all writes and most reads.
Project is optional but, when present, is a strict filter on resources that declare project_id.
Resource access always validates ownership by scope (meeting/task/artifact must belong to workspace, and project if set).
Overrides are explicit and auditable (e.g., “cross‑project search” requires a dedicated endpoint and opt‑in policy).

Enforcement Points (Planned + Existing)

gRPC Interceptors: src/noteflow/grpc/interceptors/scope.py (planned in Sprint 16) enforces scope on every RPC before handler logic.
Repository Filters: src/noteflow/infrastructure/persistence/repositories/_scoped.py (planned in Sprint 16) applies workspace_id/project_id constraints to queries.
Context Model: src/noteflow/domain/identity/context.py (planned changes in Sprint 16) adds ProjectContext and OperationContext.project_id.
Project Entities: src/noteflow/domain/entities/project.py + src/noteflow/infrastructure/persistence/models/project.py (planned in Sprint 18).

Concrete Examples (Planned)

Meeting list: repository method appends WHERE workspace_id = :workspace_id and AND project_id = :project_id (if present).
Artifacts: enforce per‑workspace by default, but allow project association with explicit policy flags.
Rules: evaluator consumes OperationContext.project_id and denies cross‑project actions unless rule explicitly opts in.

Tests (Required by Quality Standards)

tests/infrastructure/test_scoped_repos.py (planned): validates workspace/project filters.
tests/grpc/test_scope_enforcement.py (planned): verifies interceptors block out‑of‑scope access.
tests/grpc/test_projects.py (planned): ensures project CRUD respects scope.

All scope changes must satisfy docs/sprints/QUALITY_STANDARDS.md quality gates.

Quality Gates (All Sprints)

pytest tests/quality/           # 23+ test smell checks
ruff check src/noteflow         # Zero lint errors
basedpyright                    # Zero type errors
npm run lint                    # Zero frontend errors
npm run test                    # Unit tests pass

See docs/sprints/QUALITY_STANDARDS.md for full quality gates, test limits, and lint rules.

Appendix: Phase 1-3 (Complete)

Phase	Sprints	Focus
0	0	Proto & Schema Foundation
1	1-3	Core Pipeline (templates, diarization, PDF)
2	4	Intelligence (NER)
3	5-6	Integrations (calendar, webhooks)

See docs/sprints/phase-*/ for historical documentation.

40 KiB Raw Blame History Unescape Escape

NoteFlow Feature Gap Analysis & Development Roadmap

Executive Summary

Current State (Validation 2025-12-29)

Existing Infrastructure (Verified 2025-12-29)

Phase 5 Implementation Status (2025-12-29)

Sprint Status Overview

Critical Path

Parallel Execution Map

Wave 1: Foundation (Can Start Immediately)

Wave 2: Integration (After Wave 1)

Wave 3: Intelligence (After Wave 2)

Sprint Overview

Phase 4: Productization (Validation 2025-12-29)

Phase 5: Platform Evolution

Phase 4: Productization (Validated 2025-12-29)

Sprint 10: Integration Config + OAuth Types

Sprint 11: Trigger System Wiring

Sprint 12: Tauri Fallback & Offline State

Sprint 13: Simulated Transcription Toggle

Sprint 14: Preferences Sync

Phase 5: Platform Evolution

Sprint 15: Platform Hardening

Implementation Status

Delivered Components

Sprint 16: Identity Foundation

Verified Assets

Deliverables

Sprint 17: Custom OAuth Providers

Verified Assets

Deliverables

Sprint 18: Projects v1

Key Decisions (2025-12-30)

Implemented Components

Deliverables

Sprint 19: Artifacts v1

Verified Assets

Deliverables

Sprint 20: Artifacts v2 + RAG Migration

Key Decision (2025-12-30)

Verified Assets

Deliverables

Sprint 21: MCP Server Configuration

Missing Prerequisites

Missing Components

Sprint 22: Rules v1

Missing Prerequisites

Missing Components

Verified Assets

Sprint 23: Rules Auditor + Analytics

Prerequisites Status

Deliverables

Sprint 24: Entities Graph + Explore

Verified Assets

Missing Prerequisites

Deliverables

Sprint 25+: LangGraph Migration

Prerequisites Status

Verified Assets

Dependency Graph (Updated 2025-12-29)

Recommended Execution Order

Sequential Constraints (Must Follow)

Parallel Opportunities

Optimal Execution Timeline

Effort Sizing Summary

Architecture Decisions (Finalized 2025-12-29)

Risk Assessment

Appendix: Scope Lattice Enforcement (Planned)

Scope Rules (Best‑in‑Class Defaults)

Enforcement Points (Planned + Existing)

Concrete Examples (Planned)

Tests (Required by Quality Standards)

Quality Gates (All Sprints)

Appendix: Phase 1-3 (Complete)

40 KiB

Raw Blame History