Files

Travis Vasceannie 2d1a86937f Enhance observability and usage event infrastructure

- Introduced `UsageEventSink` interface for capturing application-layer usage events.
- Updated `OtelUsageEventSink` to attach span events and emit usage events from application services.
- Enhanced logging by injecting `trace_id` and `span_id` into LogBuffer entries.
- Implemented metadata persistence for `Summary.tokens_used` and `Summary.latency_ms` before saving.
- Updated documentation to reflect changes in observability components and planned features for Sprint 15.

All quality checks pass.

2025-12-29 21:07:47 +00:00

37 KiB

Raw Blame History

NoteFlow Feature Gap Analysis & Development Roadmap

Generated: 2025-12-23 | Updated: 2025-12-29 Focus: Platform evolution — auth, workspaces, artifacts, rules, AI orchestration

Executive Summary

This document identifies features not yet developed and provides a phased roadmap for NoteFlow's evolution from a single-user meeting tool to a multi-user, rule-driven knowledge platform with RAG capabilities.

Current State (Validation 2025-12-29)

Component	Status	Coverage
Transcription	✅ Complete	Real-time streaming, VAD, partial/final segments
Summarization	✅ Complete	AI generation with templates, citation verification
Diarization	✅ Complete	Streaming + offline refinement, speaker renaming
Export	✅ Complete	Markdown/HTML/PDF with styling
NER	✅ Complete	Entity extraction with segment tracking
Integrations	✅ Complete	Calendar sync, webhooks, observability
OAuth	✅ Complete	PKCE S256, deep links, encrypted token storage
Triggers	✅ Complete	Audio, foreground app, calendar (23 config fields)
Offline Mode	✅ Complete	Cached read-only mode + offline banner + guarded mutations
Simulation Toggle	✅ Complete	Toggle UI + MockTranscriptionStream + SimulationIndicator
Preferences Sync	✅ Complete	gRPC endpoints + ETag conflict detection + sync module

Existing Infrastructure (Verified 2025-12-29)

Asset	Location	Status	Implication
`WorkspaceModel`, `UserModel`, `WorkspaceMembershipModel`	`persistence/models/identity/identity.py`	✅	Multi-tenancy modeled
`MeetingModel.workspace_id`, `TaskModel.workspace_id`	ORM models	✅	Workspace scoping exists
`SegmentModel.embedding` (1536 dims)	`models/core/meeting.py:202-205`	✅	Vector infrastructure ready
`cosine_distance` similarity query	`repositories/segment_repo.py:148`	✅	RAG retrieval plumbing exists
Generic OAuth messages	`noteflow.proto` lines 54-58, 766-857	✅	OAuth extensible
`LogBuffer`	`infrastructure/logging/log_buffer.py`	✅	Observability foundation exists
`MetricsCollector`	`infrastructure/metrics/collector.py`	✅	Performance monitoring exists
`SummaryModel.tokens_used`, `latency_ms`	`models/core/summary.py`	✅	Usage tracking partial
`TriggerSettings` (23 fields)	`config/settings.py`	✅	Rules-like config exists
`parse_meeting_id_or_abort()`	`grpc/_mixins/converters.py`	✅	Error pattern exists

Phase 5 Implementation Status (2025-12-29)

Sprint Status Overview

Sprint	Name	Status	Blockers
15	Platform Hardening	⚠️ Partial	OpenTelemetry instrumentation + usage events NOT implemented; usage metadata not persisted
16	Identity Foundation	✅ Ready	Prerequisites verified
17	Custom OAuth	✅ Ready	Prerequisites verified
18	Projects v1	❌ Not started	—
19	Artifacts v1	✅ Ready	Prerequisites verified
20	Artifacts v2	🚫 Blocked	Requires Sprint 19
21	MCP Configuration	🚫 Blocked	Requires Sprint 18 (Projects)
22	Rules v1	🚫 Blocked	Requires Sprint 21 (MCP)
23	Analytics	🚫 Blocked	Requires Sprint 15 (Usage) + Sprint 22 (Rules)
24	Graph + Explore	⚠️ Partial	Core ready, awaits Sprint 18/19
25+	LangGraph	🚫 Blocked	Requires Sprint 15 (Usage) + Sprint 21 (MCP)

Critical Path

Sprint 15 (OpenTelemetry + Usage Events + Metadata) ───────────┐
                                                               │
Sprint 18 (Projects) ─────┬───────────────────────────────────┐│
                          │                                   ││
                          ▼                                   ▼▼
                   Sprint 21 (MCP) ────► Sprint 22 (Rules) ────► Sprint 23 (Analytics)
                          │
                          └──────────────────────────────────────► Sprint 25 (LangGraph)

Unblock Order: Sprint 15 + Sprint 18 → Sprint 21 → Sprint 22 → Sprint 23/25

Parallel Execution Map

Wave 1: Foundation (Can Start Immediately)

Sprint	Why Parallelizable	Dependencies
15 Platform Hardening	Backend-only, no UI	Phase 4 complete
16 Identity Foundation	Prerequisites verified	Sprint 15 (can overlap)
17 Custom OAuth	Prerequisites verified	Sprint 16
18 Projects v1	Independent of 15/17	Sprint 16
19 Artifacts v1	Vector infra ready	Sprint 18 (can start core work early)

Recommended parallel groupings:

🔀 Sprint 15 + Sprint 16 (backend focus, different subsystems)
🔀 Sprint 17 + Sprint 18 (both depend on 16, don't overlap)
🔀 Sprint 19 + Sprint 21 (both depend on 18, different concerns)

Wave 2: Integration (After Wave 1)

Sprint	Why Sequential	Dependencies
20 Artifacts v2	Requires Artifacts v1 pipeline	Sprint 19
21 MCP Configuration	Requires Project scoping	Sprint 18
22 Rules v1	Requires MCP + Identity	Sprint 16, Sprint 21

Recommended parallel groupings:

🔀 Sprint 20 + Sprint 21 (different pipelines, both after 18/19)
🔀 Sprint 22 + Sprint 24 (Rules backend + Graph frontend, minimal overlap)

Wave 3: Intelligence (After Wave 2)

Sprint	Why Sequential	Dependencies
23 Analytics	Requires Rules + Usage events	Sprint 15, Sprint 22
24 Graph + Explore	Requires Projects + Artifacts	Sprint 18, Sprint 19
25+ LangGraph	Requires MCP + Usage tracking	Sprint 15, Sprint 21

Recommended parallel groupings:

🔀 Sprint 23 + Sprint 25 (both depend on 15+21/22, different AI subsystems)
🔀 Sprint 24 can run alongside 23/25 if 18+19 complete

Sprint Overview

Phase 4: Productization (Validation 2025-12-29)

Sprint	Name	Status	Finding
10	Integration Config + OAuth	✅ Complete	All 10 components verified with line numbers
11	Trigger System Wiring	✅ Complete	All 9 components verified (23 TriggerSettings fields confirmed)
12	Tauri Fallback & Offline	✅ Complete	Cached read-only offline mode + reconnect + banner
13	Simulated Transcription	✅ Complete	All 5 components verified: toggle UI, preference storage, MockTranscriptionStream, control logic (3 locations), SimulationIndicator
14	Preferences Sync	✅ Complete	gRPC endpoints + ETag conflict detection + client sync module + PreferencesSyncStatus component

Resolved: Sprint 12 implemented; dependencies for Sprint 13/14 now satisfied.

Phase 5: Platform Evolution

Sprint	Name	Size	Prerequisites	Status	Key Deliverable
15	Platform Hardening	M	Phase 4	⚠️ Partial	Central error taxonomy, OpenTelemetry instrumentation, usage events
16	Identity Foundation	L	Sprint 15	✅ Ready	User auth mechanism, workspace enforcement
17	Custom OAuth Providers	L	Sprint 16	✅ Ready	OIDC discovery, Authentik/Authelia presets
18	Projects v1	M-L	Sprint 16	❌ Not started	Project entity, meeting/task grouping
19	Artifacts v1	XL	Sprint 18	✅ Ready	Upload + chunking + embedding pipeline
20	Artifacts v2	XL	Sprint 19	🚫 Blocked	Google Drive / OneDrive connectors
21	MCP Configuration	L	Sprint 18	🚫 Blocked	Scoped MCP registry (workspace defaults + project overrides)
22	Rules v1	XL	Sprint 16, 21	🚫 Blocked	Rules schema, auto-record, templates, outputs
23	Analytics	M-L	Sprint 15, 22	🚫 Blocked	Conflict detection, usage/evaluations tabs
24	Graph + Explore	L-XL	Sprint 18, 19	⚠️ Partial	Graph persistence, Explore visualization
25+	LangGraph Migration	XL	Sprint 15, 21	🚫 Blocked	RAG/research/Q&A graphs, eval hooks

Phase 4: Productization (Validated 2025-12-29)

Validation complete. See individual README files for detailed implementation status.

Sprint 10: Integration Config + OAuth Types

Status: ✅ COMPLETE — README

All OAuth infrastructure verified and production-ready:

InitiateOAuth, CompleteOAuth, GetOAuthConnectionStatus, DisconnectOAuth RPCs (calendar.py:113-190)
PKCE S256 implementation (oauth_manager.py:312-420)
Deep link handler with CSRF protection (tauri.conf.json + use-oauth-flow.ts)
IntegrationSecretModel with encrypted storage (integration.py:108-137)
Behavioral tests: tests/grpc/test_oauth.py (19 tests), use-oauth-flow.test.ts (19 tests)

Sprint 11: Trigger System Wiring

Status: ✅ COMPLETE (100% VERIFIED) — README

All trigger infrastructure verified with 108 passing tests:

Backend: TriggerService (17 tests), CalendarProvider (36 tests), AudioActivityProvider (12 tests), ForegroundAppProvider (17 tests)
Domain: TriggerSignal, TriggerDecision, TriggerAction (18 tests)
Settings: TriggerSettings with exactly 23 fields (8 tests)
Rust: TriggerService, audio monitor with adaptive noise floor, foreground polling
UI: Toast-based trigger prompt (tauri-event-listener.tsx:36-67)
Quality: All 48 quality gate tests pass

Sprint 12: Tauri Fallback & Offline State

Status: ✅ COMPLETE — README

All offline/read-only infrastructure implemented:

✅ Connection state tracking + context provider
✅ Cached read-only adapter + meeting cache store
✅ Offline banner + compact indicator
✅ Guarded mutation hook for write blocking
✅ Reconnection backoff logic

Sprint 13: Simulated Transcription Toggle

Status: ✅ COMPLETE — README

All components verified and fully implemented (validated 2025-12-29):

✅ Settings toggle UI (developer-options-section.tsx:45-62)
✅ Preference storage (preferences.ts:78, preferences.ts:278-282)
✅ MockTranscriptionStream (mock-transcription-stream.ts:23-133)
✅ Control logic: Recording.tsx lines 165, 207, 268 check preferences.simulate_transcription
✅ SimulationIndicator component (simulation-indicator.tsx:27-84)

End-to-end flow: Toggle in Settings → persisted to localStorage/Tauri → checked at recording start → branches to Mock or real gRPC stream → indicator shown when simulating.

Sprint 14: Preferences Sync

Status: ✅ COMPLETE — README

All sync infrastructure implemented and tested (validated 2025-12-29):

✅ PreferencesRepository with CRUD + bulk operations (preferences_repo.py)
✅ UserPreferencesModel with JSONB (settings.py:83-102)
✅ Client preferences store (preferences.ts - 499 lines)
✅ gRPC endpoints (grpc/_mixins/preferences.py) — GetPreferences/SetPreferences RPCs
✅ Proto messages (noteflow.proto) — Full sync protocol with ETag conflict detection
✅ Sync module (client/src/lib/preferences-sync.ts) — hydrate/push with conflict resolution
✅ Sync status component (client/src/components/preferences-sync-status.tsx)
✅ React hook (client/src/hooks/use-preferences-sync.ts)
✅ Backend tests (13 tests in tests/grpc/test_preferences_mixin.py)
✅ Integration tests (8 tests in tests/integration/test_preferences_repository.py)
✅ Client tests (34 tests across hook and module)

End-to-end flow: App startup → hydrateFromServer() → ETag comparison → merge preferences → pushToServer() on save → conflict detection + resolution dialog.

Phase 5: Platform Evolution

Sprint 15: Platform Hardening

Size: M | Owner: Backend | Prerequisites: Phase 4 complete Status: ⚠️ PARTIALLY IMPLEMENTED

Objective: Make the system diagnosable and stable before adding surface area.

Implementation Status

Component	Status	Location
Central error taxonomy	❌ Not implemented	`domain/errors.py` needed
Structured logging	⚠️ Partial	LogBuffer exists, but no trace/span IDs or enforced source taxonomy
OpenTelemetry instrumentation	❌ Not implemented	Planned `infrastructure/observability/otel.py`
Usage event stream (from OTel spans/metrics)	❌ Not implemented	Planned `infrastructure/observability/usage.py`
Usage metadata persistence	⚠️ Partial	Providers compute tokens/latency, but `Summary.tokens_used`/`latency_ms` are never populated before save
Correlation ID propagation	❌ Not implemented	Planned OTel context + gRPC interceptors
LogBuffer	✅ Implemented	Ring buffer with 1000 capacity
MetricsCollector	✅ Implemented	CPU, memory, disk, network (history only grows when collected)

⚠️ Blocker: Usage events infrastructure is required by Sprint 23 and Sprint 25.

Remaining Deliverables

src/noteflow/domain/errors.py — Error base types with gRPC mapping
src/noteflow/application/observability/ports.py — Usage event sink port
src/noteflow/infrastructure/observability/otel.py — OpenTelemetry setup (traces, metrics, logs)
src/noteflow/infrastructure/observability/usage.py — Usage events derived from OTel spans/metrics
src/noteflow/infrastructure/logging/context.py — Inject trace/span IDs into LogBuffer entries
src/noteflow/grpc/_interceptors/otel.py — gRPC interceptor for correlation IDs
src/noteflow/application/services/summarization_service.py — Persist tokens/latency and emit usage events

Sprint 16: Identity Foundation

Size: L | Owner: Backend + Client | Prerequisites: Sprint 15 Status: ✅ PREREQUISITES VERIFIED

Objective: Every operation has a "user POV" and tenant boundaries are enforced.

Verified Assets

Asset	Location
UserModel	`persistence/models/identity/identity.py:87-125`
WorkspaceModel	`persistence/models/identity/identity.py:27-85`
WorkspaceMembershipModel	`persistence/models/identity/identity.py:127-159`
MeetingModel.workspace_id	FK with RESTRICT delete
Keystore	3 implementations in `infrastructure/security/keystore.py`

Deliverables

Auth mechanism implementation (local-first)
CurrentUser + ScopeContext propagation (workspace + project)
Repository query filters for scope enforcement
Workspace switcher component

Sprint 17: Custom OAuth Providers

Size: L | Owner: Backend + Client | Prerequisites: Sprint 16 Status: ✅ PREREQUISITES VERIFIED

Objective: Support Authentik, Authelia, and other OIDC providers.

Verified Assets

Asset	Location
Generic OAuth messages	`noteflow.proto` lines 54-58, 766-857
OAuthManager	`infrastructure/calendar/oauth_manager.py` (431 lines, PKCE)
IntegrationModel.config	JSONB field at `integration.py:62-66`
IntegrationSecretModel	Encrypted storage at `integration.py:108-137`

Deliverables

src/noteflow/infrastructure/auth/oidc_registry.py
OidcProviderConfig domain entity
Provider management in Settings UI

Sprint 18: Projects v1

Size: M-L | Owner: Backend + Client | Prerequisites: Sprint 16 Status: ❌ NOT IMPLEMENTED

Objective: Introduce a first-class container above meetings/tasks.

Missing Components

Component	Required Location
ProjectModel	`persistence/models/project.py`
Project domain entity	`domain/entities/project.py`
Project RPCs	proto messages needed
Project UI	`client/src/pages/Projects.tsx`
Task domain entity	Currently ORM-only, no gRPC API

⚠️ Blocker: Sprint 21 (MCP Config) and Sprint 22 (Rules) depend on project scoping.

Deliverables

src/noteflow/domain/entities/project.py
src/noteflow/infrastructure/persistence/models/project.py
Project RPCs in proto
client/src/pages/Projects.tsx
src/noteflow/domain/identity/scope.py — Scope lattice (workspace → project → resource)
src/noteflow/grpc/_interceptors/scope.py — Enforce scope on all RPCs

Sprint 19: Artifacts v1

Size: XL | Owner: Backend | Prerequisites: Sprint 18 Status: ✅ PREREQUISITES VERIFIED

Objective: Get any non-meeting corpus into embeddings and retrievable context.

Verified Assets

Asset	Location
SegmentModel.embedding	`models/core/meeting.py:202-205` with Vector(1536)
cosine_distance query	`repositories/segment_repo.py:148` in `search_semantic()`
Encrypted asset storage	`infrastructure/audio/writer.py` with AES-GCM
EMBEDDING_DIM = 1536	`models/_base.py:8`

Deliverables

src/noteflow/domain/entities/artifact.py
src/noteflow/infrastructure/artifacts/ — Storage, chunking, embedding
RetrieveContext RPC for RAG queries
Artifact upload UI

Sprint 20: Artifacts v2

Size: XL | Owner: Backend + Client | Prerequisites: Sprint 19 Status: 🚫 BLOCKED

Objective: Sync from external sources (directories, cloud drives).

Verified Assets

Asset	Location
StartIntegrationSync RPC	`noteflow.proto:73`, `grpc/_mixins/sync.py:47-105`
Google OAuth	`oauth_manager.py` with PKCE
CalendarEventModel	`models/integrations/integration.py:188-255`

Blocked by: Sprint 19 (Artifacts v1) must be implemented first.

Deliverables

src/noteflow/infrastructure/artifacts/sources/ — Directory, Drive, OneDrive
ArtifactSource domain entity with sync metadata

Sprint 21: MCP Server Configuration

Size: L | Owner: Backend + Client | Prerequisites: Sprint 18 Status: 🚫 BLOCKED

Objective: Centralize "where context/tools come from" before Rules + LangGraph.

Missing Prerequisites

Prerequisite	Status
Project scoping (Sprint 18)	❌ Not implemented

Missing Components

Component	Required Location
MCP registry schema	`domain/entities/mcp_config.py`
MCP registry infrastructure	`infrastructure/mcp/registry.py`
MCP RPCs	proto messages needed
Scope precedence	Workspace defaults + project overrides + resource overrides
Credential boundary	Per-scope secrets (workspace vs project)

Action required: Complete Sprint 18 (Projects) before starting.

⚠️ Blocker: Sprint 22 (Rules) and Sprint 25 (LangGraph) depend on MCP configuration.

Sprint 22: Rules v1

Size: XL | Owner: Backend + Client | Prerequisites: Sprint 16, Sprint 21 Status: 🚫 BLOCKED — NOT IMPLEMENTED

Objective: Automate recording, templates, and outputs without chaos.

Missing Prerequisites

Prerequisite	Status
Sprint 21 (MCP Config)	🚫 Blocked by Sprint 18

Missing Components

Component	Required Location
Rules schema	`domain/rules/schema.py`
Rules evaluator	`domain/rules/evaluator.py`
Rules service	`application/services/rules_service.py`
Rules RPCs	proto messages needed
Rules UI	`client/src/pages/Rules.tsx`
Domain event schema	`domain/events/schema.py` (inputs for rules + audit)

Verified Assets

Asset	Location
TriggerSettings (23 fields)	`config/settings.py`
trigger_meeting_apps	List of 13 app names
Summarization templates	4 tone × 4 format × 4 verbosity
Export infrastructure	`infrastructure/export/`

Sprint 23: Rules Auditor + Analytics

Size: M-L | Owner: Backend + Client | Prerequisites: Sprint 15, Sprint 22 Status: 🚫 BLOCKED

Objective: Governance and feedback loops.

Missing Prerequisites

Prerequisite	Status	Impact
Usage events (Sprint 15)	❌ Not implemented	Cannot aggregate usage data
Rules schema (Sprint 22)	🚫 Blocked	Cannot audit rule execution

Action required: Complete Sprint 15 (Usage Events) and Sprint 22 (Rules Schema) before starting.

Deliverables

src/noteflow/application/services/rules_auditor.py
src/noteflow/domain/entities/evaluation.py
Analytics pages in client

Sprint 24: Entities Graph + Explore

Size: L-XL | Owner: Backend + Client | Prerequisites: Sprint 18, Sprint 19 Status: ⚠️ PARTIAL PREREQUISITES

Objective: Transform entities into a navigable knowledge layer.

Verified Assets

Asset	Location
NamedEntity with segment_ids	`domain/entities/named_entity.py:69`
Entity extraction pipeline	`application/services/ner_service.py`, `infrastructure/ner/engine.py`
People.tsx page	`client/src/pages/People.tsx`

Missing Prerequisites

Prerequisite	Status
Sprint 18 (Projects)	❌ Not implemented
Sprint 19 (Artifacts)	❌ Not implemented

Partial blocker: Core entity infrastructure exists. Graph schema can proceed, but project/artifact integration requires Sprint 18/19.

Deliverables

src/noteflow/infrastructure/graph/ — Schema, queries
client/src/pages/Explore.tsx

Sprint 25+: LangGraph Migration

Size: XL (multi-sprint) | Owner: Backend | Prerequisites: Sprint 15, Sprint 21 Status: 🚫 BLOCKED

Objective: Replace AI workflows with LangGraph after context sources exist.

Missing Prerequisites

Prerequisite	Status	Impact
MCP configuration (Sprint 21)	🚫 Blocked	Cannot configure tool sources
Usage events (Sprint 15)	❌ Not implemented	Cannot emit run metadata
Project scoping (Sprint 18)	❌ Not implemented	Blocks Sprint 21

Verified Assets

Asset	Location
Summarization service	`application/services/summarization_service.py`
RAG retrieval	`SegmentModel.embedding` + cosine_distance

Action required: Complete Sprint 18 (Projects), Sprint 21 (MCP Config), and Sprint 15 (Usage Events) before starting.

Dependency Graph (Updated 2025-12-29)

Phase 4 Complete
       │
       ▼
┌──────────────────────────────────────────────────────────────────────────┐
│                           WAVE 1: Foundation                              │
├──────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│  ┌──────────────┐                                                        │
│  │  Sprint 15   │ Platform Hardening                                     │
│  │ ⚠️ PARTIAL   │ OTel + usage events MISSING ───────────────┐           │
│  └──────┬───────┘                                            │           │
│         │                                                    │           │
│         ▼                                                    │           │
│  ┌──────────────┐                                            │           │
│  │  Sprint 16   │ Identity Foundation                        │           │
│  │ ✅ READY     │                                            │           │
│  └──────┬───────┘                                            │           │
│         │                                                    │           │
│         ├─────────────────┬──────────────────┐               │           │
│         ▼                 ▼                  ▼               │           │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐        │           │
│  │  Sprint 17   │  │  Sprint 18   │  │              │        │           │
│  │ ✅ READY     │  │ ❌ NEEDED    │  │   (Sprint    │        │           │
│  │ Custom OAuth │  │  Projects    │  │    21 MCP    │        │           │
│  └──────────────┘  └──────┬───────┘  │   blocked)   │        │           │
│                           │          └──────────────┘        │           │
└───────────────────────────┼──────────────────────────────────┼───────────┘
                            │                                  │
┌───────────────────────────┼──────────────────────────────────┼───────────┐
│                           │      WAVE 2: Integration         │           │
├───────────────────────────┼──────────────────────────────────┼───────────┤
│                           │                                  │           │
│                           ▼                                  │           │
│  ┌───────────────────────────────────────┐                   │           │
│  │              Sprint 19                │                   │           │
│  │           ✅ Artifacts v1             │                   │           │
│  │        (Prerequisites ready)          │                   │           │
│  └──────────────────┬────────────────────┘                   │           │
│                     │                                        │           │
│         ┌───────────┴───────────┐                            │           │
│         ▼                       ▼                            │           │
│  ┌──────────────┐       ┌──────────────┐                     │           │
│  │  Sprint 20   │       │  Sprint 21   │                     │           │
│  │ Artifacts v2 │       │ 🚫 BLOCKED   │◄────────────────────┤           │
│  │ (needs S19)  │       │   MCP Config │  needs Sprint 18    │           │
│  └──────────────┘       └──────┬───────┘                     │           │
│                                │                             │           │
└────────────────────────────────┼─────────────────────────────┼───────────┘
                                 │                             │
┌────────────────────────────────┼─────────────────────────────┼───────────┐
│                                │      WAVE 3: Intelligence   │           │
├────────────────────────────────┼─────────────────────────────┼───────────┤
│                                │                             │           │
│                                ▼                             │           │
│                        ┌──────────────┐                      │           │
│                        │  Sprint 22   │                      │           │
│                        │ 🚫 BLOCKED   │                      │           │
│                        │   Rules v1   │                      │           │
│                        └──────┬───────┘                      │           │
│                               │                              │           │
│                    ┌──────────┴──────────┐                   │           │
│                    ▼                     ▼                   │           │
│             ┌──────────────┐      ┌──────────────┐           │           │
│             │  Sprint 23   │      │  Sprint 25+  │◄──────────┘           │
│             │ 🚫 BLOCKED   │      │ 🚫 BLOCKED   │  needs Usage events   │
│             │  Analytics   │      │  LangGraph   │                       │
│             └──────────────┘      └──────────────┘                       │
│                                                                          │
│             ┌──────────────┐                                             │
│             │  Sprint 24   │ (can start after S18+S19)                   │
│             │ ⚠️ PARTIAL   │                                             │
│             │Graph/Explore │                                             │
│             └──────────────┘                                             │
│                                                                          │
└──────────────────────────────────────────────────────────────────────────┘

LEGEND:
  ✅ READY      = Prerequisites verified, can start
  ⚠️ PARTIAL    = Some work done, gaps remain
  ❌ NEEDED     = Blocker for downstream sprints
  🚫 BLOCKED    = Cannot start until prerequisites complete

Recommended Execution Order

Sequential Constraints (Must Follow)

Sprint 15 ─► Sprint 16 ─► Sprint 17   (Identity chain)
Sprint 16 ─► Sprint 18 ─► Sprint 19   (Projects → Artifacts)
Sprint 18 ─► Sprint 21 ─► Sprint 22   (MCP → Rules)
Sprint 19 ─► Sprint 20                (Artifacts v1 → v2)
Sprint 22 ─► Sprint 23                (Rules → Analytics)

Parallel Opportunities

Parallel Group	Sprints	Shared Resource Conflicts
Group A	15 + 16	None (different subsystems)
Group B	17 + 18	Both need Sprint 16 complete first
Group C	19 + 21	Both need Sprint 18 complete first
Group D	20 + 22	Low overlap (artifacts vs rules)
Group E	23 + 24 + 25	All need different prereqs, can parallelize

Optimal Execution Timeline

Week 1-2:   Sprint 15 (OpenTelemetry + Usage Events) + Sprint 16 (Identity) ─ parallel
Week 2-3:   Sprint 17 (OAuth) + Sprint 18 (Projects) ─ parallel
Week 3-5:   Sprint 19 (Artifacts v1) + Sprint 21 (MCP) ─ parallel
Week 5-7:   Sprint 20 (Artifacts v2) + Sprint 22 (Rules) ─ parallel
Week 7-9:   Sprint 23 + Sprint 24 + Sprint 25 ─ all can parallelize

Effort Sizing Summary

Size	Sprints	Description
S	—	Single-focus, < 1 week
M	15, 23	Multi-file, 1-2 weeks
M-L	18	Significant but bounded
L	16, 17, 21, 24	Multi-component, 2-3 weeks
XL	19, 20, 22, 25+	Major subsystem, 3+ weeks

Architecture Decisions (Finalized 2025-12-29)

Decision	Sprint	Choice	Rationale
Auth Model	16	Local-first	Single user per install, design for multi-user. Reduces Sprint 16 from L → M.
OIDC Config	17	Discovery only	Require issuer URL with `.well-known`. Covers 90%+ of providers.
Artifact Scope	19	Per-workspace	Simpler ACL, easier cross-project sharing.
MCP Storage	21	Backend DB	Enables team sharing, consistent behavior across devices.
Rules Execution	22	Backend-evaluated	Audit trails, single source of truth. Client handles triggers only.
Offline Mode	12	Cached read-only	Prevents silent divergence; no mock writes.
Observability	15	OpenTelemetry	Standardized traces/metrics/logs + correlation IDs.
Scope Lattice	16/18	Workspace → Project → Resource	Consistent scoping across repos, RPCs, and MCP.

Risk Assessment

Risk	Mitigation
Auth complexity explosion	Start local-first, defer multi-user
Artifact storage costs	Implement quotas, lazy embedding
Rules engine complexity	Start with simple precedence, iterate
LangGraph learning curve	Wrap existing code first, migrate incrementally
Breaking changes cascade	Strong proto versioning, feature flags
Sprint 18 blocking cascade	Prioritize Projects implementation to unblock 21/22/25
Usage events blocking analytics	Complete Sprint 15 OpenTelemetry + usage events early
Scope leakage across projects	Enforce `ScopeContext` filters in repos + gRPC interceptors from Sprint 16/18

Appendix: Scope Lattice Enforcement (Planned)

Goal: Ensure every request is scoped consistently: workspace → project → resource.

Scope Rules (Best‑in‑Class Defaults)

Workspace is mandatory for all writes and most reads.
Project is optional but, when present, is a strict filter on resources that declare project_id.
Resource access always validates ownership by scope (meeting/task/artifact must belong to workspace, and project if set).
Overrides are explicit and auditable (e.g., “cross‑project search” requires a dedicated endpoint and opt‑in policy).

Enforcement Points (Planned + Existing)

gRPC Interceptors: src/noteflow/grpc/interceptors/scope.py (planned in Sprint 16) enforces scope on every RPC before handler logic.
Repository Filters: src/noteflow/infrastructure/persistence/repositories/_scoped.py (planned in Sprint 16) applies workspace_id/project_id constraints to queries.
Context Model: src/noteflow/domain/identity/context.py (planned changes in Sprint 16) adds ProjectContext and OperationContext.project_id.
Project Entities: src/noteflow/domain/entities/project.py + src/noteflow/infrastructure/persistence/models/project.py (planned in Sprint 18).

Concrete Examples (Planned)

Meeting list: repository method appends WHERE workspace_id = :workspace_id and AND project_id = :project_id (if present).
Artifacts: enforce per‑workspace by default, but allow project association with explicit policy flags.
Rules: evaluator consumes OperationContext.project_id and denies cross‑project actions unless rule explicitly opts in.

Tests (Required by Quality Standards)

tests/infrastructure/test_scoped_repos.py (planned): validates workspace/project filters.
tests/grpc/test_scope_enforcement.py (planned): verifies interceptors block out‑of‑scope access.
tests/grpc/test_projects.py (planned): ensures project CRUD respects scope.

All scope changes must satisfy docs/sprints/QUALITY_STANDARDS.md quality gates.

Quality Gates (All Sprints)

pytest tests/quality/           # 23+ test smell checks
ruff check src/noteflow         # Zero lint errors
basedpyright                    # Zero type errors
npm run lint                    # Zero frontend errors
npm run test                    # Unit tests pass

See docs/sprints/QUALITY_STANDARDS.md for full quality gates, test limits, and lint rules.

Appendix: Phase 1-3 (Complete)

Phase	Sprints	Focus
0	0	Proto & Schema Foundation
1	1-3	Core Pipeline (templates, diarization, PDF)
2	4	Intelligence (NER)
3	5-6	Integrations (calendar, webhooks)

See docs/sprints/phase-*/ for historical documentation.

37 KiB Raw Blame History Unescape Escape

NoteFlow Feature Gap Analysis & Development Roadmap

Executive Summary

Current State (Validation 2025-12-29)

Existing Infrastructure (Verified 2025-12-29)

Phase 5 Implementation Status (2025-12-29)

Sprint Status Overview

Critical Path

Parallel Execution Map

Wave 1: Foundation (Can Start Immediately)

Wave 2: Integration (After Wave 1)

Wave 3: Intelligence (After Wave 2)

Sprint Overview

Phase 4: Productization (Validation 2025-12-29)

Phase 5: Platform Evolution

Phase 4: Productization (Validated 2025-12-29)

Sprint 10: Integration Config + OAuth Types

Sprint 11: Trigger System Wiring

Sprint 12: Tauri Fallback & Offline State

Sprint 13: Simulated Transcription Toggle

Sprint 14: Preferences Sync

Phase 5: Platform Evolution

Sprint 15: Platform Hardening

Implementation Status

Remaining Deliverables

Sprint 16: Identity Foundation

Verified Assets

Deliverables

Sprint 17: Custom OAuth Providers

Verified Assets

Deliverables

Sprint 18: Projects v1

Missing Components

Deliverables

Sprint 19: Artifacts v1

Verified Assets

Deliverables

Sprint 20: Artifacts v2

Verified Assets

Deliverables

Sprint 21: MCP Server Configuration

Missing Prerequisites

Missing Components

Sprint 22: Rules v1

Missing Prerequisites

Missing Components

Verified Assets

Sprint 23: Rules Auditor + Analytics

Missing Prerequisites

Deliverables

Sprint 24: Entities Graph + Explore

Verified Assets

Missing Prerequisites

Deliverables

Sprint 25+: LangGraph Migration

Missing Prerequisites

Verified Assets

Dependency Graph (Updated 2025-12-29)

Recommended Execution Order

Sequential Constraints (Must Follow)

Parallel Opportunities

Optimal Execution Timeline

Effort Sizing Summary

Architecture Decisions (Finalized 2025-12-29)

Risk Assessment

Appendix: Scope Lattice Enforcement (Planned)

Scope Rules (Best‑in‑Class Defaults)

Enforcement Points (Planned + Existing)

Concrete Examples (Planned)

Tests (Required by Quality Standards)

Quality Gates (All Sprints)

Appendix: Phase 1-3 (Complete)

37 KiB

Raw Blame History