77 Commits

Author SHA1 Message Date
f3fc8bfcb8 Implement Preferences Sync functionality and update roadmap
- Introduced gRPC endpoints for user preferences synchronization: GetPreferences and SetPreferences.
- Completed implementation of preferences sync infrastructure, including ETag conflict detection and handling.
- Updated roadmap to reflect the completion status of Sprint 14: Preferences Sync.
- Enhanced documentation for preferences sync, including detailed descriptions of request and response messages.
- Added comprehensive tests for PreferencesMixin, covering retrieval, updates, and conflict resolution scenarios.

All quality checks pass.
2025-12-29 08:53:30 -05:00
7cf0f2559b Update client submodule and remove obsolete documentation
- Updated the client submodule reference to align with the latest changes in the project.
- Deleted the feasibility and specification documents that are no longer relevant to the current implementation.
- Ensured that the repository remains clean and focused on active components, maintaining consistency across documentation and codebase.
2025-12-29 07:37:25 -05:00
44500f0b61 Update OAuth integration tests and documentation
- Added comprehensive behavioral tests for OAuth gRPC endpoints, including InitiateOAuth, CompleteOAuth, GetOAuthConnectionStatus, and DisconnectOAuth.
- Updated README documentation for Sprint 10 to reflect the implementation status of OAuth features and test coverage.
- Revised integration configuration details in the documentation to include new test files and their coverage.
- Updated client submodule reference to ensure alignment with the latest changes in the OAuth flow implementation.
2025-12-29 03:28:43 -05:00
4fabce8bf8 Add Phase 5 sprint documentation and update roadmap
- Add 11 sprint READMEs for Phase 5: Platform Evolution (sprints 15-25)
- Update roadmap with OpenTelemetry instrumentation details for Sprint 15
- Clarify Sprint 12 scope: cached read-only offline mode (not silent writes)
- Update Sprint 14 documentation for preferences sync implementation
- Add feasibility study and observability architecture docs
- Fix user_preferences migration column definitions
- Update client submodule with observability and sync infrastructure
2025-12-29 04:37:25 +00:00
b5612b9250 Validate Phase 4 sprint documentation against codebase
Add validation status sections to Sprint 10-14 READMEs with verified
file locations and line numbers. Update roadmap with accurate
implementation status.

Findings:
- Sprint 10 (OAuth): Complete - PKCE S256, deep links, token storage
- Sprint 11 (Triggers): Complete - 23 config fields, adaptive noise floor
- Sprint 12 (Offline): Not implemented - connection state infra missing
- Sprint 13 (Simulation): 50% - toggle exists but doesn't control behavior
- Sprint 14 (Preferences): Not implemented - no gRPC sync endpoints
2025-12-29 03:46:00 +00:00
4865433140 Implement integration sync orchestration and observability features
- Add gRPC endpoints for integration sync: StartIntegrationSync, GetSyncStatus, ListSyncHistory.
- Introduce SyncRun entity to track synchronization operations, including status and duration.
- Implement observability features with logging and performance metrics collection.
- Update documentation to reflect new endpoints and their usage.
- Enhance test coverage with behavioral tests for sync orchestration and observability components.

All quality checks pass.
2025-12-29 01:17:54 +00:00
2ab836282a Complete Sprint 8: Desktop Parity & Entity Persistence
Backend:
- Add UpdateEntity gRPC endpoint for text/category mutations
- Add DeleteEntity gRPC endpoint for single-entity removal
- Add entity_repo.update() and entity_repo.delete() methods
- Add 10 integration tests for entity mutations

Documentation:
- Update Sprint 8 README with COMPLETE status
- Mark all 14 acceptance criteria as done
- Update component table with implementation locations

Client submodule updated with:
- Tauri audio test commands (Rust/cpal)
- Entity store persistence functions
- EntityManagementPanel backend wiring
- Code quality fixes and new tests
2025-12-28 21:04:43 +00:00
9b224e6d83 Complete Sprint 7: Wiring & Correctness
- Add cloud consent gRPC endpoints (Grant, Revoke, GetStatus)
- Wire consent persistence via on_consent_change callback to database
- Refactor html.py export method to fix long-method violation
- Add comprehensive gRPC tests for consent endpoints
- Update sprint 7 documentation with completion status
- Update client submodule with frontend consent wiring
2025-12-28 19:00:29 +00:00
27590ca2c8 Add phase 4 productization sprint documentation
Sprint planning docs for sprints 7-14 covering wiring correctness,
desktop parity, integrations/analytics, integration config, trigger
wiring, Tauri fallback, simulated transcription, and preferences sync.
2025-12-28 17:44:45 +00:00
1b731de529 Update configuration and documentation for Noteflow
- Enhanced `example.env` with detailed sections for backend configurations, including ASR, gRPC streaming, and retention settings.
- Added new environment variables for LLM integration, webhooks, and calendar synchronization.
- Updated README files across multiple sprints to reflect validation statuses and completion of features, ensuring clarity on project progress.
- Improved test coverage for edge cases in annotation, meeting, segment, summary, and trigger entities, ensuring robustness in handling various scenarios.
- Updated the client submodule reference to indicate a dirty state, reflecting uncommitted changes.
2025-12-27 12:28:14 -05:00
d658d60241 Update dependencies and enhance calendar integration features
- Updated `pyproject.toml` to include `authlib` as a dependency for OAuth integration.
- Modified `uv.lock` to reflect the addition of `authlib` and its versioning.
- Updated documentation to clarify existing components and their statuses in the calendar sync and webhook integration sprints.
- Refactored various methods and properties for improved clarity and consistency in the meeting and export services.
- Enhanced test coverage for export functionality, including assertions for PDF format.
- Updated client submodule reference to ensure alignment with the latest changes.
2025-12-26 18:18:06 -05:00
88f843c835 Add PDF export functionality using WeasyPrint
- Introduced `PdfExporter` class for exporting meeting transcripts to PDF format.
- Updated `ExportService` to support PDF as a new export format.
- Enhanced `proto_to_export_format` to handle PDF format.
- Added WeasyPrint as a dependency in `pyproject.toml` and `uv.lock`.
- Updated documentation to reflect the new PDF export capabilities and resolved issues from Sprint 3.
- Implemented tests for PDF export functionality, ensuring valid PDF output and proper handling of meeting data.
2025-12-25 23:39:56 -05:00
d285da3d36 Enhance mypy configuration and update roadmap documentation
- Updated `pyproject.toml` to include additional mypy exclusions and plugins for improved type checking.
- Added mypy overrides for specific modules to handle complex type parameters and untyped calls.
- Revised roadmap documentation to reflect changes in the Diarization Application Service, renaming it to "Diarization Enhancements" and updating feature descriptions for clarity.
- Adjusted architectural decisions to clarify the application service layer requirements for upcoming sprints.
2025-12-25 23:21:07 -05:00
58ad140f48 Complete Sprint 0: Proto & Schema Foundation
Sprint 0 consolidates all proto schema changes and database migrations
for Sprints 1-6 into a coordinated release.

Infrastructure:
- Add named_entities table migration with updated_at trigger
- Add webhook_configs and webhook_deliveries migrations
- Add httpx dependency for webhooks
- Multi-stage Docker build with NER support (with-ner target)
- Enable depends_on for db service in compose.yaml

Proto schema:
- Add SummarizationOptions message (Sprint 1)
- Add EXPORT_FORMAT_PDF enum value (Sprint 3)
- Add ExtractEntities RPC and messages (Sprint 4)
- Add Calendar RPCs and messages (Sprint 5)
- Proto regeneration script at scripts/regenerate_proto.sh

Tests:
- Add proto compilation tests verifying Sprint 0 messages
- Add feature flag tests with env var parsing
- Add migration structure and trigger tests

Documentation:
- Fix PROTO_CHANGELOG.md CalendarProvider field names
- Update COMPLETION_PLAN.md status to 100%

Client submodule updated with Sprint 0 TypeScript types.
2025-12-26 00:35:14 +00:00
ee12bcd89e Sprint 0: Proto & Schema Foundation
Consolidate proto/DB changes for Sprints 1-6 into coordinated release:

Proto Schema (Increment 0a):
- Add SummarizationOptions message with tone/format/verbosity fields
- Add EXPORT_FORMAT_PDF = 3 to ExportFormat enum
- Add NER messages: ExtractEntitiesRequest/Response, ExtractedEntity
- Add Calendar messages: CalendarEvent, ListCalendarEvents, CalendarProvider
- Add RPCs: ExtractEntities, ListCalendarEvents, GetCalendarProviders,
  InitiateCalendarAuth, CompleteCalendarAuth
- Regenerate Python proto stubs

Database Schema (Increment 0b):
- Add named_entities table for NER storage
- Add webhook_configs and webhook_deliveries tables

Migrations & ORM (Increment 0c):
- Create migration h2c3d4e5f6g7 for named_entities
- Create migration i3d4e5f6g7h8 for webhook tables
- Add NamedEntityModel, WebhookConfigModel, WebhookDeliveryModel
- Update MeetingModel and WorkspaceModel relationships

Dependencies & Config (Increment 0d):
- Add optional deps: pdf (weasyprint), ner (spacy), calendar (google-api)
- Add FeatureFlags class with NOTEFLOW_FEATURE_ env prefix
- Add models CLI for ML model downloads (spacy)
- Update CLI dispatcher for retention/models subcommands
- Create PROTO_CHANGELOG.md

Quality Fixes:
- Add ORM model names to ALLOWED_STRINGS in test_magic_values.py
- Add assertion messages to fix assertion roulette tests

Updates client submodule to c1783f8.
2025-12-25 14:48:37 -05:00
6fa792990b Enhance summarization model attributes and database schema
- Updated the Summary entity to include provider and model names, along with tokens used and latency metrics for better tracking of summarization performance.
- Modified the ORM converters and repository methods to accommodate new attributes, ensuring backward compatibility.
- Introduced word timing position indexing to maintain order within summaries.
- Added a new SQLAlchemy model structure for improved organization of persistence layers, including core, identity, and integration models.
- Removed deprecated models and files to streamline the codebase.
2025-12-25 13:46:00 -05:00
5df60507ea Refactor gRPC client architecture and add code quality testing
Backend:
- Extract gRPC client into modular mixins (_client_mixins/)
- Add StreamingSession class for audio streaming lifecycle
- Add gRPC config and type modules
- Fix test smells across test suite

Frontend (submodule update):
- Fix code quality issues and eliminate lint warnings
- Centralize CSS class constants
- Extract Settings.tsx sections into components
- Add code quality test suite

Quality:
- Add tests/quality/ suite for code smell detection
- Add QA report and correction plan documentation
2025-12-24 17:41:01 +00:00
4e1921b939 Enhance documentation and configuration for NoteFlow
- Added a comprehensive MCP Tools Reference section in `CLAUDE.md`, detailing various tools and their use cases for web scraping, reasoning, library documentation, semantic code tools, and more.
- Updated `example.env` to reflect new configuration options for retention, diarization, encryption, and desktop client settings, improving clarity and usability.
- Introduced a new `roadmap.md` file outlining the feature gap analysis and development roadmap for NoteFlow, focusing on core pipeline completion and future enhancements.
- Updated submodule reference in the `client` directory to the latest commit.
2025-12-23 21:34:06 +00:00
4567d5a03c Update project documentation and configuration files
- Expanded `.gitignore` to include temporary files and scratch documents.
- Enhanced `AGENTS.md` with guidelines on using Conventional Commits and PR descriptions.
- Added configuration notes to `README.md` for environment variable usage.
- Updated `pyproject.toml` to reflect changes in dependencies and comments.
- Introduced new files for client configuration and testing, including Prettier settings and Playwright tests.
- Removed deprecated client components and adjusted related documentation accordingly.
- Added new tests for various components and functionalities to improve coverage.
2025-12-22 02:25:38 +00:00
703a9f2670 Refactor Repomix configuration and update documentation
- Removed the `support/` and `tests/` directories from the Repomix configuration to streamline included files.
- Updated `repomix-output.md` to reflect the new file structure and clarify included patterns.
- Deleted the outdated `triage.md` document to reduce clutter and improve project organization.
- Introduced a new `ui.md` document detailing the migration plan for transitioning to a Rust/Tauri application, including architecture overview and feature parity checklist.
2025-12-21 01:02:11 +00:00
d66aa6b958 Update configuration and enhance transcript component functionality
- Added `repomix-output.md` to `.gitignore` to exclude output files from version control.
- Updated `repomix.config.json` to include the `support/` directory in the include paths.
- Enhanced `TranscriptComponent` to improve search functionality with a debounce mechanism for input.
- Refactored search logic to filter visible segments based on the search query, ensuring better user experience.
- Updated `Meeting` and `Annotation` entities to use `utc_now()` for consistent UTC-aware timestamps.
- Introduced new repository protocols for `DiarizationJob` and `Preferences` to support additional functionalities.
- Implemented in-memory persistence for repositories to facilitate testing and development without a database.
- Added error handling and utility functions to streamline gRPC service responses and improve code maintainability.
2025-12-20 20:45:58 +00:00
a1fc7edeea Enhance recovery and summarization services with asset path management
- Added `asset_path` to the `Meeting` entity for audio asset storage.
- Implemented `AudioValidationResult` for audio integrity checks during recovery.
- Updated `RecoveryService` to validate audio file integrity for crashed meetings.
- Enhanced `SummarizationService` to include consent persistence callbacks.
- Introduced new database migrations for `diarization_jobs` and `user_preferences` tables.
- Refactored various components to support the new asset path and audio validation features.
- Improved documentation in `CLAUDE.md` to reflect changes in recovery and summarization functionalities.
2025-12-19 10:40:21 +00:00
b333ea5b23 Add initial Docker and development environment setup
- Created .dockerignore to exclude unnecessary files from Docker builds.
- Added .repomixignore for managing ignored patterns in Repomix.
- Introduced Dockerfile.dev for development environment setup with Python 3.12.
- Configured docker-compose.yaml to define services, including a PostgreSQL database.
- Established a devcontainer.json for Visual Studio Code integration.
- Implemented postCreate.sh for automatic dependency installation in the dev container.
- Added constants.py to centralize configuration constants for the project.
- Updated pyproject.toml to include new development dependencies.
- Created initial documentation files for project overview and style conventions.
- Added tests for new functionalities to ensure reliability and correctness.
2025-12-19 05:02:16 +00:00
d2f4180716 Update milestones and complete gRPC integration for summarization service
- Marked Milestone 6 as complete, indicating that all exit criteria have been met.
- Updated documentation to reflect the completion of the evidence-linked summaries feature.
- Integrated the `GenerateSummary` RPC with the `SummarizationService`, enabling real summarization functionality.
- Enhanced the gRPC server and client to auto-detect summarization service providers, improving reliability.
- Refactored the summarization logic to handle fallback scenarios when providers are unavailable.
- Improved the display of annotation text in the UI for better user experience.
2025-12-18 17:00:43 +00:00
70b13b52d9 Refactor trigger detection into a mixin for improved modularity
- Extracted trigger detection logic from `app.py` into a new `TriggerMixin` class, enhancing code organization and maintainability.
- Introduced `MeetingAudioReader` to handle reading archived audio for meetings, supporting playback review.
- Added `AnnotationDisplayComponent` to display existing annotations during meeting reviews, improving user experience.
- Updated `NoteFlowClientApp` to utilize the new mixin and components, streamlining the application structure.
- Enhanced gRPC client to retrieve meeting segments, facilitating better integration with the UI components.
2025-12-18 15:59:16 +00:00
4eef1b3be6 Add summarization and trigger services
- Introduced `SummarizationService` and `TriggerService` to orchestrate summarization and trigger detection functionalities.
- Added new modules for summarization, including citation verification and cloud-based summarization providers.
- Implemented trigger detection based on audio activity and foreground application status.
- Updated project configuration to include new dependencies for summarization and trigger functionalities.
- Created tests for summarization and trigger services to ensure functionality and reliability.
2025-12-18 00:08:51 +00:00
af1285b181 Add initial project structure and files
- Introduced .python-version for Python version management.
- Added AGENTS.md for documentation on agent usage and best practices.
- Created alembic.ini for database migration configurations.
- Implemented main.py as the entry point for the application.
- Established pyproject.toml for project dependencies and configurations.
- Initialized README.md for project overview.
- Generated uv.lock for dependency locking.
- Documented milestones and specifications in docs/milestones.md and docs/spec.md.
- Created logs/status_line.json for logging status information.
- Added initial spike implementations for UI tray hotkeys, audio capture, ASR latency, and encryption validation.
- Set up NoteFlow core structure in src/noteflow with necessary modules and services.
- Developed test suite in tests directory for application, domain, infrastructure, and integration testing.
- Included initial migration scripts in infrastructure/persistence/migrations for database setup.
- Established security protocols in infrastructure/security for key management and encryption.
- Implemented audio infrastructure for capturing and processing audio data.
- Created converters for ASR and ORM in infrastructure/converters.
- Added export functionality for different formats in infrastructure/export.
- Ensured all new files are included in the repository for future development.
2025-12-17 18:28:59 +00:00