42 Commits

Author SHA1 Message Date
d8090a98e8 ci/cd fixes
Some checks failed
CI / test-typescript (push) Has been cancelled
CI / test-rust (push) Has been cancelled
CI / test-python (push) Has been cancelled
2026-01-26 00:28:15 +00:00
2641a9fc03 optimization
Some checks failed
CI / test-python (push) Failing after 17m22s
CI / test-rust (push) Has been cancelled
CI / test-typescript (push) Has been cancelled
2026-01-25 01:40:14 +00:00
7140abfdee x
Some checks failed
CI / test-typescript (push) Has been cancelled
CI / test-rust (push) Has been cancelled
CI / test-python (push) Has been cancelled
2026-01-24 18:08:10 +00:00
acfba090e4 feat: Introduce Gitea CI/CD workflows, refactor Docker deployment with dedicated dev/prod compose files and enhanced ROCm GPU support, and update RAG documentation for new AI and ASR infrastructure. 2026-01-24 14:50:19 +00:00
d33c421756 huge 2026-01-23 07:45:20 +00:00
60540f373c merging langgraph changes into master 2026-01-22 17:34:49 +00:00
25fcd5d7f2 feat: Update gRPC logging and identity interceptors, remove stub patching script, and update project dependencies. 2026-01-20 21:13:12 +00:00
40f9adf27f feat: introduce GLiNER and spaCy backends for NER, refactor NER infrastructure, and update client-side entity extraction. 2026-01-20 03:47:12 +00:00
3222826ff7 feat: Implement PyTorch and ROCm ASR engines with GPU detection and enhance calendar OAuth integration across client and server. 2026-01-18 12:47:31 +00:00
a1bafcf578 big 2026-01-06 08:03:04 +00:00
997626de75 chore: update dependencies and improve linting results
- Added `grpc-stubs` version `1.53.0.6` to `pyproject.toml` and `uv.lock` for enhanced gRPC support.
- Cleared previous linting errors, resulting in zero errors and warnings in the linting reports.
- Updated the linter configuration to reflect recent analysis results, ensuring accurate diagnostics and reporting.
2026-01-05 06:06:20 -05:00
4fceb95438 fix: update types-grpcio dependency version and improve meeting filtering
- Changed the types-grpcio dependency from `>=1.0.0.20251009` to `==1.0.0.20251001` in `pyproject.toml` and `uv.lock` for consistency.
- Enhanced the meeting filtering logic in the `MeetingStore` and related classes to support filtering by multiple project IDs, improving query flexibility.
- Updated the gRPC proto definitions to include a new `project_ids` field for the `ListMeetingsRequest` message, allowing for more granular project filtering.
- Adjusted related repository and mixin classes to accommodate the new filtering capabilities.
2026-01-05 00:38:33 +00:00
95d1a98cc2 refactor: simplify Docker Compose configuration and add spacy dependency
- Changed server port binding from explicit `0.0.0.0:50051:50051` to `50051:50051` for standard Docker port mapping
- Commented out unused `server-full` and `client-dev` service definitions to reduce configuration complexity
- Added `spacy>=3.8.11` to project dependencies
2026-01-03 02:28:58 -05:00
15743b05a3 typing 2026-01-02 10:11:45 +00:00
e4b2c733d5 oh boy 2026-01-02 04:22:40 +00:00
f533d5aec2 x 2026-01-01 17:23:29 -05:00
8df1aa0c03 chore: update submodule and refine logging configuration
- Updated the client submodule to the latest commit for improved compatibility.
- Adjusted logging configuration in tests to utilize a more consistent format.
- Enhanced test assertions for clarity by removing unnecessary messages and ensuring consistency across various test files.

All quality checks pass.
2026-01-01 21:03:09 +00:00
b423199c5f chore: update dependencies and enhance Dockerfile for server
- Updated `pyproject.toml` to include new optional dependencies for Ollama: `anthropic`, `openai`, and `pywinctl`.
- Modified `compose.yaml` to add a new volume for the server's virtual environment.
- Enhanced the server Dockerfile to install all optional dependencies during the build process, improving flexibility and functionality.

All quality checks pass.
2025-12-31 11:31:13 -05:00
7d11e05a7f chore: add sounddevice dependency for audio processing
- Included the 'sounddevice' library in the project dependencies to enhance audio handling capabilities.
- This addition supports improved audio input/output functionalities within the application.

All quality checks pass.
2025-12-31 15:45:12 +00:00
96ed391a7c chore: update logging configuration and enhance project structure
- Added new logging configuration to improve observability across various services.
- Introduced a `.repomixignore` file to exclude unnecessary files from version control.
- Updated `pyproject.toml` to include additional paths for script discovery.
- Refreshed submodule references for the client to ensure compatibility with recent changes.

All quality checks pass.
2025-12-31 15:23:57 +00:00
e23c0555e2 chore: update development documentation and dependencies
- Added important notes regarding Docker operations to prevent disruption of the hot-reload server.
- Introduced a section outlining forbidden Docker commands without explicit user permission.
- Refactored `pyproject.toml` to categorize audio-related dependencies under an optional section.
- Updated dependencies for `weasyprint` and `spacy` to their latest versions.
- Adjusted the server Dockerfile to ensure the hot-reload server starts correctly.

All quality checks pass.
2025-12-31 05:29:33 -05:00
49354b80e9 feat: enhance project management and observability features
- Updated client submodule to the latest commit for improved integration.
- Added `types-psutil` as a dependency in `pyproject.toml` to support type checking for the `psutil` library.
- Enhanced observability by refining the `UsageEventSink` interface to include additional attributes for better event tracking.
- Improved project management documentation to reflect recent changes in project roles and settings.

All quality checks pass.
2025-12-31 04:51:39 +00:00
15da71a2dd feat: implement OIDC provider management and discovery features
- Introduced OIDC provider management capabilities, including registration, retrieval, updating, and deletion of providers.
- Added support for OIDC discovery, allowing dynamic fetching of provider configurations from `.well-known/openid-configuration` endpoints.
- Created domain entities for OIDC configuration, including `ClaimMapping`, `OidcProviderConfig`, and presets for common identity providers.
- Implemented gRPC endpoints for OIDC provider management and discovery operations.
- Enhanced tests for OIDC domain entities and gRPC mixins to ensure comprehensive coverage.

All quality checks pass.
2025-12-30 17:32:21 +00:00
3ea82ee868 feat: implement observability features and enhance error handling
- Added observability support with OpenTelemetry integration, including usage event tracking and logging enhancements.
- Introduced new `UsageEvent` and `UsageEventSink` abstractions for analytics and billing.
- Implemented `BufferedDatabaseUsageEventSink` for efficient event storage and retrieval.
- Enhanced error handling with a centralized domain error taxonomy, improving consistency across the application.
- Updated dependencies in `pyproject.toml` to include OpenTelemetry packages.

All quality checks pass.
2025-12-30 11:20:06 +00:00
3eaf9c98a8 chore: update dependencies and enhance logging
- Added `rich` library as a dependency for improved console output formatting.
- Updated `pyproject.toml` and `uv.lock` to include the new dependency.
- Enhanced logging in various services to provide clearer error handling and status updates.

All quality checks pass.
2025-12-30 09:34:06 +00:00
df7c06198f Add pytest-benchmark for performance testing and optimize audio processing
- Introduced `pytest-benchmark` dependency for performance testing of critical code paths.
- Added a new `PartialAudioBuffer` class to optimize audio chunk handling, reducing memory allocations and improving efficiency.
- Implemented benchmark tests for audio processing, validating performance improvements and establishing baselines.
- Enhanced the `Segmenter` class to utilize cached sample counts for faster duration calculations.
- Updated gRPC service to leverage consolidated streaming state, reducing multiple dictionary lookups to a single access.

All quality checks pass.
2025-12-30 00:26:41 +00:00
0c920b76bf Enhance asset management and retention services
- Introduced a new `AssetRepository` interface for managing binary assets, including a `FileSystemAssetRepository` implementation for deleting meeting assets.
- Updated the `RetentionService` and `MeetingService` to utilize the new asset management functionality, ensuring proper cleanup of associated assets during meeting deletions.
- Added versioning to the `Meeting` entity to handle concurrent modifications.
- Updated unit and integration tests to cover new asset management features and ensure robust functionality.

All quality checks pass.
2025-12-29 18:30:08 +00:00
4865433140 Implement integration sync orchestration and observability features
- Add gRPC endpoints for integration sync: StartIntegrationSync, GetSyncStatus, ListSyncHistory.
- Introduce SyncRun entity to track synchronization operations, including status and duration.
- Implement observability features with logging and performance metrics collection.
- Update documentation to reflect new endpoints and their usage.
- Enhance test coverage with behavioral tests for sync orchestration and observability components.

All quality checks pass.
2025-12-29 01:17:54 +00:00
9b224e6d83 Complete Sprint 7: Wiring & Correctness
- Add cloud consent gRPC endpoints (Grant, Revoke, GetStatus)
- Wire consent persistence via on_consent_change callback to database
- Refactor html.py export method to fix long-method violation
- Add comprehensive gRPC tests for consent endpoints
- Update sprint 7 documentation with completion status
- Update client submodule with frontend consent wiring
2025-12-28 19:00:29 +00:00
d658d60241 Update dependencies and enhance calendar integration features
- Updated `pyproject.toml` to include `authlib` as a dependency for OAuth integration.
- Modified `uv.lock` to reflect the addition of `authlib` and its versioning.
- Updated documentation to clarify existing components and their statuses in the calendar sync and webhook integration sprints.
- Refactored various methods and properties for improved clarity and consistency in the meeting and export services.
- Enhanced test coverage for export functionality, including assertions for PDF format.
- Updated client submodule reference to ensure alignment with the latest changes.
2025-12-26 18:18:06 -05:00
88f843c835 Add PDF export functionality using WeasyPrint
- Introduced `PdfExporter` class for exporting meeting transcripts to PDF format.
- Updated `ExportService` to support PDF as a new export format.
- Enhanced `proto_to_export_format` to handle PDF format.
- Added WeasyPrint as a dependency in `pyproject.toml` and `uv.lock`.
- Updated documentation to reflect the new PDF export capabilities and resolved issues from Sprint 3.
- Implemented tests for PDF export functionality, ensuring valid PDF output and proper handling of meeting data.
2025-12-25 23:39:56 -05:00
d285da3d36 Enhance mypy configuration and update roadmap documentation
- Updated `pyproject.toml` to include additional mypy exclusions and plugins for improved type checking.
- Added mypy overrides for specific modules to handle complex type parameters and untyped calls.
- Revised roadmap documentation to reflect changes in the Diarization Application Service, renaming it to "Diarization Enhancements" and updating feature descriptions for clarity.
- Adjusted architectural decisions to clarify the application service layer requirements for upcoming sprints.
2025-12-25 23:21:07 -05:00
58ad140f48 Complete Sprint 0: Proto & Schema Foundation
Sprint 0 consolidates all proto schema changes and database migrations
for Sprints 1-6 into a coordinated release.

Infrastructure:
- Add named_entities table migration with updated_at trigger
- Add webhook_configs and webhook_deliveries migrations
- Add httpx dependency for webhooks
- Multi-stage Docker build with NER support (with-ner target)
- Enable depends_on for db service in compose.yaml

Proto schema:
- Add SummarizationOptions message (Sprint 1)
- Add EXPORT_FORMAT_PDF enum value (Sprint 3)
- Add ExtractEntities RPC and messages (Sprint 4)
- Add Calendar RPCs and messages (Sprint 5)
- Proto regeneration script at scripts/regenerate_proto.sh

Tests:
- Add proto compilation tests verifying Sprint 0 messages
- Add feature flag tests with env var parsing
- Add migration structure and trigger tests

Documentation:
- Fix PROTO_CHANGELOG.md CalendarProvider field names
- Update COMPLETION_PLAN.md status to 100%

Client submodule updated with Sprint 0 TypeScript types.
2025-12-26 00:35:14 +00:00
ee12bcd89e Sprint 0: Proto & Schema Foundation
Consolidate proto/DB changes for Sprints 1-6 into coordinated release:

Proto Schema (Increment 0a):
- Add SummarizationOptions message with tone/format/verbosity fields
- Add EXPORT_FORMAT_PDF = 3 to ExportFormat enum
- Add NER messages: ExtractEntitiesRequest/Response, ExtractedEntity
- Add Calendar messages: CalendarEvent, ListCalendarEvents, CalendarProvider
- Add RPCs: ExtractEntities, ListCalendarEvents, GetCalendarProviders,
  InitiateCalendarAuth, CompleteCalendarAuth
- Regenerate Python proto stubs

Database Schema (Increment 0b):
- Add named_entities table for NER storage
- Add webhook_configs and webhook_deliveries tables

Migrations & ORM (Increment 0c):
- Create migration h2c3d4e5f6g7 for named_entities
- Create migration i3d4e5f6g7h8 for webhook tables
- Add NamedEntityModel, WebhookConfigModel, WebhookDeliveryModel
- Update MeetingModel and WorkspaceModel relationships

Dependencies & Config (Increment 0d):
- Add optional deps: pdf (weasyprint), ner (spacy), calendar (google-api)
- Add FeatureFlags class with NOTEFLOW_FEATURE_ env prefix
- Add models CLI for ML model downloads (spacy)
- Update CLI dispatcher for retention/models subcommands
- Create PROTO_CHANGELOG.md

Quality Fixes:
- Add ORM model names to ALLOWED_STRINGS in test_magic_values.py
- Add assertion messages to fix assertion roulette tests

Updates client submodule to c1783f8.
2025-12-25 14:48:37 -05:00
5df60507ea Refactor gRPC client architecture and add code quality testing
Backend:
- Extract gRPC client into modular mixins (_client_mixins/)
- Add StreamingSession class for audio streaming lifecycle
- Add gRPC config and type modules
- Fix test smells across test suite

Frontend (submodule update):
- Fix code quality issues and eliminate lint warnings
- Centralize CSS class constants
- Extract Settings.tsx sections into components
- Add code quality test suite

Quality:
- Add tests/quality/ suite for code smell detection
- Add QA report and correction plan documentation
2025-12-24 17:41:01 +00:00
4567d5a03c Update project documentation and configuration files
- Expanded `.gitignore` to include temporary files and scratch documents.
- Enhanced `AGENTS.md` with guidelines on using Conventional Commits and PR descriptions.
- Added configuration notes to `README.md` for environment variable usage.
- Updated `pyproject.toml` to reflect changes in dependencies and comments.
- Introduced new files for client configuration and testing, including Prettier settings and Playwright tests.
- Removed deprecated client components and adjusted related documentation accordingly.
- Added new tests for various components and functionalities to improve coverage.
2025-12-22 02:25:38 +00:00
cc9a7df54d Update Docker configuration, enhance dependencies, and improve migration handling
- Added environment file support and a command for the server in `compose.yaml`.
- Updated the database image in `compose.yaml` to use `pgvector/pgvector:pg15`.
- Modified `pyproject.toml` to include additional dependencies for enhanced functionality.
- Updated `uv.lock` to reflect new package versions and added new packages for improved features.
- Enhanced migration scripts to ensure the `noteflow` schema exists before running migrations.
- Improved error handling in the `OllamaSummarizer` to include a new exception type.
- Refactored the `calendar` trigger to streamline imports and improve code clarity.
- Added module-level mocks in test configurations to handle missing dependencies gracefully.
2025-12-20 16:49:47 -05:00
7f82e5f944 Update Docker configuration and enhance project dependencies
- Updated .dockerignore to include additional cache and environment files, improving build efficiency.
- Changed the Dockerfile reference in compose.yaml to point to the new server-specific Dockerfile.
- Added `diart` as a dependency in pyproject.toml and uv.lock to support enhanced diarization features.
- Created a new server.Dockerfile for a streamlined development environment setup with necessary dependencies.
2025-12-20 16:01:30 -05:00
b333ea5b23 Add initial Docker and development environment setup
- Created .dockerignore to exclude unnecessary files from Docker builds.
- Added .repomixignore for managing ignored patterns in Repomix.
- Introduced Dockerfile.dev for development environment setup with Python 3.12.
- Configured docker-compose.yaml to define services, including a PostgreSQL database.
- Established a devcontainer.json for Visual Studio Code integration.
- Implemented postCreate.sh for automatic dependency installation in the dev container.
- Added constants.py to centralize configuration constants for the project.
- Updated pyproject.toml to include new development dependencies.
- Created initial documentation files for project overview and style conventions.
- Added tests for new functionalities to ensure reliability and correctness.
2025-12-19 05:02:16 +00:00
b9297d9160 Add speaker diarization support and related configurations
- Introduced speaker diarization functionality using pyannote.audio and diart for both streaming and offline processing.
- Updated `pyproject.toml` to include new dependencies for diarization.
- Enhanced `TranscriptComponent` to handle speaker identification and color coding for speakers.
- Added new settings in `Settings` class for enabling and configuring diarization features.
- Updated `Segment` and gRPC models to include speaker ID and confidence metrics.
- Implemented a `DiarizationEngine` for managing diarization processes and speaker assignments.
- Created necessary database migrations to accommodate new speaker fields in segments.
- Added tests for audio reading and diarization functionalities to ensure reliability and correctness.
- Updated documentation to reflect new features and usage instructions for diarization.
2025-12-18 18:51:09 +00:00
4eef1b3be6 Add summarization and trigger services
- Introduced `SummarizationService` and `TriggerService` to orchestrate summarization and trigger detection functionalities.
- Added new modules for summarization, including citation verification and cloud-based summarization providers.
- Implemented trigger detection based on audio activity and foreground application status.
- Updated project configuration to include new dependencies for summarization and trigger functionalities.
- Created tests for summarization and trigger services to ensure functionality and reliability.
2025-12-18 00:08:51 +00:00
af1285b181 Add initial project structure and files
- Introduced .python-version for Python version management.
- Added AGENTS.md for documentation on agent usage and best practices.
- Created alembic.ini for database migration configurations.
- Implemented main.py as the entry point for the application.
- Established pyproject.toml for project dependencies and configurations.
- Initialized README.md for project overview.
- Generated uv.lock for dependency locking.
- Documented milestones and specifications in docs/milestones.md and docs/spec.md.
- Created logs/status_line.json for logging status information.
- Added initial spike implementations for UI tray hotkeys, audio capture, ASR latency, and encryption validation.
- Set up NoteFlow core structure in src/noteflow with necessary modules and services.
- Developed test suite in tests directory for application, domain, infrastructure, and integration testing.
- Included initial migration scripts in infrastructure/persistence/migrations for database setup.
- Established security protocols in infrastructure/security for key management and encryption.
- Implemented audio infrastructure for capturing and processing audio data.
- Created converters for ASR and ORM in infrastructure/converters.
- Added export functionality for different formats in infrastructure/export.
- Ensured all new files are included in the repository for future development.
2025-12-17 18:28:59 +00:00