noteflow/docs/sprints/phase-ongoing/deduplication/sprint_02_proto_converters.md

Sprint 02: Proto Converter Consolidation

Field	Value
Sprint Size	M (Medium)
Owner	TBD
Prerequisites	None (can run parallel to Sprint 01)
Phase	Deduplication Phase 1
Est. Pattern Reduction	20-30 patterns

Open Issues

• Determine canonical location for proto converters (single file vs organized submodules)
• Verify duplicate functions have identical behavior before consolidation
• Decide if client-side converters should share code with server-side

Validation Status

Component	Status	Notes
Server converters	✅ Exists	grpc/_mixins/converters.py
Client converters	✅ Exists	grpc/_client_mixins/converters.py
Project converters	⚠️ Duplicate	grpc/_mixins/project/_converters.py
Entity converters	⚠️ Inline	In entities.py mixin
OIDC converters	⚠️ Inline	In oidc.py mixin
Webhook converters	⚠️ Inline	In webhooks.py mixin

Objective

Consolidate scattered proto converter functions into a centralized location, eliminating duplicates and establishing consistent conversion patterns. This reduces maintenance burden and prevents conversion inconsistencies.

Key Decisions

Decision	Rationale
Centralize in grpc/_mixins/converters.py	Already established as converter location
Keep client converters separate	Different direction (proto → domain vs domain → proto)
Use *_to_proto and proto_to_* naming	Clear bidirectional naming convention
Add type stubs for proto messages	Improve type safety

What Already Exists

Current Converter Locations

Location	Functions	Type
grpc/_mixins/converters.py	9 functions	Server (domain → proto)
grpc/_client_mixins/converters.py	4 functions	Client (proto → domain)
grpc/_mixins/project/_converters.py	11 functions	Project-specific
grpc/_mixins/entities.py	1 function (inline)	Entity-specific
grpc/_mixins/oidc.py	4 functions	OIDC-specific
grpc/_mixins/webhooks.py	2 functions	Webhook-specific
grpc/_mixins/sync.py	1 function	Sync-specific
grpc/_mixins/observability.py	1 function	Metrics-specific

Identified Duplicates

export_format_to_proto:
  - grpc/_mixins/project/_converters.py
  - grpc/_client_mixins/converters.py

proto_to_export_format:
  - grpc/_mixins/converters.py
  - grpc/_mixins/project/_converters.py

Scope

Task 1: Identify and Verify Duplicates (S)

Compare duplicate functions for behavioral equivalence:

# Compare export_format_to_proto implementations
diff <(grep -A 10 "def export_format_to_proto" src/noteflow/grpc/_mixins/project/_converters.py) \
     <(grep -A 10 "def export_format_to_proto" src/noteflow/grpc/_client_mixins/converters.py)

# Compare proto_to_export_format implementations
diff <(grep -A 10 "def proto_to_export_format" src/noteflow/grpc/_mixins/converters.py) \
     <(grep -A 10 "def proto_to_export_format" src/noteflow/grpc/_mixins/project/_converters.py)

Deliverable: Document any behavioral differences before consolidation.

Task 2: Consolidate Export Format Converters (S)

Move canonical implementations to converters.py:

# In grpc/_mixins/converters.py

def export_format_to_proto(fmt: ExportFormat) -> noteflow_pb2.ExportFormat:
    """Convert domain ExportFormat to proto enum."""
    match fmt:
        case ExportFormat.MARKDOWN:
            return noteflow_pb2.ExportFormat.EXPORT_FORMAT_MARKDOWN
        case ExportFormat.HTML:
            return noteflow_pb2.ExportFormat.EXPORT_FORMAT_HTML
        case ExportFormat.PDF:
            return noteflow_pb2.ExportFormat.EXPORT_FORMAT_PDF
        case _:
            return noteflow_pb2.ExportFormat.EXPORT_FORMAT_UNSPECIFIED

def proto_to_export_format(proto_fmt: noteflow_pb2.ExportFormat) -> ExportFormat:
    """Convert proto ExportFormat enum to domain."""
    match proto_fmt:
        case noteflow_pb2.ExportFormat.EXPORT_FORMAT_MARKDOWN:
            return ExportFormat.MARKDOWN
        case noteflow_pb2.ExportFormat.EXPORT_FORMAT_HTML:
            return ExportFormat.HTML
        case noteflow_pb2.ExportFormat.EXPORT_FORMAT_PDF:
            return ExportFormat.PDF
        case _:
            return ExportFormat.MARKDOWN  # Default

Files to update:

• grpc/_mixins/converters.py - Add/verify canonical implementations
• grpc/_mixins/project/_converters.py - Remove duplicates, import from converters
• grpc/_client_mixins/converters.py - Remove duplicates, import from converters

Task 3: Consolidate Inline Converters (M)

Move inline converter functions from mixins to converters.py:

From grpc/_mixins/entities.py:

def entity_to_proto(entity: NamedEntity) -> noteflow_pb2.ExtractedEntity:
    """Convert domain NamedEntity to proto."""
    return noteflow_pb2.ExtractedEntity(
        id=str(entity.id),
        text=entity.text,
        category=entity.category.value,
        segment_ids=list(entity.segment_ids),
        confidence=entity.confidence,
        is_pinned=entity.is_pinned,
    )

From grpc/_mixins/webhooks.py:

def webhook_config_to_proto(config: WebhookConfig) -> noteflow_pb2.WebhookConfigProto:
    """Convert domain WebhookConfig to proto."""
    ...

def webhook_delivery_to_proto(delivery: WebhookDelivery) -> noteflow_pb2.WebhookDeliveryProto:
    """Convert domain WebhookDelivery to proto."""
    ...

From grpc/_mixins/oidc.py:

def claim_mapping_to_proto(mapping: ClaimMapping) -> noteflow_pb2.ClaimMappingProto:
    ...

def proto_to_claim_mapping(proto: noteflow_pb2.ClaimMappingProto) -> ClaimMapping:
    ...

def discovery_to_proto(provider: OidcProviderConfig) -> noteflow_pb2.OidcDiscoveryProto | None:
    ...

def oidc_provider_to_proto(provider: OidcProviderConfig, warnings: list[str] | None = None) -> noteflow_pb2.OidcProviderProto:
    ...

Task 4: Create Converter Organization (S)

Organize converters.py into logical sections:

"""Proto ↔ Domain converters for gRPC service layer.

This module provides bidirectional conversion between domain entities
and protobuf messages for the gRPC API.

Sections:
- Core Entity Converters (Meeting, Segment, Summary, Annotation)
- Project Converters (Project, Membership, Settings)
- Webhook Converters
- OIDC Converters
- Entity Extraction Converters
- Utility Converters (Timestamps, Enums)
"""

# =============================================================================
# Core Entity Converters
# =============================================================================

def word_to_proto(word: WordTiming) -> noteflow_pb2.WordTiming:
    ...

def meeting_to_proto(meeting: Meeting, ...) -> noteflow_pb2.Meeting:
    ...

# =============================================================================
# Project Converters
# =============================================================================

def project_to_proto(project: Project) -> noteflow_pb2.ProjectProto:
    ...

# ... etc

Task 5: Update All Import Sites (M)

Update all files importing from removed duplicate locations:

Original Import	New Import
from ..project._converters import export_format_to_proto	from ..converters import export_format_to_proto
from .entities import entity_to_proto	from .converters import entity_to_proto
from .oidc import _provider_to_proto	from .converters import oidc_provider_to_proto

Deliverables

Code Changes

• Consolidate export_format_to_proto into single canonical location
• Consolidate proto_to_export_format into single canonical location
• Move entity_to_proto from entities.py to converters.py
• Move webhook converters from webhooks.py to converters.py
• Move OIDC converters from oidc.py to converters.py
• Move sync converters from sync.py to converters.py
• Update all import statements across mixin files
• Remove leading underscore from public converter functions

Tests

• Add/update converter tests in tests/grpc/test_converters.py
• Verify round-trip conversion (domain → proto → domain)
• Test edge cases (None values, empty collections)

Test Strategy

Converter Correctness Tests

import pytest
from noteflow.grpc._mixins.converters import (
    export_format_to_proto,
    proto_to_export_format,
    entity_to_proto,
    webhook_config_to_proto,
)

class TestExportFormatConverters:
    @pytest.mark.parametrize("domain_fmt,proto_fmt", [
        (ExportFormat.MARKDOWN, noteflow_pb2.ExportFormat.EXPORT_FORMAT_MARKDOWN),
        (ExportFormat.HTML, noteflow_pb2.ExportFormat.EXPORT_FORMAT_HTML),
        (ExportFormat.PDF, noteflow_pb2.ExportFormat.EXPORT_FORMAT_PDF),
    ])
    def test_roundtrip(self, domain_fmt, proto_fmt):
        proto = export_format_to_proto(domain_fmt)
        assert proto == proto_fmt
        domain = proto_to_export_format(proto)
        assert domain == domain_fmt

class TestEntityConverter:
    def test_entity_to_proto_complete(self):
        entity = NamedEntity(
            id=UUID(int=1),
            text="Anthropic",
            category=EntityCategory.COMPANY,
            segment_ids=frozenset([1, 2, 3]),
            confidence=0.95,
            is_pinned=True,
        )
        proto = entity_to_proto(entity)
        assert proto.text == "Anthropic"
        assert proto.category == "company"
        assert proto.confidence == 0.95
        assert proto.is_pinned is True

Quality Gates

Pre-Implementation

• Document all existing converter locations
• Verify duplicate behavior is identical
• Run baseline quality test

Post-Implementation

• No duplicate converter functions remain
• All imports resolve correctly
• Quality test pattern count reduced by 10+
• All existing tests pass
• Type checking passes

Verification Commands

# Check for remaining duplicates
grep -r "def \w*_to_proto" src/noteflow/grpc/_mixins/ | grep -v converters.py | wc -l
# Should be 0

# Check for remaining proto_to_* duplicates
grep -r "def proto_to_" src/noteflow/grpc/_mixins/ | grep -v converters.py | wc -l
# Should be 0

# Run tests
pytest tests/grpc/test_converters.py -v
pytest tests/grpc/ -v

Migration Notes

Before/After Import Examples

Before (entities.py):

def entity_to_proto(entity: NamedEntity) -> noteflow_pb2.ExtractedEntity:
    ...

class EntitiesMixin:
    async def ExtractEntities(...):
        ...
        return noteflow_pb2.ExtractEntitiesResponse(
            entities=[entity_to_proto(e) for e in result.entities],
            ...
        )

After (entities.py):

from .converters import entity_to_proto

class EntitiesMixin:
    async def ExtractEntities(...):
        ...
        return noteflow_pb2.ExtractEntitiesResponse(
            entities=[entity_to_proto(e) for e in result.entities],
            ...
        )

Rollback Plan

1. Converter functions are pure functions with no side effects
2. Reverting imports to original locations is straightforward
3. Git revert restores original file structure
4. No database or state changes required

References

• src/noteflow/grpc/_mixins/converters.py - Primary converter location
• src/noteflow/grpc/_client_mixins/converters.py - Client converter location
• src/noteflow/grpc/_mixins/project/_converters.py - Project converters (to consolidate)