# Deduplication Initiative

## Overview

This initiative addresses systematic code duplication patterns across the NoteFlow codebase. The goal is to reduce the repeated code patterns count well below the current threshold of 177, improving maintainability and reducing technical debt.

## Current State Analysis

| Metric | Current Value | Target |
|--------|--------------|--------|
| Repeated Patterns (quality test threshold) | 177 | <140 |
| gRPC abort patterns | 90+ | <30 |
| Proto converter duplicates | 55+ | <40 |
| Repository CRUD duplicates | 40+ | <20 |
| Inline constant imports | 84 | <30 |

### Pattern Frequency Summary

Based on comprehensive codebase analysis:

| Pattern Category | Occurrences | Primary Locations |
|------------------|-------------|-------------------|
| `abort_invalid_argument` | 56 | project/_mixin.py (14), oidc.py (10), webhooks.py (6) |
| `supports_check` (uow.supports_*) | 47 | project_service/crud.py (10), project/_mixin.py (8) |
| `abort_not_found` | 29 | project/_mixin.py (6), entities.py (5), oidc.py (4) |
| `async with uow` pattern | 25 | project/_mixin.py (10), sync.py (6), webhooks.py (5) |
| `request.HasField` checks | 25 | oidc.py (9), project/_mixin.py (7), webhooks.py (6) |
| `select_where_id` | 25 | Across 8 repository files |
| Proto converters | 55 | Multiple mixin and converter files |
| `from_constants_import` | 84 | 58 files (inline imports) |

## Sprint Roadmap

### Phase 1: High Impact (Sprints 1-2)

| Sprint | Focus | Est. Pattern Reduction | Effort |
|--------|-------|----------------------|--------|
| [Sprint 01](./sprint_01_grpc_mixin_helpers.md) | gRPC Mixin Base Helpers | 80-120 patterns | L |
| [Sprint 02](./sprint_02_proto_converters.md) | Proto Converter Consolidation | 20-30 patterns | M |

### Phase 2: Medium Impact (Sprint 3)

| Sprint | Focus | Est. Pattern Reduction | Effort |
|--------|-------|----------------------|--------|
| [Sprint 03](./sprint_03_repository_patterns.md) | Repository Base Patterns | 30-50 patterns | L |

### Phase 3: Cleanup (Sprint 4)

| Sprint | Focus | Est. Pattern Reduction | Effort |
|--------|-------|----------------------|--------|
| [Sprint 04](./sprint_04_constant_imports.md) | Constant Import Cleanup | 30-40 patterns | S |

## Success Metrics

### Primary Goals

1. **Quality Test Threshold**: Reduce from 177 to <140 (20%+ reduction)
2. **Stretch Goal**: Reduce to <120 (30%+ reduction)
3. **No Regressions**: All existing tests must pass
4. **Type Safety**: Pyrefly and basedpyright must pass with 0 errors

### Verification Commands

```bash
# Run quality tests
pytest tests/quality/test_duplicate_code.py -v

# Run type checking
basedpyright

# Run all tests
pytest
```

## Dependencies

- Each sprint can be executed independently
- Sprint 1 provides foundation patterns that Sprint 2-4 can leverage
- No external dependencies required

## Risk Assessment

| Risk | Mitigation |
|------|------------|
| Breaking existing functionality | Comprehensive test suite coverage |
| Type errors from abstraction | Gradual migration with type verification |
| Performance regression | No dynamic imports, no runtime overhead |
| Merge conflicts with active development | Small, focused PRs |

## Related Documentation

- [Quality Standards](../../QUALITY_STANDARDS.md)
- [Test Quality Suite](../../sprint_quality_suite_hardening.md)
