noteflow/.serena/memories/client_processing_pipeline.md

NoteFlow Client Processing Pipeline After Recording Stops

Executive Summary

The client-side processing pipeline is NOT fully automated. After recording stops (stopRecording), the system:

1. Stops the audio stream and saves audio to disk ✓
2. Calls stopMeeting() on the gRPC server ✓
3. THEN NAVIGATES AWAY to the MeetingDetail page
4. BUT does NOT automatically trigger summarization or entity extraction

Call Flow After Recording Stops

Recording.tsx: stopRecording() [Lines 313-346]

const stopRecording = async () => {
  setRecordingState('stopping');
  try {
    streamRef.current?.close();  // Closes TauriTranscriptionStream
    const stoppedMeeting = await api.stopMeeting(meeting.id);  // Calls server
    setMeeting(stoppedMeeting);
    
    // IMMEDIATELY NAVIGATE AWAY - no processing triggered
    navigate(projectId ? `/projects/${projectId}/meetings/${meeting.id}` : '/projects');
  }
}

What stopMeeting() Does

Tauri Adapter (tauri-adapter.ts, line 483-489):

async stopMeeting(meetingId: string): Promise<Meeting> {
  const meeting = await invoke<Meeting>(TauriCommands.STOP_MEETING, {
    meeting_id: meetingId,
  });
  meetingCache.cacheMeeting(meeting);
  return meeting;
}

Rust Command (commands/meeting.rs, lines 80-83):

pub async fn stop_meeting(state: State<'_, Arc<AppState>>, meeting_id: String) -> Result<Meeting> {
    state.grpc_client.stop_meeting(&meeting_id).await
}

This is a pure state change - it marks the meeting as completed on the server, but does NOT:

• Generate a summary
• Extract entities
• Refine speakers (diarization)

---

What Happens After Navigation to MeetingDetail

MeetingDetail.tsx: Initial Load [Lines 79-98]

useEffect(() => {
  const loadMeeting = async () => {
    const data = await getAPI().getMeeting({
      meeting_id: id,
      include_segments: true,
      include_summary: true,  // Fetches existing summary if present
    });
    setMeeting(data);
  };
}, [id]);

This only fetches the meeting state - it does NOT trigger any processing.

---

Manual Triggering Points

1. Summarization (MeetingDetail.tsx, Lines 199-210)

Trigger: User clicks "Summary" button or calls handleGenerateSummary()

const handleGenerateSummary = async () => {
  if (!meeting) return;
  const summary = await getAPI().generateSummary(meeting.id, true);
  setMeeting({ ...meeting, summary });
};

Tauri Adapter (Lines 507-526):

async generateSummary(meetingId: string, forceRegenerate?: boolean): Promise<Summary> {
  // Get user preferences for tone/format/verbosity
  const prefs = await invoke<UserPreferences>(TauriCommands.GET_PREFERENCES);
  const options = prefs?.ai_template ? { tone, format, verbosity } : undefined;
  
  return invoke<Summary>(TauriCommands.GENERATE_SUMMARY, {
    meeting_id: meetingId,
    force_regenerate: forceRegenerate ?? false,
    options,
  });
}

Rust Command (commands/summary.rs, Lines 89-134):

• Emits progress events ("started" -> "running" -> "completed"/"failed")
• Polls with exponential backoff (1s interval, max 90% progress)
• Returns Summary on completion

2. Entity Extraction (MeetingDetail.tsx, Lines 357-366)

Trigger: User clicks "Entities" button

const extractEntities = (forceRefresh = false) => {
  getAPI().extractEntities(meetingId, forceRefresh)
    .then(response => setEntitiesFromExtraction(response.entities))
    .catch(error => toast error)
};

Tauri Adapter (Lines 687-695):

async extractEntities(meetingId: string, forceRefresh?: boolean): Promise<ExtractEntitiesResponse> {
  return invoke<ExtractEntitiesResponse>(TauriCommands.EXTRACT_ENTITIES, {
    meeting_id: meetingId,
    force_refresh: forceRefresh ?? false,
  });
}

Rust Command (commands/entities.rs, Lines 15-24):

• Direct pass-through to gRPC client
• Returns result immediately (no progress polling)
• No event emission

3. Diarization (use-diarization.ts Hook)

Note: The hook provides a start() method for refinement, but it is NOT automatically called when recording stops.

Hook capabilities:

• start(meetingId, numSpeakers) - Request diarization
• poll() - Poll job status with exponential backoff
• recover() - Recover active jobs after reconnection (Sprint GAP-004)

Not found: Auto-trigger on meeting completion

---

Entity Extraction Auto-Extraction Feature

Hook feature (use-entity-extraction.ts, Lines 116-121):

useEffect(() => {
  if (autoExtract && meetingId && meetingState === 'completed') {
    extract(false);  // Auto-extract on mount when meeting is completed
  }
}, [autoExtract, meetingId, meetingState, extract]);

Problem: This feature exists in the hook, but is NOT USED anywhere in the codebase.

In MeetingDetail.tsx (Lines 72-76):

const { extract: extractEntities } = useEntityExtraction({
  meetingId: id,
  meetingTitle: meeting?.title,
  meetingState: meeting?.state,
  // autoExtract is NOT set, defaults to false
});

---

Backend Processing Pipeline (For Comparison)

The backend automatically triggers some processing in the StopMeeting RPC (server.py gRPC mixin):

1. Always triggered:

   • Mark meeting as completed
   • Flush any pending segments
   • Close streaming connections

2. Conditionally triggered (needs investigation):

   • Diarization job submission?
   • Entity extraction?
   • Summary generation?

Unknown: Whether the backend auto-starts these or waits for explicit client requests.

---

Processing Pipeline Gaps

GAP-001: No Auto-Summarization After Recording Stops

Current behavior: User must manually click "Summary" button on MeetingDetail page Expected behavior (?): Auto-generate summary when meeting enters completed state Impact: User might forget to generate summary; requires multiple interactions

GAP-002: No Auto-Entity Extraction Despite Hook Support

Current behavior: Entity extraction requires manual button click Expected behavior (?): Could auto-extract when meeting is completed (hook supports it) Impact: Entity extraction is "hidden" feature; users may not know it exists

GAP-003: No Auto-Diarization Refinement

Current behavior: User must manually initiate diarization via Tauri UI (not found in React UI) Expected behavior (?): Could auto-refine speakers after recording Impact: Speaker identification may not be optimal without refinement

GAP-004: No Polling for Processing Status

Current behavior: Client fires requests but does NOT poll for completion Expected behavior (?): Show progress while processing happens server-side Impact: User sees no feedback while server processes transcript

---

Server-Side API Endpoints (From tauri-adapter.ts)

These are available but not automatically triggered:

RPC	Purpose	Auto-triggered?
startTranscription()	Begin audio streaming	During recording start
stopMeeting()	Mark meeting completed	On stop button
generateSummary()	Create AI summary	Manual only
extractEntities()	NER extraction	Manual only
refineSpeakers()	Start diarization job	Manual only
getDiarizationJobStatus()	Poll job progress	Manual poll

---

Recommendation: Automatic Processing Pipeline

To close these gaps, consider:

1. On Recording Stop:

   • After stopMeeting() succeeds, auto-trigger:
     • generateSummary() with user preferences
     • extractEntities() if NER is enabled
     • refineSpeakers() if diarization is enabled

2. Polling for Progress:

   • Show progress indicators while server processes
   • Use same polling strategy as diarization hook (exponential backoff)

3. UI Flow:

   • Show "Processing..." state instead of immediately navigating
   • Display progress: "Generating summary... 45%"
   • Auto-navigate when all processing completes

4. Error Handling:

   • Toast notification if processing fails
   • Allow manual retry from MeetingDetail page

---

Key Files for Implementation

File	Purpose	Lines
client/src/pages/Recording.tsx	Recording page with stop button	313-346
client/src/api/tauri-adapter.ts	API adapter with processing methods	507-695
client/src/pages/MeetingDetail.tsx	Detail page with manual triggers	199-210, 357-366
client/src/hooks/use-entity-extraction.ts	Entity extraction hook with auto-extract	116-121
client/src/hooks/use-diarization.ts	Diarization with polling (reusable pattern)	124-253
client/src-tauri/src/commands/summary.rs	Progress event emission	14-40, 96-134
client/src-tauri/src/commands/recording/mod.rs	Recording lifecycle	313-346

---

Current State Summary

What works:

• Recording and transcription streaming ✓
• Manual summarization ✓
• Manual entity extraction ✓
• Manual diarization refinement ✓

What's missing:

• Auto-trigger processing pipeline ✗
• Progress feedback during processing ✗
• Coordinated processing after recording stops ✗
• Automatic entity extraction despite hook support ✗