noteflow/docs/sprints/phase-5-evolution/sprint-25-langgraph/sprint-26-meeting-qa/README.md

Sprint 26: Meeting Q&A MVP

Size: L | Owner: Backend + Client | Phase: 5 - Platform Evolution Effort: ~1 sprint | Prerequisites: Sprint 25 (Foundation)

---

Objective

Implement single-meeting Q&A with segment citations via gRPC API and React UI.

---

Current State (After Sprint 25)

Component	Status
LangGraph infrastructure	✅ Ready
State schemas	✅ Ready
Retrieval tools	✅ Ready
Synthesis tools	✅ Ready
AssistantService shell	✅ Ready

---

Implementation Tasks

Task 1: Define MeetingQA Graph

File: src/noteflow/infrastructure/ai/graphs/meeting_qa.py

Graph flow: retrieve → verify → synthesize

from langgraph.graph import StateGraph, START, END

class MeetingQAState(TypedDict):
    # Input
    question: str
    meeting_id: UUID
    top_k: int
    
    # Internal
    retrieved_segments: list[RetrievalResult]
    verification_passed: bool
    
    # Output
    answer: str
    citations: list[SegmentCitation]


def build_meeting_qa_graph(
    embedder: EmbedderProtocol,
    segment_repo: SegmentSearchProtocol,
    llm: LLMProtocol,
    verifier: CitationVerifier,
) -> StateGraph:
    
    async def retrieve_node(state: MeetingQAState) -> dict:
        results = await retrieve_segments(
            query=state["question"],
            embedder=embedder,
            segment_repo=segment_repo,
            meeting_id=state["meeting_id"],
            top_k=state["top_k"],
        )
        return {"retrieved_segments": results}
    
    async def verify_node(state: MeetingQAState) -> dict:
        # Verify segments exist and are relevant
        valid = len(state["retrieved_segments"]) > 0
        return {"verification_passed": valid}
    
    async def synthesize_node(state: MeetingQAState) -> dict:
        if not state["verification_passed"]:
            return {
                "answer": "I couldn't find relevant information in this meeting.",
                "citations": [],
            }
        
        result = await synthesize_answer(
            question=state["question"],
            segments=state["retrieved_segments"],
            llm=llm,
        )
        
        citations = [
            SegmentCitation(
                meeting_id=state["meeting_id"],
                segment_id=seg.segment_id,
                start_time=seg.start_time,
                end_time=seg.end_time,
                text=seg.text,
                score=seg.score,
            )
            for seg in state["retrieved_segments"]
            if seg.segment_id in result.cited_segment_ids
        ]
        
        return {"answer": result.answer, "citations": citations}
    
    builder = StateGraph(MeetingQAState)
    builder.add_node("retrieve", retrieve_node)
    builder.add_node("verify", verify_node)
    builder.add_node("synthesize", synthesize_node)
    
    builder.add_edge(START, "retrieve")
    builder.add_edge("retrieve", "verify")
    builder.add_edge("verify", "synthesize")
    builder.add_edge("synthesize", END)
    
    return builder.compile()

---

Task 2: Create Citation Verifier Node

File: src/noteflow/infrastructure/ai/nodes/verification.py

from dataclasses import dataclass


@dataclass
class VerificationResult:
    is_valid: bool
    invalid_citation_indices: list[int]
    reason: str | None = None


def verify_citations(
    answer: str,
    cited_ids: list[int],
    available_ids: set[int],
) -> VerificationResult:
    """Verify all cited segment IDs exist in available segments."""
    invalid = [i for i, cid in enumerate(cited_ids) if cid not in available_ids]
    return VerificationResult(
        is_valid=len(invalid) == 0,
        invalid_citation_indices=invalid,
        reason=f"Invalid citations: {invalid}" if invalid else None,
    )

---

Task 3: Add Proto Messages

File: src/noteflow/grpc/proto/noteflow.proto

// Add to existing proto

message SegmentCitation {
  string meeting_id = 1;
  int32 segment_id = 2;
  float start_time = 3;
  float end_time = 4;
  string text = 5;
  float score = 6;
}

message AskAssistantRequest {
  string question = 1;
  optional string meeting_id = 2;
  optional string thread_id = 3;
  bool allow_web = 4;
  int32 top_k = 5;
}

message AskAssistantResponse {
  string answer = 1;
  repeated SegmentCitation citations = 2;
  repeated SuggestedAnnotation suggested_annotations = 3;
  string thread_id = 4;
}

message SuggestedAnnotation {
  string text = 1;
  AnnotationType type = 2;
  repeated int32 segment_ids = 3;
}

// Add to NoteFlowService
rpc AskAssistant(AskAssistantRequest) returns (AskAssistantResponse);

After modifying proto:

python -m grpc_tools.protoc -I src/noteflow/grpc/proto \
  --python_out=src/noteflow/grpc/proto \
  --grpc_python_out=src/noteflow/grpc/proto \
  src/noteflow/grpc/proto/noteflow.proto

python scripts/patch_grpc_stubs.py

---

Task 4: Add gRPC Mixin

File: src/noteflow/grpc/_mixins/assistant.py

from __future__ import annotations

from typing import TYPE_CHECKING

from noteflow.grpc.proto import noteflow_pb2 as pb
from noteflow.grpc._mixins.protocols import ServicerHost

if TYPE_CHECKING:
    from grpc.aio import ServicerContext


class AssistantMixin:
    """gRPC mixin for AI assistant operations."""

    async def AskAssistant(
        self: ServicerHost,
        request: pb.AskAssistantRequest,
        context: ServicerContext,
    ) -> pb.AskAssistantResponse:
        from uuid import UUID
        
        meeting_id = UUID(request.meeting_id) if request.meeting_id else None
        op_context = await self.get_operation_context(context)
        
        result = await self.assistant_service.ask(
            question=request.question,
            user_id=op_context.user.id,
            meeting_id=meeting_id,
            thread_id=request.thread_id or None,
            allow_web=request.allow_web,
            top_k=request.top_k or 8,
        )
        
        return pb.AskAssistantResponse(
            answer=result["answer"],
            citations=[
                pb.SegmentCitation(
                    meeting_id=str(c["meeting_id"]),
                    segment_id=c["segment_id"],
                    start_time=c["start_time"],
                    end_time=c["end_time"],
                    text=c["text"],
                    score=c["score"],
                )
                for c in result["citations"]
            ],
            thread_id=result["thread_id"],
        )

---

Task 5: Add Rust Command

File: client/src-tauri/src/commands/assistant.rs

use crate::grpc::client::GrpcClient;
use crate::grpc::types::assistant::{AskAssistantRequest, AskAssistantResponse};
use tauri::State;

#[tauri::command]
pub async fn ask_assistant(
    client: State<'_, GrpcClient>,
    question: String,
    meeting_id: Option<String>,
    thread_id: Option<String>,
    allow_web: bool,
    top_k: i32,
) -> Result<AskAssistantResponse, String> {
    let request = AskAssistantRequest {
        question,
        meeting_id,
        thread_id,
        allow_web,
        top_k,
    };
    
    client
        .ask_assistant(request)
        .await
        .map_err(|e| e.to_string())
}

---

Task 6: Add TypeScript Adapter

File: client/src/api/tauri-adapter.ts (add method)

async askAssistant(params: AskAssistantParams): Promise<AskAssistantResponse> {
  return invoke('ask_assistant', {
    question: params.question,
    meetingId: params.meetingId,
    threadId: params.threadId,
    allowWeb: params.allowWeb ?? false,
    topK: params.topK ?? 8,
  });
}

File: client/src/api/types/assistant.ts

export interface SegmentCitation {
  meetingId: string;
  segmentId: number;
  startTime: number;
  endTime: number;
  text: string;
  score: number;
}

export interface AskAssistantParams {
  question: string;
  meetingId?: string;
  threadId?: string;
  allowWeb?: boolean;
  topK?: number;
}

export interface AskAssistantResponse {
  answer: string;
  citations: SegmentCitation[];
  suggestedAnnotations: SuggestedAnnotation[];
  threadId: string;
}

export interface SuggestedAnnotation {
  text: string;
  type: AnnotationType;
  segmentIds: number[];
}

---

Task 7: Create Ask UI Component

File: client/src/components/meeting/AskPanel.tsx

import { useState } from 'react';
import { useAssistant } from '@/hooks/use-assistant';
import { Button } from '@/components/ui/button';
import { Textarea } from '@/components/ui/textarea';
import { Card } from '@/components/ui/card';

interface AskPanelProps {
  meetingId: string;
  onCitationClick?: (segmentId: number) => void;
}

export function AskPanel({ meetingId, onCitationClick }: AskPanelProps) {
  const [question, setQuestion] = useState('');
  const { ask, isLoading, response, error } = useAssistant();

  const handleAsk = async () => {
    if (!question.trim()) return;
    await ask({ question, meetingId });
  };

  return (
    <div className="flex flex-col gap-4 p-4">
      <Textarea
        placeholder="Ask a question about this meeting..."
        value={question}
        onChange={(e) => setQuestion(e.target.value)}
        disabled={isLoading}
      />
      <Button onClick={handleAsk} disabled={isLoading || !question.trim()}>
        {isLoading ? 'Thinking...' : 'Ask'}
      </Button>
      
      {response && (
        <Card className="p-4">
          <p className="whitespace-pre-wrap">{response.answer}</p>
          {response.citations.length > 0 && (
            <div className="mt-4 border-t pt-2">
              <p className="text-sm text-muted-foreground mb-2">Sources:</p>
              {response.citations.map((citation) => (
                <button
                  key={citation.segmentId}
                  onClick={() => onCitationClick?.(citation.segmentId)}
                  className="text-sm text-blue-600 hover:underline block"
                >
                  [{citation.startTime.toFixed(1)}s] {citation.text.slice(0, 50)}...
                </button>
              ))}
            </div>
          )}
        </Card>
      )}
      
      {error && (
        <p className="text-sm text-red-600">{error}</p>
      )}
    </div>
  );
}

File: client/src/hooks/use-assistant.ts

import { useState, useCallback } from 'react';
import { api } from '@/api';
import type { AskAssistantParams, AskAssistantResponse } from '@/api/types/assistant';

export function useAssistant() {
  const [isLoading, setIsLoading] = useState(false);
  const [response, setResponse] = useState<AskAssistantResponse | null>(null);
  const [error, setError] = useState<string | null>(null);

  const ask = useCallback(async (params: AskAssistantParams) => {
    setIsLoading(true);
    setError(null);
    try {
      const result = await api.askAssistant(params);
      setResponse(result);
      return result;
    } catch (err) {
      setError(err instanceof Error ? err.message : 'Failed to get answer');
      throw err;
    } finally {
      setIsLoading(false);
    }
  }, []);

  const reset = useCallback(() => {
    setResponse(null);
    setError(null);
  }, []);

  return { ask, isLoading, response, error, reset };
}

---

Task 8: Implement AssistantService.ask()

Complete the implementation in application/services/assistant/assistant_service.py:

async def ask(
    self,
    question: str,
    user_id: UUID,
    meeting_id: UUID | None = None,
    thread_id: str | None = None,
    allow_web: bool = False,
    top_k: int = DEFAULT_TOP_K,
) -> AssistantOutputState:
    """Ask a question about meeting transcript(s)."""
    effective_thread_id = thread_id or build_thread_id(
        meeting_id, user_id, "meeting_qa"
    )
    
    async with self.uow_factory() as uow:
        # Build and run graph
        graph = build_meeting_qa_graph(
            embedder=self._embedder,
            segment_repo=uow.segments,
            llm=self._llm,
            verifier=self._verifier,
        )
        
        config = {"configurable": {"thread_id": effective_thread_id}}
        result = await graph.ainvoke(
            {
                "question": question,
                "meeting_id": meeting_id,
                "top_k": top_k,
            },
            config,
        )
        
        # Record usage
        self.usage_events.record_simple(
            "assistant.ask",
            meeting_id=str(meeting_id) if meeting_id else None,
            question_length=len(question),
            citation_count=len(result.get("citations", [])),
        )
        
        return AssistantOutputState(
            answer=result["answer"],
            citations=[asdict(c) for c in result["citations"]],
            suggested_annotations=[],
            thread_id=effective_thread_id,
        )

---

Acceptance Criteria

• Q&A returns answers with valid segment citations
• Citations link to correct timestamps in transcript
• Feature hidden when rag_enabled=false in project rules
• Thread ID persists conversation context
• make quality passes
• pytest tests/grpc/test_assistant.py passes
• UI component displays answer and clickable citations

---

Files Created/Modified

Action	Path
Created	src/noteflow/infrastructure/ai/graphs/meeting_qa.py
Created	src/noteflow/infrastructure/ai/nodes/verification.py
Modified	src/noteflow/grpc/proto/noteflow.proto
Created	src/noteflow/grpc/_mixins/assistant.py
Modified	src/noteflow/grpc/service.py (add mixin)
Created	client/src-tauri/src/commands/assistant.rs
Modified	client/src/api/tauri-adapter.ts
Created	client/src/api/types/assistant.ts
Created	client/src/components/meeting/AskPanel.tsx
Created	client/src/hooks/use-assistant.ts
Modified	src/noteflow/application/services/assistant/assistant_service.py