guide/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

A FastAPI-based guided demo platform that automates browser interactions with Raindrop using Playwright. The app executes data-driven actions (stored in ActionRegistry) on behalf of personas that target configured browser hosts (CDP or headless). All configuration is externalized via YAML files and environment overrides.

Entry Point: python -m guide (runs src/guide/main.py → guide.app.main:app) Python Version: 3.12+ Key Dependencies: FastAPI, Playwright, Pydantic v2, PyYAML, httpx

Essential Commands

# Install dependencies
uv sync

# Type checking (required before commits)
basedpyright src

# Compile sanity check
python -m compileall src/guide

# Run development server (default: localhost:8000)
python -m guide
# or with custom host/port:
HOST=127.0.0.1 PORT=9000 python -m guide

# View API docs
# Navigate to http://localhost:8000/docs

# Key endpoints:
# GET  /healthz                           # liveness check
# GET  /actions                           # list action metadata
# POST /actions/{id}/execute              # execute action; returns ActionEnvelope with correlation_id
# GET  /config/browser-hosts              # view current default + host map

Code Structure

Root module: src/guide/app/

• actions/ — Demo action implementations with auto-discovery via @register_action decorator. Auto-wired via ActionRegistry with dependency injection. Submodules: base.py (DemoAction, CompositeAction, ActionRegistry), registry.py (auto-discovery), playbooks.py (multi-step workflows like OnboardingFlow, FullDemoFlow), auth/ (LoginAsPersonaAction), intake/ (CreateIntakeAction), sourcing/ (AddSupplierAction).
• auth/ — Pluggable MFA/auth helpers. mfa.py defines MfaCodeProvider interface; DummyMfaCodeProvider raises NotImplementedError (implement for production). session.py provides ensure_persona() for login flows.
• browser/ — BrowserPool (persistent browser instances per host) + BrowserClient (context-managed page access). Handles both CDP attach and headless launch. pool.py manages lifecycle; client.py wraps for async context manager pattern. diagnostics.py captures screenshots, HTML, console logs for error debugging.
• core/ — App bootstrap: config.py (AppSettings with Pydantic v2, env prefix RAINDROP_DEMO_, YAML + JSON override cascade), logging.py (structured logging with request-scoped context variables).
• errors/ — GuideError hierarchy (ConfigError, BrowserConnectionError, PersonaError, AuthError, MfaError, ActionExecutionError, GraphQLTransportError, GraphQLOperationError); routers normalize to HTTP responses with debug info.
• raindrop/ — GraphQL client + operations. graphql.py (httpx-based HTTP client), operations/ (intake.py, sourcing.py with query/mutation definitions), generated/ (ariadne-codegen auto-generated Pydantic models), queries/ (GraphQL query/mutation files).
• strings/ — Flattened registry (Phase 1 refactoring complete). registry.py enforces domain-keyed lookups; missing keys raise immediately. No inline UI strings in actions. Submodules: selectors/ (CSS, data-testid, aria), labels/ (UI element names), demo_texts/ (copy snippets).
• models/ — Domain and persona models using Pydantic v2. domain/models.py (ActionRequest, ActionContext, ActionResult, ActionMetadata, ActionEnvelope, DebugInfo, BrowserHostDTO, BrowserHostsResponse). personas/models.py (DemoPersona with auto-coerced enums PersonaRole, LoginMethod; unified from Phase 2A refactoring). personas/store.py (PersonaStore in-memory registry from AppSettings).
• utils/ — Shared helpers. Keep <300 LoC per file; avoid circular imports. Modules: ids.py (UUID/correlation ID), env.py (environment utilities), retry.py (backoff + jitter), timing.py (rate-limiting).
• api/ — FastAPI routers in routes/. health.py (GET /healthz), actions.py (GET /actions, POST /actions/{id}/execute), config.py (GET /config/browser-hosts). Map requests → ActionRegistry → BrowserClient → ActionEnvelope responses with error capture.

Config files (git-tracked):

• config/hosts.yaml — Browser host targets (id, kind: cdp|headless, host, port, browser type).
• config/personas.yaml — Personas (id, role, email, login_method, browser_host_id).

Config overrides (runtime only, never commit):

• RAINDROP_DEMO_BROWSER_HOSTS_JSON — JSON array overrides hosts.yaml.
• RAINDROP_DEMO_PERSONAS_JSON — JSON array overrides personas.yaml.
• RAINDROP_DEMO_RAINDROP_BASE_URL — Override default https://app.raindrop.com.

Architecture Patterns

App Initialization (main.py → create_app)

1. Load AppSettings (env + YAML + JSON overrides).
2. Build PersonaStore from config.
3. Build ActionRegistry with auto-discovered actions (via @register_action decorator + pkgutil.walk_packages()). DI context includes persona store, Raindrop URL, etc. Auto-instantiates actions with parameter matching.
4. Create BrowserPool (manages persistent browser instances per host, allocates fresh contexts/pages per request).
5. Create BrowserClient (wraps BrowserPool with async context manager).
6. Stash instances on app.state for dependency injection in routers.
7. Register error handlers (GuideError → HTTP + debug info; unhandled → 500 + logging).

Action Execution Pipeline

• Request: POST /actions/{action_id}/execute with ActionRequest (persona_id, host_id, params).
• Router resolves persona + host from config → validates persona exists.
• Router generates correlation_id (UUID) and ActionContext (includes persona, host, params, correlation_id, shared_state dict for composite actions).
• ActionRegistry.get(action_id) retrieves action (auto-discovered, dependency-injected).
• BrowserClient.open_page(host_id) → allocates fresh context + page from BrowserPool. Reuses persistent browser instance for host; creates new page/context for isolation.
• Action.run(page, context) executes logic; may call ensure_persona() (login flow) before starting. For composite actions, passes shared_state dict to child actions.
• On error: captures debug info (screenshot, HTML, console logs) and returns with DebugInfo attached.
• Response: ActionEnvelope (status, correlation_id, result/error, debug_info).

Browser Host Resolution & Pooling

• BrowserPool Architecture:

  • One persistent browser instance per host (lazy-initialized on first request).
  • Fresh context + page allocated per request for isolation (no cross-request state).
  • Proper cleanup via async context manager pattern.
  • Handles timeouts, connection errors, context cleanup.

• Host Kind Resolution:

  • kind: cdp — connect to running Raindrop instance via Chrome DevTools Protocol (requires host + port). Errors surface as BrowserConnectionError.
  • kind: headless — launch Playwright browser (chromium/firefox/webkit); set browser field in config.
  • Always use async with BrowserClient.open_page(host_id) as page: to ensure proper cleanup (context manager unwinding in BrowserPool).

GraphQL & Data Layer

• raindrop/graphql.py — HTTP client (httpx, 10s timeout).
• raindrop/operations/ — query/mutation definitions + Pydantic-validated response models.
• raindrop/generated/ — Auto-generated Pydantic models (via ariadne-codegen) from Raindrop GraphQL schema. Type-safe; auto-synced with schema.
• raindrop/queries/ — GraphQL query/mutation .graphql files (single source of truth for operations).
• Validate all responses with Pydantic models; schema mismatches → GuideError.
• Never embed tokens/URLs; pull from AppSettings (env-driven).
• Transport errors → GraphQLTransportError; operation errors → GraphQLOperationError (includes details from server).

Selector & String Management (strings/)

• Flattened Registry (Phase 1 refactoring complete): All selectors, labels, copy in strings/ submodules; no 3-layer wrappers.
• Direct field access: app_strings.intake.field (was app_strings.intake.selectors.field). Registry enforces domain-keyed lookups; missing keys raise immediately.
• Organize into: selectors/ (CSS, data-testid, aria attributes), labels/ (UI element names), demo_texts/ (copy/text snippets).
• Selectors should be reusable and labeled; avoid brittle text selectors—prefer data-testid or aria labels.
• No inline UI strings in action code; all strings centralized in strings/ for easy maintenance and i18n readiness.

Development Workflow

1. Edit code (actions, browser logic, GraphQL ops, etc.).
2. Run type check: basedpyright src (catches generic types, missing annotations).
3. Sanity compile: python -m compileall src/guide (syntax check).
4. Smoke test: python -m guide then hit /docs or manual test via curl.
5. Review error handling: ensure GuideError subclasses are raised, not generic exceptions.
6. Commit with scoped, descriptive message (e.g., feat: add auth login action, chore: tighten typing).

Type & Linting Standards

• Python 3.12+: Use PEP 604 unions (str | None), built-in generics (list[str], dict[str, JSONValue]).
• Ban Any and # type: ignore: Use type guards or Protocol instead.
• Pydantic v2: Explicit types, model_validate for parsing, model_copy for immutable updates.
• Type checker: Pyright (via basedpyright).
• Docstrings: Imperative style, document public APIs, include usage examples.

Error Handling & Logging

• Always raise GuideError subclasses (not generic Exception); routers translate to HTTP responses.
• Log via core/logging (structured, levelled). Include persona/action IDs and host targets for traceability.
• For browser flows, use Playwright traces (enabled by default in BrowserClient); disable only intentionally.
• Validate external inputs early; surface schema/connection issues as GuideError.

Testing & Quality Gates

• Minimum gate: basedpyright src + python -m compileall src/guide before merge.
• Test Coverage: 28 passing tests across unit and integration suites (in tests/ directory).
• Test Structure:
  • tests/unit/ — Unit tests for strings registry, models (persona/host validation), action registration.
  • tests/integration/ — Integration tests for BrowserClient, BrowserPool, and browser lifecycle.
  • conftest.py — Shared fixtures: mock_browser_hosts, mock_personas, app_settings, persona_store, action_registry, action_context, mock_page, mock_browser_client, mock_browser_pool.
  • Pytest configured with asyncio_mode=auto for async test support.
• Mock Playwright/GraphQL in tests; avoid real network/CDP calls.
• Require deterministic fixtures; document any env vars needed in test module docstring.
• No loops or conditionals in tests; use @pytest.mark.parametrize for multiple cases.

MFA & Auth

• Default DummyMfaCodeProvider raises NotImplementedError.
• For real runs, implement a provider and wire it in core/config.py or auth/ modules.
• ensure_persona() in actions calls the provider; stub or override for demo/test execution.

Performance & Footprint

• Keep browser sessions short-lived; close contexts to avoid handle leaks.
• Cache expensive GraphQL lookups (per-request OK, global only if safe).
• Don't widen dependencies without justification; stick to project pins in pyproject.toml.
• Promptly close Playwright contexts/browser handles (wrapped in contextmanager; keep action code lean).

Refactoring Status (100% Complete)

The project has completed 8 major refactoring phases achieving full architectural modernization:

Phase	Focus	Status
Phase 1	Strings Registry Flattening	✅ Complete
Phase 2A	Persona Model Unification	✅ Complete
Phase 2B	BrowserHostDTO Separation	✅ Complete
Phase 3	Action Registration Standardization	✅ Complete
Phase 4	GraphQL Code Generation (ariadne-codegen)	✅ Complete
Phase 5	Browser Context Isolation	✅ Complete
Phase 6-8	Error Handling & Logging Enhancements	✅ Complete

Quality Metrics:

• Zero type errors (basedpyright)
• All linting passed
• 28 tests passing
• Zero code redundancy
• ~2,898 lines of production code

Git & PR Hygiene

• Scoped, descriptive commits (e.g., feat: add sourcing action, fix: handle missing persona host).
• PRs should state changes, commands run, new config entries (hosts/personas).
• Link related issues; include screenshots/logs for UI or API behavior changes.
• Never commit credentials, MFA tokens, or sensitive config; use env overrides.

Action Registration & Dependency Injection

Registering a New Action

Use the @register_action decorator to auto-discover and register actions:

from actions.base import DemoAction, register_action

@register_action
class MyAction(DemoAction):
    id = "my-action"
    description = "Does something cool"
    category = "demo"

    # Optional: Declare dependencies (auto-injected by ActionRegistry)
    def __init__(self, persona_store: PersonaStore):
        self.persona_store = persona_store

    async def run(self, page: Page, context: ActionContext) -> ActionResult:
        # Implementation
        ...

Auto-Discovery: ActionRegistry uses pkgutil.walk_packages() to discover all modules in actions/ and collect all @register_action decorated classes. No manual registration needed.

Dependency Injection: Parameters in __init__ are matched by name against DI context dict. Example: persona_store parameter → resolved from DI context during action instantiation.

Multi-Step Workflows (CompositeAction)

For workflows spanning multiple steps with shared state:

@register_action
class MyWorkflow(CompositeAction):
    id = "my-workflow"
    description = "Multi-step workflow"
    category = "demo"

    child_actions = ("step1-action", "step2-action", "step3-action")

    async def on_step_complete(self, step_id: str, result: ActionResult) -> None:
        # Optional: Update shared_state after each step
        # Accessed in child actions via context.shared_state dict
        pass

Quick Checklist (New Feature)

• Add action in actions/ submodule (or submodule directory like actions/intake/); use @register_action decorator.
• Add action-specific logic; keep it thin and testable. Use strings/ for all selectors/copy.
• Ensure persona/host exist in config/hosts.yaml + config/personas.yaml (or use env overrides).
• If action needs GraphQL, add query/mutation to raindrop/operations/ + .graphql files in raindrop/queries/.
• If action needs UI strings, add to strings/ submodules (selectors, labels, demo_texts).
• Run basedpyright src + python -m compileall src/guide (type check + syntax check).
• Run pytest tests/ to ensure no regressions (28 tests must pass).
• Test via python -m guide + navigate to http://localhost:8000/docs to test endpoint.
• If auth flow required, implement/mock MFA provider or use DummyMfaCodeProvider for testing.
• Review error handling; raise GuideError subclasses, not generic exceptions.
• Commit with descriptive message (e.g., feat: add my-action, test: add my-action tests).