From f5022864507044a01be7424565ceac158f051894 Mon Sep 17 00:00:00 2001 From: Travis Vasceannie Date: Sun, 7 Dec 2025 14:16:27 +0000 Subject: [PATCH] Update documentation, refactor string handling, and enhance logging - Updated the development server documentation to reflect the new port configuration. - Refactored string handling by replacing the centralized registry with dedicated selectors for better modularity and type safety. - Enhanced logging throughout the application by integrating loguru for structured logging and improved context handling. - Removed outdated files and streamlined the codebase for better maintainability. - Added new HTML parsing utilities using BeautifulSoup to improve DOM traversal and element extraction. - Updated various components to utilize the new string selectors, ensuring consistency across the codebase. --- CLAUDE.md | 2 +- docs/failures.md | 170 ------- docs/sourcing_intake.md | 1 - docs/spec.md | 254 +++------- plan.md | 173 ++----- pyproject.toml | 2 + .../app/actions/contract/fill_contract.py | 107 ++-- src/guide/app/actions/demo/__init__.py | 4 +- .../actions/diagnose/messaging_selectors.py | 18 +- .../app/actions/intake/sourcing_request.py | 49 +- src/guide/app/actions/messaging/respond.py | 28 +- .../app/actions/sourcing/add_suppliers.py | 9 +- src/guide/app/api/routes/actions.py | 8 +- src/guide/app/auth/session.py | 132 ++--- src/guide/app/auth/session_manager.py | 32 +- src/guide/app/browser/client.py | 12 +- .../app/browser/elements/dropdown/_helpers.py | 49 +- src/guide/app/browser/elements/mui.py | 40 +- src/guide/app/browser/extension_client.py | 103 ++-- src/guide/app/browser/html_parser.py | 218 ++++++++ src/guide/app/browser/pool.py | 78 +-- src/guide/app/core/config.py | 2 + src/guide/app/core/logging.py | 143 ++++-- src/guide/app/main.py | 4 +- src/guide/app/raindrop/graphql.py | 8 +- src/guide/app/strings/registry.py | 478 ------------------ .../test_sourcing_intake_reconciliation.py | 28 +- tests/unit/strings/test_registry.py | 102 ++-- 28 files changed, 870 insertions(+), 1384 deletions(-) delete mode 100644 docs/failures.md delete mode 100644 docs/sourcing_intake.md create mode 100644 src/guide/app/browser/html_parser.py delete mode 100644 src/guide/app/strings/registry.py diff --git a/CLAUDE.md b/CLAUDE.md index 4e4b212..3342995 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -34,7 +34,7 @@ python -m guide HOST=127.0.0.1 PORT=9000 python -m guide # View API docs -# Navigate to http://localhost:8765/docs +# Navigate to http://localhost:8000/docs # Key endpoints: # GET /healthz # liveness check diff --git a/docs/failures.md b/docs/failures.md deleted file mode 100644 index 0e6f2c4..0000000 --- a/docs/failures.md +++ /dev/null @@ -1,170 +0,0 @@ -# MUI Autocomplete Dropdown Close - Failed Strategies - -## Problem Statement -MUI Autocomplete multi-select dropdowns stay open by design for multiple selections. After selecting options, the dropdown must be closed before interacting with other form fields. Otherwise, subsequent fields see the wrong listbox options. - -## Environment -- Browser automation via Chrome Extension (WebSocket-based, not CDP) -- Extension uses `dispatchEvent` for all interactions (events have `isTrusted: false`) -- MUI v5 Autocomplete with multi-select enabled -- React controlled components - ---- - -## Failed Strategies - -### 1. Simple Blur -```javascript -input.blur(); -document.body.focus(); -``` -**Result:** Dropdown stays open. MUI Autocomplete does not close on blur alone for multi-select. - ---- - -### 2. Tab Key Event -```javascript -const eventProps = { - key: 'Tab', - code: 'Tab', - keyCode: 9, - bubbles: true, - cancelable: true, - composed: true -}; -input.dispatchEvent(new KeyboardEvent('keydown', eventProps)); -input.dispatchEvent(new KeyboardEvent('keyup', eventProps)); -input.blur(); -``` -**Result:** Dropdown stays open. Tab key alone doesn't trigger close for multi-select autocomplete. - ---- - -### 3. Click Popup Indicator (Toggle) -```python -await page.click(f"{field_selector} .MuiAutocomplete-popupIndicator") -``` -**Result:** Inconsistent. Sometimes works, sometimes opens a different dropdown. The popup indicator is a toggle button - if the dropdown is somehow in a weird state, clicking it may re-open instead of close. - ---- - -### 4. Click Neutral Element (h2, Dialog Title) -```python -await page.click("h2") -await page.click(".MuiDialogTitle-root") -await page.click(".MuiDialogContent-root") -``` -**Result:** Dropdown stays open. The click events are dispatched but MUI ClickAwayListener doesn't respond. Likely because programmatic events have `isTrusted: false`. - ---- - -### 5. Mousedown Event on Document (ClickAwayListener) -```javascript -const event = new MouseEvent('mousedown', { - bubbles: true, - cancelable: true, - view: window, - clientX: rect.left + 5, - clientY: rect.top + 5, - button: 0 -}); -target.dispatchEvent(event); // target = h2, dialog, body -``` -**Result:** Dropdown stays open. MUI ClickAwayListener likely checks `event.isTrusted` which is `false` for programmatically dispatched events. - ---- - -### 6. Escape Key to Input -```javascript -input.focus(); -const eventProps = { - key: 'Escape', - code: 'Escape', - keyCode: 27, - bubbles: true, - cancelable: true, - composed: true -}; -input.dispatchEvent(new KeyboardEvent('keydown', eventProps)); -input.dispatchEvent(new KeyboardEvent('keyup', eventProps)); -input.blur(); -``` -**Result:** CLOSES THE PARENT MODAL. The Escape event bubbles up and closes the dialog, not just the dropdown. This breaks the entire form flow. - ---- - -### 7. Escape Key to Active Element (from _open_dropdown) -```javascript -const active = document.activeElement; -active.dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', ... })); -``` -**Result:** Same as #6 - closes the parent modal. - ---- - -### 8. Remove Listbox from DOM -```javascript -const listbox = document.querySelector('[role="listbox"]'); -listbox.remove(); -``` -**Result:** CRASHES REACT. React expects to manage the DOM - removing elements it controls causes "Cannot read properties of null" errors and breaks the form. - ---- - -### 9. Set aria-expanded to false -```javascript -input.setAttribute('aria-expanded', 'false'); -``` -**Result:** Visual state doesn't change. MUI manages this attribute internally via React state. Setting it directly has no effect on the actual dropdown visibility. - ---- - -## Key Insights - -1. **`isTrusted` Check**: MUI ClickAwayListener likely checks `event.isTrusted`. All programmatically dispatched events have `isTrusted: false`, so they're ignored. - -2. **React State Ownership**: MUI Autocomplete dropdown visibility is controlled by React state, not DOM attributes. Direct DOM manipulation doesn't work. - -3. **Multi-select Behavior**: Unlike single-select, multi-select autocomplete is designed to stay open for selecting multiple items. Standard close mechanisms are intentionally disabled. - -4. **Escape Bubbling**: Escape key events bubble to parent elements, closing modals instead of just dropdowns. - -5. **Extension Limitation**: Chrome extension automation via `dispatchEvent` cannot produce trusted events. Only actual user input or CDP Input.dispatchMouseEvent can create trusted events. - ---- - -## Successful Solution ✅ - -**Mark ALL listboxes as closed with data attribute + CSS hiding + filter in queries** - -```javascript -// In close logic (after select_multi completes): -const listboxes = document.querySelectorAll('[role="listbox"]:not([data-dropdown-closed])'); -listboxes.forEach(listbox => { - listbox.setAttribute('data-dropdown-closed', 'true'); - listbox.style.setProperty('display', 'none', 'important'); - listbox.style.setProperty('visibility', 'hidden', 'important'); - listbox.style.setProperty('pointer-events', 'none', 'important'); -}); - -// In all listbox queries: -const listbox = document.querySelector('[role="listbox"]:not([data-dropdown-closed])'); -``` - -**Why it works:** -1. **Data attribute marking**: `data-dropdown-closed="true"` survives React re-renders (React doesn't remove unknown attributes) -2. **CSS with !important**: Prevents MUI styles from overwriting our hiding -3. **Filter in ALL queries**: `check_listbox_visible`, `_get_options`, `_get_listbox_options` all use `:not([data-dropdown-closed])` selector -4. **Mark ALL listboxes**: Not just the one for current field - catches any orphaned/stale listboxes - -**Key insight:** The issue wasn't that we couldn't close the dropdown - it's that we couldn't prevent subsequent code from finding the stale listbox. By marking and filtering, we make stale listboxes invisible to our automation code. - ---- - -## Previously Attempted (Did Not Work) - -### CSS Hiding Only (Without Data Attribute) -```javascript -listbox.style.display = 'none'; -``` -**Result:** React re-renders could remove the style, and queries would still find the element. diff --git a/docs/sourcing_intake.md b/docs/sourcing_intake.md deleted file mode 100644 index f1b26b0..0000000 --- a/docs/sourcing_intake.md +++ /dev/null @@ -1 +0,0 @@ -
Board: "Operations"

Intake form: "Sourcing Request"

Required

Required

\ No newline at end of file diff --git a/docs/spec.md b/docs/spec.md index 08d6165..392bfaf 100644 --- a/docs/spec.md +++ b/docs/spec.md @@ -1,201 +1,89 @@ -This is a sophisticated automation codebase blending Playwright/CDP with high-level abstraction layers. However, to achieve your goal of **LLM-driven contextual filling** (where an LLM decides *what* to put in a field based on its label/context rather than hardcoded `fill_text` calls), the current implementation has significant gaps in how it "perceives" the form. +# Spec Validation (2025-12-07) -Here is a critique of the current approach, specific bug identifications, and a strategy to bridge the gap. - -### 1. Critique: The "Uncanny Valley" of Field Discovery - -The codebase currently sits between two approaches: -1. **Explicit Definitions:** `raindrop/operations/form_schema.py` (GraphQL schema). -2. **Heuristic DOM Scraping:** `browser/elements/form_discovery.py` (Regex/HTML parsing). - -**The Core Problem:** There is no reliable bridge between the *Schema* (which knows that field `f19` is "Estimated Value" and requires a number) and the *DOM* (which is just a `
` with `data-cy="board-item-field-number-f19"`). - -To allow an LLM to generate values, you need to construct a **Context Object** that says: -> "Field 'Description' (DOM Selector: `[data-cy=...]`) is a Text Area. It is currently empty. It corresponds to the 'Description' field in the Schema." - -#### Weaknesses in Current Logic: -* **Regex HTML Parsing is Fragile:** `extract_field_from_html` in `form_discovery.py` attempts to parse raw HTML strings with Regex to find labels and attributes. This is prone to failure with React/MUI because: - * The `label` is often a sibling or wrapped in a `FormControl`, not always nested cleanly inside the captured HTML snippet. - * MUI uses portals for dropdowns; the options aren't inside the captured HTML snippet of the field. -* **Dependency on `data-cy` structure:** The logic heavily infers types based on the string format of `data-cy` (e.g., splitting by hyphens). If the frontend team changes `board-item-field-text-...` to `field-input-...`, the inference engine breaks entirely. -* **Disconnect from ARIA:** The discovery logic looks for visual labels via Regex but ignores the Accessibility Tree. The most reliable way to identify a field for an LLM is the **Accessible Name** (computed from `label`, `aria-label`, `aria-labelledby`), which is what screen readers (and intelligent agents) should use. - -### 2. Bugs & Fragile Code Identification - -#### Bug A: Regex Parsing of React/MUI Forms -**File:** `src/guide/app/browser/elements/form_discovery.py` -**Function:** `extract_field_from_html` +Findings below are grounded in current source with inline excerpts. +## Functional Gaps +- Docling config is cosmetic: `extract_ui_elements` keeps `_docling_url`/`_api_key` params but the docstring says they are “Unused (maintained for signature compatibility)”, and the body only runs `_extract_elements_from_html(...)` on captured HTML (no HTTP call). ```python -# CURRENT BROKEN LOGIC -label_match = re.search( - r']*>(.*?)', - html, - re.IGNORECASE | re.DOTALL -) +# src/guide/app/browser/diagnostics.py +async def extract_ui_elements(..., _docling_url: str | None = None, _api_key: str | None = None): + """... docling_url: Unused (maintained for signature compatibility)""" + html = await capture_html(page) + ui_elements = _extract_elements_from_html(html) ``` -**Why it fails:** In MUI, the `