* Update tox.ini to document libmagic1 system dependency requirement
* [CI] Consolidate test reports into single PR comment
This change reduces noise in PR comments by combining multiple test
reports (runner and sdk1) into a single consolidated comment.
Changes:
- Add combine-test-reports.sh script to merge test reports
- Update CI workflow to combine reports before posting
- Replace separate runner/sdk1 comment steps with single combined step
- Summary section shows test counts (passed/failed/total) collapsed by default
- Full detailed reports available in collapsible sections
- Simplify job summary to use combined report
Benefits:
- Single PR comment instead of multiple separate comments
- Cleaner PR comment section with less clutter
- Easy-to-read summary with detailed inspection on demand
- Maintains all existing test information
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix test count extraction in combine-test-reports script
The previous version was using text pattern matching which incorrectly
extracted test counts. This fix properly parses the pytest-md-report
markdown table format by:
- Finding the table header row to determine column positions
- Locating the TOTAL row (handles both TOTAL and **TOTAL** formatting)
- Extracting values from the passed, failed, and SUBTOTAL columns
- Using proper table parsing instead of pattern matching
This resolves the issue where the summary showed incorrect counts
(23 for both runner and sdk1 instead of the actual 11 and 66).
Fixes: Test count summary in PR comments now shows correct values
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix LaTeX formatting in pytest-md-report test count extraction
pytest-md-report wraps all table values in LaTeX formatting:
$$\textcolor{...}{\tt{VALUE}}$$
The previous fix attempted to parse the table but didn't handle
the LaTeX formatting, causing it to extract 0 for all counts.
Changes:
- Add strip_latex() function to extract values from LaTeX wrappers
- Update grep pattern to match TOTAL row specifically (not SUBTOTAL in header)
- Apply LaTeX stripping to all extracted values before parsing
This fix was tested locally with tox-generated reports and correctly
shows: Runner=11 passed, SDK1=66 passed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* UN-2793 [FEAT] Add retry logic with exponential backoff to SDK1
Implemented automatic retry logic for platform and prompt service calls
with configurable exponential backoff, comprehensive test coverage, and
CI integration.
Features:
- Exponential backoff with jitter for transient failures
- Configurable via environment variables (MAX_RETRIES, MAX_TIME, BASE_DELAY, etc.)
- Retries ConnectionError, Timeout, HTTPError (502/503/504), OSError
- 67 tests with 100% pass rate
- CI integration with test reporting
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* [SECURITY] Use full commit SHA for sticky-pull-request-comment action
Replace tag reference with full commit SHA for better security:
- marocchino/sticky-pull-request-comment@v2 → @7737449 (v2.9.4)
This prevents potential supply chain attacks where tags could be moved
to point to malicious code. Commit SHAs are immutable.
Fixes SonarQube security hotspot for external GitHub action.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* [FIX] Allow retryable HTTP errors (502/503/504) to propagate for retry
Fixed HTTPError handling in _get_adapter_configuration to check status
codes and re-raise retryable errors (502, 503, 504) so the retry
decorator can handle them. Non-retryable errors are still converted
to SdkError as before.
Changes:
- Check HTTPError status code before converting to SdkError
- Re-raise HTTPError for 502/503/504 to allow retry decorator to retry
- Added parametrized test for all retryable status codes (502, 503, 504)
- All 12 platform tests pass
This fixes a bug where 502/503/504 errors were not being retried
because they were converted to SdkError before the retry decorator
could see them.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* [FIX] Use pytest.approx() for floating point comparisons in tests
Replaced direct equality comparisons (==) with pytest.approx() for
floating point values to avoid precision issues and satisfy SonarQube
code quality check (python:S1244).
Changes in test_retry_utils.py:
- test_exponential_backoff_without_jitter: Use pytest.approx() for 1.0, 2.0, 4.0, 8.0
- test_max_delay_cap: Use pytest.approx() for 5.0
This is the proper way to compare floating point values in tests,
accounting for floating point precision limitations.
All 4 TestCalculateDelay tests pass.
Fixes SonarQube: python:S1244 - Do not perform equality checks with
floating point values.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* minor: Addressed code smells, ruff fixes
* misc: Fixed tox config for sdk1 tests
* misc: Ruff issues fixed
* misc: tox tests fixed
* prompt service lock file for venv
* updated lock files for backend and prompt-service
* UN-2793 [FEAT] Update to unstract-sdk v0.78.0 with retry logic support (#1567)
[FEAT] Update unstract-sdk to v0.78.0 across all services and tools
- Updated unstract-sdk dependency from v0.77.3 to v0.78.0 in all pyproject.toml files
- Main repository, backend, workers, platform-service, prompt-service
- filesystem and tool-registry modules
- Updated tool requirements.txt files (structure, classifier, text_extractor)
- Bumped tool versions in properties.json:
- Structure tool: 0.0.88 → 0.0.89
- Classifier tool: 0.0.68 → 0.0.69
- Text extractor tool: 0.0.64 → 0.0.65
- Updated tool versions in backend/sample.env and public_tools.json
- Regenerated all uv.lock files with new SDK version
This update brings in the retry logic with exponential backoff from unstract-sdk v0.78.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
---------
Signed-off-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
* tox updated to support venv
* pytest command reverted
* pytest removed from allowlist
* tox updated
* tox updated
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Signed-off-by: Jaseem Jas <89440144+jaseemjaskp@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* python version updated from 3.9 into 3.12
* x2text-service updated with uv and python version 3.12
* x2text-service docker file updated
* Unstract packages updated with uv
* Runner updated with uv
* Promptservice updated with uv
* Platform service updated with uv
* backend service updated with uv
* root pyproject.toml file updated
* sdk version updated in services
* unstract package modules updated based on sdk version:
* docker file update
* pdm lock workflow modified to support uv
* Docs updated based on uv support
* lock automation updated
* snowflake module version updated into 3.14.0
* tox updated to support UV
* tox updated to support UV
* tox updated with pytest
* tox updated with pytest-md-report
* tox updated with module requirements
* python migration from 3.9 to 3.12
* tox updated with module requirements
* runner updated
* Commit uv.lock changes
* runner updated
* Commit uv.lock changes
* pytest.ini added
* x2text-service docker file updated
* pytest.ini removed
* environment updated to test
* docformatter commented on pre-commit
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* some pre-commit issues ignored
* some pre-commit issues ignored
* some pre-commit issues ignored
* some pre-commit issues ignored
* some pre-commit issues ignored
* pre-commit updates
* un used import removed from platfrom service controller
* tox issue fixed
* tox issue fixed
* docker files updated
* backend dockerfile updated
* open installation issue fixed
* Tools docker file updated with base python version 3.12
* python version updated into min 3.12 in pyproject.toml
* linting issue fixed
* uv version upgraded into 0.6.14
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* migrations excluded from ruff
* added PoethePoet task runner
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: Added poe tasks for services (#1248)
* Added poe tasks for services
* reverted FE change made by mistake
* updated tool-sidecar to uv and python to 3.12.9
* minor updates in pyproject descreption
* feat: platform-service logging improvements (#1255)
feat: Used flask util from core to improve logging in platform-service, added core as a dependency to platform-service:
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: Platform-service build issue and numpy issue with Python 3.12 (#1258)
* fix: Platform-service build and numpy issue with Py 3.12
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: Removed backend dockerfile install statements for numpy
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* minor: Handled scenario when cost is not calculated due to no usage
* minor: Corrected content shown for workflow input
* fix: Minor fixes, used gthread for prompt-service, runner
* Commit uv.lock changes
* Removed unused line in tool dockerfile
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Chandrasekharan M <chandrasekharan@zipstack.com>
Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com>
Co-authored-by: ali-zipstack <muhammad.ali@zipstack.com>
* implement remote-storage from sdk in backend under featureflag
* added feature flags in imports
* minor changes
* handled fetching result from filestorage
* minor changes and adding updating docs and sample envs
* addressed pr comments
* minor changes in test
* updated tox ini
* Added unit test setup for worker
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Removed coverage upload
* Fixed flake8 issue and corrected the comment file path
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Changed comment input
* Updated workflow file
* Updated workflow file
* Updated workflow file
* Added markdown report
* Added updated tox.ini
* Hardcoded file name in workflow
* Replaced commenting with summary
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Added sticky comment
* Removed other comment
* Added build on merge to main also
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kirtiman Mishra <110175055+kirtimanmishrazipstack@users.noreply.github.com>
