Files
unstract/pyproject.toml
Chandrasekharan M 0c0c8c1034 UN-2793 [FEAT] Add retry logic with exponential backoff to SDK1 (#1564)
* UN-2793 [FEAT] Add retry logic with exponential backoff to SDK1

Implemented automatic retry logic for platform and prompt service calls
with configurable exponential backoff, comprehensive test coverage, and
CI integration.

Features:
- Exponential backoff with jitter for transient failures
- Configurable via environment variables (MAX_RETRIES, MAX_TIME, BASE_DELAY, etc.)
- Retries ConnectionError, Timeout, HTTPError (502/503/504), OSError
- 67 tests with 100% pass rate
- CI integration with test reporting

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [SECURITY] Use full commit SHA for sticky-pull-request-comment action

Replace tag reference with full commit SHA for better security:
- marocchino/sticky-pull-request-comment@v2 → @7737449 (v2.9.4)

This prevents potential supply chain attacks where tags could be moved
to point to malicious code. Commit SHAs are immutable.

Fixes SonarQube security hotspot for external GitHub action.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [FIX] Allow retryable HTTP errors (502/503/504) to propagate for retry

Fixed HTTPError handling in _get_adapter_configuration to check status
codes and re-raise retryable errors (502, 503, 504) so the retry
decorator can handle them. Non-retryable errors are still converted
to SdkError as before.

Changes:
- Check HTTPError status code before converting to SdkError
- Re-raise HTTPError for 502/503/504 to allow retry decorator to retry
- Added parametrized test for all retryable status codes (502, 503, 504)
- All 12 platform tests pass

This fixes a bug where 502/503/504 errors were not being retried
because they were converted to SdkError before the retry decorator
could see them.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [FIX] Use pytest.approx() for floating point comparisons in tests

Replaced direct equality comparisons (==) with pytest.approx() for
floating point values to avoid precision issues and satisfy SonarQube
code quality check (python:S1244).

Changes in test_retry_utils.py:
- test_exponential_backoff_without_jitter: Use pytest.approx() for 1.0, 2.0, 4.0, 8.0
- test_max_delay_cap: Use pytest.approx() for 5.0

This is the proper way to compare floating point values in tests,
accounting for floating point precision limitations.

All 4 TestCalculateDelay tests pass.

Fixes SonarQube: python:S1244 - Do not perform equality checks with
floating point values.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* minor: Addressed code smells, ruff fixes

* misc: Fixed tox config for sdk1 tests

* misc: Ruff issues fixed

* misc: tox tests fixed

* prompt service lock file for venv

* updated lock files for backend and prompt-service

* UN-2793 [FEAT] Update to unstract-sdk v0.78.0 with retry logic support (#1567)

[FEAT] Update unstract-sdk to v0.78.0 across all services and tools

- Updated unstract-sdk dependency from v0.77.3 to v0.78.0 in all pyproject.toml files
  - Main repository, backend, workers, platform-service, prompt-service
  - filesystem and tool-registry modules
- Updated tool requirements.txt files (structure, classifier, text_extractor)
- Bumped tool versions in properties.json:
  - Structure tool: 0.0.88 → 0.0.89
  - Classifier tool: 0.0.68 → 0.0.69
  - Text extractor tool: 0.0.64 → 0.0.65
- Updated tool versions in backend/sample.env and public_tools.json
- Regenerated all uv.lock files with new SDK version

This update brings in the retry logic with exponential backoff from unstract-sdk v0.78.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>

---------

Signed-off-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-09 10:48:19 +05:30

231 lines
7.5 KiB
TOML

[project]
name = "unstract"
version = "0.1.0"
requires-python = ">=3.12,<3.13"
dependencies = []
[dependency-groups]
dev = [
"docutils~=0.20.1",
"mypy~=1.9.0",
"pre-commit~=3.6.2",
"yamllint>=1.35.1",
"ruff<1.0.0,>=0.2.2",
"pytest>=8.0.1",
"pycln>=2.5.0",
"types-PyMySQL~=1.1.0.1",
"types-pyOpenSSL~=24.0.0.20240311",
"types-PyYAML~=6.0.12.12",
"types-redis~=4.6.0.3",
"types-requests~=2.31.0.6",
"types-tzlocal~=5.1.0.1",
]
workers = [
"unstract-workers",
]
hook-check-django-migrations = [
"celery>=5.3.4",
"cron-descriptor==1.4.0",
"django==4.2.1",
"djangorestframework==3.14.0",
# Pinning django-celery-beat to avoid build issues
"django-celery-beat==2.5.0",
"django-cors-headers>=4.3.1",
"django-redis==5.4.0",
"django-tenants==3.5.0",
"drf-yasg==1.21.7",
"social-auth-app-django==5.3.0",
"psycopg2-binary==2.9.9",
"python-dotenv==1.0.1",
"python-magic==0.4.27",
"unstract-sdk~=0.78.0",
"unstract-connectors",
"unstract-core",
"unstract-flags",
"unstract-tool-registry",
"unstract-tool-sandbox",
"unstract-workflow-execution",
"unstract-filesystem",
]
[tool.uv.sources]
unstract-filesystem = { path = "./unstract/filesystem", editable = true }
unstract-workflow-execution = { path = "./unstract/workflow-execution", editable = true }
unstract-tool-sandbox = { path = "./unstract/tool-sandbox", editable = true }
unstract-tool-registry = { path = "./unstract/tool-registry", editable = true }
unstract-flags = { path = "./unstract/flags", editable = true }
unstract-core = { path = "./unstract/core", editable = true }
unstract-connectors = { path = "./unstract/connectors", editable = true }
# Workers
unstract-workers = { path = "./workers", editable = true }
# === Development tool configurations ===
[tool.ruff]
line-length = 90
target-version = "py312"
exclude = [
".bzr",
".direnv",
".eggs",
".git",
".git-rewrite",
".hg",
".mypy_cache",
".nox",
".pants.d",
".pytype",
".ruff_cache",
".svn",
".tox",
".venv",
"__pypackages__",
"_build",
"buck-out",
"build",
"dist",
"node_modules",
"venv",
".*migrations/.*.py",
"tests",
]
[tool.ruff.lint]
select = [
"E", # pycodestyle (formatting)
"F", # Pyflakes (static analysis)
"I", # isort (import sorting)
"B", # bugbear (security/performance)
"W", # warnings
"C90", # mccabe complexity
"N", # pep8-naming
"D", # pydocstyle
"UP", # pyupgrade
"ANN", # flake8-annotations
"TCH", # flake8-type-checking
"PYI", # flake8-pyi
]
fixable = ["ALL"]
ignore = [
"ANN001", # Missing type annotation for function argument `value`
"ANN002", # Missing type annotation for `*args`"
"ANN003", # Missing type annotation for `**kwargs`"
"ANN101", # Missing type annotation for self
"ANN102", # Missing type annotation for cls
"ANN201", # Missing return type annotation for public function
"ANN204", # Missing return type annotation for special method `__init__`,
"ANN205", # Missing return type annotation for staticmethod `list`
"ANN401", # Dynamically typed expressions (typing.Any) are disallowed in `user_login`
"B006", # Do not use mutable data structures for argument defaults
"B007", # Loop control variable `dirs` not used within loop body
"B008", # Do not perform function call `FileStorage`"
"B015", # Pointless comparison. Did you mean to assign a value? Otherwise, prepend `assert` or remove it."
"B018", # Found useless expression. Either assign it to a variable or remove it.
"B024", # `AuthService` is an abstract base class, but it has no abstract methods or properties"
"B904", # Within an `except` clause, raise exceptions with
"C901", # `handle_output` is too complex (11 > 10)"
"D100", # Missing docstring in public module
"D101", # Missing docstring in public class
"D102", # Missing docstring in public method
"D103", # Missing docstring in public function
"D104", # Missing docstring in public package
"D105", # Missing docstring in magic method
"D106", # Missing docstring in public nested class
"D107", # Missing docstring in `__init__`
"D205", # 1 blank line required between summary line and description
"D402", # First line should not be the function's signature"
"D415", # First line should end with a period, question mark, or exclamation point"
"D417", # Missing argument description in the docstring for `user_organizations`: `request`
"E501", # Line too long (91 > 90)
"N806", # Variable `organizationData` in function should be lowercase
"N802", # Function name `makeSignupRequestParams` should be lowercase
"N804", # First argument of a class method should be named `cls`"
"N805", # First argument of a method should be named `self`"
"N812", # Lowercase `exceptions` imported as non-lowercase `GoogleApiException`"
"N813", # Camelcase `Common` imported as lowercase `common`
"N816", # Variable `zipProject` in global scope should not be mixedCase"
"N817", # CamelCase `FieldLengthConstants` imported as acronym `FLC`,
"N818", # Exception name `InvalidToolPropertiesException` should be named with an Error suffix
"N999", # Invalid module name: 'FileValidator'"
"TCH001", #Move application import xxx into a type-checking block
"TCH002", # Move third-party import `rest_framework.serializers.Serializer` into a type-checking block
"TCH003", # Move standard library import `uuid` into a type-checking block"
# "TC006", # [*] Add quotes to type expression in `typing.cast()"
"ANN206", #Missing return type annotation
"ANN202", # Missing return type annotation for private function"
"B027", # `NotificationProvider.validate` is an empty method in an abstract base class, but has no abstract decorator"
"F405", # `os` may be undefined, or defined from star imports"
"UP038", # Use `X | Y` in `isinstance` call instead of `(X, Y)`"
]
[tool.ruff.lint.mccabe]
max-complexity = 10
[tool.ruff.lint.pydocstyle]
convention = "google"
[tool.ruff.format]
quote-style = "double"
indent-style = "space"
line-ending = "auto"
docstring-code-format = true
[tool.pycln]
all = true
expand-stars = true
no-gitignore = false
verbose = true
[tool.pyupgrade]
keep-runtime-typing = true
py39-plus = true
keep-dict-typing = true
[tool.pytest.ini_options]
python_files = ["tests.py", "test_*.py", "*_tests.py"]
DJANGO_SETTINGS_MODULE = "backend.settings.test_cases"
testpaths = ["tests"]
markers = [
"slow: marks tests as slow (deselect with '-m \"not slow\"')",
"integration: marks tests as integration (deselect with '-m \"not integration\"')",
]
[tool.mypy]
python_version = "3.12"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = false
no_implicit_optional = true
strict_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true
allow_untyped_globals = false
allow_redefinition = false
local_partial_types = true
implicit_reexport = true
follow_imports = "silent"
ignore_missing_imports = true
pretty = true
show_column_numbers = true
show_error_codes = true
strict = true
exclude = '''(?x)^(
.*migrations/.*\.py|
backend/prompt/.*|
unstract/connectors/tests/.*|
unstract/core/.*|
unstract/flags/src/unstract/flags/.*|
__pypackages__/.*|
)$'''
[tool.yamllint]
relaxed = true