78 Commits

Author SHA1 Message Date
Daniel.y
ac3063f417 Merge pull request #2503 from HKUDS/dependabot/github_actions/github-actions-35b2a8182b
Bump the github-actions group with 3 updates
2025-12-21 19:09:25 +08:00
yangdx
73c3c4135c Drop Python 3.13 from tests and reformat code 2025-12-19 18:15:12 +08:00
yangdx
a3b33bbc3c Remove E2E tests and update migration unit tests
- Delete E2E workflows and test files
- Remove multi-model demo example
- Update Postgres migration unit tests
- Enforce workspace requirement in tests
- Fix dimension mismatch test mocks
2025-12-19 15:20:32 +08:00
dependabot[bot]
a8bc957610 Bump the github-actions group with 3 updates
Bumps the github-actions group with 3 updates: [actions/cache](https://github.com/actions/cache), [actions/upload-artifact](https://github.com/actions/upload-artifact) and [actions/download-artifact](https://github.com/actions/download-artifact).


Updates `actions/cache` from 4 to 5
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v4...v5)

Updates `actions/upload-artifact` from 5 to 6
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v5...v6)

Updates `actions/download-artifact` from 6 to 7
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/download-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-14 18:09:18 +00:00
yangdx
19ab979a9c Merge branch 'main' into feature/vector-model-isolation 2025-12-12 10:28:59 +08:00
yangdx
6476021619 Configure Dependabot schedule with specific times and timezone
- Set Monday 2AM for GitHub Actions
- Set Wednesday 2AM for Python deps
- Set Friday 2AM for web UI deps
- Use Asia/Shanghai timezone
- Spread updates across weekdays
2025-12-02 19:21:44 +08:00
yangdx
13fc9f339a Reduce dependabot open pull request limits 2025-12-02 17:56:27 +08:00
yangdx
1f8751225d Drop Python 3.10 and 3.11 from CI test matrix 2025-12-02 15:36:18 +08:00
yangdx
883c5dc0a0 Update dependabot config with new groupings and patterns
- Add typescript-eslint grouping
- Include prettier-* patterns
- Add @types/bun dependency
- Create content-rendering group
- Group katex and mermaid updates
2025-12-02 12:03:30 +08:00
yangdx
9425277fd9 Improve dependabot config with better docs and numpy ignore rule
• Add detailed PR strategy comments
• Ignore numpy major version updates
• Group dependency updates by category
• Document update types per ecosystem
• Enhance configuration readability
2025-12-02 11:12:32 +08:00
yangdx
445adfc9cb Add name to lint-and-format job in GitHub workflow 2025-12-02 05:17:28 +08:00
Daniel.y
d0509d6f27 Merge pull request #2448 from HKUDS/dependabot/github_actions/github-actions-b6ffb444c9
Bump the github-actions group with 7 updates
2025-12-02 05:08:59 +08:00
yangdx
f93bda5859 Enable numpy updates in dependabot configuration
- Re-enable numpy in data-processing group
- Remove numpy from ignore list
- Allow minor and patch updates
- Remove breaking change comment
2025-12-02 04:50:28 +08:00
dependabot[bot]
88357675ea Bump the github-actions group with 7 updates
Bumps the github-actions group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [actions/checkout](https://github.com/actions/checkout) | `2` | `6` |
| [actions/setup-python](https://github.com/actions/setup-python) | `2` | `6` |
| [docker/build-push-action](https://github.com/docker/build-push-action) | `5` | `6` |
| [oven-sh/setup-bun](https://github.com/oven-sh/setup-bun) | `1` | `2` |
| [actions/upload-artifact](https://github.com/actions/upload-artifact) | `4` | `5` |
| [actions/download-artifact](https://github.com/actions/download-artifact) | `4` | `6` |
| [actions/stale](https://github.com/actions/stale) | `9` | `10` |


Updates `actions/checkout` from 2 to 6
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v6)

Updates `actions/setup-python` from 2 to 6
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v6)

Updates `docker/build-push-action` from 5 to 6
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5...v6)

Updates `oven-sh/setup-bun` from 1 to 2
- [Release notes](https://github.com/oven-sh/setup-bun/releases)
- [Commits](https://github.com/oven-sh/setup-bun/compare/v1...v2)

Updates `actions/upload-artifact` from 4 to 5
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v5)

Updates `actions/download-artifact` from 4 to 6
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v6)

Updates `actions/stale` from 9 to 10
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v9...v10)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/setup-python
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: docker/build-push-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: oven-sh/setup-bun
  dependency-version: '2'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/upload-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/download-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/stale
  dependency-version: '10'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-01 20:45:49 +00:00
yangdx
0f19f80fdb Configure comprehensive Dependabot for Python and frontend dependencies
- Add pip ecosystem with grouping
- Add bun ecosystem for webui
- Set weekly update schedule
- Configure cooldown periods
- Ignore numpy breaking changes
2025-12-02 04:28:59 +08:00
yangdx
ecef842cb5 Update GitHub Actions to use latest versions (v6) 2025-12-02 03:47:47 +08:00
Daniel.y
6fee81f503 Merge pull request #2435 from cclauss/patch-1
Keep GitHub Actions up to date with GitHub's Dependabot
2025-12-02 03:45:58 +08:00
Christian Clauss
d2ab7fb246 Add Python 3.13 and 3.14 to the testing 2025-11-28 16:15:16 +01:00
Christian Clauss
90e38c20ca Keep GitHub Actions up to date with GitHub's Dependabot
* [Keeping your software supply chain secure with Dependabot](https://docs.github.com/en/code-security/dependabot)
* [Keeping your actions up to date with Dependabot](https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot)
* [Configuration options for the `dependabot.yml` file - package-ecosystem](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem)

To see all GitHub Actions dependencies, type:
% `git grep 'uses: ' .github/workflows/`
2025-11-28 15:04:53 +01:00
palanisd
c233da6318 Update copilot-setup-steps.yml 2025-11-23 17:42:04 -05:00
palanisd
1b0413ee74 Create copilot-setup-steps.yml 2025-11-22 15:29:05 -05:00
BukeLy
6737ec00bc fix: improve Qdrant wait strategy in E2E tests
Why this change is needed:
Previous wait strategy used `/health` endpoint with `-f` flag and only
30 second timeout, causing timeouts in GitHub Actions.

How it solves it:
- Use root endpoint `/` instead of `/health` (Qdrant API root responds)
- Remove `-f` flag to accept any response (not just 2xx)
- Increase timeout from 30s to 60s
- Add progress output for each attempt
- Add clear error message on failure

Impact:
More reliable Qdrant service detection in E2E tests

Testing:
Will verify on GitHub Actions E2E test run
2025-11-20 00:38:28 +08:00
BukeLy
bef7577fd9 fix: correct PostgreSQL environment variable name in E2E workflow
Why this change is needed:
E2E tests were failing with:
"ValueError: Storage implementation 'PGKVStorage' requires the following
environment variables: POSTGRES_DATABASE"

The workflow was setting POSTGRES_DB but LightRAG's check_storage_env_vars()
expects POSTGRES_DATABASE (matching ClientManager.get_config()).

How it solves it:
Changed environment variable name from POSTGRES_DB to POSTGRES_DATABASE
in the "Run PostgreSQL E2E tests" step.

Impact:
- PGKVStorage, PGGraphStorage, and PGDocStatusStorage can now properly
  initialize using ClientManager's configuration
- Fixes ValueError during LightRAG initialization

Testing:
Next E2E run should pass environment variable validation and proceed
to actual test execution.
2025-11-20 00:35:03 +08:00
BukeLy
722f639fa5 fix: remove Qdrant health check in E2E workflow
Why this change is needed:
Qdrant Docker image does not have curl or wget pre-installed,
causing health check to always fail and container to be marked
as unhealthy after timeout.

How it solves it:
Remove health check from Qdrant service container configuration.
The E2E test already has a "Wait for Qdrant" step that uses curl
from the runner environment to verify service readiness before
running tests.

Impact:
- Qdrant container will start immediately without health check delays
- Service readiness still verified by test-level wait step
- Eliminates container startup failures

Testing:
Next CI run should successfully start Qdrant container and pass
the wait/verify steps in the test workflow.
2025-11-20 00:26:36 +08:00
BukeLy
66a0dfe5b7 fix: resolve E2E test failures in CI
Why this change is needed:
E2E tests were failing in GitHub Actions CI with two critical issues:
1. PostgreSQL tests failed with "ModuleNotFoundError: No module named 'qdrant_client'"
2. Qdrant container health check never became healthy

How it solves it:
1. Added qdrant-client to PostgreSQL job dependencies
   - test_e2e_multi_instance.py imports QdrantClient at module level
   - Even with -k "postgres" filter, pytest imports the whole module first
   - Both PostgreSQL and Qdrant tests now share dependencies

2. Changed Qdrant health check from curl to wget
   - Qdrant Docker image may not have curl pre-installed
   - wget is more commonly available in minimal container images
   - New command: wget --no-verbose --tries=1 --spider

Impact:
- Fixes PostgreSQL E2E test import errors
- Enables Qdrant container to pass health checks
- Allows both test suites to run successfully in CI

Testing:
- Will verify in next CI run that both jobs complete successfully
- Health check should now return "healthy" status within retry window
2025-11-20 00:25:35 +08:00
BukeLy
dc2061583f test: refactor E2E tests using complete LightRAG instances
Replaced storage-level E2E tests with comprehensive LightRAG-based tests.

Key improvements:
- Use complete LightRAG initialization (not just storage classes)
- Proper mock LLM/embedding functions matching real usage patterns
- Added tokenizer support for realistic testing

Test coverage:
1. test_legacy_migration_postgres: Automatic migration from legacy table (1536d)
2. test_multi_instance_postgres: Multiple LightRAG instances (768d + 1024d)
3. test_multi_instance_qdrant: Multiple Qdrant instances (768d + 1024d)

Scenarios tested:
- ✓ Multi-dimension support (768d, 1024d, 1536d)
- ✓ Multi-model names (model-a, model-b, text-embedding-ada-002)
- ✓ Legacy migration (backward compatibility)
- ✓ Multi-instance coexistence
- ✓ PostgreSQL and Qdrant storage backends

Removed:
- tests/test_e2e_postgres_migration.py (replaced)
- tests/test_e2e_qdrant_migration.py (replaced)

Updated:
- .github/workflows/e2e-tests.yml: Use unified test file
2025-11-20 00:13:00 +08:00
BukeLy
c32e6a4e7b test: add E2E tests with real PostgreSQL and Qdrant services
Why this change is needed:
While unit tests with mocks verify code logic, they cannot catch real-world
issues like database connectivity, SQL syntax errors, vector dimension mismatches,
or actual data migration failures. E2E tests with real database services provide
confidence that the feature works in production-like environments.

What this adds:
1. E2E workflow (.github/workflows/e2e-tests.yml):
   - PostgreSQL job with ankane/pgvector:latest service
   - Qdrant job with qdrant/qdrant:latest service
   - Runs on Python 3.10 and 3.12
   - Manual trigger + automatic on PR

2. PostgreSQL E2E tests (test_e2e_postgres_migration.py):
   - Fresh installation: Create new table with model suffix
   - Legacy migration: Migrate 10 real records from legacy table
   - Multi-model: Two models create separate tables with different dimensions
   - Tests real SQL execution, pgvector operations, data integrity

3. Qdrant E2E tests (test_e2e_qdrant_migration.py):
   - Fresh installation: Create new collection with model suffix
   - Legacy migration: Migrate 10 real vectors from legacy collection
   - Multi-model: Two models create separate collections (768d vs 1024d)
   - Tests real Qdrant API calls, collection creation, vector operations

How it solves it:
- Uses GitHub Actions services to spin up real databases
- Tests connect to actual PostgreSQL with pgvector extension
- Tests connect to actual Qdrant server with HTTP API
- Verifies complete data flow: create → migrate → verify
- Validates dimension isolation and data integrity

Impact:
- Catches database-specific issues before production
- Validates migration logic with real data
- Confirms multi-model isolation works end-to-end
- Provides high confidence for merge to main

Testing:
After this commit, E2E tests can be triggered manually from GitHub Actions UI:
  Actions → E2E Tests (Real Databases) → Run workflow

Expected results:
- PostgreSQL E2E: 3 tests pass (fresh install, migration, multi-model)
- Qdrant E2E: 3 tests pass (fresh install, migration, multi-model)
- Total: 6 E2E tests validating real database operations

Note:
E2E tests are separate from fast unit tests and only run on:
1. Manual trigger (workflow_dispatch)
2. Pull requests that modify storage implementation files
This keeps the main CI fast while providing thorough validation when needed.
2025-11-19 23:41:40 +08:00
BukeLy
209dadc0af ci: add feature branch testing workflow
Why this change is needed:
Before creating a PR, we need to validate that the vector storage model isolation
feature works correctly in the CI environment. The existing tests.yml only runs
on main/dev branches and only tests marked as 'offline'. We need a dedicated
workflow to test feature branches and specifically run migration tests.

What this adds:
- New workflow: feature-tests.yml
- Triggers on:
  1. Manual dispatch (workflow_dispatch) - can be triggered from GitHub UI
  2. Push to feature/** branches - automatic testing
  3. Pull requests to main/dev - pre-merge validation
- Runs migration tests across Python 3.10, 3.11, 3.12
- Specifically tests:
  - test_qdrant_migration.py (6 tests)
  - test_postgres_migration.py (6 tests)
- Uploads test results as artifacts

How to use:
1. Automatic: Push to feature/vector-model-isolation triggers tests
2. Manual: Go to Actions tab → Feature Branch Tests → Run workflow
3. PR: Tests run automatically when PR is created

Impact:
- Enables pre-PR validation on GitHub infrastructure
- Catches issues before code review
- Provides test results across multiple Python versions
- No need for local test environment setup

Testing:
After pushing this commit, tests will run automatically on the feature branch.
Can also be triggered manually from GitHub Actions UI.
2025-11-19 23:34:45 +08:00
Daniel.y
dfbc97363c Merge pull request #2369 from HKUDS/workspace-isolation
Feat: Add Workspace Isolation for Pipeline Status and In-memory Storage
2025-11-18 15:21:10 +08:00
yangdx
656025b75e Rename GitHub workflow from "Tests" to "Offline Unit Tests" 2025-11-18 13:36:00 +08:00
yangdx
4ea2124001 Add GitHub CI workflow and test markers for offline/integration tests
- Add GitHub Actions workflow for CI
- Mark integration tests requiring services
- Add offline test markers for isolated tests
- Skip integration tests by default
- Configure pytest markers and collection
2025-11-18 11:36:10 +08:00
yangdx
f83b475ab1 Remove Dependabot configuration file
• Delete .github/dependabot.yml
• Remove weekly pip updates
2025-11-18 01:42:15 +08:00
yangdx
daeca17f38 Change default docker image to offline version
• Add lite verion docker image with tiktoken cache
• Update docs and build scripts
2025-10-16 16:52:01 +08:00
yangdx
8cc8bbf486 Change Docker build cache mode from max to min
• Reduce cache storage usage
• Try to fix GithHub Action failure
2025-10-16 12:24:51 +08:00
yangdx
e6332ce512 Add reminder note to manual Docker build workflow 2025-10-16 11:45:27 +08:00
yangdx
19c05f9ea4 Add static 'offline' tag to Docker image metadata 2025-10-15 15:56:42 +08:00
yangdx
6d1ae40478 Add offline Docker build support with embedded models and cache
- Add offline Dockerfile with tiktoken cache
- Create GitHub workflow for offline builds
- Update dockerignore for cleaner builds
- Exclude dev dirs from package setup
- Remove tiktoken volume from compose
2025-10-15 15:40:30 +08:00
yangdx
e5cbc593f4 Optimize Docker build with multi-stage frontend compilation
• Add frontend build stage to Dockerfile
• Remove --production flag from bun install
• Fix frontend asset integration
2025-10-14 15:02:58 +08:00
yangdx
8bf411314e Standardize build commands and remove --emptyOutDir flag
• Uses exact versions from `bun.lock` without updating
• Skips devDependencies, saving time and bandwidth
• The --emptyOutDir already configured in vite.config.ts
2025-10-14 02:49:25 +08:00
yangdx
be9e6d1612 Exclude Frontend Build Artifacts from Git Repository
• Automate frontend build in CI/CD
• Add build validation checks
• Clean git repo of build artifacts
• Comprehensive build guide docs
• Smart setup.py build validation
2025-10-13 23:43:34 +08:00
yangdx
4772148901 ci: only tag latest for stable releases, skip pre-releases
• Add pre-release detection step
• Check for 'rc' and 'dev' in tags
• Conditionally apply latest tag
• Skip latest for pre-releases
• Preserve stable release tagging
2025-09-26 23:41:08 +08:00
yangdx
ff6c061aa9 Remove conditional check for latest Docker tag
- Remove is_default_branch condition
- Always apply latest tag
2025-09-08 23:23:13 +08:00
yangdx
d57fce1aac Add manual trigger to PyPI publish workflow 2025-09-08 22:03:54 +08:00
yangdx
0ccf2036c6 Refactor Docker workflows: rename and clean up build processes
• Rename workflows for clarity
• Remove debug git commands
2025-09-05 11:46:56 +08:00
yangdx
cf31d636c2 Add git tag fetching and debug output to Docker workflow 2025-09-05 11:28:28 +08:00
yangdx
9181649bae Add version sync to __init__.py in Docker workflows 2025-09-05 10:56:46 +08:00
yangdx
d85ff5b9d7 Fix variable reference in PyPI publish workflow 2025-09-05 02:37:22 +08:00
yangdx
b88ab7c04e Revert "Remove PyPI environment from publish workflow"
This reverts commit 688550a9c6.
2025-09-05 02:00:44 +08:00
yangdx
688550a9c6 Remove PyPI environment from publish workflow 2025-09-05 01:53:20 +08:00
yangdx
ed5b9b414c Add automatic version extraction from git tags to PyPI workflow
* Fetch full git history for tags
* Extract version from latest git tag
* Update __init__.py with tag version
* Display updated version for verification
2025-09-05 01:48:53 +08:00