diff --git a/.clinerules/01-basic.md b/.clinerules/01-basic.md index 566a8159..955afa83 100644 --- a/.clinerules/01-basic.md +++ b/.clinerules/01-basic.md @@ -47,6 +47,31 @@ else np.frombuffer(base64.b64decode(dp.embedding), dtype=np.float32) **Location**: Neo4j storage finalization **Impact**: Prevents application shutdown failures +### 6. Async Generator Lock Management (CRITICAL) +**Pattern**: Never hold locks across async generator yields - create snapshots instead +**Issue**: Holding locks while yielding causes deadlock when consumers need the same lock +**Location**: `lightrag/tools/migrate_llm_cache.py` - `stream_default_caches_json` +**Solution**: Create snapshot of data while holding lock, release lock, then iterate over snapshot +```python +# WRONG - Deadlock prone: +async with storage._storage_lock: + for key, value in storage._data.items(): + batch[key] = value + if len(batch) >= batch_size: + yield batch # Lock still held! + +# CORRECT - Snapshot approach: +async with storage._storage_lock: + matching_items = [(k, v) for k, v in storage._data.items() if condition] +# Lock released here +for key, value in matching_items: + batch[key] = value + if len(batch) >= batch_size: + yield batch # No lock held +``` +**Impact**: Prevents deadlocks in Json→Json migrations and similar scenarios where source/target share locks +**Applicable To**: Any async generator that needs to access shared resources while yielding + ## Architecture Patterns ### 1. Dependency Injection