* feat: enhance coverage reporting and improve tool configuration - Added support for JSON coverage reports in pyproject.toml. - Updated .gitignore to include coverage.json and task files for better management. - Introduced a new Type Safety Audit Report to document findings and recommendations for type safety improvements. - Created a comprehensive coverage configuration guide to assist in understanding coverage reporting setup. - Refactored tools configuration to utilize environment variables for concurrent scraping settings. These changes improve the project's testing and reporting capabilities while enhancing overall code quality and maintainability. * feat: enhance configuration handling and improve error logging - Introduced a new utility function `_get_env_int` for robust environment variable integer retrieval with validation. - Updated `WebToolsConfig` and `ToolsConfigModel` to utilize the new utility for environment variable defaults. - Enhanced logging in `CircuitBreaker` to provide detailed state transition information. - Improved URL handling in `url_analyzer.py` for better file extension extraction and normalization. - Added type validation and logging in `SecureInputMixin` to ensure input sanitization and validation consistency. These changes improve the reliability and maintainability of configuration management and error handling across the codebase. * refactor: update imports and enhance .gitignore for improved organization - Updated import paths in various example scripts to reflect the new structure under `biz_bud`. - Enhanced .gitignore to include clearer formatting for task files. - Removed obsolete function calls and improved error handling in several scripts. - Added public alias for backward compatibility in `upload_r2r.py`. These changes improve code organization, maintainability, and compatibility across the project. * refactor: update graph paths in langgraph.json for improved organization - Changed paths for research, catalog, paperless, and url_to_r2r graphs to reflect new directory structure. - Added new entries for analysis and scraping graphs to enhance functionality. These changes improve the organization and maintainability of the graph configurations. * fix: enhance validation and error handling in date range and scraping functions - Updated date validation in UserFiltersModel to ensure date values are strings. - Improved error messages in create_scraped_content_dict to clarify conditions for success and failure. - Enhanced test coverage for date validation and scraping content creation to ensure robustness. These changes improve input validation and error handling across the application, enhancing overall reliability. * refactor: streamline graph creation and enhance type annotations in examples - Simplified graph creation in `catalog_ingredient_research_example.py` and `catalog_tech_components_example.py` by directly compiling the graph. - Updated type annotations in `catalog_intel_with_config.py` for improved clarity and consistency. - Enhanced error handling in catalog data processing to ensure robustness against unexpected data types. These changes improve code readability, maintainability, and error resilience across example scripts. * Update src/biz_bud/nodes/extraction/extractors.py Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Update src/biz_bud/core/validation/pydantic_models.py Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * refactor: migrate Jina and Tavily clients to use ServiceFactory dependency injection * refactor: migrate URL processing to provider-based architecture with improved error handling * feat: add FirecrawlApp compatibility classes and mock implementations * fix: add thread-safe locking to LazyLoader factory management * feat: implement service restart and refactor cache decorator helpers * refactor: move r2r_direct_api_call to tools.clients.r2r_utils and improve HTTP service error handling * chore: update Sonar task IDs in report configuration --------- Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
119 lines
3.9 KiB
Python
119 lines
3.9 KiB
Python
"""Example demonstrating the new succinct logging configuration.
|
|
|
|
This shows how the new logging filters reduce verbosity while maintaining
|
|
meaningful information in the logs.
|
|
"""
|
|
|
|
import logging
|
|
import time
|
|
from pathlib import Path
|
|
|
|
# Import the logging configuration
|
|
from biz_bud.logging import get_logger, setup_logging
|
|
|
|
|
|
def simulate_verbose_logs():
|
|
"""Simulate the verbose logs that were problematic."""
|
|
setup_logging(level="DEBUG")
|
|
logger = get_logger("example_app")
|
|
|
|
# Simulate LangGraph queue stats (these will be filtered)
|
|
for i in range(20):
|
|
logger.info(
|
|
f"Stats(queue_id='test-queue-{i}', worker_id='worker-1', "
|
|
f"timestamp=datetime.datetime(2025, 1, 9, 10, 30, {i}, 123456))"
|
|
)
|
|
|
|
# Simulate HTTP retries (only first and last will show)
|
|
for attempt in range(1, 6):
|
|
logger.warning(f"Retrying HTTP request: attempt {attempt}/5")
|
|
time.sleep(0.1)
|
|
|
|
# Simulate timeout warnings (will be reduced)
|
|
for i in range(10):
|
|
logger.warning(f"Request timeout after 30 seconds - iteration {i}")
|
|
|
|
# Simulate service initialization (only first will show)
|
|
for service in ["auth", "database", "cache", "api"]:
|
|
logger.info(f"Initializing {service} service...")
|
|
logger.info(f"Service {service} initialized successfully")
|
|
|
|
|
|
def demonstrate_new_features():
|
|
"""Demonstrate the new logging features."""
|
|
setup_logging(level="INFO")
|
|
logger = get_logger("demo")
|
|
|
|
logger.info("Starting demonstration of succinct logging")
|
|
|
|
# The new filters will:
|
|
# 1. Show only every 10th queue stat
|
|
# 2. Show only first and last retry
|
|
# 3. Show timeout warnings every 5th occurrence
|
|
# 4. Show service init only once per service
|
|
# 5. Simplify datetime formatting
|
|
|
|
simulate_verbose_logs()
|
|
|
|
logger.info("Verbose logs have been filtered for clarity")
|
|
|
|
|
|
def demonstrate_yaml_config():
|
|
"""Show how to use YAML configuration for fine-grained control."""
|
|
# Try to load YAML config if available
|
|
config_path = (
|
|
Path(__file__).parent.parent
|
|
/ "packages"
|
|
/ "business-buddy-core"
|
|
/ "src"
|
|
/ "bb_core"
|
|
/ "logging"
|
|
/ "logging_config.yaml"
|
|
)
|
|
|
|
if config_path.exists():
|
|
try:
|
|
# YAML config loading not available in current implementation
|
|
logger = get_logger("config")
|
|
logger.info("Loaded YAML logging configuration")
|
|
except Exception as e:
|
|
logger = get_logger("config")
|
|
logger.warning(f"Could not load YAML config: {e}")
|
|
|
|
# Now logs will follow the YAML configuration rules
|
|
|
|
|
|
def demonstrate_custom_log_levels():
|
|
"""Show how to adjust logging levels programmatically."""
|
|
# Set different levels for different components
|
|
setup_logging(level="INFO")
|
|
|
|
# You can also set specific loggers
|
|
langgraph_logger = logging.getLogger("langgraph")
|
|
langgraph_logger.setLevel(logging.ERROR) # Only show errors from LangGraph
|
|
|
|
httpx_logger = logging.getLogger("httpx")
|
|
httpx_logger.setLevel(logging.ERROR) # Only show errors from httpx
|
|
|
|
|
|
if __name__ == "__main__":
|
|
print("=== Succinct Logging Configuration Demo ===\n")
|
|
|
|
print("1. Demonstrating filtered verbose logs:")
|
|
demonstrate_new_features()
|
|
|
|
print("\n2. Demonstrating YAML configuration:")
|
|
demonstrate_yaml_config()
|
|
|
|
print("\n3. Demonstrating custom log levels:")
|
|
demonstrate_custom_log_levels()
|
|
|
|
print("\n=== Demo Complete ===")
|
|
print("\nKey improvements:")
|
|
print("- LangGraph queue stats: Shown every 10th occurrence")
|
|
print("- HTTP retries: Only first and last attempts shown")
|
|
print("- Timeout warnings: Shown every 5th occurrence")
|
|
print("- Service init messages: Shown only once")
|
|
print("- Datetime objects: Simplified to readable format")
|
|
print("- HTTP client logs: Reduced to WARNING level by default")
|