[MEDIUM][Reliability] Circuit Breaker State Not Shared Across Client Instances

[filthyrake]

Summary

Each client instance creates its own circuit breaker. In a multi-server setup, if server1 is down, its circuit breaker opens, but server2's circuit breaker doesn't benefit from learning about the failure pattern.

Affected Projects

All projects with circuit breakers

Representative File

• idrac-mcp/src/idrac_client.py (lines 108-114)

Impact

Under cascading failures (e.g., network partition affecting all servers), each client independently discovers the failure through timeouts instead of fast-failing after the first client's circuit opens.

Recommended Fix

Use a shared circuit breaker registry:

_CIRCUIT_BREAKERS: Dict[str, CircuitBreaker] = {}
_CIRCUIT_BREAKER_LOCK = threading.Lock()

def get_circuit_breaker(name: str, **config) -> CircuitBreaker:
    """Get or create a shared circuit breaker."""
    with _CIRCUIT_BREAKER_LOCK:
        if name not in _CIRCUIT_BREAKERS:
            _CIRCUIT_BREAKERS[name] = create_circuit_breaker(name=name, **config)
        return _CIRCUIT_BREAKERS[name]

🤖 Generated with Claude Code