feat: Enhanced Health Checks with Dependency Status

[remo-lab]

Description

This PR adds a proper readiness check to Hyperledger Fabric peers while keeping the existing liveness (/healthz) behavior unchanged.

Today, /healthz mainly tells whether the process is running. In Kubernetes and production setups, it’s often useful to know whether a peer is actually ready to receive traffic (ledger open, gossip initialized, etc.). This change separates those concerns in a safe, backward-compatible way.

What changed

1. Added a new /readyz endpoint for readiness probing
2. Kept /healthz exactly as it is (liveness only)
3. Introduced a small readiness handler (without touching vendored code)
4. Added optional readiness checkers:

• Gossip: checks that the gossip service is initialized and (optionally) connected to peers
• Ledger: checks that ledgers are available and readable
• Orderer: optional connectivity check (disabled by default)

5. Added an optional /healthz/detailed endpoint with component-level status

• Disabled by default
• Protected by operations TLS / client auth
• Uses normalized states: OK, DEGRADED, UNAVAILABLE
  All new checks are opt-in and use conservative defaults to avoid false negatives

Why this is useful

• Works cleanly with Kubernetes liveness and readiness probes
• Avoids sending traffic to peers that aren’t ready yet
• Makes partial failures easier to observe without forcing restarts
• Improves operational visibility without changing core behavior

Backward compatibility

• No behavior change to existing /healthz
• No breaking config changes
• Readiness failures only happen if explicitly enabled
• Ledger lag and orderer connectivity checks are disabled by default

Configuration (example)

operations:
  healthCheck:
    gossip:
      enabled: true
      minPeers: 0
    ledger:
      enabled: true
      failOnLag: false
    orderer:
      enabled: false
    detailedHealth:
      enabled:  false

Tests

• Unit tests for readiness handler and individual checkers
• Coverage for readiness vs liveness behavior
• Basic validation for failure and success cases

screenshot of terminal after the feat is implemented:

Notes

This is intentionally scoped to readiness and safety.
Stricter checks (ledger lag enforcement, channel-level readiness, etc.) can be added incrementally later if needed.
Feedback welcome — happy to adjust based on review.