Remove readiness cache

[timmarkhuff]

Bug

After a detector's inference pod becomes ready, inference requests would intermittently fail with:

Edge predictions are required, but an edge-inference server is not available for detector_id='...'

Failures lasted ~3-5 seconds then stopped on their own. This was most visible immediately after gl.edge.set_config() returns, since that's when pods freshly roll out, but the underlying issue could surface any time pod readiness state changed (e.g. after a model update rollout).

Root cause

is_edge_inference_ready() was decorated with a 5-second per-process TTL cache. The server runs 8 uvicorn workers, each with its own private cache instance. When a pod becomes ready, whichever worker happens to check first caches True -- but the other workers still hold a stale False for up to 5 more seconds. Image queries round-robining across those workers got 503s from any worker that hadn't yet refreshed its cached negative result.

Fix

Remove the cache and the pre-flight inference_is_available() check entirely. Instead of checking /health/ready before every request, just fire the inference POST directly and handle RuntimeError on failure. This is faster on the success path (one fewer HTTP round trip per query) and eliminates the stale-cache window.

The /edge-detector-readiness endpoint (used by set_config() polling) now calls a new check_inference_ready() free function that hits /health/ready live with no caching, so readiness reporting is always accurate.

A 1-second connect timeout was added to inference POSTs so that requests to a K8s Service with no ready endpoints fail quickly rather than hanging indefinitely.

Testing

Added load-testing/fresh_pod_readiness_test.py, which reproduced the failure reliably (11/194 requests failed in a 10s burst immediately after set_config() returned). After this fix the same script should declare victory with zero failures.

[timmarkhuff]

Moving these functions to groundlight_helpers.py so that they can be reused.