[skyrl-train] IPv6-safe HTTP URLs and bind hosts for inference servers - #2158
Draft
anuragprat1k wants to merge 2 commits into
Draft
[skyrl-train] IPv6-safe HTTP URLs and bind hosts for inference servers#2158anuragprat1k wants to merge 2 commits into
anuragprat1k wants to merge 2 commits into
Conversation
On IPv6-only clusters ray.util.get_node_ip_address() returns a bare IPv6
literal; f"http://{ip}:{port}" then yields e.g. http://2602:fb33::5:8100,
which httpx rejects (InvalidURL: Invalid port) in VLLMServerActor's health
poll, ServerInfo.url, the router URL and the metrics scraper. Add
format_http_url() (brackets IPv6 per RFC 3986) and default_bind_host() ('::'
on IPv6 nodes, '0.0.0.0' otherwise) in inference_servers/common.py and use
them at every site. Companion to NovaSky-AI#612, which fixed the tcp:// rendezvous URLs
in the older code path.
…nit tests `uvx ruff@0.11.9 check` (the pinned version used by the SkyRL-CPU lint job and the pre-commit hook) flagged I001 in the five files touched by the IPv6 change: the new imports were not isort-ordered. Apply `ruff --fix` output only; no behaviour change. Add unit tests for is_ipv6_address(), format_http_url(), default_bind_host() and ServerInfo.url alongside the existing inference_servers/common tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On IPv6-only clusters (e.g. EKS with IPv6 pod networking)
ray.util.get_node_ip_address()returns a bare IPv6 literal. The inference-server stack builds HTTP URLs asf"http://{ip}:{port}", which for2602:fb33::5and port8100yieldshttp://2602:fb33::5:8100; httpx rejects it:raised from
VLLMServerActor.start()'s health poll, so every run on such a cluster dies at inference-engine startup. The same pattern exists inServerInfo.url, the vllm-router URL and the metrics scraper; the servers/router also bind on0.0.0.0, which is not reachable over IPv6.This adds two helpers to
inference_servers/common.pyand uses them at every site:format_http_url(host, port, path="")brackets IPv6 hosts per RFC 3986 (http://[2602:fb33::5]:8100/health); IPv4 and hostnames are unchanged.default_bind_host(node_ip)returns"::"when the node IP is IPv6,"0.0.0.0"otherwise, used for the vLLM server--hostdefault and the router bind host.Behaviour on IPv4 clusters is unchanged (same strings as before). Companion to #612, which fixed the
tcp://rendezvous URLs.Test plan
python -m py_compileon the five touched files againstmain;ruff/blackclean at the repo's pinned versions.format_http_url("10.0.0.5", 8100)==http://10.0.0.5:8100;format_http_url("2602:fb33::5", 8100, "/health")==http://[2602:fb33::5]:8100/health;default_bind_host("2602:fb33::5")==::.skyrl-v0.3.0: vLLM server start, health poll and router come up on the2602:fb33:...pod addresses and the first training step completes. The unpatched tag fails at the same point with thehttpx.InvalidURLabove.tests/backends/skyrl_train/inference_servers/test_common.py(TestIPv6SafeAddressing, 12 cases);pytest tests/backends/skyrl_train/inference_servers/ -m "not vllm"passes (147 tests).