Support dynamic client certificate resolution (ResolvesClientCert) by brucearctor · Pull Request #1340 · temporalio/sdk-rust

brucearctor · 2026-06-21T04:35:54Z

Support dynamic client certificate resolution (ResolvesClientCert)

What

Add a client_cert_resolver field to TlsOptions that accepts an Arc<dyn ResolvesClientCert> for dynamic, per-handshake client certificate selection. This enables transparent mTLS certificate rotation without requiring a process restart — useful for short-lived certificates managed by Vault agents, cert-manager sidecars, or HSM-backed signers.

Closes #1338

Why

The existing client_tls_options requires static (cert, key) bytes at connection time. Users with rotating certificates (e.g., Vault-issued certs with 24h TTLs) must restart the entire Temporal client to pick up new material. The Go SDK solves this with tls.Config.GetClientCertificate; this PR brings equivalent functionality to the Rust SDK.

How

Since tonic 0.14.6 does not expose rustls::ClientConfig::with_client_cert_resolver(), the implementation bypasses tonic's TLS layer when a resolver is present:

build_custom_rustls_config() — manually constructs a rustls::ClientConfig with the user's ResolvesClientCert, replicating tonic's security defaults (protocol versions, ALPN, native/custom CA roots)
DynamicTlsConnector — a tower::Service<Uri> that wraps TCP with TLS using tokio_rustls::TlsConnector, including connect timeouts and tracing
Endpoint::connect_with_connector() — used instead of connect() to wire the custom connector into tonic's channel

The resolver fires on each new TLS handshake (reconnections), not per-RPC over an existing HTTP/2 connection.

Key design decisions

Mutually exclusive with static certs — setting both client_tls_options and client_cert_resolver returns InvalidConfig
Re-exports — ResolvesClientCert, CertifiedKey, and SignatureScheme are re-exported at the crate root so users don't need to depend on tokio-rustls directly
DNS load balancing — returns a clear error (not yet supported with dynamic certs due to balance_channel API constraints)
Proxy — returns a clear error (not yet supported; would require composing connectors)
C bridge — hardcodes client_cert_resolver: None (dynamic resolution from C callers needs a callback design in a follow-up)

Changes

File	Change
`crates/client/src/options_structs.rs`	Add `client_cert_resolver` field, update `Debug` impl
`crates/client/src/lib.rs`	`TlsConfigResult` enum, `add_tls_to_channel` branching, `build_custom_rustls_config`, `DynamicTlsConnector`, proxy/DNS validation, tracing
`crates/client/src/dns.rs`	Handle `TlsConfigResult`, reject resolver + DNS LB
`crates/client/src/envconfig.rs`	Add `client_cert_resolver: None`
`crates/client/Cargo.toml`	Add `rustls-native-certs` dep
`crates/sdk-core-c-bridge/src/client.rs`	Add `client_cert_resolver: None`
`crates/sdk-core/tests/common/mod.rs`	Update test struct literals
`CHANGELOG.md`	Add entry under `[Unreleased] > Added`

Tests (15 new, 134 total pass)

Custom connector returned when resolver is set (explicit domain + inferred domain)
Mutual exclusion: static + dynamic certs → error
Resolver + custom ServerCertVerifier combination
Resolver + custom CA certificate
build_custom_rustls_config with/without resolver/verifier
Debug output correctness for TlsOptions with resolver
DynamicTlsConnector is Clone + Debug
Default TlsOptions has no resolver
No-TLS passthrough returns Standard
IP host fallback for domain extraction
Re-exports compile (CertifiedKey, SignatureScheme)

Future work

SDK-owned convenience trait (decouple users from rustls types) — Provide SDK-owned trait for dynamic client certificates (decouple from rustls) #1341
Connection max-lifetime for proactive cert rotation — Add connection max-lifetime for proactive mTLS certificate rotation #1342
DNS load balancing + dynamic certs — Support dynamic client cert resolver with DNS load balancing #1343
C bridge callback support — Expose dynamic client cert resolver via C bridge for language SDKs #1344
FileWatchingCertResolver batteries-included implementation — Provide batteries-included FileWatchingCertResolver for common mTLS rotation #1345

…ClientCert) Add a new client_cert_resolver field to TlsOptions that accepts an Arc<dyn ResolvesClientCert> for dynamic per-handshake client certificate resolution. This enables transparent mTLS certificate rotation without requiring a process restart -- useful for short-lived certificates managed by Vault agents, sidecars, or HSMs. Key changes: - Add client_cert_resolver: Option<Arc<dyn ResolvesClientCert>> to TlsOptions (mutually exclusive with static client_tls_options) - Re-export ResolvesClientCert from the crate root for ergonomic use - When a resolver is set, bypass tonic's static Identity path and build a rustls::ClientConfig manually with with_client_cert_resolver() - Introduce DynamicTlsConnector (tower::Service<Uri>) that performs TLS via tokio_rustls::TlsConnector with the custom config - Use Endpoint::connect_with_connector() to wire it into tonic's channel - Add validation: error when both static and dynamic client certs are set - DNS load balancing returns a clear error for now (not yet supported with dynamic cert resolution) - Update C bridge and envconfig with client_cert_resolver: None Tests (13 new): - Mutual exclusion validation (static + dynamic = error) - CustomConnector result with explicit and inferred domain - Resolver + custom ServerCertVerifier combination - Resolver + custom CA certificate - build_custom_rustls_config with/without resolver/verifier - Debug output for TlsOptions with resolver - DynamicTlsConnector Clone requirement - Default TlsOptions has no resolver - No-TLS passthrough still returns Standard Closes: temporalio#1338

Fixes from 4-reviewer deep code review (Temporal, Rust, Systems, Security): Must Fix: - Proxy + cert resolver: add validation to prevent silent discard of resolver when http_connect_proxy is also set (was silently ignoring the dynamic cert resolver) - Empty SNI domain: fail early with clear error instead of producing an empty string that causes a confusing late error from ServerName - Localhost fallback: DynamicTlsConnector no longer falls back to "localhost" when URI has no host — returns a clear error instead - Connect timeout: wrap TCP connect in 30s timeout to prevent hanging on unreachable hosts (tonic's built-in connector handles this but custom connectors must do it themselves) - CA cert parsing: check roots.is_empty() after add_parsable_certificates to catch cases where all provided CA certs are malformed - Dead code: remove unused CountingCertResolver test helper Should Fix: - Tracing: add debug! logging to DynamicTlsConnector for TCP connect and TLS handshake completion - Re-exports: add CertifiedKey and SignatureScheme re-exports so users can implement ResolvesClientCert without adding tokio-rustls directly - Debug impl: add manual Debug for DynamicTlsConnector showing domain - Nodelay comment: document why set_nodelay(true) is set

chris-olszewski · 2026-06-30T16:42:24Z

Will be giving this a review very soon, thanks for your patience.

brucearctor · 2026-06-30T16:58:47Z

i'm in no rush

chris-olszewski

Overall I think this design makes sense to me. The largest thing for me is making sure we feature gate it.

brucearctor · 2026-07-07T19:32:04Z

@chris-olszewski - i think addressed?

chris-olszewski

Looks good to me. Only a few questions on some of the new tests and a suggestion that we mark this variant as experimental. Thanks for your patience on the review process.

chris-olszewski · 2026-07-30T13:46:39Z

+            #[test]
+            fn dynamic_tls_connector_is_clone_and_debug() {
+                // Verify DynamicTlsConnector is Clone (required by tonic) and Debug
+                let config = build_custom_rustls_config(
+                    &TlsOptions::default(),
+                    Some(Arc::new(MockClientCertResolver)),
+                )
+                .unwrap();
+                let server_name =
+                    tokio_rustls::rustls::pki_types::ServerName::try_from("test.temporal.io")
+                        .unwrap()
+                        .to_owned();
+                let connector = DynamicTlsConnector {
+                    tls: tokio_rustls::TlsConnector::from(Arc::new(config)),
+                    domain: Arc::new(server_name),
+                };
+                let _cloned = connector.clone();
+                let debug_str = format!("{:?}", connector);
+                assert!(
+                    debug_str.contains("DynamicTlsConnector"),
+                    "Debug should show struct name: {debug_str}"
+                );
+                assert!(
+                    debug_str.contains("test.temporal.io"),
+                    "Debug should show domain: {debug_str}"
+                );
+            }


Unsure the value of a test asserting that these traits are implemented

Suggested change

#[test]

fn dynamic_tls_connector_is_clone_and_debug() {

// Verify DynamicTlsConnector is Clone (required by tonic) and Debug

let config = build_custom_rustls_config(

&TlsOptions::default(),

Some(Arc::new(MockClientCertResolver)),

)

.unwrap();

let server_name =

tokio_rustls::rustls::pki_types::ServerName::try_from("test.temporal.io")

.unwrap()

.to_owned();

let connector = DynamicTlsConnector {

tls: tokio_rustls::TlsConnector::from(Arc::new(config)),

domain: Arc::new(server_name),

};

let _cloned = connector.clone();

let debug_str = format!("{:?}", connector);

assert!(

debug_str.contains("DynamicTlsConnector"),

"Debug should show struct name: {debug_str}"

);

assert!(

debug_str.contains("test.temporal.io"),

"Debug should show domain: {debug_str}"

);

}

chris-olszewski · 2026-07-30T15:21:58Z

+            #[test]
+            fn re_exports_are_accessible() {
+                // Verify that CertifiedKey and SignatureScheme are re-exported
+                // and usable from the crate root. This is a compile-time check.
+                fn _assert_types_exist(
+                    _resolver: &dyn ResolvesClientCert,
+                    _scheme: SignatureScheme,
+                ) {
+                    // This function just needs to compile
+                }
+                let _ = SignatureScheme::ECDSA_NISTP256_SHA256;
+            }
+        }


I don't believe the test asserts what it says since use super::* gets parent's imports, not just exports. Directly referencing these via crate:: is probably what is desired, but unsure if testing this is necessary.

chris-olszewski · 2026-07-30T15:25:37Z

+    /// Standard tonic TLS was applied, endpoint is ready to connect normally.
+    Standard(Endpoint),
+    /// A custom rustls::ClientConfig is needed. The endpoint has no TLS configured;
+    /// the caller must use `connect_with_connector` with a custom TLS connector.


Suggested change

/// the caller must use `connect_with_connector` with a custom TLS connector.

/// the caller must use `connect_with_connector` with a custom TLS connector.

///

/// Experimental API subject to change

brucearctor added 2 commits June 20, 2026 21:10

brucearctor requested a review from a team as a code owner June 21, 2026 04:35

brucearctor added 2 commits June 22, 2026 16:16

Merge branch 'main' into feat/dynamic-client-certs

921cce3

Merge branch 'main' into feat/dynamic-client-certs

22f5d19

chris-olszewski requested changes Jul 5, 2026

View reviewed changes

Comment thread crates/client/src/options_structs.rs Outdated

Comment thread crates/client/Cargo.toml Outdated

Comment thread crates/client/src/lib.rs Outdated

fix(client): feature-gate dynamic cert resolver behind 'dynamic-tls'

9e434fb

chris-olszewski self-requested a review July 29, 2026 20:25

chris-olszewski requested changes Jul 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support dynamic client certificate resolution (ResolvesClientCert) - #1340

Support dynamic client certificate resolution (ResolvesClientCert)#1340
brucearctor wants to merge 5 commits into
temporalio:mainfrom
brucearctor:feat/dynamic-client-certs

brucearctor commented Jun 21, 2026 •

edited

Loading

Uh oh!

chris-olszewski commented Jun 30, 2026

Uh oh!

brucearctor commented Jun 30, 2026

Uh oh!

chris-olszewski left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brucearctor commented Jul 7, 2026

Uh oh!

chris-olszewski left a comment

Uh oh!

chris-olszewski Jul 30, 2026

Uh oh!

chris-olszewski Jul 30, 2026

Uh oh!

chris-olszewski Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

brucearctor commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How

Key design decisions

Changes

Tests (15 new, 134 total pass)

Future work

Uh oh!

chris-olszewski commented Jun 30, 2026

Uh oh!

brucearctor commented Jun 30, 2026

Uh oh!

chris-olszewski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brucearctor commented Jul 7, 2026

Uh oh!

chris-olszewski left a comment

Choose a reason for hiding this comment

Uh oh!

chris-olszewski Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

chris-olszewski Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

chris-olszewski Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

brucearctor commented Jun 21, 2026 •

edited

Loading