Skip to content

Abhishek/fix api gw peering#23454

Merged
LordAbhishek merged 3 commits intomainfrom
abhishek/api-gateway-peering
Apr 22, 2026
Merged

Abhishek/fix api gw peering#23454
LordAbhishek merged 3 commits intomainfrom
abhishek/api-gateway-peering

Conversation

@LordAbhishek
Copy link
Copy Markdown
Contributor

@LordAbhishek LordAbhishek commented Apr 13, 2026

This PR adds api gateway peering unit test cases and fixes couple of issues discussed below with “api gateway when operating in peering”.
Please note: This PR merges 3 other small PRs.
#23369
#23370
#23371

Issues details (Theoretically): API Gateway issue with peering and fix.docx

Details:

  1. Whenever we generate the endpoints config for api-gateway, within makeUpstreamLoadAssignmentForPeerService (xds/endpoints.go), it always fetch local_mesh_gateway_endpoint only from connect-proxy, instead of api-gateway cfgSnap.
    localGw, ok := cfgSnap.ConnectProxy.WatchedLocalGWEndpoints.Get(cfgSnap.Locality.String()).

    If not fixed, api-gateway’s cluster for peering will always endup with no endpoints, when meshGatewayConfig.Mode is local.

    Fixed it to fetch the endpoints based on cfgSnap kind. PR

  2. Whenever we generate the endpoint config for api-gateway, within endpointsFromDiscoveryChain (xds/endpoints.go), it fetches meshGatewayConfig to configure endpoints, if it needs to point to local meshGateway or remote meshGateway.
    Here also, this method always fetches meshGatewayConfig.Mode from connect-proxy, instead of api-gateway. So, mgwMode is always nil and nil defaults to remote meshGatewayConfig.Mode.
    So, even if we configure the mesh-gateway to local, it will always fallback to remote.

    Fixed it to fetch the meshGatewayConfig based on cfgSnap kind. PR

  3. In continuation of issue 2, where we have to fetch the meshGatewayConfig for api-gateway, we never have had the meshGatewayConfig in api-gateway upstreams. 

    Found that handleRouteConfigUpdate (proxycfg/api_gateway.go) which updates the upstreams for api-gateway, was skipping to propagate meshGatewayConfig to upstreams. 

    Fixed it by adding meshGatewayConfig within upstream object. PR

  4. Whenever we generate the cluster config for api-gateway, within makeUpstreamClustersForDiscoveryChain (xds/clusters.go), it always generates the cluster without endpoints.
    As we know that whenever any service upstream endpoints is hostname, we have to send it via clusters config to CDS, because envoy is not able to resolve hostnames as EDS. 

    If not fixed, this would results envoy config without endpoints for some clusters (those whose upstream are of hostname type).

    Fixed it to generate cluster based on upstream endpoint type as done for connect proxy or mesh gateway. PR

How the fix is tested:

Unit test cases:
$ go test ./agent/xds -run 'TestAllResourcesFromSnapshot/.*/api-gateway-with-peers-mesh-mode-(local|remote)-and-upstream-is-(hostname|static)' -count=1 -update

ok github.com/hashicorp/consul/agent/xds 0.491s
$ go test ./agent/xds
ok github.com/hashicorp/consul/agent/xds 1.924s
$ go test ./agent/proxycfg
ok github.com/hashicorp/consul/agent/proxycfg 5.105s

Consul-k8s acceptance tests:

consul-ent PR (same as this one):
Ran the consul-k8s workflow with the patched consul-ent image.
All peering test ran fine on both EKS and Kind.
Kind test: https://github.com/hashicorp/consul-k8s-workflows/actions/runs/22376072101
EKS test: https://github.com/hashicorp/consul-k8s-workflows/actions/runs/22376066354

@github-actions github-actions Bot added theme/telemetry Anything related to telemetry or observability theme/envoy/xds Related to Envoy support labels Apr 13, 2026
@LordAbhishek LordAbhishek changed the title Add api gateway peering unit test cases. (#23366) Abhishek/fix api gw peering Apr 13, 2026
@LordAbhishek LordAbhishek force-pushed the abhishek/api-gateway-peering branch from e61bd4b to 535a23e Compare April 13, 2026 03:52
@LordAbhishek LordAbhishek added backport/all Apply backports for all active releases per .release/versions.hcl backport/ent/1.18 Changes are backported to 1.18 ent backport/ent/1.21 changes are backported to 1.21 ent backport/ent/1.22 Changes are backported to 1.22 ent and removed theme/telemetry Anything related to telemetry or observability labels Apr 13, 2026
@LordAbhishek LordAbhishek marked this pull request as ready for review April 13, 2026 04:25
@LordAbhishek LordAbhishek requested review from a team as code owners April 13, 2026 04:25
@LordAbhishek LordAbhishek force-pushed the abhishek/api-gateway-peering branch from 028ef23 to 38997ae Compare April 13, 2026 05:12
@LordAbhishek LordAbhishek force-pushed the abhishek/api-gateway-peering branch from 38997ae to 5d6ffac Compare April 22, 2026 07:45
@LordAbhishek LordAbhishek force-pushed the abhishek/api-gateway-peering branch from 5d6ffac to e9fb247 Compare April 22, 2026 07:57
Copy link
Copy Markdown
Contributor

@anandmukul93 anandmukul93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM . need to see if all acceptance tests pass. and need to rerun the multiport tests as well post this.

* add unit test cases for api gateway supporting consul peering

* add compiled xds config golden files after running api-gw golden testcases

* fix(proxyCfg): propogate meshGatewayConfig to upstreams of API gateway (#23369)

* fix(proxyCfg): propogate meshGatewayConfig to upstreams of API gateway

- Fixed handleRouteConfigUpdate to properly propagate meshGatewayConfig to API gateway upstreams, which is required during XDS endpoint and cluster config generation.
- Added TestStateChangedAPIGateway test cases in state_test.go to validate API gateway update handling.
- Added API gateway-specific logging prefix (similar to mesh gateway) to help in debugging.

* fix(xds): correct endpoint config generation for API gateway in peered setups (#23370)

* fix(xds): correct endpoint config generation for API gateway in peered setups

- Previously, API gateway XDS endpoint generation incorrectly relied on cfgSnap.ConnectProxy config (instead of cgfSnap.APIGateway), which caused wrong/no endpoint configuration for peered environments.
- Changes made:
- Updated makeUpstreamLoadAssignmentForPeerService to fetch localGatewayEndpoint based on cfgSnap kind instead of always using cfgSnap.ConnectProxy.
- Updated endpointsFromDiscoveryChain to derive meshGatewayMode based on cfgSnap kind instead of always using cfgSnap.ConnectProxy.
- Recompiled golden test file to reflect fix.

* removed comment

* fix(xds): correct cluster config generation for API gateway in peered setups (#23371)

* fix(xds): correct cluster config generation for API gateway in peered setups

- Updated makeUpstreamClustersForDiscoveryChain to generate cluster config based on upstream endpoint type. Before this fix, it always generated cluster configs without endpoints, which is incorrect when the upstream endpoint type is hostname and mesh-gateway mode is remote; in such cases, endpoints must also be included in the cluster config.

- Added recompiled golden test file to reflect the fix.

* fix lint error
@LordAbhishek LordAbhishek merged commit 34b661a into main Apr 22, 2026
160 of 163 checks passed
@hc-github-team-consul-core hc-github-team-consul-core added the backport/2.0 Changes are backported to 2.0 label Apr 22, 2026
@hc-github-team-consul-core
Copy link
Copy Markdown
Collaborator

📣 Hi @LordAbhishek! a backport is missing for this PR [23454] for versions [1.18,1.21] please perform the backport manually and add the following snippet to your backport PR description:

<details>
	<summary> Overview of commits </summary>
		- <<backport commit 1>>
		- <<backport commit 2>>
		...
</details>

@LordAbhishek LordAbhishek removed the backport/ent/1.18 Changes are backported to 1.18 ent label Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport/all Apply backports for all active releases per .release/versions.hcl backport/ent/1.21 changes are backported to 1.21 ent backport/ent/1.22 Changes are backported to 1.22 ent backport/2.0 Changes are backported to 2.0 theme/envoy/xds Related to Envoy support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants