diff --git a/docs/user-guide/.pages b/docs/user-guide/.pages index f3169ab8..7a01a03f 100644 --- a/docs/user-guide/.pages +++ b/docs/user-guide/.pages @@ -13,6 +13,8 @@ nav: - grafana.md - Gateway: - gateway.md + - gateway-peering.md + - gateway-acls.md - gateway-add.md - gateway-failover.md - ... diff --git a/docs/user-guide/external.md b/docs/user-guide/external.md index b77517b9..e0f53653 100644 --- a/docs/user-guide/external.md +++ b/docs/user-guide/external.md @@ -183,7 +183,7 @@ To allow a specific VPC to have access to prefixes reachable via an Edge Device, External VPC Peering via this Fabric object is only supported for BGP-speaking externals or static externals without proxy ARP. For the proxied static external use case, or whenever NAT is required to access the target prefixes, Gateway peering should be used instead: see - [Gateway Peering for External Connections](gateway.md#gateway-peering-for-external-connections). + [Gateway Peering for External Connections](gateway-peering.md#gateway-peering-for-external-connections). ```yaml apiVersion: vpc.githedgehog.com/v1beta1 @@ -367,7 +367,7 @@ route-map HedgeOut permit 10 bgp community-list standard HedgeIn permit 5000:65102 ``` -See [Gateway Peering with NAT for External Connections](gateway.md#gateway-peering-for-external-connections) for examples on how to connect to external networks using NAT. +See [Gateway Peering with NAT for External Connections](gateway-peering.md#gateway-peering-for-external-connections) for examples on how to connect to external networks using NAT. ## Example 2: Static External diff --git a/docs/user-guide/gateway-acls.md b/docs/user-guide/gateway-acls.md new file mode 100644 index 00000000..ddbf0cbf --- /dev/null +++ b/docs/user-guide/gateway-acls.md @@ -0,0 +1,575 @@ +# Gateway Peering ACLs + +A [Gateway Peering](gateway-peering.md) can carry an **Access Control List +(ACL)**: an ordered list of rules that decides, for each new flow or packet, +whether the gateway admits it or drops it. ACLs are configured via the +optional, peering-scoped `spec.acl` field. + +ACLs are **opt-in**: a peering with no `spec.acl` behaves exactly as described +in [Gateway Peering](gateway-peering.md). + +The full list of fields is in the +[Fabric & Gateway APIs reference](../reference/fabric-api.md#peeringacl). + +## Traffic filtering in Gateway Peerings + +Several mechanisms restrict what traffic can flow between the two VPCs of a +Gateway Peering. Before any ACL rule is evaluated, the first mechanism is the +definition of the peering itself, via the prefixes in the `expose` blocks: when +the gateway receives a packet, it attempts to determine its destination VPC by +finding the right peering based on the source VPC, source IP and port, +destination IP and port (ports are used in the case of port-forwarding). If a +packet does not match prefixes from exposed blocks in any peering attached to +its source VPC, it is dropped. + +Some stateful NAT modes, such as masquerade and port-forwarding, add a second +restriction: a flow may only be initiated in one direction. When masquerade is +set-up, a flow can only be initiated from the VPC with the masqueraded +prefixes; a packet sent towards a masqueraded host will be dropped if the +gateway has no associated stateful flow entry in its table, even if the address +is otherwise valid with regards to the peering definition. Conversely, flows +can only be initiated towards, and never from, an endpoint using +port-forwarding. + +**ACL rules** come as a third mechanism, and **add explicit allow/deny** +decisions based on 5-tuples. Traffic restrictions from ACL rules come in +addition to the restrictions implied by the Gateway Peering definition. In +other words, an ACL rule will **never allow any packet that is not permitted by +the prefixes exposed** in the peering configuration. + +## ACL configuration + +To configure an ACL, use field **`spec.acl`** alongside `spec.peering`. The ACL +applies to the whole peering. The YAML definition for the `expose` blocks +remains unchanged. Here is an example: + +```{.yaml .annotate linenums="1" title="gw-acl-peer.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-app--vpc-db + namespace: default +spec: + peering: + vpc-app: + expose: + - ips: + - cidr: 10.0.1.0/24 + as: + - cidr: 192.168.1.0/24 + nat: + masquerade: + idleTimeout: 5m0s + vpc-db: + expose: + - ips: + - cidr: 10.0.2.0/24 + acl: + # Default action when no rule matches; "deny-unless-exposed" if omitted + default: deny + rules: + - name: app-to-db # Optional, used in logs and diagnostics + # Either "from" or "to" may be omitted (not both) + from: vpc-app # Side initiating the flow + to: vpc-db # Side receiving the flow + action: allow # "allow" or "deny" - required + match: + src: # From-side native (pre-NAT) prefix + - cidr: 10.0.1.0/24 + dst: # To-side advertised prefix, as exposed to the peer + - cidr: 10.0.2.0/24 + ports: ["5432"] + proto: tcp # Omit to match any protocol + scope: flow # "flow" (default; match reply traffic) or "packet" + log: false # true or false (default) +``` + +This peering lets hosts in prefix `10.0.1.0/24` from `vpc-app` open TCP +connections on port `5432` to hosts with prefix `10.0.2.0/24` from `vpc-db`, +allows return traffic to go through as well, and nothing else (`default: deny`) +in either direction. + +### Rule evaluation order + +The gateway evaluates rules in `spec.acl.rules` top-down: **the first rule +whose `from`/`to` and `match` pattern match the flow decides the verdict**, and +evaluation stops immediately. If no rule matches, `spec.acl.default` applies. +Conflicts between rules are resolved purely by order, so a broad rule placed +early will shadow the more specific rules that follow it. + +### Rule direction: `from` and `to` + +Every rule specifies a **direction**. Fields **`from`** and **`to`** must each +name one of the peering's two sides, using the same names as the keys under +`spec.peering` (externals included, prefixed with `ext.`). A rule applies to +packets emitted by `from` towards `to`. + +Because a peering always has exactly two sides, one of `from` or `to` may be +omitted, and the other side is implied. Omitting both is not allowed. + +Depending on the `scope` specified for the rule, rules may also apply to +packets of an established flow's reverse direction. Refer to [the section on +rule scope](#rule-scope) for details. + +### Matching patterns + +The `match` block defines what packets (or flows) an ACL rule applies to. + +#### NAT considerations + +The gateway evaluates ACL rules for a packet **before any NAT is applied**. +This means that the patterns in the `match` block are compared with the **real +source address** and the **destination address the initiator used** (typically, +its exposed service address). + +For example, let's consider `vpc-01` exposing `10.0.1.0/24` (without NAT), and +`vpc-02` exposing a private address `10.0.2.10` behind port-forwarding, as +public address `100.64.2.10/32`: + +- To write a rule to allow endpoint `10.0.1.1` on `vpc-01` to reach a server + exposed as a service behind `100.64.2.10` on `vpc-02`, there is no need to + account for the translated address. As ACL rules are evaluated before NAT (in + this case, port-forwarding) takes place, the destination address + `100.64.2.10` is used to write the rule, regardless of what private address + in `vpc-02` the gateway translates it to. + +- To write a rule to forbid the server on `vpc-02` to talk to some endpoint + (say, `10.0.1.4`) on `vpc-01`, the same applies, in the opposite direction: + the ACL rule evaluation comes before NAT, which means we use the private IP + address for the server, `10.0.2.10`, and the plain destination address of the + endpoint at `10.0.1.4`, to write the match pattern (although in the current + case, the server is already forbidden to initiate connections by the peering + definition, because it is exposed using port-forwarding). + +#### Match block + +The `match` block is a 5-tuple, expressed from the initiator's point of view: + +- **`src`** contains the `from` side's native address prefixes and ports +- **`dst`** contains the `to` side's advertised address prefixes and ports + (from its `expose`'s `as`, or `ips` where no NAT applies) +- **`proto`** can be `tcp`, `udp`, or a quoted [protocol number] (for example: + `"132"` for SCTP) + +`src` and `dst` are lists of +[match endpoints](../reference/fabric-api.md#peeringaclmatchendpoint). Each +entry carries an IP prefix and an optional list of ports, where a port entry is +either a single port such as `"80"` (do not omit the quotes) or an inclusive +range such as `"8000-8100"`. The prefix takes the same shape as an `expose` +entry: either a literal `cidr`, or a `vpcSubnet` reference naming a VPC subnet. +At most one of `cidr` and `vpcSubnet` can be set on a given entry. + +Here are some match block examples: + +- Match all UDP traffic towards hosts in prefix `10.0.2.0/24`: + + ```yaml + match: + dst: + - cidr: 10.0.2.0/24 + proto: udp + ``` + +- Match all TCP traffic with destination port 25, for all exposed IP addresses + on the relevant side of the peering: + + ```yaml + match: + dst: + - ports: ["25"] + proto: tcp + ``` + +- Match all TCP traffic from prefixes `10.0.1.0/24` and `10.0.2.0/24`, to + prefix `10.0.5.0/24` with destination port 5432: + + ```yaml + match: + src: + - cidr: 10.0.1.0/24 + - cidr: 10.0.2.0/24 + dst: + - cidr: 10.0.5.0/24 + ports: ["5432"] + proto: tcp + ``` + +- Match UDP traffic from IP address `10.0.2.12` with source port between 8000 + and 8100 inclusive, or equal to 9000, to any valid destination defined by the + peering: + + ```yaml + match: + src: + - cidr: 10.0.2.12/32 + ports: ["8000-8100", "9000"] + proto: udp + ``` + +- Match all traffic between `10.0.1.1` and `10.0.2.1`, regardless of protocol + and ports: + + ```yaml + match: + src: + - cidr: 10.0.1.1/32 + dst: + - cidr: 10.0.2.1/32 + ``` + +- Match all TCP traffic in the rule's direction. Non-TCP traffic does not match + the rule. + + ```yaml + match: + proto: tcp + ``` + +- Match SCTP traffic from `10.0.1.1` to `10.0.2.1`: + + ```yaml + match: + src: + - cidr: 10.0.1.1/32 + dst: + - cidr: 10.0.2.1/32 + proto: "132" + ``` + +Note that source and destination ports can only be used when the `proto` field +is set to `tcp` or `udp`. + +!!! tip + Field `proto` supports values `tcp`, `udp`, or a quoted protocol number. An + `icmp` keyword should be added in the future, with related matching options + such as error types; until then, you can use protocol number `"1"` to match + ICMP packets. + +[protocol number]: https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml + +#### Optional `match` fields + +Everything in `match` is optional, and omitting a field widens the rule: + +| Field | If omitted, then... | +| --- | --- | +| `src`, `dst` | the rule matches any source or destination address and port within the peering configuration, respectively | +| `src.{cidr,vpcSubnet}`, `dst.{cidr,vpcSubnet}` | the rule matches any source or destination IP address within the peering configuration (respectively), but can still be restricted to specified port ranges | +| `src.ports`, `dst.ports` | the rule matches all source or destination ports within the peering configuration, respectively | +| `proto` | the rule matches any protocol | + +If the `match` block is omitted altogether, the rule matches every flow in the +specified direction: this is how to write a generic allow or deny for one +direction. + +### Rule scope + +The **`scope` field** in each rule selects whether the rule is stateful: + +- **`flow`** (the default) makes the rule **bidirectional for established + connections**. Once a flow is admitted, its return traffic is matched by the + flow table entry and allowed even if it does not match any other explicit + allow rule. The connection can still only be _initiated_ in the rule's + direction. +- **`packet`** makes the rule stateless. **Return traffic is not implicitly + allowed** and must be permitted by a separate rule. + +Note that under `default: deny`, a rule with `scope: packet` and no matching +reverse rule admits the forward direction **while dropping every reply**. Add +an explicit rule for the return direction whenever you use `scope: packet` and +need bidirectional traffic. + +Also, note that reply traffic for allow rules with `scope: flow` may still be +dropped if it matches an explicit deny rule in the list: packets are evaluated +as potential "return traffic" only after going through a first direct lookup +against the ACL rules table. + +The scope does not matter in the case of `deny` rules, given that no flow +matching this rule can be initiated in the first place. + +!!! warning + Due to internal limitations in the gateway, `scope: flow` is currently + **restricted** to peerings for which **all flows use masquerade or + port-forwarding**. Flows relying on static NAT, or no NAT at all, do not + currently generate stateful entries in the gateway's table. This limitation + will be lifted in a future release. + +### Default action + +**When no rule matches, the default verdict from `spec.acl.default` is +applied**. The field can take one of two values: + +- **`deny-unless-exposed`** (the default) **denies the packet unless it + corresponds to a defined peering**. This means that the packet is rejected + unless it matches the following conditions: + + 1. Its source VPC, source IP address, and destination address match exposed + prefixes (and source or destination ports match exposed port ranges, in + the case of port-forwarding) for a peering attached to the source VPC. + 2. If the packet attempts to establish a new flow, the NAT modes in use — + as defined in the relevant `expose` blocks for the peering — do not + prevent it. + 3. No deny rule in the ACL matches the packet. + + See also the section about + [traffic filtering in Gateway Peerings](#traffic-filtering-in-gateway-peerings). + If a packet matches all these conditions, then `deny-unless-exposed` allows + it to go through. + +- **`deny`** means **that packets not explicitly allowed by a rule** (or + corresponding to return traffic for an established flow allowed by a rule + with `scope: flow`) **are dropped**. So even a packet that matches the + peering is dropped if no allow rule matches it. + +### Logging + +Set the `log` field of a rule to `true` to log all packets matching that rule +(as well as corresponding reply traffic, for rules with `scope: flow`). The +rule's name, if specified, appears in the log message. + +## Examples + +This section contains some example use cases and configuration for the peering +ACLs. + +### Restricting an exposed service + +VPC `vpc-02` only publishes a backend as `100.64.2.10:443`. + +```{.yaml .annotate linenums="1" title="gw-acl-port-fw-service.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-01--vpc-02 + namespace: default +spec: + peering: + vpc-01: + expose: + - ips: + - cidr: 10.0.1.0/24 # No NAT + vpc-02: + expose: + - ips: + - cidr: 10.0.2.10/32 + as: + - cidr: 100.64.2.10/32 # Expose 10.0.2.10:8443 as 100.64.2.10:443 + nat: + portForward: + ports: + - proto: tcp + port: "8443" + as: "443" + acl: + default: deny # Deny packets even if they match the peering + rules: + - name: vpc-01-workloads-to-web # Name of the rule, for logging + from: vpc-01 # Rule applies to packets from vpc-01 to vpc-02 + to: vpc-02 # (Optional, implied by "from: vpc-01") + action: allow # Allow matching packets + match: + dst: + - cidr: 100.64.2.10/32 # The IP vpc-01 sends to: vpc-02's "as" + ports: ["443"] # Match traffic to port 443 only... + proto: tcp # ... and for TCP only + scope: flow # Allow return traffic for flows +``` + +A flow from `10.0.1.5` to `100.64.2.10:443` is evaluated before NAT, so the +rule matches `vpc-02`'s backend address before port-forwarding takes place. +Based on the `scope`, reply traffic is automatically allowed. + +### Egress allow-list towards an external + +A private subnet reaches the Internet through masquerade, with a strict list of +allowed destination ports, and one carve-out rule placed ahead of it. + +```{.yaml .annotate linenums="1" title="gw-acl-egress.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-internal--ext + namespace: default +spec: + peering: + vpc-internal: + expose: + - ips: + - cidr: 10.0.0.0/16 + as: + - cidr: 198.51.100.7/32 + nat: + masquerade: + idleTimeout: 5m0s + ext.internet: + expose: + - default: true + acl: + default: deny # Deny packets even if they match the peering + rules: + - name: block-blackholed-ranges + from: vpc-internal # "to" is omitted: the external side is implied + action: deny # Deny traffic to the following prefixes/address + match: + dst: + - cidr: 192.0.2.0/24 + - cidr: 198.51.100.66/32 + - name: allow-web + from: vpc-internal + action: allow # Allow TCP traffic to ports 80 and 443, + match: # only for traffic not denied by the first rule + proto: tcp + dst: + - ports: ["80", "443"] + scope: flow # Also allow return traffic + - name: allow-dns + from: vpc-internal + action: allow # Allow UDP traffic to port 53, + match: # only for traffic not denied by the first rule + proto: udp + dst: + - ports: ["53"] + scope: flow # Also allow return traffic +``` + +Match field `src` is omitted throughout, so it matches any address +`vpc-internal` exposes through the peering. The `deny` rule comes first because +the first match decides the verdict: moving `allow-web` above +`block-blackholed-ranges` would allow HTTP/HTTPS towards the blackholed ranges. + +### Bidirectional traffic with static NAT + +In the following setup, `vpc-01` exposes a prefix using static NAT, and uses an +ACL to restrict communications to a few hosts within the prefix. Rules apply +per packet and not per flow (`scope: packet`), meaning that the return traffic +has to be explicitly allowed. + +```{.yaml .annotate linenums="1" title="gw-acl-bidir.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-01--vpc-02 + namespace: default +spec: + peering: + vpc-01: + expose: + - ips: + - cidr: 10.0.1.10/32 + as: + - cidr: 192.168.0.10/32 + nat: + static: {} + vpc-02: + expose: + - ips: + - cidr: 10.0.2.0/24 + acl: + default: deny # Deny packets even if they match the peering + rules: + - from: vpc-01 + action: allow # Allow traffic from 10.0.1.10 (private address, + # before source NAT) to 10.0.2.28 + match: + src: + - cidr: 10.0.1.10/32 + dst: + - cidr: 10.0.2.28/32 + scope: packet # Return traffic is not automatically allowed + - from: vpc-02 + action: allow # Allow traffic from 10.0.2.{12,28} to 192.168.0.10 + # (exposed address, before reverse destination NAT) + match: + src: + - cidr: 10.0.2.12/32 + - cidr: 10.0.2.28/32 + dst: + - cidr: 192.168.0.10/32 + scope: packet # Return traffic is not automatically allowed +``` + +Endpoint `10.0.2.28` from `vpc-02` can communicate with `10.0.1.10` from +`vpc-01` both ways, thanks to the first rule allowing traffic from `10.0.1.10`; +however, endpoint `10.0.2.12` from `vpc-02` can emit towards `10.0.1.10`, but +will not receive any return traffic. + +### Blocking sub-ranges + +Here is a peering where `vpc-01` only exposes portions of subnet `10.0.1.0/24` +to `vpc-02`. This example illustrates how ACL rules can provide finer-grained +control than the `expose` block: it offers control over direction, protocol, +and port ranges. + +```{.yaml .annotate linenums="1" title="gw-acl-subranges.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-01--vpc-02 + namespace: default +spec: + peering: + vpc-01: + expose: + - ips: + - cidr: 10.0.1.0/24 + - not: 10.0.1.64/26 + vpc-02: + expose: + - ips: + - cidr: 10.0.2.0/24 + acl: + default: deny-unless-exposed # Deny unless packets match the peering + rules: + - from: vpc-01 + action: deny # Restrict traffic coming from vpc-01 + match: + src: + - cidr: 10.0.1.128/26 # Block source IP range + - from: vpc-01 + action: deny # Restrict further traffic from vpc-01 + match: + src: + - cidr: 10.0.1.192/27 # Block TCP from 10.0.1.192/27:2000-3000 + ports: ["2000-3000"] + proto: tcp + - from: vpc-02 + action: deny # Also restrict traffic coming from vpc-02 + match: + dst: + - cidr: 10.0.1.234/32 # Block UDP traffic to 10.0.1.234:53 + ports: ["53"] + proto: udp +``` + +With this configuration, when `vpc-01` sends packets towards `vpc-02`: + +- All packets that do not come from `10.0.1.0/24`, or are not addressed to + `10.0.2.0/24`, are dropped, because they do not match the peering definition. +- From `10.0.1.0/26`: Packets match the peering and do not match any deny rule, + so `deny-unless-exposed` lets them pass. +- From `10.0.1.64/26`: Per the peering definition, this sub-prefix is not part + of the exposed block (`not:`), so packets are dropped. +- From `10.0.1.128/26`: Because of the first ACL rule, packets are dropped. +- From `10.0.1.192/27`: + - TCP traffic is dropped for source ports between 2000 and 3000 + (inclusive), because of the second rule. + - TCP traffic from other ports is allowed by `deny-unless-exposed`. + - Non-TCP traffic is allowed by `deny-unless-exposed`. +- From `10.0.1.224/27`: Packets match the peering and do not match any deny + rule, so `deny-unless-exposed` lets them pass. + +And when `vpc-02` sends packets towards `vpc-01`: + +- All packets that do not come from `10.0.2.0/24`, or are not addressed to + `10.0.1.0/24`, are dropped, because they do not match the peering definition. +- To `10.0.1.0/26`: Packets match the peering and do not match any deny rule, + so `deny-unless-exposed` lets them pass. +- To `10.0.1.64/26`: Per the peering definition, this sub-prefix is not part of + the exposed block (`not:`), so packets are dropped. +- To `10.0.1.234`: + - UDP traffic to port 53 is dropped, because of the third rule. + - UDP traffic to other ports is allowed by `deny-unless-exposed`. + - Non-UDP traffic is allowed by `deny-unless-exposed`. +- To other addresses in `10.0.1.0/24`: Packets match the peering and do not + match any deny rule — the first two rules in the list do not apply to traffic + in this direction — so `deny-unless-exposed` lets them pass. diff --git a/docs/user-guide/gateway-peering.md b/docs/user-guide/gateway-peering.md new file mode 100644 index 00000000..6339a7a9 --- /dev/null +++ b/docs/user-guide/gateway-peering.md @@ -0,0 +1,441 @@ +# Gateway Peering + +Just as [VPC Peerings](vpcs.md#vpcpeering) provide VPC-to-VPC connectivity by +way of the switches in the fabric, gateway peerings provide connectivity via +the gateway nodes. Gateway services can be inserted between a pair of VPCs or a +VPC and an external using a Gateway Peering. Each peering can be configured to +provide the necessary services for traffic that uses that peering. + +See the [Gateway overview](gateway.md) for how gateway nodes attach to the fabric and +attract the traffic that these peerings apply to. + +!!! warning + Peering the same entities via gateway and fabric at the same time results in undefined behavior. + If two entities (VPCs or externals) are already peered via the fabric, delete this peering first, + then peer them via the gateway. + +## Simple Gateway Peering Between VPCs + +A simple peering with no services deployed between the VPCs. This traffic will +transit the gateway node(s). + +```{.yaml .annotate linenums="1" title="gw-peer.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-1--vpc-2 + namespace: default +spec: + peering: + vpc-1: + expose: + - ips: + - cidr: 10.0.0.0/24 # Expose all IP address in the 10.0.0.0/24 CIDR block to vpc-2 + vpc-2: + expose: + - ips: + - cidr: 192.168.0.0/16 # Expose all IP addresses in the 192.168.0.0/16 CIDR block to vpc-1 +``` + +Note that multiple VPCs cannot expose overlapping prefixes to a given VPC. For +example, if `vpc-1` and `vpc-3` are both peered with `vpc-2`, and `vpc-1` +exposes subnet `10.1.1.0/24` to `vpc-2`, then `vpc-3` cannot expose overlapping +prefix `10.1.0.0/16` to `vpc-2`. There is one exception to this rule: exposed +prefixes can overlap with an expose block marked as `default`, see section on +[peering for external connections](#gateway-peering-for-external-connections) +for details. + +## Gateway Peering with Static (Stateless) NAT + +Static NAT translates source and/or destination IP addresses for all packets that traverse +the peering, but it does not maintain any flow state for the connection; in other words, +it is stateless. A one-to-one mapping is established between the addresses exposed in the CIDRs for +`ips` and the addresses to use represented by the CIDRs in `as`: each address from +the first group is consistently mapped to a single address from the second group. +Therefore, the total number of addresses covered by the CIDRs YAML array entries +from `ips` must be equal to the total number of addresses covered by the CIDRs from `as`. + +```{.yaml .annotate linenums="1" title="gw-static-nat-peer.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-1--static--vpc-2 + namespace: default +spec: + peering: + vpc-1: + expose: + - ips: + - cidr: 10.0.1.0/24 # IP addresses in the 10.0.1.0/24 block will be exposed ... + as: + - cidr: 10.11.11.0/24 # as IP addresses in the 10.11.11.0/24 block. + nat: + static: {} + - ips: + - cidr: 10.0.2.3/32 # This single IP address will be reachable... + as: + - cidr: 10.11.22.3/32 # as this IP address in vpc-2. + nat: + static: {} + vpc-2: + expose: + - ips: + - cidr: 10.0.2.0/24 # A /24 can be split into two ranges + as: + - cidr: 10.22.22.0/25 # and exposed back to vpc-1. + - cidr: 10.22.22.128/25 + nat: + static: {} +``` + +## Gateway Peering with Stateful NAT + +Stateful NAT uses a flow table to track established connections; the decision on the action +to take for a packet depends on both the configuration of the peering and the flow table state. + +There are two flavors of stateful NAT, depending on in which direction of the traffic the rule applies: +masquerade and port-forwarding. + +!!! note + If one side of a gateway peering (i.e. one VPC or external) uses "stateful + NAT", such as masquerade and/or port forwarding, the other side cannot also + use stateful NAT (but using stateful NAT one one side, and static NAT on + the other side of a gateway peering is supported). + This limitation will be lifted in an upcoming release. + +### Flow Table + +The gateway maintains a **flow table** to track active connections for stateful +NAT (masquerade or port-forwarding). Each unique connection (identified by its +source/destination IPs, ports, and protocol) creates an entry in the flow +table. This entry records the NAT translation to apply and the connection's +idle timer. + +Flow entries expire after a configurable period of inactivity. Set the idle timeout per +peering via the `idleTimeout` field in the NAT configuration; see +[Masquerade](#masquerade-stateful-source-nat) and +[Port-Forwarding](#port-forwarding-stateful-destination-nat) for usage. If `idleTimeout` +is omitted, a built-in default applies — set it explicitly when the timeout matters for +your workload. When a flow expires, its entry is removed and subsequent packets for that +connection are treated as a new flow. + +!!! tip + Use TCP keepalives or application-layer keepalives for long-lived connections through + stateful NAT. This prevents the flow entry from expiring due to inactivity during + idle periods. + +The flow table is maintained by the gateway node. Refer to section +[Stateful Processing](gateway.md#stateful-processing) for guidance about +configuring the flow table capacity, and considerations about gateway +fail-over. + +### Masquerade (Stateful Source NAT) + +Stateful source NAT, also referred to as _masquerade_, uses a flow table to track established connections. +When traffic is initiated from `vpc-1` to `vpc-2`, the flow table is updated +with the connection details. In the return direction (from `vpc-2` to `vpc-1` +in the following example), the flow table is consulted to determine if the packet +is part of an established connection. If it is, the packet is allowed to pass +through the peering. If it is not, the packet is dropped. +This behavior allows the use of masquerading as a simple firewall. + +```{.yaml .annotate linenums="1" title="gw-masquerade-nat-peer.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-1--masquerade--vpc-2 + namespace: default +spec: + peering: + vpc-1: + expose: + # Allow 10.0.0.0/24 addresses to talk to vpc-2 + # Because of stateful source NAT, traffic from vpc-2 to vpc-1 is only allowed if there is + # a flow table entry created by a traffic flow initiated from vpc-1 to vpc-2. + - ips: + - cidr: 10.0.0.0/24 + as: + - cidr: 10.0.1.0/31 # but, NAT those addresses using the addresses in 10.0.1.0/31 + nat: + masquerade: # Stateful source NAT: connections initiated from vpc-1 to vpc-2 will be added to the flow table + idleTimeout: 5m0s # Timeout connections after 5 minutes of inactivity (no packets received) + vpc-2: + expose: + # Allows traffic from vpc-1 to vpc-2 on these addresses. + # Connections must be initiated from vpc-1 to vpc-2 due to flow tracking. + - ips: + - cidr: 192.168.0.0/16 + # Currently, only one of the two VPCs of a peering can use stateful NAT + # (i.e. masquerade and/or port-forwarding). This restriction will be lifted in a future release. +``` + +In this example, a host in `vpc-1` with an IP address in the range `10.0.0.0/24` - for example +`10.0.0.4` - can send packets to a host in `vpc-2` with an address in the range `192.168.0.0/16` - +for example `192.168.5.32`. The host in `vpc-2` will see those packets arrive with a source address +in the range `10.0.1.0/31` - for example `10.0.1.1`; the reverse translation will be done on the gateway +for return traffic, and the hosts are going to be able to communicate with each other. +However, with masquerade, only hosts in `vpc-1` can initiate connections. If that same host behind +`192.168.5.32` in `vpc-2` attempts to connect to that same address `10.0.1.1` before the host in `vpc-1` +has initiated the connection flow - or after the idle timeout has expired without any packet being sent +between the pair - the gateway will drop those packets and the connection will not succeed. + +### Port-Forwarding (Stateful Destination NAT) + +Port-Forwarding is the complement to masquerading, in that it enables connection +flows initiated from the remote side. One use case for this would be +to allow external services to connect to a host in a VPC on a specific port +or range of ports, while rejecting connections outside of that range. + +The following YAML fragment is an example with port forwarding configured on +`vpc-1`'s side: + +```{.yaml .annotate linenums="1" title="gw-pf-peer.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-1--pf--vpc-2 + namespace: default +spec: + peering: + vpc-1: + expose: + - ips: + - cidr: 10.0.1.3/32 # the real IPs of the hosts we want to use internally in the VPC + as: + - cidr: 192.168.11.20/32 # the "public" IPs that can be used to reach those hosts + nat: + portForward: + idleTimeout: 10m0s # Timeout connections after 10 minutes of inactivity (no packets received) + ports: + - proto: tcp # one of tcp or udp, or empty for both + port: "22" # the real port (or port range) on the host where the traffic will be sent + as: "22101" # the port (or port range) the sender will use as the destination on the public IPs above + vpc-2: + expose: + - ips: + - cidr: 10.0.2.0/24 + # Currently, only one of the two VPCs of a peering can use stateful NAT + # (i.e. masquerade and/or port-forwarding). This restriction will be lifted in a future release. +``` + +In this example, hosts in `vpc-2` with an IP address in the range `10.0.2.0/24` can connect +to `192.168.11.20` via TCP on port `22101`, and this will be mapped to port `22` +of host `10.0.1.3` in `vpc-1`, e.g. allowing them to ssh into it. + +The fields `port` and `as` under `ports` accept a string with a single port or a port range; +a range should be specified with the lower and higher bounds (inclusive) separated with a dash. +Spaces are not allowed within the string for ports or port ranges. For example, both `"245"` and +`"435-512"` are valid entries, while `"340-237"`, `340` (with no quotes), `"2,3"`, or `"122 - 234"` are not. + +!!! warning + Port-Forwarding tracks the state of TCP flows. It is recommended to use TCP keepalives + or application-layer keepalives to avoid flows from expiring due to inactivity, which + would cause subsequent packets from those flows to be dropped. + + The default timeout for TCP is 30 minutes, starting from the time the flow reaches the + established state. For UDP, it is 30 seconds, starting from when there's bidirectional + communication; since UDP is stateless, this is less problematic. + +## Gateway Peering for External Connections + +Gateway peerings can also be used to peer a VPC with an [External](external.md). +The following YAML is an example configuration for a peering between VPC `vpc-02` +and External `example-ext`: + +```{.yaml .annotate linenums="1" title="gw-peer-external.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-02--example-ext + namespace: default +spec: + peering: + ext.example-ext: # NOTE the name of the external is prefixed with "ext." + expose: + - default: true # Fallback destination when no other expose matches + vpc-02: + expose: + - ips: + - cidr: 10.50.2.0/24 +``` + +Note that the name of the external is prefixed with `ext.`: this is a +requirement. + +Also, note how the external uses `default: true` instead of specifying the root +IPv4 prefix `0.0.0.0/0`. An `expose` block marked as `default: true` serves as +a default destination for all addresses _that do not otherwise match any of the +other prefixes exposed to the VPC_, whether or not these prefixes are from the +same peering. In our example, if `vpc-02` is additionally peered with `vpc-03` +which exposes `10.50.3.0/24`, all packets from `vpc-02` towards this subnet are +sent to `vpc-03`, and packets addressed to all other subnets from the IPv4 +space are sent to the external `example-ext`. + +For any given VPC, at most one remote `expose` block, among all peerings for +this VPC, can act as a `default` destination. + +## Access Control + +Defining a Gateway Peering implies defining what traffic will be allowed +between two VPCs: an address that is not exposed is not reachable. Using +masquerade or port-forwarding for some `expose` block also sets up a constraint +on the direction for which corresponding flows can be initiated. + +On top of these, a peering can also carry an optional Access Control List, +configured via the peering-scoped `spec.acl` field, to explicitly allow or deny +flows that would otherwise be allowed by the peering. For example, the +following rule denies connections from `vpc-01` to `vpc-02` on TCP port 25, +whereas all other flows in the peering are allowed: + +```{.yaml .annotate linenums="1" title="gw-acl-peer.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: acl-example + namespace: default +spec: + acl: + rules: + - from: vpc-01 # Deny connections from vpc-01 to vpc-02 on TCP port 25, + to: vpc-02 # while everything the peering exposes stays reachable + action: deny + match: + proto: tcp + dst: + - ports: ["25"] + peering: + # ... Peering definition, unchanged +``` + +Rules are evaluated top-down and the first match decides. When no rule matches, +the peering falls back to `spec.acl.default`, which defaults to +`deny-unless-exposed`: traffic that an `expose` block already permits is +admitted. + +See [Gateway Peering ACLs](gateway-acls.md) for more details. + +## Example Use Case: Gateway External Peering with Masquerade and Port-Forwarding + +Let's see how everything we have discussed so far can be used to support a reasonable +external connectivity use case. + +We will assume that our Fabric is connected to an external endpoint providing +Internet connectivity. We want hosts on one of our VPCs to be able to reach the +Internet despite having a private IP address in the VPC subnet range; additionally, +we want to expose some ports on the VPC hosts via a public IP, so that we can connect +to a locally running service from the outside. Naturally, we want to keep the hosts isolated otherwise. + +First, let's create the `External` object and its attachment to a border leaf `leaf-01`. +Let's assume the external is managed by an ISP which does not want to BGP peer with us; +instead, it provides a default route with a /31 nexthop on a VLAN connection. We will use a static external +with proxy-ARP to connect the border leaf to the external, and later we will configure +the gateway peering to masquerade traffic from our VPC hosts using the IP address that the external +believes it is directly connected to, so that the external will see the outgoing traffic as if it was coming +directly from the border leaf. + +```{.yaml .annotate linenums="1" title="static-external.yaml"} +apiVersion: vpc.githedgehog.com/v1beta1 +kind: External +metadata: + name: ext-sp-01 + namespace: default +spec: + ipv4Namespace: default + static: + prefixes: + - 0.0.0.0/0 +--- +apiVersion: vpc.githedgehog.com/v1beta1 +kind: ExternalAttachment +metadata: + name: leaf-01--ext-sp-01 + namespace: default +spec: + connection: leaf-01--external + external: ext-sp-01 + neighbor: {} + static: + proxy: true + remoteIP: 100.1.10.1 # this is the /31 address configured on the external + vlan: 10 + switch: {} +``` + +We also have some VPCs and attachments already created: here are their YAMLs for reference. + +```{.yaml .annotate linenums="1" title="vpc-01.yaml"} +apiVersion: vpc.githedgehog.com/v1beta1 +kind: VPC +metadata: + name: vpc-01 + namespace: default +spec: + ipv4Namespace: default + mode: l3vni + subnets: + subnet-01: + dhcp: + enable: true + range: + end: 10.0.1.255 + start: 10.0.1.2 + gateway: 10.0.1.1 + subnet: 10.0.1.0/24 + vlan: 1001 + vlanNamespace: default +--- +apiVersion: vpc.githedgehog.com/v1beta1 +kind: VPCAttachment +metadata: + name: server-01--unbundled--leaf-01--vpc-01--subnet-01 + namespace: default +spec: + connection: server-01--unbundled--leaf-01 + subnet: vpc-01/subnet-01 +``` + +Let's assume `server-01` has received the IP address `10.0.1.2/32` from the DHCP server. +We want to create a gateway peering between `vpc-01` and the external `ext-sp-01`, such that: + +- the external advertises a "default route" to the Internet, mapping any prefix for which there +is no explicit expose; this is done via the `default` flag on the external side +- traffic from the VPC towards the external is masqueraded using the address that the external believes +it is directly connected to, i.e. `100.1.10.0`, and the reverse translation is done for return traffic +- traffic from the external directed to the public IP address above and TCP port `22101` will be +mapped to the IP address of `server-01` and TCP port `22` +- all other inbound traffic to `server-01`, whether via its private IP or the public IP `100.1.10.0`, +will be rejected + +```{.yaml .annotate linenums="1" title="gw-peering.yaml"} +apiVersion: gateway.githedgehog.com/v1alpha1 +kind: GatewayPeering +metadata: + name: vpc-01--ext-sp-01 + namespace: default +spec: + gatewayGroup: default + peering: + ext.ext-sp-01: + expose: + - default: true + vpc-01: + expose: + - as: + - cidr: 100.1.10.0/32 + ips: + - cidr: 10.0.1.0/24 + nat: + masquerade: + idleTimeout: 2m0s + - as: + - cidr: 100.1.10.0/32 + ips: + - cidr: 10.0.1.2/32 + nat: + portForward: + idleTimeout: 2m0s + ports: + - proto: tcp + port: "22" + as: "22101" +``` diff --git a/docs/user-guide/gateway.md b/docs/user-guide/gateway.md index cd8f4ea7..d4272b00 100644 --- a/docs/user-guide/gateway.md +++ b/docs/user-guide/gateway.md @@ -103,401 +103,38 @@ style Leaves fill:none,stroke:none style Servers fill:none,stroke:none ``` -## Flow Table and Stateful Processing +## Stateful Processing -When stateful NAT (masquerade or port-forwarding) is configured on a gateway peering, -the gateway maintains a **flow table** to track active connections. Each unique connection -(identified by its source/destination IPs, ports, and protocol) creates an entry in the -flow table. This entry records the NAT translation applied and the connection's idle timer. +The gateway is a stateful device. When stateful NAT (masquerade or +port-forwarding) is configured on a gateway peering, the gateway maintains a +**flow table** in which each active connection is associated to an entry +recording the NAT translation applied and the connection's idle timer. -Key characteristics of the flow table: +The flow table can handle millions of concurrent entries depending on the +gateway node's available memory. The maximum number of flow entries can be +configured via the `flowTableCapacity` field in the Gateway spec. In most +deployments, the default is sufficient. -- **Timeout-based eviction**: Flow entries expire after a configurable period of inactivity. - Set the idle timeout per peering via the `idleTimeout` field in the NAT configuration; - see [Masquerade](#masquerade-stateful-source-nat) and - [Port-Forwarding](#port-forwarding-stateful-destination-nat) for usage. If `idleTimeout` - is omitted, a built-in default applies — set it explicitly when the timeout matters for - your workload. When a flow expires, its entry is removed and subsequent packets for that - connection are treated as a new flow. -- **Capacity**: The flow table can handle millions of concurrent entries depending on the gateway - node's available memory. The maximum number of flow entries can be configured via the - `flowTableCapacity` field in the Gateway spec. In most deployments, the default is sufficient. -- **Per-gateway state**: Each gateway maintains its own flow table independently. Flow state - is not shared between gateways. If a gateway fails and traffic is redirected to a backup - gateway (see [Gateway fail-over](gateway-failover.md)), existing stateful connections must - be re-established, as the backup gateway has no knowledge of the failed gateway's flow table. +Each gateway maintains its own flow table independently; flow state is not +shared between gateways. If a gateway fails and traffic is redirected to a +backup gateway (see [Gateway fail-over](gateway-failover.md)), existing +stateful connections must be re-established, as the backup gateway has no +knowledge of the failed gateway's flow table. -!!! tip - Use TCP keepalives or application-layer keepalives for long-lived connections through - stateful NAT. This prevents the flow entry from expiring due to inactivity during - idle periods. +Flow entries expire after a configurable period of inactivity. For the +per-peering idle timeouts that govern entries eviction, see +[Flow Table](gateway-peering.md#flow-table). ## Gateway Peering Just as [VPC Peerings](vpcs.md#vpcpeering) provide VPC-to-VPC connectivity by way of the switches in the fabric, gateway peerings provide connectivity via the gateway nodes. Gateway services can be inserted between a pair of VPCs or a VPC and an external using a Gateway Peering. Each peering can be configured to provide the necessary services for traffic -that uses that peering. +that uses that peering. -!!! warning - Peering the same entities via gateway and fabric at the same time results in undefined behavior. - If two entities (VPCs or externals) are already peered via the fabric, delete this peering first, - then peer them via the gateway. +See [Gateway Peering](gateway-peering.md) for the configuration of peerings, +including static and stateful NAT, peering with externals, and external +connectivity examples. -### Simple Gateway Peering Between VPCs - -A simple peering with no services deployed between the VPCs. This traffic will -transit the gateway node(s). - -```{.yaml .annotate linenums="1" title="gw-peer.yaml"} -apiVersion: gateway.githedgehog.com/v1alpha1 -kind: GatewayPeering -metadata: - name: vpc-1--vpc-2 - namespace: default -spec: - peering: - vpc-1: - expose: - - ips: - - cidr: 10.0.0.0/24 # Expose all IP address in the 10.0.0.0/24 CIDR block to vpc-2 - vpc-2: - expose: - - ips: - - cidr: 192.168.0.0/16 # Expose all IP addresses in the 192.168.0.0/16 CIDR block to vpc-1 -``` - -Note that multiple VPCs cannot expose overlapping prefixes to a given VPC. For -example, if `vpc-1` and `vpc-3` are both peered with `vpc-2`, and `vpc-1` -exposes subnet `10.1.1.0/24` to `vpc-2`, then `vpc-3` cannot expose overlapping -prefix `10.1.0.0/16` to `vpc-2`. There is one exception to this rule: exposed -prefixes can overlap with an expose block marked as `default`, see section on -[peering for external connections](#gateway-peering-for-external-connections) -for details. - -### Gateway Peering with Static (Stateless) NAT - -Static NAT translates source and/or destination IP addresses for all packets that traverse -the peering, but it does not maintain any flow state for the connection; in other words, -it is stateless. A one-to-one mapping is established between the addresses exposed in the CIDRs for -`ips` and the addresses to use represented by the CIDRs in `as`: each address from -the first group is consistently mapped to a single address from the second group. -Therefore, the total number of addresses covered by the CIDRs YAML array entries -from `ips` must be equal to the total number of addresses covered by the CIDRs from `as`. - -```{.yaml .annotate linenums="1" title="gw-static-nat-peer.yaml"} -apiVersion: gateway.githedgehog.com/v1alpha1 -kind: GatewayPeering -metadata: - name: vpc-1--static--vpc-2 - namespace: default -spec: - peering: - vpc-1: - expose: - - ips: - - cidr: 10.0.1.0/24 # IP addresses in the 10.0.1.0/24 block will be exposed ... - as: - - cidr: 10.11.11.0/24 # as IP addresses in the 10.11.11.0/24 block. - nat: - static: {} - - ips: - - cidr: 10.0.2.3/32 # This single IP address will be reachable... - as: - - cidr: 10.11.22.3/32 # as this IP address in vpc-2. - nat: - static: {} - vpc-2: - expose: - - ips: - - cidr: 10.0.2.0/24 # A /24 can be split into two ranges - as: - - cidr: 10.22.22.0/25 # and exposed back to vpc-1. - - cidr: 10.22.22.128/25 - nat: - static: {} -``` - -### Gateway Peering with Stateful NAT - -Stateful NAT uses a flow table to track established connections; the decision on the action -to take for a packet depends on both the configuration of the peering and the flow table state. - -There are two flavors of stateful NAT, depending on in which direction of the traffic the rule applies: -masquerade and port-forwarding. - -!!! note - If one side of a gateway peering (i.e. one VPC or external) uses "stateful NAT", such - as masquerade and/or port forwarding, the other side cannot use NAT of any type. - This limitation will be lifted in an upcoming release. - -#### Masquerade (Stateful Source NAT) - -Stateful source NAT, also referred to as _masquerade_, uses a flow table to track established connections. -When traffic is initiated from `vpc-1` to `vpc-2`, the flow table is updated -with the connection details. In the return direction (from `vpc-2` to `vpc-1` -in the following example), the flow table is consulted to determine if the packet -is part of an established connection. If it is, the packet is allowed to pass -through the peering. If it is not, the packet is dropped. -This behavior allows the use of masquerading as a simple firewall. - -```{.yaml .annotate linenums="1" title="gw-masquerade-nat-peer.yaml"} -apiVersion: gateway.githedgehog.com/v1alpha1 -kind: GatewayPeering -metadata: - name: vpc-1--masquerade--vpc-2 - namespace: default -spec: - peering: - vpc-1: - expose: - # Allow 10.0.0.0/24 addresses to talk to vpc-2 - # Because of stateful source NAT, traffic from vpc-2 to vpc-1 is only allowed if there is - # a flow table entry created by a traffic flow initiated from vpc-1 to vpc-2. - - ips: - - cidr: 10.0.0.0/24 - as: - - cidr: 10.0.1.0/31 # but, NAT those addresses using the addresses in 10.0.1.0/31 - nat: - masquerade: # Stateful source NAT: connections initiated from vpc-1 to vpc-2 will be added to the flow table - idleTimeout: 5m0s # Timeout connections after 5 minutes of inactivity (no packets received) - vpc-2: - expose: - # Allows traffic from vpc-1 to vpc-2 on these addresses. - # Connections must be initiated from vpc-1 to vpc-2 due to flow tracking. - - ips: - - cidr: 192.168.0.0/16 - # Currently, only one of the two VPCs of a peering can use stateful NAT - # (i.e. masquerade and/or port-forwarding). This restriction will be lifted in a future release. -``` - -In this example, a host in `vpc-1` with an IP address in the range `10.0.0.0/24` - for example -`10.0.0.4` - can send packets to a host in `vpc-2` with an address in the range `192.168.0.0/16` - -for example `192.168.5.32`. The host in `vpc-2` will see those packets arrive with a source address -in the range `10.0.1.0/31` - for example `10.0.1.1`; the reverse translation will be done on the gateway -for return traffic, and the hosts are going to be able to communicate with each other. -However, with masquerade, only hosts in `vpc-1` can initiate connections. If that same host behind -`192.168.5.32` in `vpc-2` attempts to connect to that same address `10.0.1.1` before the host in `vpc-1` -has initiated the connection flow - or after the idle timeout has expired without any packet being sent -between the pair - the gateway will drop those packets and the connection will not succeed. - -#### Port-Forwarding (Stateful Destination NAT) - -Port-Forwarding is the complement to masquerading, in that it enables connection -flows initiated from the remote side. One use case for this would be -to allow external services to connect to a host in a VPC on a specific port -or range of ports, while rejecting connections outside of that range. - -The following YAML fragment is an example with port forwarding configured on -`vpc-1`'s side: - -```{.yaml .annotate linenums="1" title="gw-pf-peer.yaml"} -apiVersion: gateway.githedgehog.com/v1alpha1 -kind: GatewayPeering -metadata: - name: vpc-1--pf--vpc-2 - namespace: default -spec: - peering: - vpc-1: - expose: - - ips: - - cidr: 10.0.1.3/32 # the real IPs of the hosts we want to use internally in the VPC - as: - - cidr: 192.168.11.20/32 # the "public" IPs that can be used to reach those hosts - nat: - portForward: - idleTimeout: 10m0s # Timeout connections after 10 minutes of inactivity (no packets received) - ports: - - proto: tcp # one of tcp or udp, or empty for both - port: "22" # the real port (or port range) on the host where the traffic will be sent - as: "22101" # the port (or port range) the sender will use as the destination on the public IPs above - vpc-2: - expose: - - ips: - - cidr: 10.0.2.0/24 - # Currently, only one of the two VPCs of a peering can use stateful NAT - # (i.e. masquerade and/or port-forwarding). This restriction will be lifted in a future release. -``` - -In this example, hosts in `vpc-2` with an IP address in the range `10.0.2.0/24` can connect -to `192.168.11.20` via TCP on port `22101`, and this will be mapped to port `22` -of host `10.0.1.3` in `vpc-1`, e.g. allowing them to ssh into it. - -The fields `port` and `as` under `ports` accept a string with a single port or a port range; -a range should be specified with the lower and higher bounds (inclusive) separated with a dash. -Spaces are not allowed within the string for ports or port ranges. For example, both `"245"` and -`"435-512"` are valid entries, while `"340-237"`, `340` (with no quotes), `"2,3"`, or `"122 - 234"` are not. - -!!! warning - Port-Forwarding tracks the state of TCP flows. It is recommended to use TCP keepalives - or application-layer keepalives to avoid flows from expiring due to inactivity, which - would cause subsequent packets from those flows to be dropped. - - The default timeout for TCP is 30 minutes, starting from the time the flow reaches the - established state. For UDP, it is 30 seconds, starting from when there's bidirectional - communication; since UDP is stateless, this is less problematic. - -### Gateway Peering for External Connections - -Gateway peerings can also be used to peer a VPC with an [External](external.md). -The following YAML is an example configuration for a peering between VPC `vpc-02` -and External `example-ext`: - -```{.yaml .annotate linenums="1" title="gw-peer-external.yaml"} -apiVersion: gateway.githedgehog.com/v1alpha1 -kind: GatewayPeering -metadata: - name: vpc-02--example-ext - namespace: default -spec: - peering: - ext.example-ext: # NOTE the name of the external is prefixed with "ext." - expose: - - default: true # Fallback destination when no other expose matches - vpc-02: - expose: - - ips: - - cidr: 10.50.2.0/24 -``` - -Note that the name of the external is prefixed with `ext.`: this is a -requirement. - -Also, note how the external uses `default: true` instead of specifying the root -IPv4 prefix `0.0.0.0/0`. An `expose` block marked as `default: true` serves as -a default destination for all addresses _that do not otherwise match any of the -other prefixes exposed to the VPC_, whether or not these prefixes are from the -same peering. In our example, if `vpc-02` is additionally peered with `vpc-03` -which exposes `10.50.3.0/24`, all packets from `vpc-02` towards this subnet are -sent to `vpc-03`, and packets addressed to all other subnets from the IPv4 -space are sent to the external `example-ext`. - -For any given VPC, at most one remote `expose` block, among all peerings for -this VPC, can act as a `default` destination. - -## Example Use Case: Gateway External Peering with Masquerade and Port-Forwarding - -Let's see how everything we have discussed so far can be used to support a reasonable -external connectivity use case. - -We will assume that our Fabric is connected to an external endpoint providing -Internet connectivity. We want hosts on one of our VPCs to be able to reach the -Internet despite having a private IP address in the VPC subnet range; additionally, -we want to expose some ports on the VPC hosts via a public IP, so that we can connect -to a locally running service from the outside. Naturally, we want to keep the hosts isolated otherwise. - -First, let's create the `External` object and its attachment to a border leaf `leaf-01`. -Let's assume the external is managed by an ISP which does not want to BGP peer with us; -instead, it provides a default route with a /31 nexthop on a VLAN connection. We will use a static external -with proxy-ARP to connect the border leaf to the external, and later we will configure -the gateway peering to masquerade traffic from our VPC hosts using the IP address that the external -believes it is directly connected to, so that the external will see the outgoing traffic as if it was coming -directly from the border leaf. - -```{.yaml .annotate linenums="1" title="static-external.yaml"} -apiVersion: vpc.githedgehog.com/v1beta1 -kind: External -metadata: - name: ext-sp-01 - namespace: default -spec: - ipv4Namespace: default - static: - prefixes: - - 0.0.0.0/0 ---- -apiVersion: vpc.githedgehog.com/v1beta1 -kind: ExternalAttachment -metadata: - name: leaf-01--ext-sp-01 - namespace: default -spec: - connection: leaf-01--external - external: ext-sp-01 - neighbor: {} - static: - proxy: true - remoteIP: 100.1.10.1 # this is the /31 address configured on the external - vlan: 10 - switch: {} -``` - -We also have some VPCs and attachments already created: here are their YAMLs for reference. - -```{.yaml .annotate linenums="1" title="vpc-01.yaml"} -apiVersion: vpc.githedgehog.com/v1beta1 -kind: VPC -metadata: - name: vpc-01 - namespace: default -spec: - ipv4Namespace: default - mode: l3vni - subnets: - subnet-01: - dhcp: - enable: true - range: - end: 10.0.1.255 - start: 10.0.1.2 - gateway: 10.0.1.1 - subnet: 10.0.1.0/24 - vlan: 1001 - vlanNamespace: default ---- -apiVersion: vpc.githedgehog.com/v1beta1 -kind: VPCAttachment -metadata: - name: server-01--unbundled--leaf-01--vpc-01--subnet-01 - namespace: default -spec: - connection: server-01--unbundled--leaf-01 - subnet: vpc-01/subnet-01 -``` - -Let's assume `server-01` has received the IP address `10.0.1.2/32` from the DHCP server. -We want to create a gateway peering between `vpc-01` and the external `ext-sp-01`, such that: - -- the external advertises a "default route" to the Internet, mapping any prefix for which there -is no explicit expose; this is done via the `default` flag on the external side -- traffic from the VPC towards the external is masqueraded using the address that the external believes -it is directly connected to, i.e. `100.1.10.0`, and the reverse translation is done for return traffic -- traffic from the external directed to the public IP address above and TCP port `22101` will be -mapped to the IP address of `server-01` and TCP port `22` -- all other inbound traffic to `server-01`, whether via its private IP or the public IP `100.1.10.0`, -will be rejected - -```{.yaml .annotate linenums="1" title="gw-peering.yaml"} -apiVersion: gateway.githedgehog.com/v1alpha1 -kind: GatewayPeering -metadata: - name: vpc-01--ext-sp-01 - namespace: default -spec: - gatewayGroup: default - peering: - ext.ext-sp-01: - expose: - - default: true - vpc-01: - expose: - - as: - - cidr: 100.1.10.0/32 - ips: - - cidr: 10.0.1.0/24 - nat: - masquerade: - idleTimeout: 2m0s - - as: - - cidr: 100.1.10.0/32 - ips: - - cidr: 10.0.1.2/32 - nat: - portForward: - idleTimeout: 2m0s - ports: - - proto: tcp - port: "22" - as: "22101" -``` +See [Gateway Peering ACLs](gateway-acls.md) for restricting the traffic that a +peering admits. diff --git a/docs/user-guide/overview.md b/docs/user-guide/overview.md index 9a1d5eb3..deaf90ce 100644 --- a/docs/user-guide/overview.md +++ b/docs/user-guide/overview.md @@ -6,7 +6,9 @@ This chapter gives an overview of the main features of Hedgehog Fabric and their - [**Connections**](connections.md): Describes the different types of network links, such as workload server connections, switch interconnects, and external peering links. - [**VPCs and Namespaces**](vpcs.md): Explains virtual private cloud configurations, subnet management, and peering mechanisms. - [**External Peering**](external.md): Details how to connect the fabric to external networks and services through border leaf peering. -- [**Gateway**](gateway.md): Explains gateway configuration and gateway based Peerings for NAT and firewall functionality. +- [**Gateway**](gateway.md): Explains how gateway nodes attach to the fabric and how they process traffic. +- [**Gateway Peering**](gateway-peering.md): Details gateway based Peerings for NAT and firewall functionality. +- [**Gateway Peering ACLs**](gateway-acls.md): Explains how to restrict the traffic admitted by a gateway peering. - [**Adding a gateway to Fabric**](gateway-add.md): Explains how to add a gateway to a Fabric. - [**Gateway fail-over**](gateway-failover.md): Explains fail-over strategies when using multiple gateways. - [**Fabric Shrink/Expand**](shrink-expand.md): Guides users on adding, removing, or replacing switches within the fabric. diff --git a/docs/user-guide/vpcs.md b/docs/user-guide/vpcs.md index 3111f3b3..252aa696 100644 --- a/docs/user-guide/vpcs.md +++ b/docs/user-guide/vpcs.md @@ -119,7 +119,7 @@ It is possible to configure DHCP relay for a VPC subnet towards a DHCP server in - configure `spec.subnets..dhcp.relay` with the address of the host where the DHCP server will be running - configure `spec.subnets..dhcp.relayVPC` with the name of the VPC where the DHCP server lives - make sure that the DHCP server has a route back to the client's VPC subnet -- create a [VPC Peering](#vpcpeering) or [Gateway Peering](gateway.md#gateway-peering) between the VPCs of the DHCP server and client +- create a [VPC Peering](#vpcpeering) or [Gateway Peering](gateway-peering.md) between the VPCs of the DHCP server and client The relay sets two sub-options of DHCP Option 82 (RelayAgentInfo) in the packet: diff --git a/docs/vlab/demo.md b/docs/vlab/demo.md index 7f5a5acb..f94f66d8 100644 --- a/docs/vlab/demo.md +++ b/docs/vlab/demo.md @@ -580,7 +580,7 @@ hhfab vlab setup-peerings 1+2:gw ``` Alternatively, you can create the peering manually in the control node, using the examples in the -[gateway section of the user guide](../user-guide/gateway.md#gateway-peering) as a base, e.g.: +[Gateway Peering section of the user guide](../user-guide/gateway-peering.md) as a base, e.g.: ``` core@control-1 ~ $ cat < vpc-01--vpc-02--gw.yaml diff --git a/includes/abbreviations.md b/includes/abbreviations.md index 93ecbe8f..4cb026f3 100644 --- a/includes/abbreviations.md +++ b/includes/abbreviations.md @@ -20,3 +20,5 @@ *[DHCP]: Dynamic Host Configuration Protocol *[BGP]: Border Gateway Protocol *[ESLAG]: Ethernet Segment Link Aggregation (RFC 7432) +*[NAT]: Network Address Translation +*[ACL]: Access Control List