feat(infra): add cost optimization and performance variables for Cloud Run and Redis VPC connector

[saurabh-google]

Motivation

Custom Data Commons setups already provide knobs like cpu_idle to optimize infrastructure costs for dev and low-traffic environments. Bringing these same customization options to DCP enables teams to reduce baseline cloud spend while maintaining fine-grained control over performance.

Summary

This PR exposes configurable Terraform variables to enable cost optimization and performance tuning for DCP (Data Commons Platform) deployments.

Changes

1. Cloud Run Resource Controls (cpu_idle & startup_cpu_boost):

   • Added cpu_idle (default false) and startup_cpu_boost (default true) configurations to datacommons_services and ingestion/helper_service Cloud Run services.
   • Setting cpu_idle = true allows CPU to only be allocated during request processing, reducing running costs for lower-traffic environments.
   • Enabling startup_cpu_boost mitigates cold start latencies during container boots.

2. Redis VPC Access Connector Scaling:

   • Parameterized vpc_connector_min_instances (default 2) and vpc_connector_max_instances (default 10) in the redis module.
   • Allows deployments to adjust baseline VPC connector instance counts to optimize infrastructure costs.

3. Templates & Variables:

   • Added commented examples in terraform.tfvars.template.
   • Threaded configurations through root variables.tf, main.tf, and the stack module with backwards-compatible defaults.

Testing

• Validated Terraform syntax and module wiring across modified modules.

[gemini-code-assist]

Code Review

This pull request introduces configuration options for CPU idle mode and startup CPU boost for Cloud Run services (datacommons_services and ingestion_helper), as well as configurable scaling parameters (min_instances and max_instances) for the Redis VPC Access Connector. The review feedback highlights that the newly added cpu_idle and startup_cpu_boost variables in the ingestion_helper module are not threaded through the parent stack module, rendering them unconfigurable. Additionally, it is recommended to add Terraform validation blocks to both the root and module-level VPC connector instance variables to enforce GCP's strict limit requirements (minimum between 2 and 9, maximum between 3 and 10) during the planning phase.

[dwnoble]

Thank you @saurabh-google !

Were you able to test this in a live environment?

[saurabh-google]

Thank you @saurabh-google !

Were you able to test this in a live environment?

I have tested it for a live deployment and I can see the Cloudrun > Revision History > Billing change to Request-based from Instance-based.