Skip to content

OT-14 chore setup terraform and cicd #238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโ€™ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
arlen02-01 merged 10 commits into from
May 17, 2026
Merged

OT-14 chore setup terraform and cicd #238

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
853ab90
[OT-14] [CHORE]: deploy-ai AWS ์ธ์ฆ์„ OIDC role assume ๋ฐฉ์‹์œผ๋กœ ์ „ํ™˜
arlen02-01 May 15, 2026
cb4d1e7
[OT-14] [FEAT]: deploy-ec2-docker ์ธํ”„๋ผ ํŒŒ๋ผ๋ฏธํ„ฐยทDB ๋น„๋ฐ€ ์กฐํšŒ ๊ธฐ๋ฐ˜ ๋ฐฐํฌ๋กœ ๊ฐœ์„ 
arlen02-01 May 15, 2026
97608ad
[OT-14] [FEAT]: monitoring ๋ฐฐํฌ ํƒ€๊นƒ ์ž๋™ ํ•ด์„ ๋ฐ Grafana Secret Manager ์—ฐ๋™
arlen02-01 May 15, 2026
23aed85
[OT-14] [FEAT]: rabbitmq ์‚ฌ์šฉ์ž ์„ค์ •์„ Secrets Manager ๋™๊ธฐํ™” ๋ฐฉ์‹์œผ๋กœ ์ „ํ™˜
arlen02-01 May 15, 2026
7061e4c
[OT-14] [CHORE]: dev ์ธํ”„๋ผ ๊ธฐ์ค€ ๋ฐฐํฌ ์›Œํฌํ”Œ๋กœ์šฐ ์ •ํ•ฉ์„ฑ ๋ฐ ๋ณด์•ˆ ๊ฐœ์„ 
arlen02-01 May 15, 2026
3e12138
[OT-14] [FIX]: deploy-ai monitoring ์กฐํšŒ๋ฅผ ํ”„๋กœ์ ํŠธ/ํ™˜๊ฒฝ ํƒœ๊ทธ๋กœ ํ•œ์ •
arlen02-01 May 15, 2026
c456878
[OT-14] fix(ci): harden SSM deploys and align workflows with terraforโ€ฆ
arlen02-01 May 15, 2026
b254580
[OT-14] fix(ci): align ai env parameter path with dev scoped ssm
arlen02-01 May 15, 2026
4abe536
[OT-14] [CHORE]: deploy-ec2-docker OIDC ๋””๋ฒ„๊ทธ ๊ฒ€์ฆ ์Šคํ… ์ถ”๊ฐ€
arlen02-01 May 15, 2026
d6cd836
[OT-14] ci: wait for ec2 running and ssm online before send-command
arlen02-01 May 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .github/workflows/deploy-ai.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ on:
types:
- closed

permissions:
id-token: write
contents: read

env:
AWS_REGION: ap-northeast-2
SERVICE_NAME: machine
Expand All @@ -28,8 +32,7 @@ jobs:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
role-to-assume: ${{ secrets.AWS_GITHUB_ACTIONS_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}

- name: Login to ECR
Expand Down Expand Up @@ -73,8 +76,7 @@ jobs:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
role-to-assume: ${{ secrets.AWS_GITHUB_ACTIONS_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
Comment on lines 76 to 80

@coderabbitai coderabbitai Bot May 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

โš ๏ธ Potential issue | ๐ŸŸ  Major | ๐Ÿ—๏ธ Heavy lift

๐Ÿงฉ Analysis chain

๐Ÿ Script executed:

cat -n .github/workflows/deploy-ai.yml | head -100

Repository: OpenTheTaste/backend

Length of output: 4350

๐Ÿ Script executed:

cat -n .github/workflows/deploy-ai.yml | sed -n '200,280p'

Repository: OpenTheTaste/backend

Length of output: 4593

๐ŸŒ Web query:

aws-actions configure-aws-credentials@v4 default role-duration-seconds OIDC session

๐Ÿ’ก Result:

In the aws-actions/configure-aws-credentials action (including v4), the role-duration-seconds input controls the duration of the assumed role session in seconds [1][2]. Key details regarding this setting include: Default Value: The default duration is 1 hour (3600 seconds) [1][3]. Configurability: You can adjust this value by providing an integer to the role-duration-seconds input [1][3]. Constraints: The requested duration cannot exceed the MaxSessionDuration setting configured for the specific IAM Role in AWS [1][4]. Acceptable values generally range from 15 minutes (900 seconds) to 12 hours (43200 seconds), depending on the IAM role's configuration [3][5]. OIDC Context: When using OIDC to assume a role, the same 1-hour default applies [6][7]. If you require a session longer than 1 hour, you must explicitly set role-duration-seconds to your desired value (up to the role's maximum) [6]. If you encounter an error stating that the requested duration exceeds the maximum, ensure that the IAM Role's MaxSessionDuration in the AWS console or via CLI/API is set high enough to accommodate your requested duration [4][5].

Citations:

OIDC ์„ธ์…˜ ๊ธธ์ด๋ฅผ ๋ฐฐํฌ ์ตœ๋Œ€ ์‹œ๊ฐ„์— ๋งž์ถฐ ๋ช…์‹œํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

ํ˜„์žฌ configure-aws-credentials ์•ก์…˜์˜ ๊ธฐ๋ณธ OIDC ์„ธ์…˜ ๊ธฐ๊ฐ„์€ 3,600์ดˆ(1์‹œ๊ฐ„)์ด์ง€๋งŒ, ๋ผ์ธ 235์˜ SSM ๋ช…๋ น ํƒ€์ž„์•„์›ƒ์€ 7,200์ดˆ์ด๊ณ  ๋ผ์ธ 244-270์˜ ํด๋ง ๋ฃจํ”„๋„ ์ตœ๋Œ€ 7,200์ดˆ๊นŒ์ง€ ์ง€์†๋ฉ๋‹ˆ๋‹ค. ์ด ๊ฒฝ์šฐ ๋ฐฐํฌ๊ฐ€ 1์‹œ๊ฐ„์„ ์ดˆ๊ณผํ•˜๋ฉด ๋ผ์ธ 245, 259, 273์˜ AWS API ํ˜ธ์ถœ์ด ExpiredToken ์˜ค๋ฅ˜๋กœ ์‹คํŒจํ•ฉ๋‹ˆ๋‹ค. role-duration-seconds๋ฅผ 7,200์ดˆ ์ด์ƒ์œผ๋กœ ์„ค์ •ํ•˜๊ณ , ํ•ด๋‹น IAM ์—ญํ• ์˜ MaxSessionDuration์ด ์ถฉ๋ถ„ํžˆ ๊ตฌ์„ฑ๋˜์–ด ์žˆ๋Š”์ง€ ํ™•์ธํ•˜์„ธ์š”.

๐Ÿค– Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/deploy-ai.yml around lines 76 - 80, The OIDC session
length is too short (default 3600s) for your deployment which can exceed that;
update the GitHub Action step that uses aws-actions/configure-aws-credentials@v4
(the "Configure AWS credentials" step) to set with: role-duration-seconds to at
least 7200 (or a bit higher to match your SSM/polling timeouts) and verify the
target IAM role's MaxSessionDuration is configured to the same or greater value
so AssumeRole tokens won't expire during the long-running SSM/polling
operations.


- name: Deploy AI service to EC2 via SSM
Expand Down
66 changes: 46 additions & 20 deletions .github/workflows/deploy-ec2-docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ on:
types:
- closed

permissions:
id-token: write
contents: read

env:
AWS_REGION: ap-northeast-2

Expand All @@ -36,8 +40,7 @@ jobs:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
role-to-assume: ${{ secrets.AWS_GITHUB_ACTIONS_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}

- name: Login to ECR
Expand All @@ -55,6 +58,7 @@ jobs:
run: |
IMAGE_TAG="${IMAGE_TAG_INPUT:-${GITHUB_SHA}}"
IMAGE_URI="${ECR_REGISTRY}/${{ matrix.ecr_repo }}:${IMAGE_TAG}"

IMAGE_URI_LATEST="${ECR_REGISTRY}/${{ matrix.ecr_repo }}:latest"

docker build \
Expand All @@ -75,8 +79,7 @@ jobs:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
role-to-assume: ${{ secrets.AWS_GITHUB_ACTIONS_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}

- name: Deploy to EC2 instances via SSM
Expand All @@ -85,28 +88,49 @@ jobs:
IMAGE_TAG_INPUT: ${{ github.event.inputs.image_tag }}
PROJECT_NAME: oplust
DB_NAME: oplust
SSM_RDS_ENDPOINT_PARAM: /oplust/common/rds-endpoint
SSM_DB_USERNAME_PARAM: /oplust/common/db-username
SSM_DB_PASSWORD_PARAM: /oplust/common/db-password
RDS_INSTANCE_IDENTIFIER: oplust-dev-db
NODE_EXPORTER_TARGET_SSM_PARAM: /oplust/monitoring/targets/node-exporter
Comment thread
arlen02-01 marked this conversation as resolved.
ECS_CLUSTER_NAME: fluffy-flamingo-5ag1uq
ECS_SERVICE_NAME: oplust-transcoder-service
SSM_ECS_CLUSTER_NAME_PARAM: /oplust/dev/lambda/worker/ecs_cluster_name
SSM_ECS_SERVICE_NAME_PARAM: /oplust/dev/lambda/worker/ecs_service_name
TRANSCODER_REPOSITORY: oplust-transcoder
TRANSCODER_CONTAINER_NAME: oplust-transcoder
SCALER_LAMBDA_NAME: oplust-transcoder-scaler
ENV_NAME: dev
run: |
set -euo pipefail

IMAGE_TAG="${IMAGE_TAG_INPUT:-${GITHUB_SHA}}"

ECS_CLUSTER_NAME=$(aws ssm get-parameter --region "$AWS_REGION" --name "$SSM_ECS_CLUSTER_NAME_PARAM" --with-decryption --query 'Parameter.Value' --output text)
ECS_SERVICE_NAME=$(aws ssm get-parameter --region "$AWS_REGION" --name "$SSM_ECS_SERVICE_NAME_PARAM" --with-decryption --query 'Parameter.Value' --output text)
SCALER_LAMBDA_NAME="${PROJECT_NAME}-${ENV_NAME}-worker"

if [ -z "$ECS_CLUSTER_NAME" ] || [ "$ECS_CLUSTER_NAME" = "None" ] || [ -z "$ECS_SERVICE_NAME" ] || [ "$ECS_SERVICE_NAME" = "None" ]; then
echo "Failed to resolve ECS cluster/service from SSM parameters" >&2
exit 1
fi

DB_HOST=$(aws rds describe-db-instances --region "$AWS_REGION" --db-instance-identifier "$RDS_INSTANCE_IDENTIFIER" --query 'DBInstances[0].Endpoint.Address' --output text)
DB_SECRET_ARN=$(aws rds describe-db-instances --region "$AWS_REGION" --db-instance-identifier "$RDS_INSTANCE_IDENTIFIER" --query 'DBInstances[0].MasterUserSecret.SecretArn' --output text)
DB_SECRET_JSON=$(aws secretsmanager get-secret-value --region "$AWS_REGION" --secret-id "$DB_SECRET_ARN" --query 'SecretString' --output text)
DB_USER=$(echo "$DB_SECRET_JSON" | jq -r '.username')
DB_PASS=$(echo "$DB_SECRET_JSON" | jq -r '.password')

if [ -z "$DB_HOST" ] || [ "$DB_HOST" = "None" ] || [ -z "$DB_USER" ] || [ "$DB_USER" = "null" ] || [ -z "$DB_PASS" ] || [ "$DB_PASS" = "null" ]; then
echo "Failed to resolve DB connection values from RDS/SecretsManager" >&2
exit 1
fi

DB_USER_B64=$(printf '%s' "$DB_USER" | base64 | tr -d '\n')
DB_PASS_B64=$(printf '%s' "$DB_PASS" | base64 | tr -d '\n')

MONITORING_PRIVATE_IP=$(aws ec2 describe-instances \
--region "$AWS_REGION" \
--filters "Name=tag:Name,Values=${PROJECT_NAME}-monitoring-ec2" "Name=instance-state-name,Values=running" \
--filters "Name=tag:Name,Values=monitoring" "Name=instance-state-name,Values=running" \
--query "Reservations[0].Instances[0].PrivateIpAddress" \
--output text)

if [ -z "$MONITORING_PRIVATE_IP" ] || [ "$MONITORING_PRIVATE_IP" = "None" ]; then
echo "No running monitoring instance found for tag: ${PROJECT_NAME}-monitoring-ec2" >&2
echo "No running monitoring instance found for tag: monitoring" >&2
Comment thread
arlen02-01 marked this conversation as resolved.
Outdated
exit 1
fi

Expand Down Expand Up @@ -187,13 +211,10 @@ jobs:
commands=(
"set -e"
"sudo mkdir -p /etc/oplust"
"DB_HOST=\$(aws ssm get-parameter --region $AWS_REGION --name '$SSM_RDS_ENDPOINT_PARAM' --with-decryption --query 'Parameter.Value' --output text)"
"DB_USER=\$(aws ssm get-parameter --region $AWS_REGION --name '$SSM_DB_USERNAME_PARAM' --with-decryption --query 'Parameter.Value' --output text)"
"DB_PASS=\$(aws ssm get-parameter --region $AWS_REGION --name '$SSM_DB_PASSWORD_PARAM' --with-decryption --query 'Parameter.Value' --output text)"
"SERVICE_ENV=\$(aws ssm get-parameter --region $AWS_REGION --name '${service_env_param}' --with-decryption --query 'Parameter.Value' --output text)"
"echo \"SPRING_DATASOURCE_URL=jdbc:mysql://\$DB_HOST:3306/${DB_NAME}\" | sudo tee ${env_file} >/dev/null"
"echo \"SPRING_DATASOURCE_USERNAME=\$DB_USER\" | sudo tee -a ${env_file} >/dev/null"
"echo \"SPRING_DATASOURCE_PASSWORD=\$DB_PASS\" | sudo tee -a ${env_file} >/dev/null"
"echo \"SPRING_DATASOURCE_URL=jdbc:mysql://${DB_HOST}:3306/${DB_NAME}\" | sudo tee ${env_file} >/dev/null"
"echo \"SPRING_DATASOURCE_USERNAME=\$(echo '${DB_USER_B64}' | base64 -d)\" | sudo tee -a ${env_file} >/dev/null"
"echo \"SPRING_DATASOURCE_PASSWORD=\$(echo '${DB_PASS_B64}' | base64 -d)\" | sudo tee -a ${env_file} >/dev/null"
Comment thread
arlen02-01 marked this conversation as resolved.
Outdated
"printf '%s\n' \"\$SERVICE_ENV\" | sudo tee -a ${env_file} >/dev/null"
"sudo chmod 600 ${env_file}"
"aws ecr get-login-password --region $AWS_REGION | sudo docker login --username AWS --password-stdin $ECR_REGISTRY"
Expand Down Expand Up @@ -469,6 +490,11 @@ jobs:
rm -f "$td_file" "$new_td_file"
}

deploy_service "${PROJECT_NAME}-user-ec2" "${ECR_REGISTRY}/oplust-api-user:${IMAGE_TAG}" "oplust-api-user" "/etc/oplust/api-user.env" "8080" "/oplust/api-user/env" "user-api"
deploy_service "${PROJECT_NAME}-admin-ec2" "${ECR_REGISTRY}/oplust-api-admin:${IMAGE_TAG}" "oplust-api-admin" "/etc/oplust/api-admin.env" "8081" "/oplust/api-admin/env" "admin-api"
deploy_service "user" "${ECR_REGISTRY}/oplust-api-user:${IMAGE_TAG}" "oplust-api-user" "/etc/oplust/api-user.env" "8080" "/oplust/${ENV_NAME}/api-user/env" "user-api"
deploy_service "admin" "${ECR_REGISTRY}/oplust-api-admin:${IMAGE_TAG}" "oplust-api-admin" "/etc/oplust/api-admin.env" "8081" "/oplust/${ENV_NAME}/api-admin/env" "admin-api"
deploy_transcoder_ecs





175 changes: 69 additions & 106 deletions .github/workflows/deploy-monitoring.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,43 +6,28 @@ on:
monitoring_instance_tag:
description: "EC2 Name tag for monitoring server"
required: true
default: "oplust-monitoring-ec2"
default: "monitoring"
type: string
user_api_target_ssm_param:
description: "SSM parameter name for user-api target (host:port)"
project_name:
description: "Project name"
required: true
default: "/oplust/monitoring/targets/user-api"
default: "oplust"
type: string
admin_api_target_ssm_param:
description: "SSM parameter name for admin-api target (host:port)"
environment:
description: "Environment name"
required: true
default: "/oplust/monitoring/targets/admin-api"
default: "dev"
type: string
transcoder_target_ssm_param:
description: "SSM parameter name for transcoder target (host:port)"
grafana_admin_secret_id:
description: "Secrets Manager secret id/arn for Grafana admin credentials"
required: true
default: "/oplust/monitoring/targets/transcoder"
type: string
node_exporter_target_ssm_param:
description: "SSM parameter name for node-exporter target (host:port)"
required: true
default: "/oplust/monitoring/targets/node-exporter"
type: string
machine_node_exporter_target_ssm_param:
description: "SSM parameter name for machine node-exporter target (host:port)"
required: true
default: "/oplust/monitoring/targets/machine-node-exporter"
type: string
grafana_password_ssm_param:
description: "SSM SecureString parameter name for Grafana admin password"
required: true
default: "/oplust/monitoring/grafana-admin-password"
type: string
grafana_admin_password:
description: "Optional override password (leave blank to use SSM)"
required: false
default: "oplust/dev/monitoring/grafana-admin-credentials"
type: string

permissions:
id-token: write
contents: read

env:
AWS_REGION: ap-northeast-2
MONITORING_ROOT: /opt/oplust-monitoring
Expand All @@ -58,82 +43,66 @@ jobs:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
role-to-assume: ${{ secrets.AWS_GITHUB_ACTIONS_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}

- name: Resolve scrape targets from SSM
- name: Resolve targets from Terraform naming convention
env:
USER_API_TARGET_SSM_PARAM: ${{ github.event.inputs.user_api_target_ssm_param }}
ADMIN_API_TARGET_SSM_PARAM: ${{ github.event.inputs.admin_api_target_ssm_param }}
TRANSCODER_TARGET_SSM_PARAM: ${{ github.event.inputs.transcoder_target_ssm_param }}
NODE_EXPORTER_TARGET_SSM_PARAM: ${{ github.event.inputs.node_exporter_target_ssm_param }}
MACHINE_NODE_EXPORTER_TARGET_SSM_PARAM: ${{ github.event.inputs.machine_node_exporter_target_ssm_param }}
PROJECT_NAME: ${{ github.event.inputs.project_name }}
ENV_NAME: ${{ github.event.inputs.environment }}
INSTANCE_TAG: ${{ github.event.inputs.monitoring_instance_tag }}
run: |
set -euo pipefail

USER_API_TARGET=$(aws ssm get-parameter \
--region "$AWS_REGION" \
--name "$USER_API_TARGET_SSM_PARAM" \
--with-decryption \
--query 'Parameter.Value' \
--output text)
get_private_ip_by_name() {
local name_tag="$1"
aws ec2 describe-instances \
--region "$AWS_REGION" \
--filters "Name=tag:Name,Values=${name_tag}" "Name=instance-state-name,Values=running" \
--query "Reservations[0].Instances[0].PrivateIpAddress" \
--output text
}

ADMIN_API_TARGET=$(aws ssm get-parameter \
--region "$AWS_REGION" \
--name "$ADMIN_API_TARGET_SSM_PARAM" \
--with-decryption \
--query 'Parameter.Value' \
--output text)
USER_API_IP=$(get_private_ip_by_name "user")
ADMIN_API_IP=$(get_private_ip_by_name "admin")
MONITORING_IP=$(get_private_ip_by_name "${INSTANCE_TAG}")
Comment thread
arlen02-01 marked this conversation as resolved.

TRANSCODER_TARGET=$(aws ssm get-parameter \
--region "$AWS_REGION" \
--name "$TRANSCODER_TARGET_SSM_PARAM" \
--with-decryption \
--query 'Parameter.Value' \
--output text)
if [ -z "$USER_API_IP" ] || [ "$USER_API_IP" = "None" ] || [ -z "$ADMIN_API_IP" ] || [ "$ADMIN_API_IP" = "None" ] || [ -z "$MONITORING_IP" ] || [ "$MONITORING_IP" = "None" ]; then
echo "Failed to resolve required EC2 private IPs (user/admin/monitoring)." >&2
exit 1
fi

NODE_EXPORTER_TARGET=$(aws ssm get-parameter \
--region "$AWS_REGION" \
--name "$NODE_EXPORTER_TARGET_SSM_PARAM" \
--with-decryption \
--query 'Parameter.Value' \
--output text)
USER_API_TARGET="${USER_API_IP}:8080"
ADMIN_API_TARGET="${ADMIN_API_IP}:8081"
MACHINE_NODE_EXPORTER_TARGET="${MONITORING_IP}:9100"

MACHINE_NODE_EXPORTER_TARGET=$(aws ssm get-parameter \
--region "$AWS_REGION" \
--name "$MACHINE_NODE_EXPORTER_TARGET_SSM_PARAM" \
--with-decryption \
--query 'Parameter.Value' \
--output text)
NODE_EXPORTER_TARGETS_YAML="\"${USER_API_IP}:9100\",\"${ADMIN_API_IP}:9100\",\"${MONITORING_IP}:9100\""

ECS_CLUSTER_PARAM="/${PROJECT_NAME}/${ENV_NAME}/lambda/worker/ecs_cluster_name"
ECS_SERVICE_PARAM="/${PROJECT_NAME}/${ENV_NAME}/lambda/worker/ecs_service_name"

if [ -z "$USER_API_TARGET" ] || [ "$USER_API_TARGET" = "None" ] || \
[ -z "$ADMIN_API_TARGET" ] || [ "$ADMIN_API_TARGET" = "None" ] || \
[ -z "$TRANSCODER_TARGET" ] || [ "$TRANSCODER_TARGET" = "None" ] || \
[ -z "$NODE_EXPORTER_TARGET" ] || [ "$NODE_EXPORTER_TARGET" = "None" ] || \
[ -z "$MACHINE_NODE_EXPORTER_TARGET" ] || [ "$MACHINE_NODE_EXPORTER_TARGET" = "None" ]; then
echo "One or more scrape targets are empty. Check SSM parameter values." >&2
ECS_CLUSTER_NAME=$(aws ssm get-parameter --region "$AWS_REGION" --name "$ECS_CLUSTER_PARAM" --with-decryption --query 'Parameter.Value' --output text)
ECS_SERVICE_NAME=$(aws ssm get-parameter --region "$AWS_REGION" --name "$ECS_SERVICE_PARAM" --with-decryption --query 'Parameter.Value' --output text)

if [ -z "$ECS_CLUSTER_NAME" ] || [ "$ECS_CLUSTER_NAME" = "None" ] || [ -z "$ECS_SERVICE_NAME" ] || [ "$ECS_SERVICE_NAME" = "None" ]; then
echo "Failed to resolve ECS cluster/service from SSM." >&2
exit 1
fi

normalize_targets() {
# Accept comma/newline/space separated values and render a YAML inline list payload:
# "host1:port","host2:port"
printf '%s' "$1" \
| tr ',\r\n\t' ' ' \
| xargs -n1 \
| sed '/^$/d' \
| sort -u \
| awk '{printf "\"%s\",", $0}' \
| sed 's/,$//'
}
TASK_ARNS=$(aws ecs list-tasks --region "$AWS_REGION" --cluster "$ECS_CLUSTER_NAME" --service-name "$ECS_SERVICE_NAME" --desired-status RUNNING --query 'taskArns' --output text || true)
TRANSCODER_TARGETS_YAML=""

TRANSCODER_TARGETS_YAML=$(normalize_targets "$TRANSCODER_TARGET")
NODE_EXPORTER_TARGETS_YAML=$(normalize_targets "$NODE_EXPORTER_TARGET")
if [ -n "${TASK_ARNS:-}" ]; then
ENI_IDS=$(aws ecs describe-tasks --region "$AWS_REGION" --cluster "$ECS_CLUSTER_NAME" --tasks $TASK_ARNS \
--query 'tasks[].attachments[].details[?name==`networkInterfaceId`].value' --output text || true)

if [ -z "$TRANSCODER_TARGETS_YAML" ] || [ -z "$NODE_EXPORTER_TARGETS_YAML" ]; then
echo "Rendered target list is empty. Check SSM parameter values." >&2
exit 1
if [ -n "${ENI_IDS:-}" ]; then
TRANSCODER_IPS=$(aws ec2 describe-network-interfaces --region "$AWS_REGION" --network-interface-ids $ENI_IDS \
--query 'NetworkInterfaces[].PrivateIpAddress' --output text || true)
if [ -n "${TRANSCODER_IPS:-}" ]; then
TRANSCODER_TARGETS_YAML=$(printf '%s\n' $TRANSCODER_IPS | sed '/^$/d' | sort -u | awk '{printf "\"%s:8080\",", $0}' | sed 's/,$//')
fi
fi
fi
Comment thread
arlen02-01 marked this conversation as resolved.
Outdated

echo "USER_API_TARGET=$USER_API_TARGET" >> "$GITHUB_ENV"
Expand All @@ -154,26 +123,22 @@ jobs:
-e "s|__MACHINE_NODE_EXPORTER_TARGET__|${MACHINE_NODE_EXPORTER_TARGET}|g" \
apps/monitoring/prometheus/prometheus.prod.yml.tpl > apps/monitoring/prometheus/prometheus.prod.yml

- name: Resolve Grafana admin password
- name: Resolve Grafana admin password from Secrets Manager
env:
GRAFANA_ADMIN_PASSWORD_INPUT: ${{ github.event.inputs.grafana_admin_password }}
GRAFANA_PASSWORD_SSM_PARAM: ${{ github.event.inputs.grafana_password_ssm_param }}
GRAFANA_ADMIN_SECRET_ID: ${{ github.event.inputs.grafana_admin_secret_id }}
run: |
set -euo pipefail

if [ -n "$GRAFANA_ADMIN_PASSWORD_INPUT" ]; then
GRAFANA_PASSWORD="$GRAFANA_ADMIN_PASSWORD_INPUT"
else
GRAFANA_PASSWORD=$(aws ssm get-parameter \
--region "$AWS_REGION" \
--name "$GRAFANA_PASSWORD_SSM_PARAM" \
--with-decryption \
--query 'Parameter.Value' \
--output text)
fi
SECRET_JSON=$(aws secretsmanager get-secret-value \
--region "$AWS_REGION" \
--secret-id "$GRAFANA_ADMIN_SECRET_ID" \
--query 'SecretString' \
--output text)

GRAFANA_PASSWORD=$(echo "$SECRET_JSON" | jq -r '.password')

if [ -z "$GRAFANA_PASSWORD" ] || [ "$GRAFANA_PASSWORD" = "None" ]; then
echo "Grafana admin password is empty. Check input or SSM parameter." >&2
if [ -z "$GRAFANA_PASSWORD" ] || [ "$GRAFANA_PASSWORD" = "null" ]; then
echo "Grafana admin password is empty in secret: $GRAFANA_ADMIN_SECRET_ID" >&2
exit 1
fi

Expand Down Expand Up @@ -265,5 +230,3 @@ jobs:

echo "Monitoring deployment timed out" >&2
exit 1


Loading