Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 27 additions & 1 deletion docs/services/redshift.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,31 @@ For `AWS::Redshift::Cluster`:

Replacement occurs if `ClusterIdentifier`, `DBName`, `MasterUsername`, or `ClusterSubnetGroupName` changes. Other properties (such as `NodeType`, `MasterUserPassword`, `ClusterParameterGroupName`, and `VpcSecurityGroupIds`) update in place.

`SnapshotIdentifier` restores the snapshot's SQL dump into the new cluster rather than creating
an empty database. The snapshot must exist in the current account. The restored cluster inherits
the snapshot's admin username and password, so `MasterUserPassword` can be omitted. The template's
`MasterUsername` and `MasterUserPassword` do not override snapshot credentials during creation;
a later in-place update can change the password. `ManageMasterPassword` instead generates a new
password and stores it in a Redshift-owned Secrets Manager secret.

Changing, adding, or removing `SnapshotIdentifier` replaces the cluster. Updates that keep the
same snapshot modify the existing cluster without replaying the dump, preserving rows written
since restoration. A failed replacing update restores the original physical id and attributes
through the existing replacement rollback mechanism. `Ref`, `Endpoint.Address`, `Endpoint.Port`,
and `ClusterNamespaceArn` identify the restored cluster.

```yaml
Warehouse:
Type: AWS::Redshift::Cluster
Properties:
ClusterIdentifier: restored-warehouse
NodeType: ra3.large
ClusterType: single-node
DBName: dev
MasterUsername: admin
SnapshotIdentifier: warehouse-backup
```

For `AWS::Redshift::ClusterParameterGroup`, `Parameters` is applied via `ModifyClusterParameterGroup` on both create and update. `Description` and `ParameterGroupFamily` are replacement properties, matching AWS: changing either creates a new parameter group instead of reusing the prior one.

### Gaps and Limitations
Expand All @@ -244,7 +269,8 @@ For `AWS::Redshift::ClusterParameterGroup`, `Parameters` is applied via `ModifyC
- `ManageMasterPassword` creates a Redshift-owned Secrets Manager secret. The secret contains the
managed username, password, endpoint, port, and database name. `MasterPasswordSecretKmsKeyId`
selects the KMS key used for the secret metadata and is validated through KMS.
- `SnapshotIdentifier` is ignored: a fresh cluster is created instead of restoring from a snapshot.
- Snapshot restoration has the same limitations as `RestoreFromClusterSnapshot`: snapshots are
local SQL dumps, not cross-account backups or a full Redshift engine image.
- `AWS::Redshift::ClusterSecurityGroup` is accepted as metadata: Floci does not emulate the legacy EC2-Classic security group model.

## Configuration
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,16 @@ private void provisionCluster(StackResource r, JsonNode props, ProvisionContext
String nodeType = ctx.resolveOptional(props, "NodeType");
String masterUsername = ctx.resolveOptional(props, "MasterUsername");
String masterUserPassword = ctx.resolveOptional(props, "MasterUserPassword");
if (!manageMasterPassword && (masterUserPassword == null || masterUserPassword.isBlank())) {
String snapshotIdentifier = ctx.resolveOptional(props, "SnapshotIdentifier");
if (snapshotIdentifier != null && snapshotIdentifier.isBlank()) {
throw new AwsException("ValidationException", "SnapshotIdentifier must not be blank", 400);
}
if (snapshotIdentifier != null && manageMasterPassword && masterUserPassword != null) {
throw new AwsException("ValidationException",
"MasterUserPassword cannot be used with ManageMasterPassword", 400);
}
if (snapshotIdentifier == null && !manageMasterPassword
&& (masterUserPassword == null || masterUserPassword.isBlank())) {
throw new AwsException("ValidationException",
"MasterUserPassword is required unless ManageMasterPassword is set", 400);
}
Expand All @@ -113,18 +122,25 @@ private void provisionCluster(StackResource r, JsonNode props, ProvisionContext

String explicitId = ctx.resolveOptional(props, "ClusterIdentifier");
Cluster prior = ctx.isUpdate() ? findExistingCluster(ctx.priorPhysicalId()) : null;
boolean createOnlyChanged = dbNameChanged || (prior != null && (
(masterUsername != null && !masterUsername.equals(prior.getMasterUsername()))
|| (subnetGroup != null && !Objects.equals(subnetGroup, prior.getClusterSubnetGroupName()))
));
boolean snapshotChanged = ctx.isUpdate()
&& !Objects.equals(snapshotIdentifier, attributesBefore.get("Floci::SnapshotIdentifier"));
boolean masterUsernameChanged = snapshotIdentifier == null
? prior != null && masterUsername != null && !masterUsername.equals(prior.getMasterUsername())
: ctx.isUpdate() && !Objects.equals(masterUsername, attributesBefore.get("Floci::RestoreMasterUsername"));
boolean createOnlyChanged = dbNameChanged || snapshotChanged || masterUsernameChanged
|| prior != null && subnetGroup != null && !Objects.equals(subnetGroup, prior.getClusterSubnetGroupName());
boolean sameTemplateIdentifier = explicitId == null || explicitId.equals(ctx.priorPhysicalId())
|| explicitId.equals(attributesBefore.get("Floci::ClusterIdentifier"));

String id;
if (createOnlyChanged && (explicitId == null || explicitId.equals(ctx.priorPhysicalId()))) {
if (createOnlyChanged && sameTemplateIdentifier) {
if (explicitId != null && !explicitId.isBlank()) {
id = replacementId(explicitId, IDENTIFIER_MAX_LENGTH, true);
} else {
id = ctx.generatePhysicalName(r.getLogicalId(), IDENTIFIER_MAX_LENGTH, true);
}
} else if (ctx.isUpdate() && sameTemplateIdentifier) {
id = ctx.priorPhysicalId();
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
Resolved
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
Resolved
} else {
id = ctx.stablePhysicalName(explicitId, r.getLogicalId(), IDENTIFIER_MAX_LENGTH, true);
}
Expand All @@ -138,6 +154,24 @@ private void provisionCluster(StackResource r, JsonNode props, ProvisionContext
cluster = redshiftService.modifyCluster(id, nodeType, numberOfNodes(props, ctx),
masterUserPassword, ctx.resolveOptional(props, "ClusterParameterGroupName"),
securityGroups);
} else if (snapshotIdentifier != null) {
cluster = redshiftService.restoreFromClusterSnapshot(id, snapshotIdentifier, nodeType, subnetGroup,
manageMasterPassword, ctx.resolveOptional(props, "MasterPasswordSecretKmsKeyId"), ctx.region());
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
Resolved
String parameterGroup = ctx.resolveOptional(props, "ClusterParameterGroupName");
if (parameterGroup != null || !securityGroups.isEmpty()) {
try {
cluster = redshiftService.modifyCluster(id, nodeType, numberOfNodes(props, ctx),
null, parameterGroup, securityGroups);
} catch (RuntimeException failure) {
try {
redshiftService.deleteCluster(id);
} catch (RuntimeException cleanupFailure) {
failure.addSuppressed(cleanupFailure);
LOG.warnv(cleanupFailure, "Failed to clean up restored cluster {0}", id);
}
throw failure;
}
}
} else if (manageMasterPassword) {
String kmsKeyId = props.path("MasterPasswordSecretKmsKeyId").asText(null);
cluster = redshiftService.createClusterWithManagedMasterPassword(id, nodeType, masterUsername,
Expand All @@ -149,10 +183,35 @@ private void provisionCluster(StackResource r, JsonNode props, ProvisionContext

Map<String, String> tags = ctx.resolveTags(props, "Tags");
if (!tags.isEmpty()) {
redshiftService.createTags(id, tags);
try {
String arn = AwsArnUtils.Arn.of("redshift", ctx.region(), ctx.accountId(), "cluster:" + id).toString();
redshiftService.createTags(arn, tags);
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
Resolved
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
Resolved
} catch (RuntimeException failure) {
if (snapshotIdentifier != null && !ctx.reusesPriorEntity(id)) {
try {
redshiftService.deleteCluster(id);
} catch (RuntimeException cleanupFailure) {
failure.addSuppressed(cleanupFailure);
LOG.warnv(cleanupFailure, "Failed to clean up restored cluster {0}", id);
}
}
throw failure;
}
}

r.setPhysicalId(id);
r.getAttributes().remove("Floci::ClusterIdentifier");
if (explicitId != null) {
r.getAttributes().put("Floci::ClusterIdentifier", explicitId);
}
r.getAttributes().remove("Floci::SnapshotIdentifier");
r.getAttributes().remove("Floci::RestoreMasterUsername");
if (snapshotIdentifier != null) {
r.getAttributes().put("Floci::SnapshotIdentifier", snapshotIdentifier);
if (masterUsername != null) {
r.getAttributes().put("Floci::RestoreMasterUsername", masterUsername);
}
}
if (dbName != null && !dbName.isBlank()) {
r.getAttributes().put("Floci::DBName", dbName);
}
Expand All @@ -162,6 +221,8 @@ private void provisionCluster(StackResource r, JsonNode props, ProvisionContext
}
if (cluster.getMasterPasswordSecretArn() != null) {
r.getAttributes().put("MasterPasswordSecretArn", cluster.getMasterPasswordSecretArn());
} else {
r.getAttributes().remove("MasterPasswordSecretArn");
}
r.getAttributes().put("ClusterNamespaceArn", namespaceArn(ctx, id));
}
Expand All @@ -186,7 +247,8 @@ private void warnUnsupported(JsonNode props, ProvisionContext ctx, String id) {
if (ctx.resolveOptional(props, "Port") != null) {
LOG.warnv("Cluster {0}: Port ignored, Floci assigns the host proxy port", id);
}
if (!ctx.isUpdate() && ctx.resolveOptional(props, "ClusterParameterGroupName") != null) {
if (!ctx.isUpdate() && ctx.resolveOptional(props, "SnapshotIdentifier") == null
&& ctx.resolveOptional(props, "ClusterParameterGroupName") != null) {
LOG.warnv("Cluster {0}: ClusterParameterGroupName is only applied on update", id);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -341,17 +341,21 @@ private Cluster createClusterWithManagedMasterPasswordLocked(
String password = RandomPasswordGenerator.generate(objectMapper.createObjectNode());
Cluster cluster = createCluster(identifier, nodeType, username, password,
clusterSubnetGroupName, vpcSecurityGroupIds, iamRoleArns, defaultIamRoleArn);
String secretName = "redshift/" + identifier;
String secretString = managedMasterSecret(cluster, password);
Secret secret;
try {
secret = secretsManagerService.createSecret(secretName, secretString, null,
"Managed master user secret for Redshift cluster " + identifier,
kmsKeyId, List.of(), "redshift", region);
return createManagedMasterSecret(cluster, kmsKeyId, region);
} catch (RuntimeException e) {
rollbackManagedMasterPasswordCluster(cluster);
throw e;
}
}

private Cluster createManagedMasterSecret(Cluster cluster, String kmsKeyId, String region) {
String identifier = cluster.getClusterIdentifier();
String secretName = "redshift/" + identifier;
String secretString = managedMasterSecret(cluster, cluster.getMasterPassword());
Secret secret = secretsManagerService.createSecret(secretName, secretString, null,
"Managed master user secret for Redshift cluster " + identifier,
kmsKeyId, List.of(), "redshift", region);
cluster.setMasterPasswordSecretArn(secret.getArn());
cluster.setMasterPasswordSecretKmsKeyId(kmsKeyId);
clusters.put(identifier, cluster);
Expand Down Expand Up @@ -1199,6 +1203,37 @@ public Cluster restoreFromClusterSnapshot(String clusterIdentifier, String snaps
() -> mutate(() -> restoreSnapshot(clusterIdentifier, snapshotIdentifier, nodeType))));
}

public Cluster restoreFromClusterSnapshot(String clusterIdentifier, String snapshotIdentifier, String nodeType,
String clusterSubnetGroupName, boolean manageMasterPassword,
String kmsKeyId, String region) {
if (manageMasterPassword && secretsManagerService == null) {
throw new AwsException("InternalFailure", "Secrets Manager is unavailable", 500);
}
return withResourceLock("cluster", clusterIdentifier, () -> withResourceLock("snapshot", snapshotIdentifier,
() -> mutate(() -> {
Cluster cluster = restoreSnapshot(clusterIdentifier, snapshotIdentifier, nodeType);
Comment thread
qodo-code-review[bot] marked this conversation as resolved.
Resolved
try {
cluster.setClusterSubnetGroupName(clusterSubnetGroupName);
if (manageMasterPassword) {
String password = RandomPasswordGenerator.generate(objectMapper.createObjectNode());
modifyCluster(clusterIdentifier, null, null, password, null, List.of());
return createManagedMasterSecret(cluster, kmsKeyId, region);
}
clusters.put(clusterIdentifier, cluster);
clusters.flush();
return cluster;
} catch (RuntimeException failure) {
try {
deleteCluster(clusterIdentifier);
} catch (RuntimeException cleanupFailure) {
failure.addSuppressed(cleanupFailure);
LOG.warnv(cleanupFailure, "Failed to clean up restored cluster {0}", clusterIdentifier);
}
throw failure;
}
})));
}

private Cluster restoreSnapshot(String clusterIdentifier, String snapshotIdentifier, String nodeType) {
if (clusters.get(clusterIdentifier).isPresent()) {
throw new AwsException("ClusterAlreadyExists", "Cluster " + clusterIdentifier + " already exists", 400);
Expand Down
Loading
Loading