Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion daprdocs/content/en/concepts/dapr-services/sentry.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ linkTitle: "Sentry"
description: "Overview of the Dapr sentry service"
---

The Dapr Sentry service manages mTLS between services and acts as a certificate authority. It generates mTLS certificates and distributes them to any running sidecars. This allows sidecars to communicate with encrypted, mTLS traffic. For more information read the [sidecar-to-sidecar communication overview]({{% ref "security-concept#sidecar-to-sidecar-communication" %}}).
The Dapr Sentry service manages mTLS between services and acts as a certificate authority. It generates mTLS certificates and distributes them to any running sidecars. This allows sidecars to communicate with encrypted, mTLS traffic. For more information read the [sidecar-to-sidecar communication overview]({{% ref "security-concept#sidecar-to-sidecar-communication" %}}).

On Kubernetes, Sentry also stamps the requesting pod's [container image references into each issued workload certificate]({{% ref "security-concept#container-image-references-in-workload-certificates" %}}) as a custom X.509 extension, giving downstream systems verifiable supply-chain provenance of the software behind each workload identity.

## Self-hosted mode

Expand Down
65 changes: 65 additions & 0 deletions daprdocs/content/en/concepts/security-concept.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,71 @@ Ed25519 (Curve25519) may not satisfy FIPS 140 requirements; consult your complia
Sentry writes the Ed25519-keyed CA bundle to the `dapr-trust-bundle` Kubernetes secret. Sentry versions **before 1.17.7** cannot parse this bundle and crash on startup. Do not roll back from 1.18 to a version earlier than 1.17.7 without first rotating the CA. See the [1.18 release notes](https://github.com/dapr/dapr/releases/tag/v1.18.0) for the safe rollback path.
{{% /alert %}}

### Container image references in workload certificates

Starting with Dapr **1.19**, when running on Kubernetes, Sentry stamps the container image references of the requesting pod into every workload certificate it issues, as a custom X.509 extension. Each certificate then carries a verifiable record of exactly which software (the application container images and the daprd sidecar image) was scheduled in the pod that holds the certificate.

This gives security and platform operators supply-chain provenance at the identity layer:

- Any system that receives a Dapr workload certificate (an mTLS peer, an audit log that stores certificate chains, an attestation record) can determine which container images the workload was running, without any cooperation from the workload itself.
- Certificates, and records derived from them, can be cross-referenced against CVE advisories to flag workloads that ran a known-vulnerable image.
- [Workflow history signing]({{% ref "workflow-history-signing.md#software-provenance-which-images-executed-the-workflow" %}}) persists signing certificates with each workflow's history, so signed workflows automatically carry a cryptographically bound record of which software executed every workflow, activity, and child workflow.

#### Trust model

The image data is sourced from the Kubernetes pod object which Sentry fetches from the Kubernetes API server while validating the certificate signing request. The workload never self-reports its images: the certificate request carries no image fields, and the pod is bound to the requestor by Sentry's existing checks (Kubernetes TokenReview of the pod's projected service account token, service account match, and app-id match against the pod's annotations). A workload cannot forge the image list in its own certificate; it records what Kubernetes actually scheduled.

#### What is included

| Containers | Included |
|---|---|
| Regular containers (`spec.containers`) | Always, including the `daprd` sidecar |
| Native sidecar init containers (`spec.initContainers` with `restartPolicy: Always`) | Always, this captures daprd when [native sidecar mode]({{% ref "arguments-annotations-overview" %}}) is enabled |
| One-shot init containers | Never |

Each entry records the container's role (`daprd` for the Dapr sidecar container, `app` for everything else), its name, its image reference exactly as deployed (the tag reference from the pod spec), and best effort, the resolved image digest from the pod's container statuses. The digest can be empty when the container has not yet started at issuance time (daprd requests its certificate early in the pod lifecycle); since workload certificates are short-lived and rotated continuously, subsequent certificates are re-stamped with the digest populated.

#### Wire format

The extension is identified by OID `1.3.6.1.4.1.57683.100.1`, a Dapr sub-arc under the [Cloud Native Computing Foundation Private Enterprise Number](https://www.iana.org/assignments/enterprise-numbers/?q=57683) (57683). The extension is **non-critical**, so every existing X.509 and SPIFFE consumer which does not understand it ignores it.

The extension value is a DER OCTET STRING whose contents are a JSON array of objects with the following fields:

| Field | Type | Description |
|---|---|---|
| `role` | string | `daprd` for the Dapr sidecar container, `app` for all other containers |
| `containerName` | string | The container's name in the pod spec |
| `image` | string | The image reference as deployed, for example `ghcr.io/dapr/daprd:1.19.0` |
| `digest` | string, optional | The resolved image digest, for example `sha256:1f8d...`; omitted when unknown at issuance time |

To decode: locate the extension by OID, unwrap the inner DER OCTET STRING, and parse the resulting bytes as JSON. In Go:

```go
oid := asn1.ObjectIdentifier{1, 3, 6, 1, 4, 1, 57683, 100, 1}
for _, ext := range cert.Extensions {
if !ext.Id.Equal(oid) {
continue
}
var jsonBytes []byte
if _, err := asn1.Unmarshal(ext.Value, &jsonBytes); err != nil {
return err
}
var images []struct {
Role string `json:"role"`
ContainerName string `json:"containerName"`
Image string `json:"image"`
Digest string `json:"digest,omitempty"`
}
if err := json.Unmarshal(jsonBytes, &images); err != nil {
return err
}
}
```

#### Availability

The extension is added on the Kubernetes token validator only. Certificates issued through the insecure validator (self-hosted mode) or the JWKS validator omit the extension, as there is no pod to query for image data.

### Configuring mTLS

mTLS can be turned on/off by editing the default configuration deployed with Dapr via the `spec.mtls.enabled` field.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,11 @@ Sentry and uses it both for mTLS and for signing workflow history.
it to sign each history batch.
- **Trust roots**: every SVID chains to a Sentry CA. Verifiers accept a
signature only if its certificate chains to a CA in the trust bundle.
- **Software provenance** (Kubernetes, Dapr 1.19+): Sentry stamps the
[container image references of the workload's pod]({{% ref "security-concept.md#container-image-references-in-workload-certificates" %}})
into every SVID as a custom X.509 extension. Because signing certificates
are persisted with the workflow history, this extends to
[provenance of the software that executed the workflow](#software-provenance-which-images-executed-the-workflow).

For background on Sentry, mTLS, and trust domains, see [setup & configure
mTLS]({{% ref "mtls.md" %}}) and [security concepts]({{% ref "security-concept.md" %}}).
Expand Down Expand Up @@ -471,6 +476,61 @@ Verified foreign certificates are absorbed into the same `ext-sigcert-NNNNNN`
table used by completion attestations, so downstream attestation lookups can
content-address them.

## Software provenance: which images executed the workflow

On Kubernetes, from Dapr **1.19**, every SVID that Sentry issues carries the
[container image references of the requesting pod]({{% ref "security-concept.md#container-image-references-in-workload-certificates" %}})
as a custom X.509 extension (OID `1.3.6.1.4.1.57683.100.1`): the daprd sidecar
image, every application container image, and best effort resolved digests.
Sentry reads this data from the Kubernetes API server's pod object during
certificate issuance; the workload cannot influence it.

Because workflow history signing persists the full certificate chain of every
signer alongside the history, each signed workflow automatically accumulates a
cryptographically bound record of **exactly which software executed it**:

- The `sigcert-NNNNNN` table holds the certificates the workflow's own
executor signed with. Every signature references its certificate, and every
certificate names the daprd and app images of the pod that held the private
key. For any range of history events, you can determine the images of the
pod that produced them.
- The `ext-sigcert-NNNNNN` table holds the verified certificates of foreign
identities: every attested child workflow and activity completion, and every
verified lineage chunk, references a certificate in this table. For each
cross-identity completion, you can determine the images of the app that
executed that child workflow or activity.
- Certificate rotation preserves the timeline. Each rotation (restart, SVID
expiry, replica migration, or a deployment rollout to a new image) appends a
new certificate entry, so a long-running workflow carries a dated record of
every software version that touched it over its lifetime.

This enables, without any application changes:

- **CVE response**: scan the certificate tables of stored workflows for image
digests matching a vulnerability advisory, and identify precisely which
workflow instances, and which event ranges within them, were executed by the
affected software.
- **Audit**: answer "what software processed this workflow, activity, or
child workflow" from the state store alone, for the full retention window of
the workflow state.
- **Anomaly detection**: flag executors whose certificates name images that
were never part of an approved deployment.

To inspect, decode any certificate from the `sigcert-NNNNNN` or
`ext-sigcert-NNNNNN` entries (protobuf `SigningCertificate`, DER-encoded X.509
chain) and read the extension from the leaf certificate as described in the
[security concepts page]({{% ref "security-concept.md#container-image-references-in-workload-certificates" %}}).

{{% alert title="Scope of the guarantee" color="primary" %}}
The extension records the images Kubernetes scheduled for the pod, as observed
by Sentry at certificate issuance. It is supply-chain provenance for lookup
and flagging, not runtime integrity attestation or image signature
verification.
In self-hosted mode certificates carry no extension, and signing and
verification behave exactly as before: the extension is informational and is
not part of the verification checks.
{{% /alert %}}

## What happens when verification fails

When signature verification fails, Dapr's response depends on whether the
Expand Down Expand Up @@ -606,6 +666,7 @@ operation, ensuring atomicity.
| **Lineage integrity** | Forwarded `PropagatedHistoryChunk`s are individually signed and verified against the producing app's SPIFFE identity |
| **Immutable history** | Dapr never modifies the existing workflow history, signatures, or inbox; the only write on tamper detection is an unsigned terminal `ExecutionCompleted` event with error type `DAPR_WORKFLOW_HISTORY_TAMPERED` |
| **One-way commitment** | Signing cannot be disabled for signed workflows or enabled for unsigned workflows |
| **Software provenance** (Kubernetes, 1.19+) | Every persisted signing certificate names the container images of the pod that executed the signed events, stamped by Sentry from the Kubernetes pod object and unforgeable by the workload |

## Frequently asked questions

Expand Down Expand Up @@ -657,6 +718,16 @@ certificate is stored in the certificate table and the signature chain remains
valid. All certificates are verified as belonging to the same app ID via SPIFFE
identity binding.

### Can I tell which software executed a workflow?

**Yes, on Kubernetes with Dapr 1.19+.** Every signing certificate persisted
with the workflow (both the workflow's own in `sigcert-NNNNNN` and foreign
executors' in `ext-sigcert-NNNNNN`) carries the container image references of
the pod that held the signing key, stamped by Sentry from the Kubernetes pod
object. Decode the extension from the stored certificates to see which daprd
and application images executed each part of the workflow. See
[software provenance](#software-provenance-which-images-executed-the-workflow).

### What state store backends are supported?

History signing works with any state store that supports the actor state
Expand All @@ -668,5 +739,6 @@ alongside the existing workflow state.
- [Workflow overview]({{% ref workflow-overview.md %}})
- [Workflow architecture]({{% ref workflow-architecture.md %}})
- [Setup & configure mTLS]({{% ref mtls.md %}})
- [Container image references in workload certificates]({{% ref "security-concept.md#container-image-references-in-workload-certificates" %}})
- [Multi-app workflows]({{% ref workflow-multi-app.md %}})
- [History retention policy]({{% ref workflow-history-retention-policy.md %}})
23 changes: 23 additions & 0 deletions daprdocs/content/en/operations/security/mtls.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,29 @@ For a full explanation of what changed, why, and the compatibility matrix (inclu
When you supply your own root and issuer certificates, Sentry signs workload CSRs using the algorithm of the **issuer key you provide**, not Ed25519. If your CA key is RSA or ECDSA, workload certs will be signed with that algorithm. This is the recommended path for FIPS-compliant environments.
{{% /alert %}}

## Container image references in workload certificates (Kubernetes, Dapr 1.19+) {#container-image-references}

On Kubernetes, Sentry embeds the requesting pod's container image references (the daprd sidecar image and all application container images, with best effort resolved digests) into every workload certificate as a non-critical X.509 extension under OID `1.3.6.1.4.1.57683.100.1`. The data comes from the pod object Sentry reads from the Kubernetes API server, not from the workload, so it cannot be forged by the certificate requestor.

No configuration is needed: the extension is always present in certificates issued via the Kubernetes validator, and absent in self-hosted mode. Existing consumers are unaffected because the extension is non-critical.
Comment thread
JoshVanL marked this conversation as resolved.
Outdated

For the trust model, the exact wire format, and how to decode the extension, see [Container image references in workload certificates]({{% ref "security-concept.md#container-image-references-in-workload-certificates" %}}) in the security concepts page.

To see the raw extension on an issued certificate, inspect the workload certificate with OpenSSL; the extension appears under its OID in the output:

```bash
openssl x509 -in workload-cert.pem -noout -text
```

```
X509v3 extensions:
...
1.3.6.1.4.1.57683.100.1:
.z[{"role":"daprd","containerName":"daprd","image":"ghcr.io/dapr/daprd:1.19.0","digest":"sha256:..."},...
```

The leading bytes before the JSON are the inner DER OCTET STRING header; decode as described in the concepts page for programmatic use.

If custom certificates have not been provided, Dapr automatically creates and persist self-signed certs valid for one year.
In Kubernetes, the certs are persisted to a secret that resides in the namespace of the Dapr system pods, accessible only to them.
In self-hosted mode, the certs are persisted to disk.
Expand Down
Loading