Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/processing-activities-kebab.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@transcend-io/cli': minor
'@transcend-io/sdk': minor
---

Complete processing-activities RoPA pull/push: create-by-title then update relationships, skip empty `processing-activities: []` on push, and soft-warn unresolved `teamNames`.
7 changes: 4 additions & 3 deletions packages/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2472,7 +2472,7 @@ The API key permissions for this command vary based on the `resources` argument:
| `enrichers` | `enrichers` | The Privacy Request enricher configurations. | View Identity Verification Settings | [DSR Automation -> Identifiers](https://app.transcend.io/privacy-requests/identifiers) |
| `dataFlows` | `data-flows` | Consent Manager Data Flow definitions. | View Data Flows | [Consent Management -> Data Flows](https://app.transcend.io/consent-manager/data-flows/approved) |
| `businessEntities` | `business-entities` | The business entities in the Data Inventory. | View Data Inventory | [Data Inventory -> Business Entities](https://app.transcend.io/data-map/data-inventory/business-entities) |
| `processingActivities` | `processing-activities` | The processing activities in the Data Inventory. | View Data Inventory | [Data Inventory -> Processing Activities](https://app.transcend.io/data-map/data-inventory/processing-activities) |
| `processingActivities` | `processing-activities` | The processing activities in the Data Inventory (RoPA). Empty `processing-activities: []` does not request Manage Data Map scopes. | View Data Inventory | [Data Inventory -> Processing Activities](https://app.transcend.io/data-map/data-inventory/processing-activities) |
| `actions` | `actions` | The privacy request action settings. | View Data Subject Request Settings | [DSR Automation -> Request Settings -> Data Actions](https://app.transcend.io/privacy-requests/settings/data-actions) |
| `dataSubjects` | `data-subjects` | The privacy request data subject settings. | View Data Subject Request Settings | [DSR Automation -> Request Settings -> Data Subjects](https://app.transcend.io/privacy-requests/settings/data-subjects) |
| `identifiers` | `identifiers` | The privacy request identifier configurations. | View Identity Verification Settings | [DSR Automation -> Identifiers](https://app.transcend.io/privacy-requests/identifiers) |
Expand Down Expand Up @@ -2674,7 +2674,8 @@ FLAGS

#### Scopes

The API key permissions for this command vary based on the resources declared as top-level keys in your [`transcend.yml`](#transcendyml) file:
The API key permissions for this command vary based on the resources declared as top-level keys in your [`transcend.yml`](#transcendyml) file.
Empty arrays (e.g. `processing-activities: []`) do **not** request scopes for that resource.

| Resource | Key in `transcend.yml` | Description | Scopes | Link |
| ----------------------- | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
Expand All @@ -2685,7 +2686,7 @@ The API key permissions for this command vary based on the resources declared as
| `enrichers` | `enrichers` | The Privacy Request enricher configurations. | Manage Request Identity Verification | [DSR Automation -> Identifiers](https://app.transcend.io/privacy-requests/identifiers) |
| `dataFlows` | `data-flows` | Consent Manager Data Flow definitions. | Manage Data Flows | [Consent Management -> Data Flows](https://app.transcend.io/consent-manager/data-flows/approved) |
| `businessEntities` | `business-entities` | The business entities in the Data Inventory. | Manage Data Inventory | [Data Inventory -> Business Entities](https://app.transcend.io/data-map/data-inventory/business-entities) |
| `processingActivities` | `processing-activities` | The processing activities in the Data Inventory. | Manage Data Map | [Data Inventory -> Processing Activities](https://app.transcend.io/data-map/data-inventory/processing-activities) |
| `processingActivities` | `processing-activities` | The processing activities in the Data Inventory (RoPA). Empty `processing-activities: []` does not request Manage Data Map scopes. | Manage Data Map | [Data Inventory -> Processing Activities](https://app.transcend.io/data-map/data-inventory/processing-activities) |
| `actions` | `actions` | The privacy request action settings. | Manage Data Subject Request Settings | [DSR Automation -> Request Settings -> Data Actions](https://app.transcend.io/privacy-requests/settings/data-actions) |
| `dataSubjects` | `data-subjects` | The privacy request data subject settings. | Manage Data Subject Request Settings | [DSR Automation -> Request Settings -> Data Subjects](https://app.transcend.io/privacy-requests/settings/data-subjects) |
| `identifiers` | `identifiers` | The privacy request identifier configurations. | Manage Request Identity Verification | [DSR Automation -> Identifiers](https://app.transcend.io/privacy-requests/identifiers) |
Expand Down
31 changes: 31 additions & 0 deletions packages/cli/examples/processing-activities.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,32 @@
# Processing activities (RoPA) for Data Inventory.
# Admin UI: https://app.transcend.io/data-map/data-inventory/processing-activities
#
# Pull:
# transcend inventory pull --auth="$TRANSCEND_API_KEY" --resources=processingActivities
# Push:
# transcend inventory push --auth="$TRANSCEND_API_KEY" --resources=processingActivities
processing-activities:
# TWDC CCTV RoPA acceptance fixture — round-trips via pull/push
- title: CCTV — Disney Italia Theme Park Entrances
description: On-premise CCTV surveillance…
controllerships: [CONTROLLER]
retentionType: LIMITED
dataProtectionImpactAssessmentStatus: MISSING
dataSubjectTypes: [customer, employee]
teamNames: [Security Operations]
processingSubPurposes:
- purpose: OPERATION_SECURITY
- purpose: ESSENTIAL
dataSubCategories:
- category: OTHER
name: Security-Physical
- category: CONTACT
name: Image
storageRegions:
- country: IT
transferRegions:
- country: EU

- title: Chatbot AI
description: >-
A product that allows employees to chat with an informational AI Chatbot
Expand Down Expand Up @@ -26,6 +54,7 @@ processing-activities:
name: Interaction Events
saaSCategories:
- Artificial Intelligence

- title: Marketing Analytics
description: Data around how our website is used
controllerships:
Expand All @@ -47,6 +76,7 @@ processing-activities:
processingSubPurposes:
- purpose: ANALYTICS
- purpose: MARKETING

- title: Onboarding
description: >-
The process of onboarding new hires through a number of interactive
Expand All @@ -61,6 +91,7 @@ processing-activities:
- charlie@acme.com
processingSubPurposes:
- purpose: HR

- title: Recruiting
description: >-
Our recruitment pipeline includes the use of several different tools and
Expand Down
3 changes: 2 additions & 1 deletion packages/cli/src/commands/inventory/push/readme.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@ const examples = buildExamples<PushCommandFlags>(

export default `#### Scopes

The API key permissions for this command vary based on the resources declared as top-level keys in your [\`transcend.yml\`](#transcendyml) file:
The API key permissions for this command vary based on the resources declared as top-level keys in your [\`transcend.yml\`](#transcendyml) file.
Empty arrays (e.g. \`processing-activities: []\`) do **not** request scopes for that resource.

${createPullResourceScopesTable(TR_PUSH_RESOURCE_SCOPE_MAP)}

Expand Down
3 changes: 2 additions & 1 deletion packages/cli/src/lib/docgen/createPullResourceScopesTable.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ const RESOURCE_DOCUMENTATION: Record<
],
},
[TranscendPullResource.ProcessingActivities]: {
description: 'The processing activities in the Data Inventory.',
description:
'The processing activities in the Data Inventory (RoPA). Empty `processing-activities: []` does not request Manage Data Map scopes.',
markdownLinks: [
'[Data Inventory -> Processing Activities](https://app.transcend.io/data-map/data-inventory/processing-activities)',
],
Expand Down
17 changes: 11 additions & 6 deletions packages/cli/src/lib/graphql/syncConfigurationToTranscend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ import { GraphQLClient } from 'graphql-request';
/* eslint-disable max-lines */
import { TranscendInput } from '../../codecs.js';
import { logger } from '../../logger.js';
import { hasTranscendConfigSection } from '../helpers/hasTranscendConfigSection.js';
import { validatePreferenceManagementSlugs } from '../preference-management/validatePreferenceManagementSlugs.js';
import { ensureAllDataSubjectsExist } from './ensureAllDataSubjectsExist.js';
import { syncDataSilos } from './syncDataSilos.js';
Expand Down Expand Up @@ -151,7 +152,7 @@ export async function syncConfigurationToTranscend(
})
: ({} as { [k in string]: Identifier }),
// Grab all data subjects in the organization
dataSilos || dataSubjects || enrichers || processingActivities
dataSilos || dataSubjects || enrichers || hasTranscendConfigSection(processingActivities)
? ensureAllDataSubjectsExist(input, client)
: {},
// Grab API keys
Expand Down Expand Up @@ -551,11 +552,15 @@ export async function syncConfigurationToTranscend(
await syncDataSiloDependencies(client, { input: dependencyUpdates, logger });
}

// Update processing activities
if (processingActivities) {
const processingActivitySuccess = await syncProcessingActivities(client, processingActivities, {
logger,
});
// Update processing activities (skip empty `processing-activities: []`)
if (hasTranscendConfigSection(processingActivities)) {
const processingActivitySuccess = await syncProcessingActivities(
client,
processingActivities!,
{
logger,
},
);
encounteredError = encounteredError || !processingActivitySuccess;
}

Expand Down
71 changes: 71 additions & 0 deletions packages/cli/src/lib/helpers/hasTranscendConfigSection.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
import type { ScopeName } from '@transcend-io/privacy-types';
import { uniq } from 'lodash-es';

import type { TranscendInput } from '../../codecs.js';
import { TR_PUSH_RESOURCE_SCOPE_MAP, TR_YML_RESOURCE_TO_FIELD_NAME } from '../../constants.js';
import { TranscendPullResource } from '../../enums.js';

/**
* Whether a transcend.yml top-level section should trigger push scopes / sync.
* Empty arrays like `"processing-activities": []` must not request ManageDataMap.
*
* @param value - Top-level YAML section value
* @returns True when the section has content to sync
*/
export function hasTranscendConfigSection(value: unknown): boolean {
if (value === undefined || value === null) {
return false;
}
if (Array.isArray(value)) {
return value.length > 0;
}
if (typeof value === 'object') {
return Object.keys(value).length > 0;
}
return true;
}

/**
* Remove empty transcend.yml sections before decode / scope derivation.
* Empty arrays like `"action-items": []` still count as present keys and can
* cause unnecessary API scopes or sync attempts.
*
* @param config - Raw YAML object
* @returns Config with empty top-level sections removed
*/
export function stripEmptyTranscendConfigSections(
config: Record<string, unknown>,
): Record<string, unknown> {
return Object.fromEntries(
Object.entries(config).filter(([, value]) => hasTranscendConfigSection(value)),
);
}

/**
* Derive push scopes only for non-empty config sections.
*
* @param input - Decoded transcend.yml input
* @returns Unique ScopeName values required for the push
*/
export function derivePushScopesFromTranscendInput(input: TranscendInput): ScopeName[] {
return uniq(
Object.values(TranscendPullResource).flatMap((resource) =>
hasTranscendConfigSection(input[TR_YML_RESOURCE_TO_FIELD_NAME[resource]])
? TR_PUSH_RESOURCE_SCOPE_MAP[resource]
: [],
),
);
}

/**
* Summarize which transcend.yml sections are included in a push.
*
* @param input - Decoded transcend.yml input
* @returns Human-readable section summary
*/
export function summarizeTranscendConfigSections(input: TranscendInput): string {
return Object.entries(input)
.filter(([, value]) => hasTranscendConfigSection(value))
.map(([key, value]) => `${key}: (${Array.isArray(value) ? value.length : 1})`)
.join(', ');
}
1 change: 1 addition & 0 deletions packages/cli/src/lib/helpers/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ export * from './collectParquetFilesOrExit.js';
export * from './collectCsvFilesOrExit.js';
export * from './parquetToCsvOneFile.js';
export * from './writeCsv.js';
export * from './hasTranscendConfigSection.js';
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import { DefaultConsentOption, ScopeName } from '@transcend-io/privacy-types';
import { describe, expect, it } from 'vitest';

import {
derivePushScopesFromTranscendInput,
hasTranscendConfigSection,
stripEmptyTranscendConfigSections,
} from '../hasTranscendConfigSection.js';

describe('hasTranscendConfigSection', () => {
it('returns false for empty arrays and empty objects', () => {
expect(hasTranscendConfigSection([])).toBe(false);
expect(hasTranscendConfigSection({})).toBe(false);
expect(hasTranscendConfigSection(undefined)).toBe(false);
expect(hasTranscendConfigSection(null)).toBe(false);
});

it('returns true for non-empty arrays and objects', () => {
expect(hasTranscendConfigSection([{ title: 'CCTV' }])).toBe(true);
expect(hasTranscendConfigSection({ unknownRequestPolicy: 'ALLOW' })).toBe(true);
});
});

describe('stripEmptyTranscendConfigSections', () => {
it('removes empty top-level sections', () => {
expect(
stripEmptyTranscendConfigSections({
purposes: [{ title: 'Advertising' }],
'processing-activities': [],
'action-items': [],
}),
).toEqual({
purposes: [{ title: 'Advertising' }],
});
});
});

describe('derivePushScopesFromTranscendInput', () => {
it('does not request ManageDataMap for empty processing-activities arrays', () => {
const scopes = derivePushScopesFromTranscendInput({
'processing-activities': [],
purposes: [
{
trackingType: 'Advertising',
title: 'Advertising',
name: 'Advertising',
'default-consent': DefaultConsentOption.OptOut,
},
],
});

expect(scopes).not.toContain(ScopeName.ManageDataMap);
expect(scopes).toContain(ScopeName.ManageConsentManager);
});

it('requests ManageDataMap when processing-activities has entries', () => {
const scopes = derivePushScopesFromTranscendInput({
'processing-activities': [{ title: 'CCTV — Disney Italia Theme Park Entrances' }],
});

expect(scopes).toContain(ScopeName.ManageDataMap);
});
});
Loading
Loading