v1.13.1 - Hosted adjuncts, `maxWidth` and `openFullMax`

Summary

• maxWidth and openFullMax
  • Orchestrator respects maxWidth and openFullMax, see ADR 0010 and ADR 0011
  • Orchestrator strictly parses IIIF ImageApi 2.1 and 3.0 requests.
  • Orchestrator will always refresh cached asset before generating info.json.
  • Display maxWidth and openFullMax in Portal
• Support hosted adjuncts, updates to API/Orchestrator/Engine, see https://dlcs.github.io/public-docs/api-doc/adjuncts/
• Introduce adjunct batches
• Handling of 'stub' assets. Space 0 repurposed, see RFC 022
• Delete CustomerStorage record when space deleted.

Bugfixes

• Catch edge-case where not-found image returns incorrect status code #1197
• Handle s3 objects larger than 50GB #1200

What's Changed

• Hosted Adjunct - API Management by @p-kaczynski in #1102
• Set missing ingesting/error prop on adjunct hydra output model by @p-kaczynski in #1108
• Add openFullMax and maxWidth (API only) by @donaldgray in #1105
• Update ThumbCreator to use maxUnauth replacements by @donaldgray in #1107
• Bump to localstack:4.13 by @donaldgray in #1109
• Update docker compose readme by @donaldgray in #1101
• Replace maxUnauthorised in Orchestrator by @donaldgray in #1116
• Refresh OrchestrationImage info.json create by @donaldgray in #1119
• Add maxWidth and openFullMax to portal by @donaldgray in #1120
• Engine handles hosted adjuncts by @p-kaczynski in #1115
• PullRequest template by @donaldgray in #1126
• Add InlineAnnotation to the API by @JackLewis-digirati in #1118
• Add support to orchestrator for inline annotations by @JackLewis-digirati in #1124
• Add /version endpoint to API and Orchestrator by @donaldgray in #1131
• Handle very small maxWidth values and "sizes" in info.json by @donaldgray in #1130
• Initial RFC for stub assets by @donaldgray in #1129
• Bugfixes for inline annotation work by @JackLewis-digirati in #1139
• Orchestrator serves adjuncts by @p-kaczynski in #1136
• Use orchestrator url for hosted adjuncts id in manifest builder by @p-kaczynski in #1146
• Add ability to delete hosted adjuncts by @JackLewis-digirati in #1138
• Fix hosted adjunct being updated getting deleted by @JackLewis-digirati in #1162
• Add concurrency to github action by @donaldgray in #1156
• Support ?include=adjuncts query parameter by @donaldgray in #1165
• Add provides to adjunct by @JackLewis-digirati in #1168
• Allow POSTing of multiple Adjuncts in one request for creation by @p-kaczynski in #1167
• Implement bulk delete of adjuncts by @JackLewis-digirati in #1169
• Immediate completion of none channel by @JackLewis-digirati in #1170
• Create batch of adjuncts by @donaldgray in #1171
• Publish notification AdjunctBatch completion by @donaldgray in #1172
• Add GET adjunct batch endpoint by @donaldgray in #1173
• Specify actual type when serialising completion by @donaldgray in #1177
• Always return single adjunct on PUT by @donaldgray in #1175
• Include 'type' in batch completion notifications by @donaldgray in #1179
• Prevent invalid adjunct ids in deletion by @donaldgray in #1181
• Support ?include on POST allImages by @donaldgray in #1185
• UpdateHandlers abort if item not in DB by @donaldgray in #1178
• Set certain fields to optional for the 'None' channel by @JackLewis-digirati in #1187
• Exclude query-parameters in generated adjunct ids by @donaldgray in #1190
• Allow for stub assets to be added to space 0 by @JackLewis-digirati in #1191
• Allow create batch of images to work with a minimal payload in the stub asset space by @JackLewis-digirati in #1193
• Generated manifests include adjuncts for 'none' by @donaldgray in #1196
• Improve OrchestrationImage caching - avoid unnecessary proxying by @donaldgray in #1198
• Handle bucket copy for files greater than 50GB by @donaldgray in #1202
• Validate "annotations" adjuncts are JSON by @donaldgray in #1204
• Use null for the global space storage record by @JackLewis-digirati in #1203
• Update wording of adjunct error message by @JackLewis-digirati in #1205
• Disallow spaces that aren't a positive integer by @JackLewis-digirati in #1206
• RFC: Hosted Annotation Adjuncts id Property Handling by @p-kaczynski in #1176
• Allow modifiying the id of a hosted annotation id by @JackLewis-digirati in #1209
• Decrement image storage for adjuncts by @JackLewis-digirati in #1211
• Delete CustomerStorage on space delete by @donaldgray in #1213
• Text-services ADR by @donaldgray in #1210
• Update main branch by @donaldgray in #1214

Warning

When deploying to an existing environment, the MaxWidth appSetting value for API, Engine and Orchestrator may need to be set depending on existing image-server configuration. The backing image-server config may need to be set to allow this MaxWidth, or disabled to allow orchestrator to control.

OldestAllowedInfoJson will also need to be set to ensure info.json files are refreshed. This may result in an increase in Orchestration operations.

Warning

Contains migrations

• New columns in "Images" and UPDATE statement. This could be slow / expensive in some environments! #1105
• Adjunct-related #1115, #1118, #1153, #1171
• Repurposing space0 #1191, #1203

Note

This PR requires infrastructure changes.

• Required:
  • SQS queues for adjunct updating and deletion
  • SNS topic for adjunct modification
• Optional: AdjunctBatch completion SNS topic. If present API or Engine will publish a notification when an AdjunctBatch is completed. If missing a warning will be logged.
• Optional: For environments where larger AV files are expected, timebased queue visibility_timeout could be increased to allow for handling. In practice this won't have much affect as messages are only retried once.

Note

This PR introduces configuration changes.

Service	AppSetting	Required?	Description	Default
CleanupHandler	AWS:SQS:AdjunctDeleteNotificationQueueName	N	Delete notification queue name	null
CleanupHandler	AWS:SQS:AdjunctUpdateNotificationQueueName	N	Update notification queue name	null
API	AWS:SNS:AdjunctModifiedNotificationTopicArn	N	Update notification topic	null
Engine	AWS:SNS:AdjunctBatchCompletedTopicArn	N	ARN for AdjunctBatch Completion SNS topic	null
API	AWS:SNS:AdjunctBatchCompletedTopicArn	N	ARN for AdjunctBatch Completion SNS topic	null
Engine	AWS:S3:CopyPartConcurrency	N	Number of concurrent part uploads	4
Orchestrator	MaxAdjunctSizeBytes	N	Max permitted annotation adjunct size	5 * 1024 * 1024

Full Changelog: v1.12.1...v1.13.1

v1.12.1 - Bugfix large file copy

Summary

• Resolve issue where s3 -> s3 copies of files exceeding 50GB would fail #1200

What's Changed

• Handle bucket copy for files greater than 50GB by @donaldgray in #1201

Infrastructure Changes

Note

This release has optional infrastructure changes.

• Optional: For environments where larger AV files are expected, timebased queue visibility_timeout could be increased to allow for handling. In practice this won't have much affect as messages are only retried once.

Configuration Changes

Note

This release introduces configuration changes.

Service	AppSetting	Required?	Description	Default
Engine	AWS:S3:CopyPartConcurrency	N	Number of concurrent part uploads	4

Full Changelog: v1.12.0...v1.12.1

v1.13.1-rc2 - Orchestrator supports maxWidth/openFullMax

Summary

• Orchestrator respects maxWidth and openFullMax, see ADR 0010 and ADR 0011
• Orchestrator strictly parses IIIF ImageApi 2.1 and 3.0 requests.
• Orchestrator will always refresh cached asset before generating info.json.
• Display maxWidth and openFullMax in Portal

What's Changed

• Replace maxUnauthorised in Orchestrator by @donaldgray in #1116
• Refresh OrchestrationImage info.json create by @donaldgray in #1119
• Add maxWidth and openFullMax to portal by @donaldgray in #1120

Warning

When deploying to an existing environment the MaxWidth appSetting value for API, Engine and Orchestrator may need to be set depending on existing image-server configuration. OR the image-server config may need to be set to allow this MaxWidth

OldestAllowedInfoJson will also need to be set to ensure info.json files are refreshed. This may result in an increase in Orchestration operations.

Full Changelog: v1.13.1-rc1...v1.13.1-rc2

v1.13.1-rc1 - HostedAdjuncts API and initial `maxUnauthorised` replacement API + Engine

Summary

• Hosted adjuncts - API changes
• Introduce openFullMax and maxWidth properties - API and Engine handling.

What's Changed

• Hosted Adjunct - API Management by @p-kaczynski in #1102
• Set missing ingesting/error prop on adjunct hydra output model by @p-kaczynski in #1108
• Add openFullMax and maxWidth (API only) by @donaldgray in #1105
• Update ThumbCreator to use maxUnauth replacements by @donaldgray in #1107
• Bump to localstack:4.13 by @donaldgray in #1109
• Update docker compose readme by @donaldgray in #1101

Warning

Contains a migration. New columns in "Images" and UPDATE statement.
This could be slow / expensive in some environments!

Full Changelog: v1.12.0...v1.13.1-rc1

v1.12.0 - External adjuncts

Summary

• Initial adjuncts work, supporting External adjuncts only. #1073 and #1074

What's Changed

• Add initial external assets adjuncts work by @JackLewis-digirati in #1082
• Allow migrations to be run in all envs by @donaldgray in #1083
• Adjunct bugfixes by @donaldgray in #1086
• Bump actions/cache from 4 to 5 by @dependabot[bot] in #1081
• Bump actions/setup-dotnet from 4 to 5 by @dependabot[bot] in #1040
• Bump actions/setup-java from 4 to 5 by @dependabot[bot] in #1037
• Bump packages in github actions by @donaldgray in #1087
• Allow adjunct @type to change by @donaldgray in #1088
• Output external adjunct on Manifests by @donaldgray in #1089
• ADR for maxWidth and openFullMax by @donaldgray in #1090
• Ensure adjunct ordering on Manifest by @donaldgray in #1091
• Merge branch 'develop' by @JackLewis-digirati in #1097

Important

This release contains migrations. Non-breaking, new adjuncts table added.

Warning

"RestrictedAssetIdCharacterString" config value renamed to "RestrictedResourceIdCharacterString". Defaults are set as part of docker image, allowing it to be removed from deployment environments.

Full Changelog: v1.11.5...v1.12.0

v1.12.0-rc2 - External adjuncts on Manifests

Summary

• Output external adjuncts on generated Manifests #1074

What's Changed

• Output external adjunct on Manifests by @donaldgray in #1089
• ADR for maxWidth and openFullMax by @donaldgray in #1090
• Ensure adjunct ordering on Manifest by @donaldgray in #1091

Full Changelog: v1.12.0-rc1...v1.12.0-rc2

v1.12.0-rc1 - External adjuncts API work

Summary

• Initial API work for adjuncts, supporting External adjuncts only. #1073

What's Changed

• Add initial external assets adjuncts work by @JackLewis-digirati in #1082
• Allow migrations to be run in all envs by @donaldgray in #1083
• Adjunct bugfixes by @donaldgray in #1086
• Bump actions/cache from 4 to 5 by @dependabot[bot] in #1081
• Bump actions/setup-dotnet from 4 to 5 by @dependabot[bot] in #1040
• Bump actions/setup-java from 4 to 5 by @dependabot[bot] in #1037
• Bump packages in github actions by @donaldgray in #1087
• Allow adjunct @type to change by @donaldgray in #1088

Important

This release contains migrations. Non-breaking, new adjuncts table added.

Warning

"RestrictedAssetIdCharacterString" config value renamed to "RestrictedResourceIdCharacterString". Defaults are set as part of docker image, allowing it to be removed from deployment environments.

Full Changelog: v1.11.5...v1.12.0-rc1

v1.11.5 - Tidy manifest output

Summary

• Small tweaks to generated Manifests - tidy ordering of dimensions and include width + height for thumbnails.
• Prevent duplicates saved values when PATCHing "manifests"values.

What's Changed

• Modify BulkAssetPatcher to use a distinct add by @JackLewis-digirati in #1056
• Bump iiif-net package by @donaldgray in #1069
• Output dimension on v3 manifest thumbnails by @donaldgray in #1071

Full Changelog: v1.11.4...v1.11.5

v1.11.5-rc1 - Tidy manifest output

Summary

• Small tweaks to generated Manifests - tidy ordering of dimensions and include width + height for thumbnails.
• Prevent duplicates saved values when PATCHing "manifests"values.

What's Changed

• Modify BulkAssetPatcher to use a distinct add by @JackLewis-digirati in #1056
• Bump iiif-net package by @donaldgray in #1069
• Output dimension on v3 manifest thumbnails by @donaldgray in #1071

Full Changelog: v1.11.4...v1.11.5-1

v1.11.4 - Bugfix for batch /test

What's Changed

• Prevent incorrect batch completion for /test by @donaldgray in #1062

Full Changelog: v1.11.3...v1.11.4