From a3a1e7934baec161e186a0710abf251619ea21af Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Fri, 24 Jul 2026 23:45:20 +0800 Subject: [PATCH 1/9] docs: provider resource declaration design spec Two-granularity provider declarations (modalities + open ResourceKind vocabulary), kind-aware core routing with provider id whitelist, and MCP dynamic schema/description generation from registered declarations. --- ...24-provider-resource-declaration-design.md | 177 ++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-24-provider-resource-declaration-design.md diff --git a/docs/superpowers/specs/2026-07-24-provider-resource-declaration-design.md b/docs/superpowers/specs/2026-07-24-provider-resource-declaration-design.md new file mode 100644 index 0000000..da7f32c --- /dev/null +++ b/docs/superpowers/specs/2026-07-24-provider-resource-declaration-design.md @@ -0,0 +1,177 @@ +# Provider Resource Declaration Design + +## Goal + +Let every provider declare *what it provides* at two granularities — coarse `modalities` +(existing, closed enum) and fine `kinds` (new, open vocabulary) — plus a one-line content +`description`. Core routes queries by these declarations (skipping providers that cannot +have the requested resource), and the MCP server collects them at startup into its tool +schema and description so an LLM can fill the classification parameters and pick sources. + +## Scope + +Three layers plus two approved extensions: + +- **Layer 0 — declarations (core types):** open `ResourceKind` vocabulary; optional + `kinds` and `description` on `ReferenceProvider`; optional `kind` on `Reference`; + widen `SearchControls.media.kind` from the closed five-value enum to `ResourceKind`. +- **Layer 1 — core routing:** extend provider selection to honor `kinds`; new skip + reasons in explain metadata; new `providers` (id whitelist) search input. +- **Layer 2 — MCP dynamic exposure:** derive the `modalities` / `media.kind` / + `providers` schemas and a per-provider source list in the tool description from the + registered providers at `createRefkitMcpServer` time. +- **Extension 1:** `ReferenceProvider.description` — one-line content-domain summary, + surfaced in the MCP source list so the LLM can judge topical fit. +- **Extension 2:** `providers?: string[]` search input — explicit source subset + selection, exposed as an MCP parameter with a dynamic id enum. + +Non-goals: no change to the `Modality` enum; no statistical / learned routing; no +negative caching (can layer on later without schema changes). + +## Layer 0 — Declarations + +In `packages/core/src/provider.ts`: + +```ts +export type WellKnownKind = + | 'photo' | 'illustration' | 'vector' | 'icon' | 'artwork' + | 'texture' | 'hdri' | '3d-model' + | 'film' | 'animation' + | 'music' | 'sound-effect' + | 'ebook' | 'poem' +export type ResourceKind = WellKnownKind | (string & {}) +``` + +`ResourceKind` is an **open** vocabulary: the union keeps autocomplete for well-known +values while `(string & {})` admits any custom kind a third-party provider invents. +Well-known values are hints, not validation — nothing in core rejects unknown kinds. + +`ReferenceProvider` gains two optional fields: + +```ts +kinds?: readonly ResourceKind[] // what fine-grained kinds this provider offers +description?: string // one-line content-domain summary, e.g. + // "Dutch Golden Age art from the Rijksmuseum" +``` + +`Reference` gains `kind?: string` (zod: `z.string().optional()` in +`packages/core/src/reference.ts`) so results can carry their fine-grained kind. + +`SearchMediaControls.kind` widens from +`'photo' | 'illustration' | 'vector' | 'film' | 'animation'` to `ResourceKind`. This is +a type-level widening — every previously valid value stays valid. + +**Declarations vs. controls are orthogonal:** `kinds` states what a provider *offers* +(routing: whether to query it at all); the `media.kind` entry in `capabilities.controls` +states that a provider can *filter* its own content by kind (translation: whether to +forward the filter upstream, e.g. pixabay's `image_type`). A single-kind provider like +polyhaven declares `kinds: ['texture']` and never needs the control. + +## Layer 1 — Core Routing + +Provider selection in `client.ts` (currently modality-intersection only) becomes, in +order: + +1. **Id whitelist** — if `input.providers` is set, keep only providers whose `id` is in + it. Unknown ids append a warning to `meta.warnings` (LLMs and humans typo; valid + remainder still runs). +2. **Modality intersection** — unchanged. +3. **Kind match** — if `input.controls.media.kind` is set: keep providers that either + do **not** declare `kinds` (conservative inclusion, same progressive philosophy as + `capabilities` in `query.ts`) or declare a `kinds` array containing the requested + value. + +If selection ends empty, throw — extending the existing +`refkit.search: no registered provider supports …` error with the reason (whitelist, +modality, or kind). + +`SearchMeta.providers[].reason` gains two values alongside `'unsupported-modality'`: +`'not-selected'` (excluded by the id whitelist) and `'unsupported-kind'`. Every +registered provider still appears in `meta.providers`, so explain output shows exactly +who was skipped and why. + +`SearchInput` gains `providers?: readonly string[]`. + +## Layer 2 — MCP Dynamic Exposure + +`createRefkitMcpServer(refkit)` already receives the configured client and +`refkit.providers` is public — all data below is read from provider declarations at +server construction; there is no second registry to maintain. + +- **`modalities` parameter:** dynamic `z.enum` over the union of registered providers' + `modalities`. (Also fixes today's failure mode where the LLM picks a modality no + registered provider supports and the search throws.) +- **`controls.media.kind` parameter:** dynamic `z.enum` over (union of declared + `kinds`) ∪ (the five legacy control values `photo / illustration / vector / film / + animation`). The legacy values stay because they remain meaningful as upstream + filter translations for providers that support the `media.kind` control but do not + declare `kinds`; including them also keeps the schema a superset of today's. +- **`providers` parameter:** `z.array(z.enum(registeredProviderIds))` — a closed enum + is correct here because the registered set *is* the universe. +- **Tool description source list:** appended to the `search_references` description, + one provider per line, generated as + `id (modality[/modality]·kind[,kind]): description`; kinds segment omitted when + undeclared, description omitted when absent. Example: + + ``` + Configured sources: + - unsplash (image·photo): high-quality stock photography + - pixabay (image·photo,illustration,vector): free stock images + - polyhaven (image·texture): CC0 PBR textures for 3D work + - freesound (audio·sound-effect): collaborative sound-effect archive + ``` + +- **Meta schema sync:** the `reason` enum in the MCP `searchMetaSchema` adds + `'not-selected'` and `'unsupported-kind'`. + +## Provider Declarations (this iteration) + +All first-party providers gain a `description`. `kinds` is declared where accurate; +broad/aggregator sources stay undeclared (conservative inclusion is the honest +answer): + +| Provider | kinds | Rationale | +|---|---|---| +| unsplash, pexels-image, flickr | `['photo']` | photo-only sources | +| pixabay-image | `['photo','illustration','vector']` | matches its `image_type` filter | +| pexels-video | `['film']` | live-action stock video | +| pixabay-video | `['film','animation']` | matches its video types | +| polyhaven | `['texture']` or `['hdri']` per `assetType` config | factory-computed | +| ambientcg | `['texture']` | PBR materials only | +| met, artic, rijksmuseum, smithsonian, europeana | `['artwork']` | museum/heritage collections | +| freesound | `['sound-effect']` | effects archive | +| jamendo | `['music']` | music platform | +| openverse-audio | `['music','sound-effect']` | aggregates both | +| gutendex | `['ebook']` | Project Gutenberg | +| poetrydb | `['poem']` | poems only | +| internet-archive | `['film','ebook']` | maps its two modalities | +| brave, wikimedia-commons, openverse-image | *undeclared* | open-web / omnibus content | + +**Result annotation:** single-kind providers stamp their kind on every `Reference` +statically. pixabay maps the upstream `type` field per item. Multi-kind providers +without upstream type data leave `kind` unset. + +**Testkit:** `provider-testkit` adds one conformance check — when a provider declares +non-empty `kinds` and a returned reference carries `kind`, that value must be in the +declared set. + +## Backward Compatibility + +Every new field is optional; undeclared third-party providers behave exactly as today +(conservative inclusion on kind-filtered queries). `media.kind` widening and the MCP +dynamic enums are supersets of current accepted values. The `reason` enum extension is +additive (a `minor` for `@refkit/core` and `@refkit/mcp`; provider packages get a +`minor` for the new declarations). + +## Error Handling + +Unknown ids in `providers` warn and continue; an empty selection after filtering +throws with the narrowing reason. No change to provider-level error semantics. + +## Verification + +TDD per production change. Core: selection-predicate units (kind match, undeclared +inclusion, whitelist, unknown-id warning, empty-selection error). MCP: construct a +server over two fake providers and assert the generated description contains the +source list and the dynamic enums carry the expected values. Final gate: +`pnpm typecheck`, `pnpm test:run`, `pnpm build`. From a34db6342cf3c42a526dbe18ec8d063beb0e63f2 Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Fri, 24 Jul 2026 23:56:31 +0800 Subject: [PATCH 2/9] docs: provider resource declaration implementation plan --- ...026-07-24-provider-resource-declaration.md | 1020 +++++++++++++++++ 1 file changed, 1020 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-24-provider-resource-declaration.md diff --git a/docs/superpowers/plans/2026-07-24-provider-resource-declaration.md b/docs/superpowers/plans/2026-07-24-provider-resource-declaration.md new file mode 100644 index 0000000..439864e --- /dev/null +++ b/docs/superpowers/plans/2026-07-24-provider-resource-declaration.md @@ -0,0 +1,1020 @@ +# Provider Resource Declaration Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Providers declare what they provide at two granularities (`modalities` + open `kinds` vocabulary) plus a one-line `description`; core routes by these declarations (including a new `providers` id whitelist), and the MCP server derives its tool schema and description from them at startup. + +**Architecture:** All declaration data lives on the `ReferenceProvider` object (single source of truth). Core's provider-selection predicate in `client.ts` grows two narrowing steps (id whitelist, kind match) with explain-metadata skip reasons. `createRefkitMcpServer` reads `refkit.providers` at construction to build dynamic zod enums and a per-provider source list in the tool description. + +**Tech Stack:** TypeScript, zod, vitest, pnpm workspaces, changesets. Spec: `docs/superpowers/specs/2026-07-24-provider-resource-declaration-design.md`. + +**Conventions:** Run all commands from the repo root. Test commands use `pnpm vitest run ` (vitest workspace resolves the right project config). Every task ends with a commit. + +--- + +### Task 1: core types — `ResourceKind`, `ReferenceProvider.kinds` / `.description` + +**Files:** +- Modify: `packages/core/src/provider.ts` (after the `SearchSafety` line, and inside `ReferenceProvider` / `SearchMediaControls`) +- Modify: `packages/core/src/index.ts` (type exports) +- Test: `packages/core/src/__tests__/provider.test.ts` + +- [ ] **Step 1: Write the failing test** + +Append to `packages/core/src/__tests__/provider.test.ts` (it already imports `defineProvider` — if not, add `import { defineProvider } from '../provider'`): + +```ts +describe('resource declarations', () => { + it('defineProvider preserves kinds and description (open vocabulary)', () => { + const p = defineProvider({ + id: 'x', + modalities: ['image'], + kinds: ['texture', 'my-custom-kind'], // well-known + custom must both typecheck + description: 'CC0 textures for tests', + search: async () => [], + }) + expect(p.kinds).toEqual(['texture', 'my-custom-kind']) + expect(p.description).toBe('CC0 textures for tests') + }) +}) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pnpm vitest run packages/core/src/__tests__/provider.test.ts` +Expected: FAIL — TS error: `kinds` does not exist on the provider type. + +- [ ] **Step 3: Implement** + +In `packages/core/src/provider.ts`, after the `export type SearchSafety = …` line (line 16), add: + +```ts +/** Fine-grained resource kind. Open vocabulary: well-known values get + * autocomplete; any other string is a valid custom kind. Well-known values are + * hints, not validation — core never rejects unknown kinds. */ +export type WellKnownKind = + | 'photo' | 'illustration' | 'vector' | 'icon' | 'artwork' + | 'texture' | 'hdri' | '3d-model' + | 'film' | 'animation' + | 'music' | 'sound-effect' + | 'ebook' | 'poem' +export type ResourceKind = WellKnownKind | (string & {}) +``` + +In `SearchMediaControls`, widen the `kind` field (keep the other fields untouched): + +```ts +export interface SearchMediaControls { + kind?: ResourceKind + size?: 'small' | 'medium' | 'large' + minWidth?: number + minHeight?: number + duration?: 'short' | 'medium' | 'long' +} +``` + +In `ReferenceProvider`, after the `modalities: Modality[]` line, add: + +```ts + /** Fine-grained kinds this provider offers (routing: a kind-filtered search + * skips providers whose declared kinds lack the requested value; undeclared + * providers are conservatively included). Orthogonal to the `media.kind` + * entry in capabilities.controls, which declares upstream FILTER support. */ + kinds?: readonly ResourceKind[] + /** One-line content-domain summary, surfaced in the MCP tool's source list + * so an agent can judge topical fit (e.g. "CC0 PBR textures for 3D work"). */ + description?: string +``` + +In `packages/core/src/index.ts`, add `WellKnownKind` and `ResourceKind` to the type-export block from `'./provider'` (the block at lines 26–45): + +```ts + WellKnownKind, + ResourceKind, +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pnpm vitest run packages/core/src/__tests__/provider.test.ts` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add packages/core/src/provider.ts packages/core/src/index.ts packages/core/src/__tests__/provider.test.ts +git commit -m "feat(core): ResourceKind vocabulary + provider kinds/description declarations" +``` + +--- + +### Task 2: core — `Reference.kind` + +**Files:** +- Modify: `packages/core/src/reference.ts` +- Test: `packages/core/src/__tests__/reference.test.ts` + +- [ ] **Step 1: Write the failing test** + +Append to `packages/core/src/__tests__/reference.test.ts` (reuse the file's existing valid-reference fixture if one exists; otherwise use this base): + +```ts +it('accepts an optional fine-grained kind', () => { + const base = { + id: 'p:1', + modality: 'image', + source: { providerId: 'p', sourceUrl: 'https://p/1' }, + canonicalUrl: 'https://p/1', + rights: { license: 'CC0-1.0', rehostPolicy: 'cache-allowed', raw: { sourceTerms: 't', sourceUrl: 'https://p/1' } }, + verifiedAt: '2026-07-24T00:00:00.000Z', + relevance: 0, + } + expect(parseReference({ ...base, kind: 'texture' }).kind).toBe('texture') + expect(parseReference(base).kind).toBeUndefined() +}) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pnpm vitest run packages/core/src/__tests__/reference.test.ts` +Expected: FAIL — TS: `kind` does not exist on type `Reference` (the `.kind` read on the parse result). + +- [ ] **Step 3: Implement** + +In `packages/core/src/reference.ts`, add to the `Reference` interface after `modality: Modality`: + +```ts + /** Fine-grained kind (open vocabulary, see ResourceKind), e.g. 'photo', 'texture'. */ + kind?: string +``` + +And in `referenceSchema`, after `modality: modalitySchema,`: + +```ts + kind: z.string().optional(), +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pnpm vitest run packages/core/src/__tests__/reference.test.ts` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add packages/core/src/reference.ts packages/core/src/__tests__/reference.test.ts +git commit -m "feat(core): optional Reference.kind" +``` + +--- + +### Task 3: core routing — kind match + `unsupported-kind` skip reason + +**Files:** +- Modify: `packages/core/src/client.ts` (lines 64–74 `ProviderSearchStatus`, 196–199 selection, 239–242 status seeding, 379 meta fallback) +- Test: `packages/core/src/__tests__/client.test.ts` + +- [ ] **Step 1: Write the failing tests** + +Append to `packages/core/src/__tests__/client.test.ts` (the file already defines `ref(…)` and `provider(…)` helpers at the top — reuse `ref`): + +```ts +describe('kind-aware routing', () => { + const kindProvider = (id: string, kinds: readonly string[], refs: Reference[]) => + defineProvider({ id, modalities: ['image'], kinds, search: async () => refs }) + + it('skips providers whose declared kinds lack the requested value', async () => { + const rk = createRefkit({ providers: [ + kindProvider('tex', ['texture'], [ref('tex-1', 'https://t/1')]), + kindProvider('ph', ['photo'], [ref('ph-1', 'https://p/1')]), + ] }) + const { references, meta } = await rk.searchWithMeta({ + query: 'x', modalities: ['image'], controls: { media: { kind: 'texture' } }, + }) + expect(references.map(r => r.canonicalUrl)).toEqual(['https://t/1']) + expect(meta.providers.find(p => p.providerId === 'ph')) + .toMatchObject({ status: 'skipped', reason: 'unsupported-kind' }) + }) + + it('conservatively includes providers that declare no kinds', async () => { + const rk = createRefkit({ providers: [ + provider('legacy', [ref('legacy-1', 'https://l/1')]), // no kinds declared + kindProvider('ph', ['photo'], [ref('ph-1', 'https://p/1')]), + ] }) + const out = await rk.search({ + query: 'x', modalities: ['image'], controls: { media: { kind: 'texture' } }, + }) + expect(out.map(r => r.canonicalUrl)).toEqual(['https://l/1']) + }) + + it('throws with the kind in the message when nothing matches', async () => { + const rk = createRefkit({ providers: [kindProvider('ph', ['photo'], [])] }) + await expect(rk.search({ + query: 'x', modalities: ['image'], controls: { media: { kind: 'texture' } }, + })).rejects.toThrow('kind "texture"') + }) +}) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `pnpm vitest run packages/core/src/__tests__/client.test.ts` +Expected: the three new tests FAIL (all providers still queried; no `unsupported-kind` reason). Pre-existing tests PASS. + +- [ ] **Step 3: Implement** + +In `packages/core/src/client.ts`: + +(a) Widen `ProviderSearchStatus.reason` (line 70): + +```ts + reason?: 'unsupported-modality' | 'unsupported-kind' +``` + +(b) Replace the selection block (lines 196–199): + +```ts + const kindFilter = input.controls?.media?.kind + const skipReasonFor = (p: ReferenceProvider): NonNullable | undefined => { + if (!p.modalities.some(m => input.modalities.includes(m))) return 'unsupported-modality' + if (kindFilter !== undefined && p.kinds && !p.kinds.includes(kindFilter)) return 'unsupported-kind' + return undefined + } + const skipReasons = new Map>() + for (const p of options.providers) { + const reason = skipReasonFor(p) + if (reason) skipReasons.set(p.id, reason) + } + const chosen = options.providers.filter(p => !skipReasons.has(p.id)) + if (chosen.length === 0) { + throw new Error( + `refkit.search: no registered provider supports modalities [${input.modalities.join(', ')}]` + + (kindFilter !== undefined ? ` with kind "${kindFilter}"` : ''), + ) + } +``` + +Note: the modality-only message is byte-identical to today's — the existing +assertion at `client.test.ts:266` must keep passing. + +(c) Replace the status-seeding loop inside `runPass` (lines 240–242): + +```ts + for (const p of options.providers) { + const reason = skipReasons.get(p.id) + if (reason) statusByProvider.set(p.id, { providerId: p.id, status: 'skipped', reason }) + } +``` + +(d) Update the meta fallback (line 379): + +```ts + providers: options.providers.map(p => pass.statusByProvider.get(p.id) + ?? { providerId: p.id, status: 'skipped', reason: skipReasons.get(p.id) ?? 'unsupported-modality' }), +``` + +- [ ] **Step 4: Run the full core suite** + +Run: `pnpm vitest run packages/core` +Expected: PASS (including the pre-existing exact-message test at line 266). + +- [ ] **Step 5: Commit** + +```bash +git add packages/core/src/client.ts packages/core/src/__tests__/client.test.ts +git commit -m "feat(core): kind-aware provider routing with unsupported-kind skip reason" +``` + +--- + +### Task 4: core routing — `providers` id whitelist + +**Files:** +- Modify: `packages/core/src/client.ts` (`SearchInput`, `ProviderSearchStatus.reason`, selection block, warnings init at line 361) +- Test: `packages/core/src/__tests__/client.test.ts` + +- [ ] **Step 1: Write the failing tests** + +Append to `packages/core/src/__tests__/client.test.ts`: + +```ts +describe('providers whitelist', () => { + it('restricts fan-out and reports not-selected in meta', async () => { + const rk = createRefkit({ providers: [ + provider('a', [ref('a-1', 'https://a/1')]), + provider('b', [ref('b-1', 'https://b/1')]), + ] }) + const { references, meta } = await rk.searchWithMeta({ query: 'x', modalities: ['image'], providers: ['a'] }) + expect(references.map(r => r.canonicalUrl)).toEqual(['https://a/1']) + expect(meta.providers.find(p => p.providerId === 'b')) + .toMatchObject({ status: 'skipped', reason: 'not-selected' }) + }) + + it('warns on unknown ids and still runs the valid remainder', async () => { + const rk = createRefkit({ providers: [provider('a', [ref('a-1', 'https://a/1')])] }) + const { references, meta } = await rk.searchWithMeta({ query: 'x', modalities: ['image'], providers: ['a', 'nope'] }) + expect(references).toHaveLength(1) + expect(meta.warnings.some(w => w.includes('"nope"'))).toBe(true) + }) + + it('throws when the whitelist selects nothing', async () => { + const rk = createRefkit({ providers: [provider('a', [])] }) + await expect(rk.search({ query: 'x', modalities: ['image'], providers: ['nope'] })) + .rejects.toThrow(/no registered provider/) + }) +}) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `pnpm vitest run packages/core/src/__tests__/client.test.ts` +Expected: the three new tests FAIL (TS: `providers` not on `SearchInput`). + +- [ ] **Step 3: Implement** + +In `packages/core/src/client.ts`: + +(a) `SearchInput` — after the `providerOptions` field: + +```ts + /** Restrict this search to these provider ids. Unknown ids append a warning + * to meta.warnings and are otherwise ignored; excluded providers appear in + * meta.providers as skipped with reason 'not-selected'. */ + providers?: readonly string[] +``` + +(b) `ProviderSearchStatus.reason` — add the third value: + +```ts + reason?: 'unsupported-modality' | 'unsupported-kind' | 'not-selected' +``` + +(c) In the selection block from Task 3, insert before `const kindFilter = …`: + +```ts + const idWhitelist = input.providers + const preWarnings: string[] = [] + if (idWhitelist) { + const known = new Set(options.providers.map(p => p.id)) + for (const id of idWhitelist) { + if (!known.has(id)) preWarnings.push(`unknown provider id in providers: "${id}"`) + } + } +``` + +and make the whitelist the FIRST check inside `skipReasonFor` (spec: reason +reflects the first step that excluded the provider): + +```ts + if (idWhitelist && !idWhitelist.includes(p.id)) return 'not-selected' +``` + +Extend the empty-selection error with the whitelist segment (append after the kind segment): + +```ts + + (idWhitelist ? ` within providers [${idWhitelist.join(', ')}]` : ''), +``` + +(d) Warnings init (line 361) — carry the pre-warnings: + +```ts + const warnings: string[] = [...preWarnings] +``` + +- [ ] **Step 4: Run the full core suite** + +Run: `pnpm vitest run packages/core` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add packages/core/src/client.ts packages/core/src/__tests__/client.test.ts +git commit -m "feat(core): providers id whitelist with not-selected skip reason and unknown-id warnings" +``` + +--- + +### Task 5: testkit — declared-kinds consistency check + +**Files:** +- Modify: `packages/provider-testkit/src/index.ts` (inside `searchConformant`'s per-item loop, after the `source.providerId` check at line 57–59) +- Test: `packages/provider-testkit/src/__tests__/testkit.test.ts` + +- [ ] **Step 1: Write the failing test** + +Append to `packages/provider-testkit/src/__tests__/testkit.test.ts` (reuse the file's existing fake-provider/fetch helpers if present; this block is self-contained either way): + +```ts +import { defineProvider, type Reference } from '@refkit/core' +import { searchConformant } from '../index' + +const kindRef = (kind?: string): Reference => ({ + id: 'kp:1', + modality: 'image', + ...(kind ? { kind } : {}), + source: { providerId: 'kp', sourceUrl: 'https://kp/1' }, + canonicalUrl: 'https://kp/1', + rights: { license: 'CC0-1.0', rehostPolicy: 'cache-allowed', raw: { sourceTerms: 't', sourceUrl: 'https://kp/1' } }, + verifiedAt: '2026-07-24T00:00:00.000Z', + relevance: 0, +}) + +describe('declared-kinds consistency', () => { + const neverFetch = (async () => { throw new Error('no network') }) as unknown as typeof fetch + + it('rejects a result whose kind is outside the declared set', async () => { + const p = defineProvider({ id: 'kp', modalities: ['image'], kinds: ['texture'], search: async () => [kindRef('photo')] }) + await expect(searchConformant(p, neverFetch)).rejects.toThrow(/kind "photo" is not in the provider's declared kinds/) + }) + + it('accepts a matching kind and a missing kind', async () => { + const p = defineProvider({ id: 'kp', modalities: ['image'], kinds: ['texture'], search: async () => [kindRef('texture'), kindRef()] }) + await expect(searchConformant(p, neverFetch)).resolves.toHaveLength(2) + }) +}) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pnpm vitest run packages/provider-testkit` +Expected: the first new test FAILS (no error thrown for the mismatched kind). + +- [ ] **Step 3: Implement** + +In `packages/provider-testkit/src/index.ts`, inside the `raw.map((item, i) => { … })` loop, after the `source.providerId` check (line 57–59), add: + +```ts + // Declared-kinds consistency: a provider stating what it offers must not + // emit results outside that set. Missing kind is allowed (annotation is + // optional); only a contradicting value is a violation. + if (provider.kinds && provider.kinds.length > 0 && ref.kind !== undefined && !provider.kinds.includes(ref.kind)) { + throw new Error(`[${provider.id}] result #${i} kind "${ref.kind}" is not in the provider's declared kinds [${provider.kinds.join(', ')}]`) + } +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pnpm vitest run packages/provider-testkit` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add packages/provider-testkit/src/index.ts packages/provider-testkit/src/__tests__/testkit.test.ts +git commit -m "feat(testkit): enforce declared-kinds consistency in searchConformant" +``` + +--- + +### Task 6: MCP — dynamic schema, source list, `providers` parameter, `kind` in output + +**Files:** +- Modify: `packages/mcp/src/index.ts` +- Test: `packages/mcp/src/__tests__/mcp.test.ts` + +- [ ] **Step 1: Write the failing test** + +Append to `packages/mcp/src/__tests__/mcp.test.ts` (file already imports `createRefkit`, `defineProvider`, `createRefkitMcpServer`, `Client`, `InMemoryTransport`): + +```ts +describe('dynamic declaration-derived schema', () => { + async function declClient() { + const tex = defineProvider({ + id: 'texsrc', modalities: ['image'], kinds: ['texture', 'custom-kind'], + description: 'CC0 textures for tests', search: async () => [], + }) + const plain = defineProvider({ id: 'plain', modalities: ['audio'], search: async () => [] }) + const refkit = createRefkit({ providers: [tex, plain], fetch: (async () => new Response('{}')) as typeof fetch }) + const server = createRefkitMcpServer(refkit) + const [clientT, serverT] = InMemoryTransport.createLinkedPair() + const client = new Client({ name: 'test', version: '1.0.0' }) + await Promise.all([client.connect(clientT), server.connect(serverT)]) + return client + } + + it('appends a per-provider source list to the tool description', async () => { + const client = await declClient() + const { tools } = await client.listTools() + const tool = tools.find(t => t.name === 'search_references')! + expect(tool.description).toContain('Configured sources:') + expect(tool.description).toContain('- texsrc (image·texture,custom-kind): CC0 textures for tests') + expect(tool.description).toContain('- plain (audio)') + await client.close() + }) + + it('derives modalities / media.kind / providers enums from declarations', async () => { + const client = await declClient() + const { tools } = await client.listTools() + const schema = tools.find(t => t.name === 'search_references')!.inputSchema as Record + const modalityEnum = schema.properties.modalities.items.enum as string[] + expect(modalityEnum.sort()).toEqual(['audio', 'image']) + const providerEnum = schema.properties.providers.items.enum as string[] + expect(providerEnum).toEqual(['texsrc', 'plain']) + const kindEnum = schema.properties.controls.properties.media.properties.kind.enum as string[] + expect(kindEnum).toEqual(expect.arrayContaining(['photo', 'illustration', 'vector', 'film', 'animation', 'texture', 'custom-kind'])) + await client.close() + }) +}) +``` + +If the JSON-schema property paths differ from the SDK's actual zod conversion, +`console.log(JSON.stringify(schema, null, 2))` once, correct the paths in the +test, and remove the log. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pnpm vitest run packages/mcp` +Expected: the two new tests FAIL (static description, static enums, no `providers` property). + +- [ ] **Step 3: Implement** + +In `packages/mcp/src/index.ts`: + +(a) After the `ORIENTATIONS` constant, add: + +```ts +// Legacy media.kind control values — kept in the dynamic enum because they stay +// meaningful as upstream filter translations for providers that support the +// media.kind control without declaring kinds. +const BASE_MEDIA_KINDS = ['photo', 'illustration', 'vector', 'film', 'animation'] as const +``` + +(b) Turn the module-level `searchControlsSchema` into a factory (replace the `const searchControlsSchema = z.object({ … })` block; the body is identical except the `media.kind` line): + +```ts +function buildSearchControlsSchema(kindValues: [string, ...string[]]) { + return z.object({ + orientation: z.enum(ORIENTATIONS).optional(), + color: z.string().optional(), + language: z.string().optional(), + sort: z.enum(['relevance', 'latest', 'popular', 'interesting']).optional(), + safety: z.enum(['strict', 'moderate', 'off']).optional(), + license: z.object({ + commercial: z.boolean().optional(), + modification: z.boolean().optional(), + allowUnknown: z.boolean().optional(), + }).optional(), + media: z.object({ + kind: z.enum(kindValues).optional(), + size: z.enum(['small', 'medium', 'large']).optional(), + minWidth: z.number().int().nonnegative().optional(), + minHeight: z.number().int().nonnegative().optional(), + duration: z.enum(['short', 'medium', 'long']).optional(), + }).optional(), + creator: z.object({ + id: z.string().optional(), + name: z.string().optional(), + }).optional(), + text: z.object({ + copyright: z.enum(['public-domain', 'copyrighted', 'any']).optional(), + }).optional(), + page: z.number().int().positive().optional(), + }) +} +``` + +(c) `toAgentRef` — add `kind: r.kind,` to the `base` object (after `modality`), and add to `agentRefSchema` after `modality: z.string(),`: + +```ts + kind: z.string().optional().describe('fine-grained resource kind, e.g. photo / texture / ebook'), +``` + +(d) `searchMetaSchema` — widen the provider skip-reason enum (line 140): + +```ts + reason: z.enum(['unsupported-modality', 'unsupported-kind', 'not-selected']).optional(), +``` + +(e) In `createRefkitMcpServer`, before `server.registerTool('search_references', …)`, derive everything from declarations: + +```ts + const registered = refkit.providers + const modalityValues = [...new Set(registered.flatMap(p => p.modalities))] as [Modality, ...Modality[]] + const kindValues = [...new Set([...BASE_MEDIA_KINDS, ...registered.flatMap(p => p.kinds ?? [])])] as [string, ...string[]] + const providerIds = registered.map(p => p.id) as [string, ...string[]] + const sourceList = registered.map(p => { + const kinds = p.kinds?.length ? `·${p.kinds.join(',')}` : '' + const desc = p.description ? `: ${p.description}` : '' + return `- ${p.id} (${p.modalities.join('/')}${kinds})${desc}` + }).join('\n') + const searchControlsSchema = buildSearchControlsSchema(kindValues) +``` + +Add `import type { Modality } from '@refkit/core'` if `Modality` is not already imported. + +(f) In the `search_references` registration: +- description: append `` + '\n\nConfigured sources:\n' + sourceList `` to the existing string. +- `modalities` input: `z.array(z.enum(modalityValues)).optional().describe('default ["image"]')` + and drop the now-unused static `MODALITIES` reference there (keep the + `MODALITIES` constant — `searchMetaSchema` still uses it). +- add after `providerOptions`: + +```ts + providers: z.array(z.enum(providerIds)).optional().describe('restrict the search to these source ids (see Configured sources in this tool description)'), +``` + +- handler: add `providers` to the destructured args and pass `providers` through in `searchInput`. + +- [ ] **Step 4: Run the full mcp suite** + +Run: `pnpm vitest run packages/mcp` +Expected: PASS (new tests and all pre-existing ones — the pre-existing tests exercise the openverse-only server, whose derived enums are supersets of what they use). + +- [ ] **Step 5: Commit** + +```bash +git add packages/mcp/src/index.ts packages/mcp/src/__tests__/mcp.test.ts +git commit -m "feat(mcp): declaration-derived schema, source list, providers parameter, kind in output" +``` + +--- + +### Task 7: provider declarations — photo/stock group + +**Files:** +- Modify: `packages/provider-unsplash/src/index.ts` (factory at line 61, `toReference` modality line 45) +- Modify: `packages/provider-pexels/src/index.ts` (factories at 72 and 132, modality lines 56 and 115) +- Modify: `packages/provider-pixabay/src/index.ts` (factories at 100 and 183, modality lines 84 and 165, `PixabayHit` interface) +- Modify: `packages/provider-flickr/src/index.ts` (factory at 182, modality line 165) +- Test: each package's main test file (`unsplash.test.ts`, `pexels.test.ts`, `pixabay.test.ts`, `flickr.test.ts`) + +Pattern for every provider in Tasks 7–10: insert `kinds` + `description` in the +`defineProvider({ … })` literal directly after its `modalities` line; insert +`kind` in the `Reference`-building literal directly after its `modality` line. + +- [ ] **Step 1: Write the failing metadata tests** + +Append one block per file (factory call must match the file's existing test setup — copy the config argument from an existing test in the same file): + +`packages/provider-unsplash/src/__tests__/unsplash.test.ts`: + +```ts +it('declares kinds and description', () => { + const p = unsplash({ accessKey: 'k' }) + expect(p.kinds).toEqual(['photo']) + expect(p.description).toMatch(/photo/i) +}) +``` + +`packages/provider-pexels/src/__tests__/pexels.test.ts` (file already imports `pexels, pexelsVideo`): + +```ts +it('declares kinds and description (image + video factories)', () => { + expect(pexels({ apiKey: 'k' }).kinds).toEqual(['photo']) + expect(pexelsVideo({ apiKey: 'k' }).kinds).toEqual(['film']) +}) +``` + +`packages/provider-pixabay/src/__tests__/pixabay.test.ts` (file already imports `pixabay, pixabayVideo`): + +```ts +it('declares kinds and description (image + video factories)', () => { + expect(pixabay({ key: 'k' }).kinds).toEqual(['photo', 'illustration', 'vector']) + expect(pixabayVideo({ key: 'k' }).kinds).toEqual(['film', 'animation']) +}) + +it('maps the upstream hit type to Reference.kind', async () => { + const hit = { + id: 1, tags: 'x', user: 'u', pageURL: 'https://pixabay.com/p/1', + previewURL: 'https://cdn/p.jpg', previewWidth: 10, previewHeight: 10, + webformatURL: 'https://cdn/w.jpg', largeImageURL: 'https://cdn/l.jpg', + imageWidth: 100, imageHeight: 100, type: 'vectors/svg', + } + const fakeFetch = (async () => new Response(JSON.stringify({ hits: [hit] }), { status: 200 })) as typeof fetch + const refs = await pixabay({ key: 'k' }).search({ text: 'x', modalities: ['image'] }, { fetch: fakeFetch }) + expect(refs[0].kind).toBe('vector') +}) +``` + +`packages/provider-flickr/src/__tests__/flickr.test.ts`: + +```ts +it('declares kinds and description', () => { + const p = flickr({ apiKey: 'k' }) + expect(p.kinds).toEqual(['photo']) + expect(p.description).toBeTruthy() +}) +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `pnpm vitest run packages/provider-unsplash packages/provider-pexels packages/provider-pixabay packages/provider-flickr` +Expected: new tests FAIL (`kinds` undefined). + +- [ ] **Step 3: Implement** + +Factory declarations (insert after each `modalities:` line): + +| File / factory | insert | +|---|---| +| unsplash (line 61) | `kinds: ['photo'],` · `description: 'High-quality free stock photography (Unsplash)',` | +| pexels image (72) | `kinds: ['photo'],` · `description: 'Free stock photos (Pexels)',` | +| pexels video (132) | `kinds: ['film'],` · `description: 'Free stock videos (Pexels)',` | +| pixabay image (100) | `kinds: ['photo', 'illustration', 'vector'],` · `description: 'Free stock photos, illustrations and vectors (Pixabay)',` | +| pixabay video (183) | `kinds: ['film', 'animation'],` · `description: 'Free stock videos and animations (Pixabay)',` | +| flickr (182) | `kinds: ['photo'],` · `description: 'Community photography with per-item CC licensing (Flickr)',` | + +Result annotation (insert after each `modality:` line in the Reference literal): + +- unsplash (45), pexels image (56), flickr (165): `kind: 'photo',` +- pexels video (115): `kind: 'film',` +- pixabay video (165): no annotation (multi-kind, no per-item type data — spec). +- pixabay image (84): per-item mapping. Add `type?: string` to `PixabayHit`, add above `toReference`: + +```ts +// Upstream image `type` values: 'photo' | 'illustration' | 'vectors/svg'. +function pixabayKind(t: string | undefined): string | undefined { + if (t === 'photo' || t === 'illustration') return t + if (t === 'vectors/svg' || t === 'vector/svg') return 'vector' + return undefined +} +``` + +and in the image `toReference` literal after `modality: 'image',`: + +```ts + ...(pixabayKind(h.type) ? { kind: pixabayKind(h.type) } : {}), +``` + +- [ ] **Step 4: Run to verify they pass** + +Run: `pnpm vitest run packages/provider-unsplash packages/provider-pexels packages/provider-pixabay packages/provider-flickr` +Expected: PASS (existing fixture tests keep passing — `kind` is additive). + +- [ ] **Step 5: Commit** + +```bash +git add packages/provider-unsplash packages/provider-pexels packages/provider-pixabay packages/provider-flickr +git commit -m "feat(providers): kinds/description declarations for photo/stock providers" +``` + +--- + +### Task 8: provider declarations — museum/heritage group + +**Files:** +- Modify: `packages/provider-met/src/index.ts` (factory 72, modality 56) +- Modify: `packages/provider-artic/src/index.ts` (factory 73, modality 46) +- Modify: `packages/provider-rijksmuseum/src/index.ts` (factory 140, modality 120) +- Modify: `packages/provider-smithsonian/src/index.ts` (factory 70, modality 54) +- Modify: `packages/provider-europeana/src/index.ts` (factory 99, modality 83) +- Test: each package's main test file + +- [ ] **Step 1: Write the failing metadata tests** + +Append to each of `met.test.ts`, `artic.test.ts`, `rijksmuseum.test.ts`, `smithsonian.test.ts`, `europeana.test.ts` this block, substituting the factory call per file — `met()`, `artic()`, `rijksmuseum()`, `smithsonian({ apiKey: 'k' })`, `europeana({ apiKey: 'k' })` (verified against each file's existing imports): + +```ts +it('declares kinds and description', () => { + const p = met() // ← per-file factory call from the list above + expect(p.kinds).toEqual(['artwork']) + expect(p.description).toBeTruthy() +}) +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `pnpm vitest run packages/provider-met packages/provider-artic packages/provider-rijksmuseum packages/provider-smithsonian packages/provider-europeana` +Expected: new tests FAIL. + +- [ ] **Step 3: Implement** + +All five factories get `kinds: ['artwork'],` after `modalities:`, plus: + +| Factory | description | +|---|---| +| met | `'Open Access artworks from the Metropolitan Museum of Art'` | +| artic | `'CC0 artworks from the Art Institute of Chicago'` | +| rijksmuseum | `'Rijksmuseum collection artworks, incl. the Dutch Golden Age'` | +| smithsonian | `'Open Access objects from Smithsonian museums'` | +| europeana | `'European cultural heritage from museums, libraries and archives (Europeana)'` | + +All five Reference literals get `kind: 'artwork',` after `modality: 'image',`. + +- [ ] **Step 4: Run to verify they pass** + +Run: `pnpm vitest run packages/provider-met packages/provider-artic packages/provider-rijksmuseum packages/provider-smithsonian packages/provider-europeana` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add packages/provider-met packages/provider-artic packages/provider-rijksmuseum packages/provider-smithsonian packages/provider-europeana +git commit -m "feat(providers): artwork kind declarations for museum/heritage providers" +``` + +--- + +### Task 9: provider declarations — audio/text group + +**Files:** +- Modify: `packages/provider-freesound/src/index.ts` (factory 96, modality 81) +- Modify: `packages/provider-jamendo/src/index.ts` (factory 94, modality 77) +- Modify: `packages/provider-openverse/src/index.ts` (audio factory 225 only; image factory 154 gets description only, Task 10) +- Modify: `packages/provider-gutendex/src/index.ts` (factory 82, modality 64) +- Modify: `packages/provider-poetrydb/src/index.ts` (factory 109, modality 32) +- Modify: `packages/provider-internet-archive/src/index.ts` (factory 94, Reference literal at 79 where `modality` is a variable) +- Test: each package's main test file + +- [ ] **Step 1: Write the failing metadata tests** + +Same `it('declares kinds and description', …)` pattern as Task 8, one per file, with these exact factory calls and expectations (all names verified against each file's existing imports; the openverse block goes in `openverse.test.ts` which already imports `openverseAudio`): + +```ts +expect(freesound({ apiKey: 'k' }).kinds).toEqual(['sound-effect']) +expect(jamendo({ clientId: 'cid' }).kinds).toEqual(['music']) +expect(openverseAudio().kinds).toEqual(['music', 'sound-effect']) +expect(gutendex().kinds).toEqual(['ebook']) +expect(poetrydb().kinds).toEqual(['poem']) +expect(internetArchive().kinds).toEqual(['film', 'ebook']) +``` + +Each block also asserts `expect(p.description).toBeTruthy()`. + +- [ ] **Step 2: Run to verify they fail** + +Run: `pnpm vitest run packages/provider-freesound packages/provider-jamendo packages/provider-openverse packages/provider-gutendex packages/provider-poetrydb packages/provider-internet-archive` +Expected: new tests FAIL. + +- [ ] **Step 3: Implement** + +| Factory | kinds | description | result annotation | +|---|---|---|---| +| freesound | `['sound-effect']` | `'Collaborative archive of CC-licensed sounds (Freesound)'` | `kind: 'sound-effect',` after modality line 81 | +| jamendo | `['music']` | `'CC-licensed independent music (Jamendo)'` | `kind: 'music',` after modality line 77 | +| openverse-audio (225) | `['music', 'sound-effect']` | `'Aggregated openly licensed music and sound effects (Openverse)'` | none (multi-kind, no per-item split) | +| gutendex | `['ebook']` | `'Public-domain ebooks from Project Gutenberg (Gutendex)'` | `kind: 'ebook',` after modality line 64 | +| poetrydb | `['poem']` | `'Classic public-domain poetry (PoetryDB)'` | `kind: 'poem',` after modality line 32 | +| internet-archive | `['film', 'ebook']` | `'Public-domain and CC films and texts from the Internet Archive'` | after `modality,` at line 79: `kind: modality === 'video' ? 'film' : 'ebook',` | + +- [ ] **Step 4: Run to verify they pass** + +Run: `pnpm vitest run packages/provider-freesound packages/provider-jamendo packages/provider-openverse packages/provider-gutendex packages/provider-poetrydb packages/provider-internet-archive` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add packages/provider-freesound packages/provider-jamendo packages/provider-openverse packages/provider-gutendex packages/provider-poetrydb packages/provider-internet-archive +git commit -m "feat(providers): kind declarations for audio and text providers" +``` + +--- + +### Task 10: provider declarations — 3D group + description-only omnibus group + +**Files:** +- Modify: `packages/provider-polyhaven/src/index.ts` (polyhaven factory 82–87, `toReference` signature at ~55–80 and modality line 67; ambientcg factory 172–176, `acgToReference` modality line 159) +- Modify: `packages/provider-brave/src/index.ts` (factory 61 — description only) +- Modify: `packages/provider-wikimedia-commons/src/index.ts` (factory 145 — description only) +- Modify: `packages/provider-openverse/src/index.ts` (image factory 154 — description only) +- Test: `polyhaven.test.ts`, `ambientcg.test.ts`, `brave.test.ts`, `wikimedia-commons.test.ts`, `openverse.test.ts` + +- [ ] **Step 1: Write the failing metadata tests** + +`packages/provider-polyhaven/src/__tests__/polyhaven.test.ts`: + +```ts +it('declares kinds per assetType', () => { + expect(polyhaven().kinds).toEqual(['texture']) + expect(polyhaven({ assetType: 'hdris' }).kinds).toEqual(['hdri']) +}) +``` + +`packages/provider-polyhaven/src/__tests__/ambientcg.test.ts`: + +```ts +it('declares kinds and description', () => { + expect(ambientcg().kinds).toEqual(['texture']) + expect(ambientcg().description).toMatch(/texture|material/i) +}) +``` + +`brave.test.ts` / `wikimedia-commons.test.ts` / `openverse.test.ts` — same block with the per-file factory call: `brave({ token: 'SECRET' })`, `wikimediaCommons()`, `openverse()` (verified against each file's existing imports): + +```ts +it('declares a description but no kinds (omnibus source)', () => { + const p = brave({ token: 'SECRET' }) // ← per-file factory call from the list above + expect(p.kinds).toBeUndefined() + expect(p.description).toBeTruthy() +}) +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `pnpm vitest run packages/provider-polyhaven packages/provider-brave packages/provider-wikimedia-commons packages/provider-openverse` +Expected: new tests FAIL. + +- [ ] **Step 3: Implement** + +polyhaven factory (inside `export function polyhaven`, after `modalities: ['image'],` at line 86 — `assetType` is already in scope at line 83): + +```ts + kinds: assetType === 'hdris' ? ['hdri'] : ['texture'], + description: assetType === 'hdris' + ? 'CC0 HDRI environments for 3D lighting (Poly Haven)' + : 'CC0 PBR textures for 3D work (Poly Haven)', +``` + +polyhaven result annotation: `toReference(id, asset, imageUrl)` is module-level +while `assetType` lives in the factory closure — add a fourth parameter +`kind: string` to `toReference`, insert `kind,` after `modality: 'image',` +(line 67), and update the call at line 115 to +`toReference(id, asset, imageUrl, assetType === 'hdris' ? 'hdri' : 'texture')`. + +ambientcg factory (after `modalities: ['image'],` at line 175): + +```ts + kinds: ['texture'], + description: 'CC0 PBR materials and textures (ambientCG)', +``` + +ambientcg result annotation: `kind: 'texture',` after `modality: 'image',` at line 159. + +Description-only (no `kinds`, after each `modalities:` line): + +| Factory | description | +|---|---| +| brave (61) | `'Open-web image search (Brave)'` | +| wikimedia-commons (145) | `'Freely licensed media from the Wikimedia Commons archive'` | +| openverse image (154) | `'Aggregated openly licensed images from many sources (Openverse)'` | + +- [ ] **Step 4: Run to verify they pass** + +Run: `pnpm vitest run packages/provider-polyhaven packages/provider-brave packages/provider-wikimedia-commons packages/provider-openverse` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add packages/provider-polyhaven packages/provider-brave packages/provider-wikimedia-commons packages/provider-openverse +git commit -m "feat(providers): 3D kind declarations and omnibus descriptions" +``` + +--- + +### Task 11: changeset + full verification + +**Files:** +- Create: `.changeset/provider-resource-declaration.md` + +- [ ] **Step 1: Confirm exact package names** + +Run: `grep -h '"name"' packages/*/package.json` +Use the exact names below (adjust if any differ). + +- [ ] **Step 2: Write the changeset** + +Create `.changeset/provider-resource-declaration.md`: + +```md +--- +'@refkit/core': minor +'@refkit/mcp': minor +'@refkit/provider-testkit': minor +'@refkit/provider-unsplash': minor +'@refkit/provider-pexels': minor +'@refkit/provider-pixabay': minor +'@refkit/provider-flickr': minor +'@refkit/provider-brave': minor +'@refkit/provider-wikimedia-commons': minor +'@refkit/provider-openverse': minor +'@refkit/provider-met': minor +'@refkit/provider-artic': minor +'@refkit/provider-rijksmuseum': minor +'@refkit/provider-smithsonian': minor +'@refkit/provider-europeana': minor +'@refkit/provider-freesound': minor +'@refkit/provider-jamendo': minor +'@refkit/provider-gutendex': minor +'@refkit/provider-poetrydb': minor +'@refkit/provider-internet-archive': minor +'@refkit/provider-polyhaven': minor +--- + +Provider resource declarations: open `ResourceKind` vocabulary with optional +`kinds` + `description` on providers and `kind` on references; kind-aware +routing and a `providers` id whitelist with new skip reasons +(`unsupported-kind`, `not-selected`); MCP tool schema, source list, and enums +now derived from registered provider declarations at startup. +``` + +- [ ] **Step 3: Full verification** + +Run, in order, and require success for each: + +```bash +pnpm typecheck +pnpm test:run +pnpm build +git diff --check +``` + +Expected: all pass. If `pnpm typecheck` / `test:run` / `build` scripts do not exist at the root, use the equivalents in the root `package.json` scripts section. + +- [ ] **Step 4: Commit** + +```bash +git add .changeset/provider-resource-declaration.md +git commit -m "chore: changeset for provider resource declarations" +``` From 3c3db792b0a83c929f2e8563927077225214026a Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Sat, 25 Jul 2026 00:01:37 +0800 Subject: [PATCH 3/9] feat(core): ResourceKind vocabulary + provider kinds/description declarations --- packages/core/src/__tests__/provider.test.ts | 14 +++++++++++++ packages/core/src/index.ts | 2 ++ packages/core/src/provider.ts | 21 +++++++++++++++++++- 3 files changed, 36 insertions(+), 1 deletion(-) diff --git a/packages/core/src/__tests__/provider.test.ts b/packages/core/src/__tests__/provider.test.ts index dcd7797..897669d 100644 --- a/packages/core/src/__tests__/provider.test.ts +++ b/packages/core/src/__tests__/provider.test.ts @@ -46,3 +46,17 @@ describe('ReferenceProvider / defineProvider', () => { expect(p.capabilities?.controls).toEqual(['orientation', 'color', 'safety']) }) }) + +describe('resource declarations', () => { + it('defineProvider preserves kinds and description (open vocabulary)', () => { + const p = defineProvider({ + id: 'x', + modalities: ['image'], + kinds: ['texture', 'my-custom-kind'], // well-known + custom must both typecheck + description: 'CC0 textures for tests', + search: async () => [], + }) + expect(p.kinds).toEqual(['texture', 'my-custom-kind']) + expect(p.description).toBe('CC0 textures for tests') + }) +}) diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 2c579b1..9f37df5 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -42,6 +42,8 @@ export type { ProviderOptions, ProviderOptionsById, KeyValueCache, + WellKnownKind, + ResourceKind, } from './provider' export { setIfString, setIfBoolean, setIfStringList, diff --git a/packages/core/src/provider.ts b/packages/core/src/provider.ts index 0b77867..e05164e 100644 --- a/packages/core/src/provider.ts +++ b/packages/core/src/provider.ts @@ -15,6 +15,17 @@ export type QueryFeature = export type SearchSort = 'relevance' | 'latest' | 'popular' | 'interesting' export type SearchSafety = 'strict' | 'moderate' | 'off' +/** Fine-grained resource kind. Open vocabulary: well-known values get + * autocomplete; any other string is a valid custom kind. Well-known values are + * hints, not validation — core never rejects unknown kinds. */ +export type WellKnownKind = + | 'photo' | 'illustration' | 'vector' | 'icon' | 'artwork' + | 'texture' | 'hdri' | '3d-model' + | 'film' | 'animation' + | 'music' | 'sound-effect' + | 'ebook' | 'poem' +export type ResourceKind = WellKnownKind | (string & {}) + export interface SearchLicenseControls { commercial?: boolean modification?: boolean @@ -22,7 +33,7 @@ export interface SearchLicenseControls { } export interface SearchMediaControls { - kind?: 'photo' | 'illustration' | 'vector' | 'film' | 'animation' + kind?: ResourceKind size?: 'small' | 'medium' | 'large' minWidth?: number minHeight?: number @@ -126,6 +137,14 @@ export interface ProviderContext { export interface ReferenceProvider { id: string modalities: Modality[] + /** Fine-grained kinds this provider offers (routing: a kind-filtered search + * skips providers whose declared kinds lack the requested value; undeclared + * providers are conservatively included). Orthogonal to the `media.kind` + * entry in capabilities.controls, which declares upstream FILTER support. */ + kinds?: readonly ResourceKind[] + /** One-line content-domain summary, surfaced in the MCP tool's source list + * so an agent can judge topical fit (e.g. "CC0 PBR textures for 3D work"). */ + description?: string /** @deprecated Not read by core anymore — declare `capabilities.controls`. */ queryFeatures?: QueryFeature[] capabilities?: ProviderCapabilities From 6031f6e5d469bb75504294419e281ebb50d7d698 Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Sat, 25 Jul 2026 00:08:13 +0800 Subject: [PATCH 4/9] feat(core): optional Reference.kind --- packages/core/src/__tests__/reference.test.ts | 14 ++++++++++++++ packages/core/src/reference.ts | 3 +++ 2 files changed, 17 insertions(+) diff --git a/packages/core/src/__tests__/reference.test.ts b/packages/core/src/__tests__/reference.test.ts index 8cd4eb2..f6ecf18 100644 --- a/packages/core/src/__tests__/reference.test.ts +++ b/packages/core/src/__tests__/reference.test.ts @@ -40,4 +40,18 @@ describe('referenceSchema / parseReference', () => { const out = parseReference({ ...ref, thumbnail: { url: 'https://x/thumb.jpg' } }) expect(out.thumbnail).toEqual({ url: 'https://x/thumb.jpg' }) }) + + it('accepts an optional fine-grained kind', () => { + const base = { + id: 'p:1', + modality: 'image', + source: { providerId: 'p', sourceUrl: 'https://p/1' }, + canonicalUrl: 'https://p/1', + rights: { license: 'CC0-1.0', rehostPolicy: 'cache-allowed', raw: { sourceTerms: 't', sourceUrl: 'https://p/1' } }, + verifiedAt: '2026-07-24T00:00:00.000Z', + relevance: 0, + } + expect(parseReference({ ...base, kind: 'texture' }).kind).toBe('texture') + expect(parseReference(base).kind).toBeUndefined() + }) }) diff --git a/packages/core/src/reference.ts b/packages/core/src/reference.ts index bbe9f88..ce4fb9e 100644 --- a/packages/core/src/reference.ts +++ b/packages/core/src/reference.ts @@ -15,6 +15,8 @@ export interface Reference { // content-addressed, stable within a single result set only (core is zero-storage). id: string modality: Modality + /** Fine-grained kind (open vocabulary, see ResourceKind), e.g. 'photo', 'texture'. */ + kind?: string title?: string // — provenance (required; a result missing any of these never enters the set) — source: { providerId: string; sourceUrl: string } @@ -42,6 +44,7 @@ const modalitySchema: z.ZodType = z.enum(['image', 'video', 'audio', ' export const referenceSchema: z.ZodType = z.object({ id: z.string().min(1), modality: modalitySchema, + kind: z.string().optional(), title: z.string().optional(), source: z.object({ providerId: z.string().min(1), sourceUrl: z.string().min(1) }), canonicalUrl: z.string().min(1), From 37909774e475ee8061500dadb4622323145ad6f3 Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Sat, 25 Jul 2026 00:30:52 +0800 Subject: [PATCH 5/9] feat(core): kind-aware routing + providers whitelist with skip reasons --- packages/core/src/__tests__/client.test.ts | 62 ++++++++++++++++++++++ packages/core/src/client.ts | 42 ++++++++++++--- 2 files changed, 98 insertions(+), 6 deletions(-) diff --git a/packages/core/src/__tests__/client.test.ts b/packages/core/src/__tests__/client.test.ts index 2575800..8cf5347 100644 --- a/packages/core/src/__tests__/client.test.ts +++ b/packages/core/src/__tests__/client.test.ts @@ -704,3 +704,65 @@ describe('createRefkit', () => { expect(calls).toBe(1) // second search hits the same cache entry as the first }) }) + +describe('kind-aware routing', () => { + const kindProvider = (id: string, kinds: readonly string[], refs: Reference[]) => + defineProvider({ id, modalities: ['image'], kinds, search: async () => refs }) + + it('skips providers whose declared kinds lack the requested value', async () => { + const rk = createRefkit({ providers: [ + kindProvider('tex', ['texture'], [ref('tex-1', 'https://t/1')]), + kindProvider('ph', ['photo'], [ref('ph-1', 'https://p/1')]), + ] }) + const { references, meta } = await rk.searchWithMeta({ + query: 'x', modalities: ['image'], controls: { media: { kind: 'texture' } }, + }) + expect(references.map(r => r.canonicalUrl)).toEqual(['https://t/1']) + expect(meta.providers.find(p => p.providerId === 'ph')) + .toMatchObject({ status: 'skipped', reason: 'unsupported-kind' }) + }) + + it('conservatively includes providers that declare no kinds', async () => { + const rk = createRefkit({ providers: [ + provider('legacy', [ref('legacy-1', 'https://l/1')]), // no kinds declared + kindProvider('ph', ['photo'], [ref('ph-1', 'https://p/1')]), + ] }) + const out = await rk.search({ + query: 'x', modalities: ['image'], controls: { media: { kind: 'texture' } }, + }) + expect(out.map(r => r.canonicalUrl)).toEqual(['https://l/1']) + }) + + it('throws with the kind in the message when nothing matches', async () => { + const rk = createRefkit({ providers: [kindProvider('ph', ['photo'], [])] }) + await expect(rk.search({ + query: 'x', modalities: ['image'], controls: { media: { kind: 'texture' } }, + })).rejects.toThrow('kind "texture"') + }) +}) + +describe('providers whitelist', () => { + it('restricts fan-out and reports not-selected in meta', async () => { + const rk = createRefkit({ providers: [ + provider('a', [ref('a-1', 'https://a/1')]), + provider('b', [ref('b-1', 'https://b/1')]), + ] }) + const { references, meta } = await rk.searchWithMeta({ query: 'x', modalities: ['image'], providers: ['a'] }) + expect(references.map(r => r.canonicalUrl)).toEqual(['https://a/1']) + expect(meta.providers.find(p => p.providerId === 'b')) + .toMatchObject({ status: 'skipped', reason: 'not-selected' }) + }) + + it('warns on unknown ids and still runs the valid remainder', async () => { + const rk = createRefkit({ providers: [provider('a', [ref('a-1', 'https://a/1')])] }) + const { references, meta } = await rk.searchWithMeta({ query: 'x', modalities: ['image'], providers: ['a', 'nope'] }) + expect(references).toHaveLength(1) + expect(meta.warnings.some(w => w.includes('"nope"'))).toBe(true) + }) + + it('throws when the whitelist selects nothing', async () => { + const rk = createRefkit({ providers: [provider('a', [])] }) + await expect(rk.search({ query: 'x', modalities: ['image'], providers: ['nope'] })) + .rejects.toThrow(/no registered provider/) + }) +}) diff --git a/packages/core/src/client.ts b/packages/core/src/client.ts index 64afe95..8d4f4b3 100644 --- a/packages/core/src/client.ts +++ b/packages/core/src/client.ts @@ -67,7 +67,7 @@ export interface ProviderSearchStatus { returned?: number accepted?: number rejected?: number - reason?: 'unsupported-modality' + reason?: 'unsupported-modality' | 'unsupported-kind' | 'not-selected' error?: string latencyMs?: number cached?: boolean @@ -119,6 +119,10 @@ export interface SearchInput { /** Provider-specific search controls keyed by provider id. Core routes only the * matching entry to each provider; providers whitelist what they translate. */ providerOptions?: ProviderOptionsById + /** Restrict this search to these provider ids. Unknown ids append a warning + * to meta.warnings and are otherwise ignored; excluded providers appear in + * meta.providers as skipped with reason 'not-selected'. */ + providers?: readonly string[] limit?: number /** Opaque cursor from a previous search's `meta.nextCursor`. Resumes the * provider-local page (overriding `controls.page`), filters out results @@ -193,9 +197,33 @@ export function createRefkit(options: RefkitOptions): RefkitClient { if (typeof doFetch !== 'function') { throw new Error('createRefkit: no fetch available — pass options.fetch') } - const chosen = options.providers.filter(p => p.modalities.some(m => input.modalities.includes(m))) + const idWhitelist = input.providers + const preWarnings: string[] = [] + if (idWhitelist) { + const known = new Set(options.providers.map(p => p.id)) + for (const id of idWhitelist) { + if (!known.has(id)) preWarnings.push(`unknown provider id in providers: "${id}"`) + } + } + const kindFilter = input.controls?.media?.kind + const skipReasonFor = (p: ReferenceProvider): NonNullable | undefined => { + if (idWhitelist && !idWhitelist.includes(p.id)) return 'not-selected' + if (!p.modalities.some(m => input.modalities.includes(m))) return 'unsupported-modality' + if (kindFilter !== undefined && p.kinds && !p.kinds.includes(kindFilter)) return 'unsupported-kind' + return undefined + } + const skipReasons = new Map>() + for (const p of options.providers) { + const reason = skipReasonFor(p) + if (reason) skipReasons.set(p.id, reason) + } + const chosen = options.providers.filter(p => !skipReasons.has(p.id)) if (chosen.length === 0) { - throw new Error(`refkit.search: no registered provider supports modalities [${input.modalities.join(', ')}]`) + throw new Error( + `refkit.search: no registered provider supports modalities [${input.modalities.join(', ')}]` + + (kindFilter !== undefined ? ` with kind "${kindFilter}"` : '') + + (idWhitelist ? ` within providers [${idWhitelist.join(', ')}]` : ''), + ) } const limit = input.limit ?? DEFAULT_LIMIT const poolFactor = Math.max(1, Number.isFinite(input.poolFactor) ? (input.poolFactor as number) : DEFAULT_POOL_FACTOR) @@ -238,7 +266,8 @@ export function createRefkit(options: RefkitOptions): RefkitClient { } : undefined const statusByProvider = new Map() for (const p of options.providers) { - if (!chosen.includes(p)) statusByProvider.set(p.id, { providerId: p.id, status: 'skipped', reason: 'unsupported-modality' }) + const reason = skipReasons.get(p.id) + if (reason) statusByProvider.set(p.id, { providerId: p.id, status: 'skipped', reason }) } const runProvider = (p: ReferenceProvider) => { @@ -358,7 +387,7 @@ export function createRefkit(options: RefkitOptions): RefkitClient { seen: [...(cursorState?.seen ?? []), ...references.map(r => cursorSeenKey(r.canonicalUrl))].slice(-maxCursorSeen), }) : undefined - const warnings: string[] = [] + const warnings: string[] = [...preWarnings] const failedCount = [...pass.statusByProvider.values()].filter(s => s.status === 'failed').length if (failedCount > 0) warnings.push(`${failedCount} provider(s) failed; returning partial results.`) for (const c of pass.rightsConflicts) { @@ -376,7 +405,8 @@ export function createRefkit(options: RefkitOptions): RefkitClient { ...(input.filters ? { appliedFilters: input.filters } : {}), ...(pass.controlsMeta ? { controls: pass.controlsMeta } : {}), ...(input.providerOptions ? { providerOptions: Object.keys(input.providerOptions) } : {}), - providers: options.providers.map(p => pass.statusByProvider.get(p.id) ?? { providerId: p.id, status: 'skipped', reason: 'unsupported-modality' }), + providers: options.providers.map(p => pass.statusByProvider.get(p.id) + ?? { providerId: p.id, status: 'skipped', reason: skipReasons.get(p.id) ?? 'unsupported-modality' }), ...(pass.gate ? { gate: pass.gate } : {}), ...(nextCursor ? { nextCursor } : {}), warnings, From 4a6832bc81ec26387b38d3763c96157494327ff3 Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Sat, 25 Jul 2026 00:30:52 +0800 Subject: [PATCH 6/9] feat(testkit): enforce declared-kinds consistency in searchConformant --- .../src/__tests__/testkit.test.ts | 25 +++++++++++++++++++ packages/provider-testkit/src/index.ts | 6 +++++ 2 files changed, 31 insertions(+) diff --git a/packages/provider-testkit/src/__tests__/testkit.test.ts b/packages/provider-testkit/src/__tests__/testkit.test.ts index 80a92e7..49ba237 100644 --- a/packages/provider-testkit/src/__tests__/testkit.test.ts +++ b/packages/provider-testkit/src/__tests__/testkit.test.ts @@ -97,6 +97,31 @@ describe('searchConformant', () => { }) }) +const kindRef = (kind?: string): Reference => ({ + id: 'kp:1', + modality: 'image', + ...(kind ? { kind } : {}), + source: { providerId: 'kp', sourceUrl: 'https://kp/1' }, + canonicalUrl: 'https://kp/1', + rights: { license: 'CC0-1.0', rehostPolicy: 'cache-allowed', raw: { sourceTerms: 't', sourceUrl: 'https://kp/1' } }, + verifiedAt: '2026-07-24T00:00:00.000Z', + relevance: 0, +}) + +describe('declared-kinds consistency', () => { + const neverFetch = (async () => { throw new Error('no network') }) as unknown as typeof fetch + + it('rejects a result whose kind is outside the declared set', async () => { + const p = defineProvider({ id: 'kp', modalities: ['image'], kinds: ['texture'], search: async () => [kindRef('photo')] }) + await expect(searchConformant(p, neverFetch)).rejects.toThrow(/kind "photo" is not in the provider's declared kinds/) + }) + + it('accepts a matching kind and a missing kind', async () => { + const p = defineProvider({ id: 'kp', modalities: ['image'], kinds: ['texture'], search: async () => [kindRef('texture'), kindRef()] }) + await expect(searchConformant(p, neverFetch)).resolves.toHaveLength(2) + }) +}) + describe('expectLicenseMap', () => { it('passes silently on exact matches', () => { expect(() => diff --git a/packages/provider-testkit/src/index.ts b/packages/provider-testkit/src/index.ts index 9790a32..b5f09fd 100644 --- a/packages/provider-testkit/src/index.ts +++ b/packages/provider-testkit/src/index.ts @@ -57,6 +57,12 @@ export async function searchConformant( if (ref.source.providerId !== provider.id) { throw new Error(`[${provider.id}] result #${i} source.providerId does not match the provider (source.providerId=${ref.source.providerId}, provider.id=${provider.id})`) } + // Declared-kinds consistency: a provider stating what it offers must not + // emit results outside that set. Missing kind is allowed (annotation is + // optional); only a contradicting value is a violation. + if (provider.kinds && provider.kinds.length > 0 && ref.kind !== undefined && !provider.kinds.includes(ref.kind)) { + throw new Error(`[${provider.id}] result #${i} kind "${ref.kind}" is not in the provider's declared kinds [${provider.kinds.join(', ')}]`) + } if (ref.rights.licenseVersion !== undefined && !VERSIONED.has(ref.rights.license)) { throw new Error(`[${provider.id}] result #${i} carries licenseVersion on non-CC-family license ${ref.rights.license}`) } From 38a1b95a24bdba7ebdc1eaf63dcd564fbc662b24 Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Sat, 25 Jul 2026 00:30:52 +0800 Subject: [PATCH 7/9] feat(providers): kinds/description declarations across all providers --- .../provider-artic/src/__tests__/artic.test.ts | 6 ++++++ packages/provider-artic/src/index.ts | 3 +++ .../provider-brave/src/__tests__/brave.test.ts | 6 ++++++ packages/provider-brave/src/index.ts | 1 + .../src/__tests__/europeana.test.ts | 6 ++++++ packages/provider-europeana/src/index.ts | 3 +++ .../src/__tests__/flickr.test.ts | 6 ++++++ packages/provider-flickr/src/index.ts | 3 +++ .../src/__tests__/freesound.test.ts | 6 ++++++ packages/provider-freesound/src/index.ts | 3 +++ .../src/__tests__/gutendex.test.ts | 6 ++++++ packages/provider-gutendex/src/index.ts | 3 +++ .../src/__tests__/internet-archive.test.ts | 6 ++++++ packages/provider-internet-archive/src/index.ts | 3 +++ .../src/__tests__/jamendo.test.ts | 6 ++++++ packages/provider-jamendo/src/index.ts | 3 +++ packages/provider-met/src/__tests__/met.test.ts | 6 ++++++ packages/provider-met/src/index.ts | 3 +++ .../src/__tests__/openverse.test.ts | 10 ++++++++++ packages/provider-openverse/src/index.ts | 3 +++ .../src/__tests__/pexels.test.ts | 5 +++++ packages/provider-pexels/src/index.ts | 6 ++++++ .../src/__tests__/pixabay.test.ts | 17 +++++++++++++++++ packages/provider-pixabay/src/index.ts | 13 +++++++++++++ .../src/__tests__/poetrydb.test.ts | 6 ++++++ packages/provider-poetrydb/src/index.ts | 3 +++ .../src/__tests__/ambientcg.test.ts | 5 +++++ .../src/__tests__/polyhaven.test.ts | 5 +++++ packages/provider-polyhaven/src/index.ts | 12 ++++++++++-- .../src/__tests__/rijksmuseum.test.ts | 6 ++++++ packages/provider-rijksmuseum/src/index.ts | 3 +++ .../src/__tests__/smithsonian.test.ts | 6 ++++++ packages/provider-smithsonian/src/index.ts | 3 +++ .../src/__tests__/unsplash.test.ts | 6 ++++++ packages/provider-unsplash/src/index.ts | 3 +++ .../src/__tests__/wikimedia-commons.test.ts | 6 ++++++ .../provider-wikimedia-commons/src/index.ts | 1 + 37 files changed, 196 insertions(+), 2 deletions(-) diff --git a/packages/provider-artic/src/__tests__/artic.test.ts b/packages/provider-artic/src/__tests__/artic.test.ts index ffbc5cb..b2dabc4 100644 --- a/packages/provider-artic/src/__tests__/artic.test.ts +++ b/packages/provider-artic/src/__tests__/artic.test.ts @@ -60,4 +60,10 @@ describe('artic provider', () => { expect(url.searchParams.get('fields')).toBe('id,title,image_id,is_public_domain,artist_display,date_display') expect(url.searchParams.get('query[term][is_public_domain]')).toBe('true') }) + + it('declares kinds and description', () => { + const p = artic() + expect(p.kinds).toEqual(['artwork']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-artic/src/index.ts b/packages/provider-artic/src/index.ts index 0f6ce5a..4f00426 100644 --- a/packages/provider-artic/src/index.ts +++ b/packages/provider-artic/src/index.ts @@ -44,6 +44,7 @@ function toReference(a: ArticArtwork, iiifUrl: string): Reference | null { return { id: referenceId('artic', canonicalUrl), modality: 'image', + kind: 'artwork', title: a.title || undefined, source: { providerId: 'artic', sourceUrl: canonicalUrl }, canonicalUrl, @@ -71,6 +72,8 @@ export function artic() { return defineProvider({ id: 'artic', modalities: ['image'], + kinds: ['artwork'], + description: 'CC0 artworks from the Art Institute of Chicago', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://api.artic.edu/api/v1/artworks/search') diff --git a/packages/provider-brave/src/__tests__/brave.test.ts b/packages/provider-brave/src/__tests__/brave.test.ts index 878c211..6339215 100644 --- a/packages/provider-brave/src/__tests__/brave.test.ts +++ b/packages/provider-brave/src/__tests__/brave.test.ts @@ -126,4 +126,10 @@ describe('brave provider', () => { const url = new URL(calledUrl) expect(url.searchParams.get('safesearch')).toBe('off') }) + + it('declares a description but no kinds (omnibus source)', () => { + const p = brave({ token: 'SECRET' }) + expect(p.kinds).toBeUndefined() + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-brave/src/index.ts b/packages/provider-brave/src/index.ts index 5bb09a9..6aaa4aa 100644 --- a/packages/provider-brave/src/index.ts +++ b/packages/provider-brave/src/index.ts @@ -59,6 +59,7 @@ export function brave(config: BraveConfig) { return defineProvider({ id: 'brave', modalities: ['image'], + description: 'Open-web image search (Brave)', capabilities: { controls: ['safety'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://api.search.brave.com/res/v1/images/search') diff --git a/packages/provider-europeana/src/__tests__/europeana.test.ts b/packages/provider-europeana/src/__tests__/europeana.test.ts index 5a9f8e8..10b1e23 100644 --- a/packages/provider-europeana/src/__tests__/europeana.test.ts +++ b/packages/provider-europeana/src/__tests__/europeana.test.ts @@ -206,4 +206,10 @@ describe('europeana search request', () => { } await expect(europeana({ apiKey: 'bad' }).search({ text: 'x', modalities: ['image'] }, ctx)).rejects.toThrow(/europeana search failed: 401/) }) + + it('declares kinds and description', () => { + const p = europeana({ apiKey: 'k' }) + expect(p.kinds).toEqual(['artwork']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-europeana/src/index.ts b/packages/provider-europeana/src/index.ts index 2d9cd10..1a6909c 100644 --- a/packages/provider-europeana/src/index.ts +++ b/packages/provider-europeana/src/index.ts @@ -81,6 +81,7 @@ function toReference(it: EuropeanaItem): Reference | null { return { id: referenceId('europeana', canonicalUrl), modality: 'image', + kind: 'artwork', title: first(it.title) || undefined, source: { providerId: 'europeana', sourceUrl: canonicalUrl }, canonicalUrl, @@ -97,6 +98,8 @@ export function europeana(config: EuropeanaConfig) { return defineProvider({ id: 'europeana', modalities: ['image'], + kinds: ['artwork'], + description: 'European cultural heritage from museums, libraries and archives (Europeana)', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL(BASE) diff --git a/packages/provider-flickr/src/__tests__/flickr.test.ts b/packages/provider-flickr/src/__tests__/flickr.test.ts index ff5efb5..d3005d5 100644 --- a/packages/provider-flickr/src/__tests__/flickr.test.ts +++ b/packages/provider-flickr/src/__tests__/flickr.test.ts @@ -222,4 +222,10 @@ describe('flickr provider', () => { expect(url.searchParams.get('safe_search')).toBe('3') expect(url.searchParams.get('user_id')).toBe('option-user') }) + + it('declares kinds and description', () => { + const p = flickr({ apiKey: 'k' }) + expect(p.kinds).toEqual(['photo']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-flickr/src/index.ts b/packages/provider-flickr/src/index.ts index 9d440d6..ea9c182 100644 --- a/packages/provider-flickr/src/index.ts +++ b/packages/provider-flickr/src/index.ts @@ -163,6 +163,7 @@ function toReference(p: FlickrPhoto): Reference { return { id: referenceId('flickr', canonicalUrl), modality: 'image', + kind: 'photo', title: p.title || undefined, source: { providerId: 'flickr', sourceUrl: canonicalUrl }, canonicalUrl, @@ -180,6 +181,8 @@ export function flickr(config: FlickrConfig) { return defineProvider({ id: 'flickr', modalities: ['image'], + kinds: ['photo'], + description: 'Community photography with per-item CC licensing (Flickr)', capabilities: { controls: ['sort', 'safety', 'license.commercial', 'license.modification', 'license.allowUnknown', 'creator.id', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const opts = q.providerOptions as FlickrSearchOptions | undefined diff --git a/packages/provider-freesound/src/__tests__/freesound.test.ts b/packages/provider-freesound/src/__tests__/freesound.test.ts index dcc5d7e..67ea105 100644 --- a/packages/provider-freesound/src/__tests__/freesound.test.ts +++ b/packages/provider-freesound/src/__tests__/freesound.test.ts @@ -109,4 +109,10 @@ describe('mapFreesoundLicense', () => { expect(mapFreesoundLicense('Weird Custom License')).toEqual({ license: 'unknown' }) expect(mapFreesoundLicense('')).toEqual({ license: 'unknown' }) }) + + it('declares kinds and description', () => { + const p = freesound({ apiKey: 'k' }) + expect(p.kinds).toEqual(['sound-effect']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-freesound/src/index.ts b/packages/provider-freesound/src/index.ts index 9cc662f..390e8c1 100644 --- a/packages/provider-freesound/src/index.ts +++ b/packages/provider-freesound/src/index.ts @@ -79,6 +79,7 @@ function toAudioReference(r: FreesoundResult): Reference | null { return { id: referenceId('freesound', canonicalUrl), modality: 'audio', + kind: 'sound-effect', title: r.name || undefined, source: { providerId: 'freesound', sourceUrl: canonicalUrl }, canonicalUrl, @@ -94,6 +95,8 @@ export function freesound(config: FreesoundConfig) { return defineProvider({ id: 'freesound', modalities: ['audio'], + kinds: ['sound-effect'], + description: 'Collaborative archive of CC-licensed sounds (Freesound)', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const opts = q.providerOptions as FreesoundSearchOptions | undefined diff --git a/packages/provider-gutendex/src/__tests__/gutendex.test.ts b/packages/provider-gutendex/src/__tests__/gutendex.test.ts index 043ff1f..6f292c8 100644 --- a/packages/provider-gutendex/src/__tests__/gutendex.test.ts +++ b/packages/provider-gutendex/src/__tests__/gutendex.test.ts @@ -166,4 +166,10 @@ describe('gutendex provider', () => { expect(result.rights.license).toBe('unknown') expect(evaluateUse(result.rights, 'commercial-product').decision).toBe('needs-review') }) + + it('declares kinds and description', () => { + const p = gutendex() + expect(p.kinds).toEqual(['ebook']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-gutendex/src/index.ts b/packages/provider-gutendex/src/index.ts index 453e17e..b4c8c3b 100644 --- a/packages/provider-gutendex/src/index.ts +++ b/packages/provider-gutendex/src/index.ts @@ -62,6 +62,7 @@ function toReference(r: GutendexResult): Reference { return { id: referenceId('gutendex', canonicalUrl), modality: 'text', + kind: 'ebook', title: r.title, source: { providerId: 'gutendex', sourceUrl: canonicalUrl }, canonicalUrl, @@ -80,6 +81,8 @@ export function gutendex(config: GutendexConfig = {}) { return defineProvider({ id: 'gutendex', modalities: ['text'], + kinds: ['ebook'], + description: 'Public-domain ebooks from Project Gutenberg (Gutendex)', capabilities: { controls: ['language', 'text.copyright', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const base = config.baseUrl ?? 'https://gutendex.com' diff --git a/packages/provider-internet-archive/src/__tests__/internet-archive.test.ts b/packages/provider-internet-archive/src/__tests__/internet-archive.test.ts index d607471..ec5d91b 100644 --- a/packages/provider-internet-archive/src/__tests__/internet-archive.test.ts +++ b/packages/provider-internet-archive/src/__tests__/internet-archive.test.ts @@ -197,4 +197,10 @@ describe('internetArchive search', () => { const q = new URL(seen).searchParams.get('q')! expect(q).toBe('(x\\) OR \\(mediatype\\:audio) AND mediatype:(movies OR texts)') }) + + it('declares kinds and description', () => { + const p = internetArchive() + expect(p.kinds).toEqual(['film', 'ebook']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-internet-archive/src/index.ts b/packages/provider-internet-archive/src/index.ts index 6b53fa0..3457c2e 100644 --- a/packages/provider-internet-archive/src/index.ts +++ b/packages/provider-internet-archive/src/index.ts @@ -77,6 +77,7 @@ export function toReference(doc: IaDoc): Reference | null { return { id: referenceId('internet-archive', canonicalUrl), modality, + kind: modality === 'video' ? 'film' : 'ebook', title: title || undefined, source: { providerId: 'internet-archive', sourceUrl: canonicalUrl }, canonicalUrl, @@ -92,6 +93,8 @@ export function internetArchive(config: InternetArchiveConfig = {}) { return defineProvider({ id: 'internet-archive', modalities: ['video', 'text'], + kinds: ['film', 'ebook'], + description: 'Public-domain and CC films and texts from the Internet Archive', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL(BASE) diff --git a/packages/provider-jamendo/src/__tests__/jamendo.test.ts b/packages/provider-jamendo/src/__tests__/jamendo.test.ts index 336ece1..35c48cd 100644 --- a/packages/provider-jamendo/src/__tests__/jamendo.test.ts +++ b/packages/provider-jamendo/src/__tests__/jamendo.test.ts @@ -135,4 +135,10 @@ describe('jamendo provider', () => { const refs = await jamendo({ clientId: 'cid' }).search({ text: 'zzzz', modalities: ['audio'] }, ctx) expect(refs).toEqual([]) }) + + it('declares kinds and description', () => { + const p = jamendo({ clientId: 'cid' }) + expect(p.kinds).toEqual(['music']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-jamendo/src/index.ts b/packages/provider-jamendo/src/index.ts index 1cf1c9e..017c446 100644 --- a/packages/provider-jamendo/src/index.ts +++ b/packages/provider-jamendo/src/index.ts @@ -75,6 +75,7 @@ function toAudioReference(t: JamendoTrack, mediaType: string): Reference | null return { id: referenceId('jamendo', canonicalUrl), modality: 'audio', + kind: 'music', title: t.name || undefined, source: { providerId: 'jamendo', sourceUrl: canonicalUrl }, canonicalUrl, @@ -92,6 +93,8 @@ export function jamendo(config: JamendoConfig) { return defineProvider({ id: 'jamendo', modalities: ['audio'], + kinds: ['music'], + description: 'CC-licensed independent music (Jamendo)', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL(BASE) diff --git a/packages/provider-met/src/__tests__/met.test.ts b/packages/provider-met/src/__tests__/met.test.ts index 46f2662..9a2bbdb 100644 --- a/packages/provider-met/src/__tests__/met.test.ts +++ b/packages/provider-met/src/__tests__/met.test.ts @@ -88,4 +88,10 @@ describe('met provider', () => { expect(url.searchParams.get('dateEnd')).toBe('1800') expect(url.searchParams.get('hasImages')).toBe('true') }) + + it('declares kinds and description', () => { + const p = met() + expect(p.kinds).toEqual(['artwork']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-met/src/index.ts b/packages/provider-met/src/index.ts index 7c086a7..a4c125c 100644 --- a/packages/provider-met/src/index.ts +++ b/packages/provider-met/src/index.ts @@ -54,6 +54,7 @@ function toReference(o: MetObject): Reference | null { return { id: referenceId('met', o.objectURL), modality: 'image', + kind: 'artwork', title: o.title || undefined, source: { providerId: 'met', sourceUrl: o.objectURL }, canonicalUrl: o.objectURL, @@ -70,6 +71,8 @@ export function met(config: MetConfig = {}) { return defineProvider({ id: 'met', modalities: ['image'], + kinds: ['artwork'], + description: 'Open Access artworks from the Metropolitan Museum of Art', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const searchUrl = new URL(`${BASE}/search`) diff --git a/packages/provider-openverse/src/__tests__/openverse.test.ts b/packages/provider-openverse/src/__tests__/openverse.test.ts index c14bebc..8119037 100644 --- a/packages/provider-openverse/src/__tests__/openverse.test.ts +++ b/packages/provider-openverse/src/__tests__/openverse.test.ts @@ -281,4 +281,14 @@ describe('openverseAudio provider', () => { expect(refs[0].rights.licenseVersion).toBe('3.0') expect(evaluateUse(refs[0].rights, 'commercial-product').decision).toBe('denied') }) + + it('declares a description but no kinds (omnibus source) for images', () => { + const p = openverse() + expect(p.kinds).toBeUndefined() + expect(p.description).toBeTruthy() + }) + + it('declares kinds and description for audio', () => { + expect(openverseAudio().kinds).toEqual(['music', 'sound-effect']) + }) }) diff --git a/packages/provider-openverse/src/index.ts b/packages/provider-openverse/src/index.ts index e8d0f15..3501aab 100644 --- a/packages/provider-openverse/src/index.ts +++ b/packages/provider-openverse/src/index.ts @@ -152,6 +152,7 @@ export function openverse(config: OpenverseConfig = {}) { return defineProvider({ id: 'openverse', modalities: ['image'], + description: 'Aggregated openly licensed images from many sources (Openverse)', capabilities: { controls: ['license.commercial', 'license.modification', 'license.allowUnknown', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://api.openverse.org/v1/images/') @@ -223,6 +224,8 @@ export function openverseAudio(config: OpenverseConfig = {}) { return defineProvider({ id: 'openverse-audio', modalities: ['audio'], + kinds: ['music', 'sound-effect'], + description: 'Aggregated openly licensed music and sound effects (Openverse)', capabilities: { controls: ['license.commercial', 'license.modification', 'license.allowUnknown', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://api.openverse.org/v1/audio/') diff --git a/packages/provider-pexels/src/__tests__/pexels.test.ts b/packages/provider-pexels/src/__tests__/pexels.test.ts index 9576f6a..4949b21 100644 --- a/packages/provider-pexels/src/__tests__/pexels.test.ts +++ b/packages/provider-pexels/src/__tests__/pexels.test.ts @@ -187,4 +187,9 @@ describe('pexelsVideo provider', () => { expect(url.searchParams.get('page')).toBe('3') expect(url.searchParams.get('color')).toBeNull() }) + + it('declares kinds and description (image + video factories)', () => { + expect(pexels({ apiKey: 'k' }).kinds).toEqual(['photo']) + expect(pexelsVideo({ apiKey: 'k' }).kinds).toEqual(['film']) + }) }) diff --git a/packages/provider-pexels/src/index.ts b/packages/provider-pexels/src/index.ts index 815b813..961209a 100644 --- a/packages/provider-pexels/src/index.ts +++ b/packages/provider-pexels/src/index.ts @@ -54,6 +54,7 @@ function toReference(p: PexelsPhoto): Reference { return { id: referenceId('pexels', p.url), modality: 'image', + kind: 'photo', title: p.alt || undefined, source: { providerId: 'pexels', sourceUrl: p.url }, canonicalUrl: p.url, @@ -70,6 +71,8 @@ export function pexels(config: PexelsConfig) { return defineProvider({ id: 'pexels', modalities: ['image'], + kinds: ['photo'], + description: 'Free stock photos (Pexels)', capabilities: { controls: ['orientation', 'color', 'language', 'media.size', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://api.pexels.com/v1/search') @@ -113,6 +116,7 @@ function toVideoReference(v: PexelsVideo): Reference { return { id: referenceId('pexels-video', v.url), modality: 'video', + kind: 'film', source: { providerId: 'pexels-video', sourceUrl: v.url }, canonicalUrl: v.url, rights, @@ -130,6 +134,8 @@ export function pexelsVideo(config: PexelsConfig) { return defineProvider({ id: 'pexels-video', modalities: ['video'], + kinds: ['film'], + description: 'Free stock videos (Pexels)', capabilities: { controls: ['orientation', 'language', 'media.size', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://api.pexels.com/videos/search') diff --git a/packages/provider-pixabay/src/__tests__/pixabay.test.ts b/packages/provider-pixabay/src/__tests__/pixabay.test.ts index 4cc8875..234aade 100644 --- a/packages/provider-pixabay/src/__tests__/pixabay.test.ts +++ b/packages/provider-pixabay/src/__tests__/pixabay.test.ts @@ -187,4 +187,21 @@ describe('pixabayVideo provider', () => { expect(url.searchParams.get('page')).toBe('5') expect(url.searchParams.get('per_page')).toBe('44') }) + + it('declares kinds and description (image + video factories)', () => { + expect(pixabay({ key: 'k' }).kinds).toEqual(['photo', 'illustration', 'vector']) + expect(pixabayVideo({ key: 'k' }).kinds).toEqual(['film', 'animation']) + }) + + it('maps the upstream hit type to Reference.kind', async () => { + const hit = { + id: 1, tags: 'x', user: 'u', pageURL: 'https://pixabay.com/p/1', + previewURL: 'https://cdn/p.jpg', previewWidth: 10, previewHeight: 10, + webformatURL: 'https://cdn/w.jpg', largeImageURL: 'https://cdn/l.jpg', + imageWidth: 100, imageHeight: 100, type: 'vectors/svg', + } + const fakeFetch = (async () => new Response(JSON.stringify({ hits: [hit] }), { status: 200 })) as typeof fetch + const refs = await pixabay({ key: 'k' }).search({ text: 'x', modalities: ['image'] }, { fetch: fakeFetch }) + expect(refs[0].kind).toBe('vector') + }) }) diff --git a/packages/provider-pixabay/src/index.ts b/packages/provider-pixabay/src/index.ts index 73a4ee3..a4401f9 100644 --- a/packages/provider-pixabay/src/index.ts +++ b/packages/provider-pixabay/src/index.ts @@ -48,6 +48,7 @@ interface PixabayHit { largeImageURL: string imageWidth: number imageHeight: number + type?: string } interface PixabayResponse { hits: PixabayHit[] } @@ -72,6 +73,13 @@ function pixabayOrientation(orientation: string | undefined): string | undefined return undefined } +// Upstream image `type` values: 'photo' | 'illustration' | 'vectors/svg'. +function pixabayKind(t: string | undefined): string | undefined { + if (t === 'photo' || t === 'illustration') return t + if (t === 'vectors/svg' || t === 'vector/svg') return 'vector' + return undefined +} + function toReference(h: PixabayHit): Reference { const rights: RightsRecord = { license: 'pixabay', @@ -82,6 +90,7 @@ function toReference(h: PixabayHit): Reference { return { id: referenceId('pixabay', h.pageURL), modality: 'image', + ...(pixabayKind(h.type) ? { kind: pixabayKind(h.type) } : {}), title: h.tags || undefined, // no title field; tags is the only descriptive text source: { providerId: 'pixabay', sourceUrl: h.pageURL }, canonicalUrl: h.pageURL, @@ -98,6 +107,8 @@ export function pixabay(config: PixabayConfig) { return defineProvider({ id: 'pixabay', modalities: ['image'], + kinds: ['photo', 'illustration', 'vector'], + description: 'Free stock photos, illustrations and vectors (Pixabay)', capabilities: { controls: ['orientation', 'color', 'language', 'sort', 'safety', 'media.kind', 'media.minWidth', 'media.minHeight', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://pixabay.com/api/') @@ -181,6 +192,8 @@ export function pixabayVideo(config: PixabayConfig) { return defineProvider({ id: 'pixabay-video', modalities: ['video'], + kinds: ['film', 'animation'], + description: 'Free stock videos and animations (Pixabay)', capabilities: { controls: ['language', 'sort', 'safety', 'media.kind', 'media.minWidth', 'media.minHeight', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://pixabay.com/api/videos/') diff --git a/packages/provider-poetrydb/src/__tests__/poetrydb.test.ts b/packages/provider-poetrydb/src/__tests__/poetrydb.test.ts index 2142f1b..eca4b66 100644 --- a/packages/provider-poetrydb/src/__tests__/poetrydb.test.ts +++ b/packages/provider-poetrydb/src/__tests__/poetrydb.test.ts @@ -174,4 +174,10 @@ describe('poetrydb provider', () => { }, ctx) expect(calledUrl).toBe('https://poetrydb.org/title,author,poemcount/Winter;William%20Shakespeare;2:abs/author,title,lines,linecount') }) + + it('declares kinds and description', () => { + const p = poetrydb() + expect(p.kinds).toEqual(['poem']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-poetrydb/src/index.ts b/packages/provider-poetrydb/src/index.ts index d346bbf..58cdc8d 100644 --- a/packages/provider-poetrydb/src/index.ts +++ b/packages/provider-poetrydb/src/index.ts @@ -30,6 +30,7 @@ function toReference(p: PoetryDbPoem): Reference { return { id: referenceId('poetrydb', `${p.author}:${p.title}`), modality: 'text', + kind: 'poem', title: p.title, source: { providerId: 'poetrydb', sourceUrl: canonicalUrl }, canonicalUrl, @@ -107,6 +108,8 @@ export function poetrydb() { return defineProvider({ id: 'poetrydb', modalities: ['text'], + kinds: ['poem'], + description: 'Classic public-domain poetry (PoetryDB)', capabilities: { controls: [] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { // /lines/ finds poems whose line content contains the term (closest to keyword search) diff --git a/packages/provider-polyhaven/src/__tests__/ambientcg.test.ts b/packages/provider-polyhaven/src/__tests__/ambientcg.test.ts index 26b98f7..388d531 100644 --- a/packages/provider-polyhaven/src/__tests__/ambientcg.test.ts +++ b/packages/provider-polyhaven/src/__tests__/ambientcg.test.ts @@ -53,4 +53,9 @@ describe('ambientcg provider', () => { const refs = await ambientcg().search({ text: 'x', modalities: ['image'] }, ctxJson(FOUND_NO_IMAGE)) expect(refs).toEqual([]) }) + + it('declares kinds and description', () => { + expect(ambientcg().kinds).toEqual(['texture']) + expect(ambientcg().description).toMatch(/texture|material/i) + }) }) diff --git a/packages/provider-polyhaven/src/__tests__/polyhaven.test.ts b/packages/provider-polyhaven/src/__tests__/polyhaven.test.ts index 8e3cd51..feb1c19 100644 --- a/packages/provider-polyhaven/src/__tests__/polyhaven.test.ts +++ b/packages/provider-polyhaven/src/__tests__/polyhaven.test.ts @@ -56,4 +56,9 @@ describe('polyhaven provider', () => { const refs = await polyhaven().search({ text: 'zzz', modalities: ['image'] }, ctxRouting({}, {})) expect(refs).toEqual([]) }) + + it('declares kinds per assetType', () => { + expect(polyhaven().kinds).toEqual(['texture']) + expect(polyhaven({ assetType: 'hdris' }).kinds).toEqual(['hdri']) + }) }) diff --git a/packages/provider-polyhaven/src/index.ts b/packages/provider-polyhaven/src/index.ts index b965cfc..4d3a9e4 100644 --- a/packages/provider-polyhaven/src/index.ts +++ b/packages/provider-polyhaven/src/index.ts @@ -54,7 +54,7 @@ function firstAuthor(authors?: Record): string | undefined { return names.length ? names.join(', ') : undefined } -function toReference(id: string, asset: PolyHavenAsset, imageUrl: string): Reference { +function toReference(id: string, asset: PolyHavenAsset, imageUrl: string, kind: string): Reference { const canonical = `https://polyhaven.com/a/${id}` const rights: RightsRecord = { license: 'CC0-1.0', @@ -65,6 +65,7 @@ function toReference(id: string, asset: PolyHavenAsset, imageUrl: string): Refer return { id: referenceId('polyhaven', canonical), modality: 'image', + kind, title: asset.name || undefined, source: { providerId: 'polyhaven', sourceUrl: canonical }, canonicalUrl: canonical, @@ -84,6 +85,10 @@ export function polyhaven(config: PolyHavenConfig = {}) { return defineProvider({ id: 'polyhaven', modalities: ['image'], + kinds: assetType === 'hdris' ? ['hdri'] : ['texture'], + description: assetType === 'hdris' + ? 'CC0 HDRI environments for 3D lighting (Poly Haven)' + : 'CC0 PBR textures for 3D work (Poly Haven)', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const listUrl = new URL(`${PH_BASE}/assets`) @@ -112,7 +117,7 @@ export function polyhaven(config: PolyHavenConfig = {}) { const files = (await fr.json()) as PolyHavenFiles const imageUrl = assetType === 'hdris' ? hdriImageUrl(files) : textureImageUrl(files) if (!imageUrl) return null // no image-format file → skip (D1) - return toReference(id, asset, imageUrl) + return toReference(id, asset, imageUrl, assetType === 'hdris' ? 'hdri' : 'texture') } catch { return null // one bad files fetch must not drop the whole batch } @@ -157,6 +162,7 @@ function acgToReference(a: AmbientCgAsset, imageUrl: string): Reference { return { id: referenceId('ambientcg', canonical), modality: 'image', + kind: 'texture', title: a.displayName || undefined, source: { providerId: 'ambientcg', sourceUrl: canonical }, canonicalUrl: canonical, @@ -173,6 +179,8 @@ export function ambientcg(config: AmbientCgConfig = {}) { return defineProvider({ id: 'ambientcg', modalities: ['image'], + kinds: ['texture'], + description: 'CC0 PBR materials and textures (ambientCG)', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL(ACG_BASE) diff --git a/packages/provider-rijksmuseum/src/__tests__/rijksmuseum.test.ts b/packages/provider-rijksmuseum/src/__tests__/rijksmuseum.test.ts index 4b6ebd8..a7f9219 100644 --- a/packages/provider-rijksmuseum/src/__tests__/rijksmuseum.test.ts +++ b/packages/provider-rijksmuseum/src/__tests__/rijksmuseum.test.ts @@ -349,4 +349,10 @@ describe('rijksmuseum provider', () => { expect(recordUrls).toHaveLength(2) expect(recordUrls.some(recordUrl => recordUrl.includes('/3?'))).toBe(false) }) + + it('declares kinds and description', () => { + const p = rijksmuseum() + expect(p.kinds).toEqual(['artwork']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-rijksmuseum/src/index.ts b/packages/provider-rijksmuseum/src/index.ts index 5dc13c1..db36557 100644 --- a/packages/provider-rijksmuseum/src/index.ts +++ b/packages/provider-rijksmuseum/src/index.ts @@ -118,6 +118,7 @@ function toReference(rec: EdmRecord): Reference | null { return { id: referenceId('rijksmuseum', canonicalUrl), modality: 'image', + kind: 'artwork', title: firstLocalized(rec.aggregatedCHO?.title, ['en', 'nl']), source: { providerId: 'rijksmuseum', sourceUrl }, canonicalUrl, @@ -138,6 +139,8 @@ export function rijksmuseum(config: RijksmuseumConfig = {}) { return defineProvider({ id: 'rijksmuseum', modalities: ['image'], + kinds: ['artwork'], + description: 'Rijksmuseum collection artworks, incl. the Dutch Golden Age', capabilities: { controls: [] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const opts = q.providerOptions as RijksmuseumSearchOptions | undefined diff --git a/packages/provider-smithsonian/src/__tests__/smithsonian.test.ts b/packages/provider-smithsonian/src/__tests__/smithsonian.test.ts index 2584578..47192b0 100644 --- a/packages/provider-smithsonian/src/__tests__/smithsonian.test.ts +++ b/packages/provider-smithsonian/src/__tests__/smithsonian.test.ts @@ -83,4 +83,10 @@ describe('smithsonian provider', () => { expect(url.searchParams.get('row_group')).toBe('archives') expect(url.searchParams.get('fq')).toBe('online_media_type:"Images" AND media_usage:"CC0" AND topic:"Cats"') }) + + it('declares kinds and description', () => { + const p = smithsonian({ apiKey: 'k' }) + expect(p.kinds).toEqual(['artwork']) + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-smithsonian/src/index.ts b/packages/provider-smithsonian/src/index.ts index 50a577a..bebee51 100644 --- a/packages/provider-smithsonian/src/index.ts +++ b/packages/provider-smithsonian/src/index.ts @@ -52,6 +52,7 @@ function toReference(row: SiRow): Reference | null { return { id: referenceId('smithsonian', canonicalUrl), modality: 'image', + kind: 'artwork', title: dnr?.title?.content || row.title || undefined, source: { providerId: 'smithsonian', sourceUrl: canonicalUrl }, canonicalUrl, @@ -68,6 +69,8 @@ export function smithsonian(config: SmithsonianConfig) { return defineProvider({ id: 'smithsonian', modalities: ['image'], + kinds: ['artwork'], + description: 'Open Access objects from Smithsonian museums', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://api.si.edu/openaccess/api/v1.0/search') diff --git a/packages/provider-unsplash/src/__tests__/unsplash.test.ts b/packages/provider-unsplash/src/__tests__/unsplash.test.ts index c5f6238..85a7998 100644 --- a/packages/provider-unsplash/src/__tests__/unsplash.test.ts +++ b/packages/provider-unsplash/src/__tests__/unsplash.test.ts @@ -104,4 +104,10 @@ describe('unsplash provider', () => { expect(url.searchParams.get('orientation')).toBe('squarish') expect(url.searchParams.get('lang')).toBe('zh-Hans') }) + + it('declares kinds and description', () => { + const p = unsplash({ accessKey: 'k' }) + expect(p.kinds).toEqual(['photo']) + expect(p.description).toMatch(/photo/i) + }) }) diff --git a/packages/provider-unsplash/src/index.ts b/packages/provider-unsplash/src/index.ts index 74d5cf4..db39eea 100644 --- a/packages/provider-unsplash/src/index.ts +++ b/packages/provider-unsplash/src/index.ts @@ -43,6 +43,7 @@ function toReference(r: UnsplashResult): Reference { return { id: referenceId('unsplash', r.links.html), modality: 'image', + kind: 'photo', title: r.description ?? r.alt_description ?? undefined, source: { providerId: 'unsplash', sourceUrl: r.links.html }, canonicalUrl: r.links.html, @@ -59,6 +60,8 @@ export function unsplash(config: UnsplashConfig) { return defineProvider({ id: 'unsplash', modalities: ['image'], + kinds: ['photo'], + description: 'High-quality free stock photography (Unsplash)', capabilities: { controls: ['orientation', 'color', 'language', 'sort', 'safety', 'page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://api.unsplash.com/search/photos') diff --git a/packages/provider-wikimedia-commons/src/__tests__/wikimedia-commons.test.ts b/packages/provider-wikimedia-commons/src/__tests__/wikimedia-commons.test.ts index 499e41a..b5a6f0b 100644 --- a/packages/provider-wikimedia-commons/src/__tests__/wikimedia-commons.test.ts +++ b/packages/provider-wikimedia-commons/src/__tests__/wikimedia-commons.test.ts @@ -158,4 +158,10 @@ describe('wikimedia-commons provider', () => { expect(refs[0].title).toBe('Paul Bril 002') expect(refs[0].rights.license).toBe('PD') }) + + it('declares a description but no kinds (omnibus source)', () => { + const p = wikimediaCommons() + expect(p.kinds).toBeUndefined() + expect(p.description).toBeTruthy() + }) }) diff --git a/packages/provider-wikimedia-commons/src/index.ts b/packages/provider-wikimedia-commons/src/index.ts index b3068d6..daaa02b 100644 --- a/packages/provider-wikimedia-commons/src/index.ts +++ b/packages/provider-wikimedia-commons/src/index.ts @@ -143,6 +143,7 @@ export function wikimediaCommons(config: WikimediaCommonsConfig = {}) { return defineProvider({ id: 'wikimedia-commons', modalities: ['image'], + description: 'Freely licensed media from the Wikimedia Commons archive', capabilities: { controls: ['page'] }, async search(q: NormalizedQuery, ctx: ProviderContext): Promise { const url = new URL('https://commons.wikimedia.org/w/api.php') From e4aa2bc75173ef9e37d2403e04ac3496f454d72a Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Sat, 25 Jul 2026 00:30:52 +0800 Subject: [PATCH 8/9] feat(mcp): declaration-derived schema, source list, providers parameter, kind in output --- packages/mcp/src/__tests__/mcp.test.ts | 39 ++++++++++++ packages/mcp/src/index.ts | 86 ++++++++++++++++---------- 2 files changed, 93 insertions(+), 32 deletions(-) diff --git a/packages/mcp/src/__tests__/mcp.test.ts b/packages/mcp/src/__tests__/mcp.test.ts index d9f9714..e2a12de 100644 --- a/packages/mcp/src/__tests__/mcp.test.ts +++ b/packages/mcp/src/__tests__/mcp.test.ts @@ -477,3 +477,42 @@ describe('defaultProviders (zero-config CLI wiring)', () => { for (const p of table) expect(pkg.dependencies ?? {}).not.toHaveProperty(p) }) }) + +describe('dynamic declaration-derived schema', () => { + async function declClient() { + const tex = defineProvider({ + id: 'texsrc', modalities: ['image'], kinds: ['texture', 'custom-kind'], + description: 'CC0 textures for tests', search: async () => [], + }) + const plain = defineProvider({ id: 'plain', modalities: ['audio'], search: async () => [] }) + const refkit = createRefkit({ providers: [tex, plain], fetch: (async () => new Response('{}')) as typeof fetch }) + const server = createRefkitMcpServer(refkit) + const [clientT, serverT] = InMemoryTransport.createLinkedPair() + const client = new Client({ name: 'test', version: '1.0.0' }) + await Promise.all([client.connect(clientT), server.connect(serverT)]) + return client + } + + it('appends a per-provider source list to the tool description', async () => { + const client = await declClient() + const { tools } = await client.listTools() + const tool = tools.find(t => t.name === 'search_references')! + expect(tool.description).toContain('Configured sources:') + expect(tool.description).toContain('- texsrc (image·texture,custom-kind): CC0 textures for tests') + expect(tool.description).toContain('- plain (audio)') + await client.close() + }) + + it('derives modalities / media.kind / providers enums from declarations', async () => { + const client = await declClient() + const { tools } = await client.listTools() + const schema = tools.find(t => t.name === 'search_references')!.inputSchema as Record + const modalityEnum = schema.properties.modalities.items.enum as string[] + expect([...modalityEnum].sort()).toEqual(['audio', 'image']) + const providerEnum = schema.properties.providers.items.enum as string[] + expect(providerEnum).toEqual(['texsrc', 'plain']) + const kindEnum = schema.properties.controls.properties.media.properties.kind.enum as string[] + expect(kindEnum).toEqual(expect.arrayContaining(['photo', 'illustration', 'vector', 'film', 'animation', 'texture', 'custom-kind'])) + await client.close() + }) +}) diff --git a/packages/mcp/src/index.ts b/packages/mcp/src/index.ts index 122388f..64f53e3 100644 --- a/packages/mcp/src/index.ts +++ b/packages/mcp/src/index.ts @@ -3,10 +3,14 @@ import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js' import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js' import { z } from 'zod' import { LICENSE_IDS, INTENTS, evaluateUse, buildAttribution, ccVersionFor, lexicalReranker } from '@refkit/core' -import type { RefkitClient, Reference, Verdict, Attribution, SearchFilters, SearchControls, SearchControlKey, ProviderOptionsById, SearchMeta, RightsRecord } from '@refkit/core' +import type { RefkitClient, Reference, Verdict, Attribution, SearchFilters, SearchControls, SearchControlKey, ProviderOptionsById, SearchMeta, RightsRecord, Modality } from '@refkit/core' const MODALITIES = ['image', 'video', 'audio', 'text'] as const const ORIENTATIONS = ['landscape', 'portrait', 'square'] as const +// Legacy media.kind control values — kept in the dynamic enum because they stay +// meaningful as upstream filter translations for providers that support the +// media.kind control without declaring kinds. +const BASE_MEDIA_KINDS = ['photo', 'illustration', 'vector', 'film', 'animation'] as const const SEARCH_CONTROL_KEYS = [ 'orientation', 'color', @@ -34,33 +38,35 @@ const filtersSchema = z.object({ }) const searchControlKeySchema = z.enum(SEARCH_CONTROL_KEYS) -const searchControlsSchema = z.object({ - orientation: z.enum(ORIENTATIONS).optional(), - color: z.string().optional(), - language: z.string().optional(), - sort: z.enum(['relevance', 'latest', 'popular', 'interesting']).optional(), - safety: z.enum(['strict', 'moderate', 'off']).optional(), - license: z.object({ - commercial: z.boolean().optional(), - modification: z.boolean().optional(), - allowUnknown: z.boolean().optional(), - }).optional(), - media: z.object({ - kind: z.enum(['photo', 'illustration', 'vector', 'film', 'animation']).optional(), - size: z.enum(['small', 'medium', 'large']).optional(), - minWidth: z.number().int().nonnegative().optional(), - minHeight: z.number().int().nonnegative().optional(), - duration: z.enum(['short', 'medium', 'long']).optional(), - }).optional(), - creator: z.object({ - id: z.string().optional(), - name: z.string().optional(), - }).optional(), - text: z.object({ - copyright: z.enum(['public-domain', 'copyrighted', 'any']).optional(), - }).optional(), - page: z.number().int().positive().optional(), -}) +function buildSearchControlsSchema(kindValues: [string, ...string[]]) { + return z.object({ + orientation: z.enum(ORIENTATIONS).optional(), + color: z.string().optional(), + language: z.string().optional(), + sort: z.enum(['relevance', 'latest', 'popular', 'interesting']).optional(), + safety: z.enum(['strict', 'moderate', 'off']).optional(), + license: z.object({ + commercial: z.boolean().optional(), + modification: z.boolean().optional(), + allowUnknown: z.boolean().optional(), + }).optional(), + media: z.object({ + kind: z.enum(kindValues).optional(), + size: z.enum(['small', 'medium', 'large']).optional(), + minWidth: z.number().int().nonnegative().optional(), + minHeight: z.number().int().nonnegative().optional(), + duration: z.enum(['short', 'medium', 'long']).optional(), + }).optional(), + creator: z.object({ + id: z.string().optional(), + name: z.string().optional(), + }).optional(), + text: z.object({ + copyright: z.enum(['public-domain', 'copyrighted', 'any']).optional(), + }).optional(), + page: z.number().int().positive().optional(), + }) +} const providerOptionValueSchema = z.union([z.string(), z.number(), z.boolean(), z.array(z.string())]) const providerOptionsSchema = z.record(z.string(), z.record(z.string(), providerOptionValueSchema)) @@ -83,6 +89,7 @@ function toAgentRef(r: Reference, assessment?: { verdict: Verdict; attribution: id: r.id, title: r.title, modality: r.modality, + kind: r.kind, provider: r.source.providerId, canonicalUrl: r.canonicalUrl, license: r.rights.license, @@ -105,6 +112,7 @@ const agentRefSchema = z.object({ id: z.string(), title: z.string().optional(), modality: z.string(), + kind: z.string().optional().describe('fine-grained resource kind, e.g. photo / texture / ebook'), provider: z.string(), canonicalUrl: z.string(), license: z.string(), @@ -137,7 +145,7 @@ const searchMetaSchema: z.ZodType = z.object({ returned: z.number().optional(), accepted: z.number().optional(), rejected: z.number().optional(), - reason: z.enum(['unsupported-modality']).optional(), + reason: z.enum(['unsupported-modality', 'unsupported-kind', 'not-selected']).optional(), error: z.string().optional(), latencyMs: z.number().optional(), cached: z.boolean().optional(), @@ -156,6 +164,17 @@ const searchMetaSchema: z.ZodType = z.object({ export function createRefkitMcpServer(refkit: RefkitClient): McpServer { const server = new McpServer({ name: 'refkit', version: VERSION }) + const registered = refkit.providers + const modalityValues = [...new Set(registered.flatMap(p => p.modalities))] as [Modality, ...Modality[]] + const kindValues = [...new Set([...BASE_MEDIA_KINDS, ...registered.flatMap(p => p.kinds ?? [])])] as [string, ...string[]] + const providerIds = registered.map(p => p.id) as [string, ...string[]] + const sourceList = registered.map(p => { + const kinds = p.kinds?.length ? `·${p.kinds.join(',')}` : '' + const desc = p.description ? `: ${p.description}` : '' + return `- ${p.id} (${p.modalities.join('/')}${kinds})${desc}` + }).join('\n') + const searchControlsSchema = buildSearchControlsSchema(kindValues) + server.registerTool( 'search_references', { @@ -164,13 +183,15 @@ export function createRefkitMcpServer(refkit: RefkitClient): McpServer { 'Search license-normalized reference material (image / video / audio / text) across the configured sources. ' + 'Every result carries a license id + canonical source link. Pass `intent` to annotate each result with a ' + 'use-verdict (may I use this, is attribution required) WITHOUT filtering; pass `gateFor` to instead return ' + - 'only results whose license allows that intent. Results are references, not rights clearance — not legal advice.', + 'only results whose license allows that intent. Results are references, not rights clearance — not legal advice.' + + '\n\nConfigured sources:\n' + sourceList, inputSchema: { query: z.string().describe('what to search for, e.g. "cyberpunk alley at night"'), - modalities: z.array(z.enum(MODALITIES)).optional().describe('default ["image"]'), + modalities: z.array(z.enum(modalityValues)).optional().describe('default ["image"]'), filters: filtersSchema.optional().describe('compatibility alias for controls.orientation, controls.color, and controls.language'), controls: searchControlsSchema.optional().describe('provider-neutral search controls; providers translate supported controls and report ignored controls in explain metadata'), providerOptions: providerOptionsSchema.optional().describe('provider-specific search controls keyed by provider id; each provider whitelists supported keys'), + providers: z.array(z.enum(providerIds)).optional().describe('restrict the search to these source ids (see Configured sources in this tool description)'), explain: z.boolean().optional().describe('include provider status, applied and ignored controls, warnings, gate/drop metadata, and the load-more cursor'), limit: z.number().int().positive().optional(), cursor: z.string().optional().describe('opaque cursor from a previous result\'s nextCursor — fetches the next batch, deduped against earlier batches'), @@ -184,13 +205,14 @@ export function createRefkitMcpServer(refkit: RefkitClient): McpServer { meta: searchMetaSchema.optional(), }, }, - async ({ query, modalities, filters, controls, providerOptions, explain, limit, cursor, rerank, intent, gateFor }) => { + async ({ query, modalities, filters, controls, providerOptions, providers, explain, limit, cursor, rerank, intent, gateFor }) => { const searchInput = { query, modalities: modalities ?? ['image'], filters: filters as SearchFilters | undefined, controls: controls as SearchControls | undefined, providerOptions: providerOptions as ProviderOptionsById | undefined, + providers, limit, cursor, ...(rerank ? { rerank: lexicalReranker() } : {}), From 431d8341cc91a418d3f7453b250c52e7bcb5d277 Mon Sep 17 00:00:00 2001 From: lizhixuan Date: Sat, 25 Jul 2026 00:36:56 +0800 Subject: [PATCH 9/9] chore: changeset for provider resource declarations --- .changeset/provider-resource-declaration.md | 36 +++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 .changeset/provider-resource-declaration.md diff --git a/.changeset/provider-resource-declaration.md b/.changeset/provider-resource-declaration.md new file mode 100644 index 0000000..2ecda7a --- /dev/null +++ b/.changeset/provider-resource-declaration.md @@ -0,0 +1,36 @@ +--- +'@refkit/core': minor +'@refkit/mcp': minor +'@refkit/provider-testkit': minor +'@refkit/provider-unsplash': minor +'@refkit/provider-pexels': minor +'@refkit/provider-pixabay': minor +'@refkit/provider-flickr': minor +'@refkit/provider-brave': minor +'@refkit/provider-wikimedia-commons': minor +'@refkit/provider-openverse': minor +'@refkit/provider-met': minor +'@refkit/provider-artic': minor +'@refkit/provider-rijksmuseum': minor +'@refkit/provider-smithsonian': minor +'@refkit/provider-europeana': minor +'@refkit/provider-freesound': minor +'@refkit/provider-jamendo': minor +'@refkit/provider-gutendex': minor +'@refkit/provider-poetrydb': minor +'@refkit/provider-internet-archive': minor +'@refkit/provider-polyhaven': minor +--- + +Provider resource declarations: open `ResourceKind` vocabulary with optional +`kinds` + `description` on providers and `kind` on references; kind-aware +routing and a `providers` id whitelist with new skip reasons +(`unsupported-kind`, `not-selected`); MCP tool schema, source list, and enums +now derived from registered provider declarations at startup. + +Note: the MCP `modalities` and new `providers` input enums are now +deployment-dependent (derived from the registered providers). A request naming +a modality no registered provider supports is now rejected at the schema +boundary — previously a fully-unsupported request threw at search time, and a +mixed request (e.g. image+audio against an image-only deployment) returned the +supported subset. Fail-loud at the boundary is intentional.