Skip to content

Commit 3e67b36

Browse files
sepcntsepcnt
committed
[world] Add workflow-world-surrealdb
Signed-off-by: sepcnt <30561671+sepcnt@users.noreply.github.com>
1 parent 9cb1fc9 commit 3e67b36

File tree

6 files changed

+318
-3
lines changed

6 files changed

+318
-3
lines changed

‎.github/workflows/benchmark-community-world.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,21 @@ jobs:
8686
sleep 2
8787
done
8888
89+
- name: Start SurrealDB
90+
if: ${{ inputs.service-type == 'surrealdb' }}
91+
run: |
92+
docker run -d --name surrealdb -p 8000:8000 \
93+
surrealdb/surrealdb:v3 \
94+
start --user root --pass root --bind 0.0.0.0:8000 memory
95+
echo "Waiting for SurrealDB to be ready..."
96+
for i in {1..30}; do
97+
if curl -sf http://localhost:8000/health &>/dev/null; then
98+
echo "SurrealDB is ready"
99+
break
100+
fi
101+
sleep 2
102+
done
103+
89104
- name: Setup environment
90105
uses: ./.github/actions/setup-workflow-dev
91106
with:
@@ -143,3 +158,4 @@ jobs:
143158
run: |
144159
docker stop mongodb 2>/dev/null || true
145160
docker stop redis 2>/dev/null || true
161+
docker stop surrealdb 2>/dev/null || true

‎.github/workflows/e2e-community-world.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,21 @@ jobs:
8282
sleep 2
8383
done
8484
85+
- name: Start SurrealDB
86+
if: ${{ inputs.service-type == 'surrealdb' }}
87+
run: |
88+
docker run -d --name surrealdb -p 8000:8000 \
89+
surrealdb/surrealdb:v3 \
90+
start --user root --pass root --bind 0.0.0.0:8000 memory
91+
echo "Waiting for SurrealDB to be ready..."
92+
for i in {1..30}; do
93+
if curl -sf http://localhost:8000/health &>/dev/null; then
94+
echo "SurrealDB is ready"
95+
break
96+
fi
97+
sleep 2
98+
done
99+
85100
- name: Setup environment
86101
uses: ./.github/actions/setup-workflow-dev
87102
with:
@@ -135,3 +150,4 @@ jobs:
135150
run: |
136151
docker stop mongodb 2>/dev/null || true
137152
docker stop redis 2>/dev/null || true
153+
docker stop surrealdb 2>/dev/null || true

‎docs/content/docs/deploying/world/meta.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
{
22
"title": "World",
3-
"pages": ["local-world", "vercel-world", "postgres-world"]
3+
"pages": ["local-world", "vercel-world", "postgres-world", "surrealdb-world"]
44
}

‎docs/content/docs/deploying/world/surrealdb-world.mdx‎

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
---
2+
title: SurrealDB World
3+
description: Community world using SurrealDB with LIVE SELECT for queueing and real-time streaming.
4+
type: integration
5+
summary: Deploy workflows to your own infrastructure using SurrealDB.
6+
prerequisites:
7+
- /docs/deploying
8+
related:
9+
- /docs/deploying/world/local-world
10+
- /docs/deploying/world/postgres-world
11+
---
12+
13+
The SurrealDB World is a community-maintained backend for self-hosted deployments. It uses [SurrealDB](https://surrealdb.com) for durable storage and leverages `LIVE SELECT` for reactive queue processing and real-time streaming.
14+
15+
Use the SurrealDB World when you want a single-binary database that handles storage, queuing, and streaming without additional infrastructure like a separate job queue.
16+
17+
<Callout type="info">
18+
This is a community world maintained at [sepcnt/workflow-world-surrealdb](https://github.com/sepcnt/workflow-world-surrealdb). For bug reports and feature requests, please use that repository.
19+
</Callout>
20+
21+
## Installation
22+
23+
Install the SurrealDB World package in your workflow project:
24+
25+
```package-install
26+
workflow-world-surrealdb
27+
```
28+
29+
Configure the required environment variables:
30+
31+
```bash title=".env"
32+
WORKFLOW_TARGET_WORLD="workflow-world-surrealdb"
33+
WORKFLOW_SURREAL_URL="ws://127.0.0.1:8000/rpc"
34+
WORKFLOW_SURREAL_NAMESPACE="workflow"
35+
WORKFLOW_SURREAL_DATABASE="workflow"
36+
WORKFLOW_SURREAL_USERNAME="root"
37+
WORKFLOW_SURREAL_PASSWORD="root"
38+
```
39+
40+
Run the setup script to create the necessary tables and functions in your database:
41+
42+
```bash
43+
npx workflow-surreal-setup
44+
```
45+
46+
<Callout type="info">
47+
The setup is idempotent and can safely be run as a post-deployment lifecycle script.
48+
</Callout>
49+
50+
## Running SurrealDB
51+
52+
Start a SurrealDB instance using Docker:
53+
54+
```bash
55+
docker run -d -p 8000:8000 \
56+
surrealdb/surrealdb:v3 \
57+
start --user root --pass root --bind 0.0.0.0:8000 memory
58+
```
59+
60+
Or use the included `docker-compose.yml`:
61+
62+
```bash
63+
docker compose up -d
64+
```
65+
66+
For persistent storage, replace `memory` with a file path:
67+
68+
```bash
69+
docker run -d -p 8000:8000 -v surrealdb-data:/data \
70+
surrealdb/surrealdb:v3 \
71+
start --user root --pass root --bind 0.0.0.0:8000 \
72+
"surrealkv:///data/workflow.db?versioned=true"
73+
```
74+
75+
## Starting the World
76+
77+
To subscribe to the queue via LIVE SELECT, your workflow app needs to start the world on server start. Here are examples for a few frameworks:
78+
79+
<Tabs items={["Next.js", "SvelteKit", "Nitro"]}>
80+
81+
<Tab value="Next.js">
82+
83+
Create an `instrumentation.ts` file in your project root:
84+
85+
```ts title="instrumentation.ts" lineNumbers
86+
export async function register() {
87+
if (process.env.NEXT_RUNTIME !== "edge") {
88+
const { getWorld } = await import("workflow/runtime");
89+
await getWorld().start?.();
90+
}
91+
}
92+
```
93+
94+
<Callout type="info">
95+
Learn more about [Next.js Instrumentation](https://nextjs.org/docs/app/guides/instrumentation).
96+
</Callout>
97+
98+
</Tab>
99+
100+
<Tab value="SvelteKit">
101+
102+
Create a `src/hooks.server.ts` file:
103+
104+
```ts title="src/hooks.server.ts" lineNumbers
105+
import type { ServerInit } from "@sveltejs/kit";
106+
107+
export const init: ServerInit = async () => {
108+
const { getWorld } = await import("workflow/runtime");
109+
await getWorld().start?.();
110+
};
111+
```
112+
113+
<Callout type="info">
114+
Learn more about [SvelteKit Hooks](https://svelte.dev/docs/kit/hooks).
115+
</Callout>
116+
117+
</Tab>
118+
119+
<Tab value="Nitro">
120+
121+
Create a plugin to start the world on server initialization:
122+
123+
```ts title="plugins/start-surreal-world.ts" lineNumbers
124+
import { defineNitroPlugin } from "nitro/~internal/runtime/plugin";
125+
126+
export default defineNitroPlugin(async () => {
127+
const { getWorld } = await import("workflow/runtime");
128+
await getWorld().start?.();
129+
});
130+
```
131+
132+
Register the plugin in your config:
133+
134+
```ts title="nitro.config.ts"
135+
import { defineNitroConfig } from "nitropack";
136+
137+
export default defineNitroConfig({
138+
modules: ["workflow/nitro"],
139+
plugins: ["plugins/start-surreal-world.ts"],
140+
});
141+
```
142+
143+
<Callout type="info">
144+
Learn more about [Nitro Plugins](https://v3.nitro.build/docs/plugins).
145+
</Callout>
146+
147+
</Tab>
148+
149+
</Tabs>
150+
151+
<Callout type="info">
152+
The SurrealDB World requires a long-lived worker process that maintains LIVE SELECT subscriptions. This does not work on serverless environments. For Vercel deployments, use the [Vercel World](/worlds/vercel) instead.
153+
</Callout>
154+
155+
## Configuration
156+
157+
All configuration options can be set via environment variables or programmatically via `createWorld()`.
158+
159+
### `WORKFLOW_SURREAL_URL`
160+
161+
SurrealDB WebSocket URL. Supports `ws://`, `wss://`, `http://`, and `https://` protocols (HTTP is auto-converted to WS).
162+
163+
Default: `ws://127.0.0.1:8000/rpc`
164+
165+
### `WORKFLOW_SURREAL_NAMESPACE`
166+
167+
SurrealDB namespace. Default: `workflow`
168+
169+
### `WORKFLOW_SURREAL_DATABASE`
170+
171+
SurrealDB database. Default: `workflow`
172+
173+
### `WORKFLOW_SURREAL_USERNAME` / `WORKFLOW_SURREAL_PASSWORD`
174+
175+
Root or namespace-level credentials for authentication.
176+
177+
### `WORKFLOW_SURREAL_TOKEN`
178+
179+
Alternative to username/password. Use a JWT token for authentication.
180+
181+
### `WORKFLOW_SURREAL_QUEUE_CONCURRENCY`
182+
183+
Number of concurrent jobs to process. Default: `10`
184+
185+
### `WORKFLOW_SURREAL_QUEUE_LANE_SHARDS`
186+
187+
Number of queue lane partitions for horizontal scalability. Default: `16`
188+
189+
### `WORKFLOW_SURREAL_QUEUE_LEASE_MS`
190+
191+
Job lease duration in milliseconds before a job is considered stale. Default: `300000` (5 minutes)
192+
193+
### `WORKFLOW_SURREAL_QUEUE_HEARTBEAT_MS`
194+
195+
Heartbeat interval for extending job leases. Default: `10000` (10 seconds)
196+
197+
### Programmatic configuration
198+
199+
{/* @skip-typecheck: incomplete code sample */}
200+
```typescript title="workflow.config.ts" lineNumbers
201+
import { createWorld } from "workflow-world-surrealdb";
202+
203+
const world = createWorld({
204+
url: "ws://127.0.0.1:8000/rpc",
205+
namespace: "workflow",
206+
database: "workflow",
207+
auth: {
208+
username: "root",
209+
password: "root",
210+
},
211+
queueConcurrency: 20,
212+
queueLaneShards: 32,
213+
});
214+
```
215+
216+
## How It Works
217+
218+
The SurrealDB World uses SurrealDB as a unified backend:
219+
220+
- **Storage** — Workflow runs, events, steps, and hooks are stored in SurrealDB tables with SCHEMAFULL validation
221+
- **Job Queue** — Lane-sharded queue with `LIVE SELECT` subscriptions for instant wake-up and automatic lease recovery
222+
- **Streaming** — `LIVE SELECT` on stream chunk tables enables real-time event distribution with periodic reconciliation
223+
224+
This architecture keeps the infrastructure minimal — a single SurrealDB instance handles storage, queuing, and streaming without additional services. For implementation details, see the [source code](https://github.com/sepcnt/workflow-world-surrealdb).
225+
226+
## Deployment
227+
228+
Deploy your application to any cloud that supports long-running servers:
229+
230+
- Docker containers
231+
- Kubernetes clusters
232+
- Virtual machines
233+
- Platform-as-a-Service providers (Railway, Render, Fly.io, etc.)
234+
235+
Ensure your deployment has:
236+
1. Network access to your SurrealDB instance
237+
2. Environment variables configured correctly
238+
3. The `start()` function called on server initialization
239+
240+
<Callout type="info">
241+
The SurrealDB World is not compatible with Vercel deployments. On Vercel, workflows automatically use the [Vercel World](/worlds/vercel) with zero configuration.
242+
</Callout>
243+
244+
## Limitations
245+
246+
- **Requires long-running process** — Must call `start()` on server initialization; not compatible with serverless platforms
247+
- **SurrealDB infrastructure** — Requires a SurrealDB instance (self-hosted or [Surreal Cloud](https://surrealdb.com/cloud))
248+
- **Not compatible with Vercel** — Use the [Vercel World](/worlds/vercel) for Vercel deployments
249+
- **Community-maintained** — Not an official Vercel package; support via [GitHub Issues](https://github.com/sepcnt/workflow-world-surrealdb/issues)
250+
251+
For local development, use the [Local World](/worlds/local) which requires no external services.

‎scripts/create-community-worlds-matrix.mjs‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,9 @@ const matrix = {
5050
let serviceType = 'none';
5151
if (world.services && world.services.length > 0) {
5252
// Use the first service's name as the service type
53-
// Currently supports: mongodb, redis
53+
// Currently supports: mongodb, redis, surrealdb
5454
const serviceName = world.services[0].name;
55-
if (['mongodb', 'redis'].includes(serviceName)) {
55+
if (['mongodb', 'redis', 'surrealdb'].includes(serviceName)) {
5656
serviceType = serviceName;
5757
}
5858
}

‎worlds-manifest.json‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,38 @@
129129
}
130130
]
131131
},
132+
{
133+
"id": "surrealdb",
134+
"type": "community",
135+
"package": "workflow-world-surrealdb",
136+
"name": "SurrealDB",
137+
"description": "SurrealDB world using LIVE SELECT for queueing and real-time streaming",
138+
"repository": "https://github.com/sepcnt/workflow-world-surrealdb",
139+
"docs": "/docs/deploying/world/surrealdb-world",
140+
"features": [],
141+
"env": {
142+
"WORKFLOW_TARGET_WORLD": "workflow-world-surrealdb",
143+
"WORKFLOW_SURREAL_URL": "ws://127.0.0.1:8000/rpc",
144+
"WORKFLOW_SURREAL_NAMESPACE": "workflow",
145+
"WORKFLOW_SURREAL_DATABASE": "workflow",
146+
"WORKFLOW_SURREAL_USERNAME": "root",
147+
"WORKFLOW_SURREAL_PASSWORD": "root"
148+
},
149+
"services": [
150+
{
151+
"name": "surrealdb",
152+
"image": "surrealdb/surrealdb:v3",
153+
"ports": ["8000:8000"],
154+
"healthCheck": {
155+
"cmd": "curl -sf http://localhost:8000/health || exit 1",
156+
"interval": "5s",
157+
"timeout": "3s",
158+
"retries": 10
159+
}
160+
}
161+
],
162+
"setup": "pnpm exec workflow-surreal-setup"
163+
},
132164
{
133165
"id": "jazz",
134166
"type": "community",

0 commit comments

Comments
 (0)