Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions packages/tools/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Babylon.js MCP Tools

This directory contains the Babylon.js Model Context Protocol tooling packages used to expose Babylon.js authoring workflows to MCP-compatible clients.

## Packages

| Package | Purpose |
| -------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `mcp-server-core` | Shared internal helpers for MCP response shaping, schema fragments, validation, and file handoff behavior. |
| `nme-mcp-server` | Node Material graph authoring and import/export workflows. |
| `flow-graph-mcp-server` | Flow Graph authoring and coordinator JSON export/import workflows. |
| `gui-mcp-server` | Babylon.js GUI authoring, layout, export/import, and snippet flows. |
| `nge-mcp-server` | Node Geometry graph authoring and export/import workflows. |
| `nrge-mcp-server` | Node Render Graph authoring and render-pipeline export/import workflows. |
| `npe-mcp-server` | Node Particle graph authoring and export/import workflows. |
| `gltf-mcp-server` | glTF/glb asset authoring, inspection, validation, and export. |
| `smart-filters-mcp-server` | Smart Filters graph authoring and export/import workflows. |

## How The Packages Fit Together

The MCP packages are organized as specialized graph or authoring servers, each managing one Babylon.js subsystem in memory. Each server can independently create, edit, validate, and export its graph format.

A future Scene MCP server will act as an orchestrator, consuming exported JSON from these servers to produce runnable scenes.

## Typical Workflow

Each server follows the same general pattern:

```text
1. Create a graph/document in memory
2. Add blocks/controls/nodes and configure them
3. Connect ports or set properties
4. Validate the graph
5. Export to JSON (inline or to a file via outputFile)
```

## Common Development Flow

Most MCP server packages in this folder support the same development commands:

```bash
npm run build -w @tools/<package-name>
npm run start -w @tools/<package-name>
```

The MCP servers are built with Rollup and consume the shared helpers from `@tools/mcp-server-core`.

## Shared Conventions

- JSON export tools generally support `outputFile`
- JSON import tools generally support `json` and `jsonFile`
- snippet-enabled servers generally support `snippetId`
- shared schema, validation, and response helpers live in `mcp-server-core`

## Workspace MCP Configuration

The workspace-level MCP server command mapping lives in `.vscode/mcp.json` at the repository root. That file is useful when testing the servers locally from VS Code or another MCP-aware client.
42 changes: 42 additions & 0 deletions packages/tools/flow-graph-mcp-server/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# @tools/flow-graph-mcp-server

MCP server for AI-driven Babylon.js Flow Graph authoring.

## Provides

- create, inspect, validate, and delete flow graphs
- add blocks and connect data or signal ports
- update block properties and context variables
- export coordinator JSON or graph-only JSON
- import previously exported flow graph JSON

## Typical Workflow

```text
create_graph -> add_block -> connect_data/connect_signal -> set_block_properties -> validate_graph -> export_graph_json
```

Use the full coordinator JSON when handing the result to Scene MCP.

## Binary

```bash
babylonjs-flow-graph
```

## Build And Run

```bash
npm run build -w @tools/flow-graph-mcp-server
npm run start -w @tools/flow-graph-mcp-server
```

## Integration

The exported coordinator JSON can be attached to the Scene MCP server through `attach_flow_graph`, either inline or via `coordinatorJsonFile`.

## Related Files

- `src/index.ts`: MCP tool registration
- `src/flowGraphManager.ts`: graph manager and export/import logic
- `src/blockRegistry.ts`: Flow Graph block catalog
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
{
"_flowGraphs": [
{
"allBlocks": [
{
"className": "FlowGraphSceneReadyEventBlock",
"config": {},
"uniqueId": "fg-00000002",
"dataInputs": [],
"dataOutputs": [],
"signalInputs": [
{
"uniqueId": "fg-00000003",
"name": "in",
"_connectionType": 0,
"connectedPointIds": []
}
],
"signalOutputs": [
{
"uniqueId": "fg-00000004",
"name": "out",
"_connectionType": 1,
"connectedPointIds": []
},
{
"uniqueId": "fg-00000005",
"name": "done",
"_connectionType": 1,
"connectedPointIds": [
"fg-00000008"
]
},
{
"uniqueId": "fg-00000006",
"name": "error",
"_connectionType": 1,
"connectedPointIds": []
}
],
"metadata": {
"displayName": "onReady"
}
},
{
"className": "FlowGraphPlayAnimationBlock",
"config": {
"targetMesh": "hero"
},
"uniqueId": "fg-00000007",
"dataInputs": [
{
"uniqueId": "fg-0000000f",
"name": "speed",
"_connectionType": 0,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "number",
"defaultValue": 0
},
"optional": false
},
{
"uniqueId": "fg-00000010",
"name": "loop",
"_connectionType": 0,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "boolean",
"defaultValue": false
},
"optional": false
},
{
"uniqueId": "fg-00000011",
"name": "from",
"_connectionType": 0,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "number",
"defaultValue": 0
},
"optional": false
},
{
"uniqueId": "fg-00000012",
"name": "to",
"_connectionType": 0,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "number",
"defaultValue": 0
},
"optional": false
},
{
"uniqueId": "fg-00000013",
"name": "animationGroup",
"_connectionType": 0,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "any"
},
"optional": false
},
{
"uniqueId": "fg-00000014",
"name": "animation",
"_connectionType": 0,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "any"
},
"optional": true
},
{
"uniqueId": "fg-00000015",
"name": "object",
"_connectionType": 0,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "any"
},
"optional": true
}
],
"dataOutputs": [
{
"uniqueId": "fg-00000016",
"name": "currentFrame",
"_connectionType": 1,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "number",
"defaultValue": 0
}
},
{
"uniqueId": "fg-00000017",
"name": "currentTime",
"_connectionType": 1,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "number",
"defaultValue": 0
}
},
{
"uniqueId": "fg-00000018",
"name": "currentAnimationGroup",
"_connectionType": 1,
"connectedPointIds": [],
"className": "FlowGraphDataConnection",
"richType": {
"typeName": "any"
}
}
],
"signalInputs": [
{
"uniqueId": "fg-00000008",
"name": "in",
"_connectionType": 0,
"connectedPointIds": []
}
],
"signalOutputs": [
{
"uniqueId": "fg-00000009",
"name": "out",
"_connectionType": 1,
"connectedPointIds": []
},
{
"uniqueId": "fg-0000000a",
"name": "done",
"_connectionType": 1,
"connectedPointIds": []
},
{
"uniqueId": "fg-0000000b",
"name": "error",
"_connectionType": 1,
"connectedPointIds": []
},
{
"uniqueId": "fg-0000000c",
"name": "animationLoopEvent",
"_connectionType": 1,
"connectedPointIds": []
},
{
"uniqueId": "fg-0000000d",
"name": "animationEndEvent",
"_connectionType": 1,
"connectedPointIds": []
},
{
"uniqueId": "fg-0000000e",
"name": "animationGroupLoopEvent",
"_connectionType": 1,
"connectedPointIds": []
}
],
"metadata": {
"displayName": "playAnim"
}
}
],
"executionContexts": [
{
"uniqueId": "fg-00000001",
"_userVariables": {},
"_connectionValues": {}
}
]
}
],
"dispatchEventsSynchronously": false
}
Loading