A versatile data extraction plugin for the xyOps Workflow Automation System.
This Plugin uses the amazing open-source Xberg library to perform structured data extractions on files. It can pull text and markdown out of PDFs, Word Docs or Powerpoint slides, cell grids or CSV out of Excel spreadsheets, perform OCR inside images to extract text, you name it!
To use this Plugin in your workflows, it requires one or more files as inputs. Once the data is extracted, it is returned as job output data (which is passed to subsequent jobs in linked chains and workflows).
- docker
This Plugin can extract structured data from many different formats, including:
| Format | Extensions | MIME Type | Extraction Method | OCR Support | Special Features |
|---|---|---|---|---|---|
.pdf |
application/pdf |
Native Rust | Yes | Metadata extraction, image extraction, text layer detection | |
| Excel | .xlsx, .xlsm, .xlsb, .xls, .xlam, .xla, .ods |
Various Excel MIME types | Native Rust (calamine) | No | Multi-sheet support, formula preservation |
| PowerPoint | .pptx, .pptm, .ppsx |
application/vnd.openxmlformats-officedocument.presentationml.presentation |
Native Rust (roxmltree) | Yes (for embedded images) | Slide extraction, image OCR, table detection |
| Word (Modern) | .docx |
application/vnd.openxmlformats-officedocument.wordprocessingml.document |
Native Rust | No | Preserves formatting, extracts metadata |
| Word (Legacy) | .doc |
application/msword |
Native Rust | No | Legacy Word text extraction |
| PowerPoint (Legacy) | .ppt |
application/vnd.ms-powerpoint |
Native Rust | No | Legacy PowerPoint text extraction |
| OpenDocument Text | .odt |
application/vnd.oasis.opendocument.text |
Native Rust | No | Full OpenDocument support |
| OpenDocument Spreadsheet | .ods |
application/vnd.oasis.opendocument.spreadsheet |
Native Rust (calamine) | No | Multi-sheet support |
| Format | Extensions | MIME Type | Extraction Method | OCR Support | Special Features |
|---|---|---|---|---|---|
| Plain Text | .txt |
text/plain |
Native Rust (streaming) | No | Line/word/character counting, memory-efficient streaming |
| Markdown | .md, .markdown |
text/markdown, text/x-markdown |
Native Rust (streaming) | No | Header extraction, link detection, code block detection |
| HTML | .html, .htm |
text/html, application/xhtml+xml |
Native Rust (html-to-markdown-rs) | No | Converts to Markdown, metadata extraction |
| XML | .xml |
application/xml, text/xml |
Native Rust (quick-xml streaming) | No | Element counting, unique element tracking |
| SVG | .svg |
image/svg+xml |
Native Rust (XML parser) | No | Treated as XML document |
| reStructuredText | .rst |
text/x-rst |
Native (rst-parser) | No | Full reST syntax support |
| Org Mode | .org |
text/x-org |
Native (org) | No | Emacs Org mode support |
| Rich Text Format | .rtf |
application/rtf, text/rtf |
Native (rtf-parser) | No | RTF 1.x support |
| Format | Extensions | MIME Type | Extraction Method | OCR Support | Special Features |
|---|---|---|---|---|---|
| JSON | .json |
application/json, text/json |
Native Rust (serde_json) | No | Field counting, nested structure extraction |
| YAML | .yaml |
application/x-yaml, text/yaml, text/x-yaml |
Native Rust (serde_yaml) | No | Multi-document support, field counting |
| TOML | .toml |
application/toml, text/toml |
Native Rust (toml crate) | No | Configuration file support |
| CSV | .csv |
text/csv |
Native Rust | No | Tabular data extraction |
| TSV | .tsv |
text/tab-separated-values |
Native Rust | No | Tab-separated data extraction |
| Format | Extensions | MIME Type | Extraction Method | OCR Support | Special Features |
|---|---|---|---|---|---|
| EML | .eml |
message/rfc822 |
Native Rust (mail-parser) | No | Header extraction, attachment listing, body text |
| MSG | .msg |
application/vnd.ms-outlook |
Native Rust (mail-parser) | No | Outlook message support, metadata extraction |
All image formats support OCR when configured with ocr parameter in ExtractionConfig.
| Format | Extensions | MIME Type | Extraction Method | OCR Support | Special Features |
|---|---|---|---|---|---|
| PNG | .png |
image/png |
Native Rust (image-rs) | Yes | EXIF metadata extraction |
| JPEG | .jpg, .jpeg |
image/jpeg, image/jpg |
Native Rust (image-rs) | Yes | EXIF metadata extraction |
| WebP | .webp |
image/webp |
Native Rust (image-rs) | Yes | Modern format support |
| BMP | .bmp |
image/bmp, image/x-bmp, image/x-ms-bmp |
Native Rust (image-rs) | Yes | Uncompressed format |
| TIFF | .tiff, .tif |
image/tiff, image/x-tiff |
Native Rust (image-rs) | Yes | Multi-page support |
| GIF | .gif |
image/gif |
Native Rust (image-rs) | Yes | Animation frame extraction |
| JPEG 2000 | .jp2, .jpx, .jpm, .mj2 |
image/jp2, image/jpx, image/jpm, image/mj2 |
Native Rust (image-rs) | Yes | Advanced JPEG format |
| PNM Family | .pnm, .pbm, .pgm, .ppm |
image/x-portable-anymap, etc. |
Native Rust (image-rs) | Yes | NetPBM formats |
| Format | Extensions | MIME Type | Extraction Method | OCR Support | Special Features |
|---|---|---|---|---|---|
| ZIP | .zip |
application/zip, application/x-zip-compressed |
Native Rust (zip crate) | No | File listing, text content extraction |
| TAR | .tar, .tgz |
application/x-tar, application/tar, application/x-gtar, application/x-ustar |
Native Rust (tar crate) | No | Unix archive support, compression detection |
| 7-Zip | .7z |
application/x-7z-compressed |
Native Rust (sevenz-rust) | No | High compression format support |
| Gzip | .gz |
application/gzip |
Native Rust | No | Gzip compression support |
| Format | Extensions | MIME Type | Extraction Method | OCR Support | Special Features |
|---|---|---|---|---|---|
| LaTeX | .tex, .latex |
application/x-latex, text/x-tex |
Native (manual parser) | No | Full LaTeX document support |
| EPUB | .epub |
application/epub+zip |
Native (zip + roxmltree + html-to-markdown-rs) | No | E-book format, metadata extraction |
| BibTeX | .bib |
application/x-bibtex, application/x-biblatex |
Native (biblatex) | No | Bibliography database support |
| Typst | .typst |
application/x-typst |
Native (typst-syntax) | No | Modern typesetting format |
| Jupyter Notebook | .ipynb |
application/x-ipynb+json |
Native (JSON parsing) | No | Code cells, markdown cells, output extraction |
| FictionBook | - | application/x-fictionbook+xml |
Native (fb2) | No | XML-based e-book format |
| DocBook | - | application/docbook+xml |
Native (roxmltree) | No | Technical documentation format |
| JATS | - | application/x-jats+xml |
Native (roxmltree) | No | Journal article XML format |
| OPML | - | application/x-opml+xml |
Native (roxmltree) | No | Outline format |
| RIS | - | application/x-research-info-systems |
Native (ris-parser) | No | Citation format |
| EndNote XML | - | application/x-endnote+xml |
Native (XML parser) | No | Reference manager format |
| CSL JSON | - | application/csl+json |
Native (JSON parser) | No | Citation Style Language JSON |
| Format | MIME Type | Extraction Method | Special Features |
|---|---|---|---|
| CommonMark | text/x-commonmark |
Native (pulldown-cmark) | Standard Markdown spec |
| GitHub Flavored Markdown | text/x-gfm |
Native (pulldown-cmark) | GFM extensions (tables, strikethrough, etc.) |
| MultiMarkdown | text/x-multimarkdown |
Native (pulldown-cmark) | MMD extensions |
| Markdown Extra | text/x-markdown-extra |
Native (pulldown-cmark) | PHP Markdown Extra extensions |
| Format | MIME Type | Extraction Method | Special Features |
|---|---|---|---|
| Man Pages | text/x-mdoc |
Native (mdoc-parser) | Unix manual page format |
| Troff | text/troff |
Native (troff-parser) | Unix document format |
| POD | text/x-pod |
Native (pod-parser) | Perl documentation format |
| DokuWiki | text/x-dokuwiki |
Native (dokuwiki-parser) | Wiki markup format |
See Xberg Format Support for more details.
Your extracted file data will be in the job output data, which can then be used in linked jobs or attached workflow nodes. The plugin unwraps Xberg's result envelope and adds the input filename. Here is an abbreviated PDF example:
{
"filename": "document.pdf",
"content": "123 Somewhere St, Melbourne VIC 3000\n\n(03) 1234 5678\n\nInvoice Number: #20130304\n...",
"mimeType": "application/pdf",
"metadata": {
"title": "Sunny Farm Invoice Sample",
"format": {
"formatType": "pdf",
"pdfVersion": "1.5",
"producer": "Prince 16 (www.princexml.com)",
"isEncrypted": false,
"width": 595,
"height": 842,
"pageCount": 1
},
"outputFormat": "plain",
"ocrUsed": false
},
"extractionMethod": "native",
"tables": []
}Additional fields depend on the input and extraction settings. Optional fields may be omitted. See Xberg's Node.js API for the full result types.
When batch mode is enabled, or when multiple files are provided, each file's extractions are pushed onto an extractions array. Example:
{
"extractions": [
{
"filename": "document1.pdf",
"content": "..."
},
{
"filename": "document2.pdf",
"content": "..."
}
]
} If you also check the "Attach Files" checkbox in the plugin parameters, this will attach all text output as files when the job completes.
Version 3 uses Xberg, the successor to Kreuzberg. To use the new release, update the image tag in your installed Plugin command to ghcr.io/pixlcore/xyplug-extract:v3.0.0. Your existing extraction configuration can stay in place: the plugin silently upgrades the applicable legacy settings in the job's configuration object every time it runs. It does not rewrite the saved Plugin or event in xyOps.
The runtime conversions include:
| Legacy setting | Runtime conversion |
|---|---|
ocr.language: "eng" or "eng+fra" |
ocr.language: ["eng"] or ["eng", "fra"] |
ocr.tesseractConfig.language as a string |
An array of language codes |
Boolean ocr.tesseractConfig.thresholdingMethod |
false becomes 0; true becomes 1 |
chunking.maxChars or chunking.chunkSize |
chunking.maxCharacters |
chunking.maxOverlap or chunking.chunkOverlap |
chunking.overlap |
chunking.enabled: false |
Removes the runtime chunking block to keep chunking disabled |
chunking.sizingType, sizingModel, sizingCacheDir |
A chunking.sizing object with type, model, and cacheDir |
Embedding selectors using modelType and value |
Xberg's type plus name, modelId, or llm.model |
keywords.ngramRange: [1, 3] |
keywords.ngramRange: { "min": 1, "max": 3 } |
layout.preset: "fast" or "accurate" |
layout.strategy: "always", using Xberg's current layout model |
Explicit Xberg settings take precedence over legacy aliases. Conversions are limited to the settings above; removed engines or features without an equivalent are not recreated. Advanced OCR, layout, tokenizer, and embedding configurations may require additional models or runtime libraries. The supplied image includes English Tesseract language data for the default OCR settings.
For new configurations, use Xberg's supported format directly. The default is:
{
"ocr": {
"backend": "tesseract",
"language": ["eng"],
"tesseractConfig": {
"psm": 3
}
}
}The xyOps output wrapper is unchanged: one file returns a single extraction unless Batch Mode is enabled, and multiple files return an extractions array. Attachments, progress updates, and per-file error handling are preserved. If some input files fail, their entries contain an error and the job succeeds when at least one file succeeds. If every file fails, the job fails.
Xberg's document output can differ from Kreuzberg's. Text ordering and table detection may change, and format metadata is now nested under metadata.format. For example, workflows reading metadata.pageCount should read metadata.format.pageCount instead. Review custom settings against the Xberg configuration reference and migration guide.
Use Node.js 24 for local testing, matching the Docker image. Install dependencies with npm install, then run npm test for the configuration compatibility and plugin integration checks.
Use this sample JSON to pass a local file into the Plugin. Set cwd to the directory containing the input file:
{
"xy": 1,
"cwd": "/path/to/your/files",
"input": {
"files": [
{ "filename": "document.pdf" }
]
}
}Example pipe command:
node -e 'console.log(JSON.stringify({ xy: 1, cwd: process.cwd(), input: { files: [{ filename: "document.pdf" }] } }))' | node index.jsMIT
