Add Firecrawl web monitoring example

[htahir1]

What this does

Adds examples/firecrawl_web_monitor, an example that turns each Firecrawl monitor.page event into a ZenML pipeline run. Firecrawl monitors and scrapes a page; ZenML versions the raw event, normalized diff, change analysis, and final report as independently named artifacts (firecrawl_monitor_event, web_page_change, web_change_analysis, web_monitoring_report), so every check is comparable in the dashboard.

The pipeline:

ingest_firecrawl_event -> normalize_page_change -> analyze_page_change -> build_monitoring_report -> optionally_notify_slack

• Event delivery: pull-based — run.py --monitor-id fetches the latest completed check from Firecrawl's API and runs the pipeline once per page result. No webhook infrastructure needed locally; the README documents triggering a pipeline snapshot via the ZenML REST API as the event-driven production path.
• Analysis: uses an OpenAI model when an API key is stored in a ZenML secret (--llm-secret); otherwise falls back to Firecrawl's own meaningful-change judgment, and failing that a deterministic diff summary — each clearly labeled via the analyzer field. No credentials are needed for the first run (a realistic sample payload is bundled).
• Alerting: meaningful changes can optionally be posted through the active stack's alerter (--notify-slack).
• Extras: a Firecrawl monitor provisioning script (create_firecrawl_monitor.py, webhook optional) and an artifact history utility (history.py).

Why

Shows an event-driven, non-ML use of ZenML's artifact versioning and lineage: monitor-triggered runs, Pydantic contracts as artifacts, secrets-based LLM configuration, and portable Docker settings sourced from the example's pyproject.toml.

Implementation notes

• zenml init is run in the example directory and the README instructs running all commands from there, so the source root and the relative pyproject_path in DockerSettings resolve correctly for remote image builds.
• The example's pyproject installs no modules into site-packages (py-modules = []) — it exists purely to declare dependencies. This avoids shadowing generic module names and a Socket supply-chain false positive that matched the local constants.py against the obfuscated constants package on PyPI.
• An earlier revision shipped a custom FastAPI webhook receiver; it was removed in favor of API pulling locally and snapshot triggers in production, dropping the fastapi/uvicorn dependencies.
• Reviewed with a high-effort multi-agent code review; confirmed issues were fixed. Also validated live against a real Firecrawl monitor on the HN new-stories feed (which caught a null-diff edge case in baseline checks).
• Every documented flow was executed successfully: fresh-clone quick start on released ZenML (0.96.1), local runs on an S3-backed stack, live --monitor-id pulls, snapshot creation via an AWS CodeBuild image builder, snapshot runs on a Kubernetes stack triggered both via Client().trigger_pipeline (including live payload + LLM secret as step-level parameter overrides) and via a raw external POST to the snapshot runs endpoint.
• No test suite by design — examples in this repo are validated by running them.

Reviewer attention

• The check-pulling path (fetch_firecrawl_check.py) maps Firecrawl's check-details response onto the same envelope as monitor.page webhook deliveries; per-page objects are identical per Firecrawl's docs, with monitorId/checkId merged in from the parent check.
• Each page result becomes its own pipeline run, matching Firecrawl's one-event-per-page webhook semantics.

🤖 Generated with Claude Code

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	pypi/​wheel@​0.47.0

View full report

[socket-security]

All alerts resolved. Learn more about Socket for GitHub.

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report

[github-actions]

Documentation Link Check Results

✅ Absolute links check passed
✅ Relative links check passed
_{Last checked: 2026-07-17 09:46:13 UTC}