diff --git a/examples/firecrawl_web_monitor/README.md b/examples/firecrawl_web_monitor/README.md new file mode 100644 index 00000000000..9e41d199ade --- /dev/null +++ b/examples/firecrawl_web_monitor/README.md @@ -0,0 +1,117 @@ +# Firecrawl Web Monitoring with ZenML + +This example turns each Firecrawl `monitor.page` event into a ZenML pipeline run. Firecrawl monitors and scrapes the page; ZenML preserves the event, diff, analysis, and report as independently versioned artifacts. An LLM interprets the change when an OpenAI key is stored in a ZenML secret and passed via `--llm-secret`, and a configured ZenML alerter can optionally post meaningful changes to Slack. + +```text +Firecrawl monitor.page event (pulled via API or delivered by webhook) + | + v +raw event -> normalized diff -> LLM analysis -> report -> optional Slack + v1 v1 v1 v1 + v2 v2 v2 v2 +``` + +The stable artifact names make every run easy to compare in the ZenML dashboard: + +- `firecrawl_monitor_event` stores the source webhook. +- `web_page_change` stores the normalized diff and Firecrawl scrape IDs. +- `web_change_analysis` stores the business interpretation. +- `web_monitoring_report` joins the evidence and analysis for downstream use. + +## Quick start + +This example assumes that a [ZenML stack](https://docs.zenml.io/stacks) is already configured. From this directory, install the example dependencies and run the bundled realistic event: + +```bash +uv pip install -e . # or: pip install -e . +zenml init +zenml stack set +python run.py +``` + +Run everything from this directory: `zenml init` sets the ZenML source root here (and the pipeline's Docker settings reference this `pyproject.toml`, so remote image builds pick up the right dependencies). Set the stack *after* `zenml init` — init scopes the active stack to this directory and resets it to `default`. If your stack uses an S3 artifact store, also run `zenml integration install s3 --uv` once before the first run. + +No Firecrawl, OpenAI, or Slack credentials are required for that first run. Without an LLM secret, the analysis uses Firecrawl's meaningful-change judgment and clearly labels itself as a fallback. Store the OpenAI key in ZenML to run the same pipeline with an LLM on both local and remote orchestrators: + +```bash +zenml secret create firecrawl-monitoring \ + --OPENAI_API_KEY= \ + --OPENAI_MODEL=gpt-5-mini +python run.py --llm-secret firecrawl-monitoring +``` + +Run the example twice, optionally editing `sample_payload.json`, then inspect the artifact versions in the dashboard or from the terminal: + +```bash +python history.py +``` + +## Monitor a real page + +Create a Firecrawl `scrape` monitor for the page you care about — no webhook infrastructure needed: + +```bash +export FIRECRAWL_API_KEY= +python create_firecrawl_monitor.py \ + --target-url https://news.ycombinator.com/newest \ + --goal "Alert when new AI or developer-tooling stories are posted" \ + --schedule "every 5 minutes" +``` + +A fast-changing page with a short cadence produces meaningful diffs within minutes — ideal for a first demo; dial it back with `--schedule hourly` or `--schedule daily` for real monitoring. A live monitor keeps running and consuming Firecrawl credits on every scheduled check until you delete it, and `--llm-secret` runs spend OpenAI tokens per analyzed change — so remember to clean up after a demo (see below). Once at least one check has completed, pull it straight from the Firecrawl API and analyze it — one pipeline run per monitored page: + +```bash +python run.py --monitor-id +``` + +This fetches the latest completed check (or a specific one with `--check-id`) and feeds each page result through the same pipeline as the bundled sample, so the artifact history mixes local experiments and real checks seamlessly. Firecrawl emits both unified markdown diffs and structured JSON field diffs; the pipeline preserves both, so a monitor configured for JSON change tracking can compare fields such as prices or availability without parsing prose. + +When you are done, delete the monitor so it stops running checks and billing: + +```bash +curl -X DELETE https://api.firecrawl.dev/v2/monitor/ \ + -H "Authorization: Bearer $FIRECRAWL_API_KEY" +``` + +## Production: trigger a pipeline snapshot + +For an event-driven setup, ZenML supports this natively — no custom receiver required. Publish the pipeline as a [snapshot](https://docs.zenml.io/concepts/snapshots) on a remote stack with a remote orchestrator, a container registry, and a remote artifact store (for example a [Kubernetes orchestrator](https://docs.zenml.io/stacks/stack-components/orchestrators/kubernetes)); the pipeline's Docker settings are already sourced from this `pyproject.toml`, so the code and artifact contracts do not change: + +```bash +zenml stack set +zenml pipeline snapshot create pipelines.monitoring.firecrawl_web_monitor_pipeline \ + --name firecrawl-web-monitor \ + --config configs/snapshot.yaml +``` + +`configs/snapshot.yaml` provides the default parameters baked into the snapshot (the bundled sample payload); triggers can override them per step. Note that server-triggered runs of static pipelines take step-level parameters, not pipeline-level ones. Trigger a run from Python: + +```python +from zenml.client import Client + +Client().trigger_pipeline( + snapshot_name_or_id="firecrawl-web-monitor", + run_configuration={ + "steps": { + "ingest_firecrawl_event": { + "parameters": {"payload": } + } + } + }, +) +``` + +Snapshots can also be [triggered from external systems](https://docs.zenml.io/user-guides/tutorial/trigger-pipelines-from-external-systems) with a single authenticated REST call, which is where Firecrawl's webhook plugs in. + +Tip for Apple Silicon: images build for `linux/amd64` under emulation, where `uv` can crash. If the build fails with exit code 139, override the installer with `settings: {docker: {python_package_installer: pip}}` in the snapshot config. + +## Optional Slack alerts + +ZenML's [Slack alerter](https://docs.zenml.io/stacks/stack-components/alerters/slack) is a stack component registered once with a Slack bot token and channel; any pipeline on the stack can then post to Slack without handling Slack credentials in code. Attach one to the active stack, then enable notifications for CLI runs: + +```bash +python run.py --notify-slack +``` + +Only reports marked meaningful are sent. If notification is enabled but the active stack has no alerter, the step logs a warning and the run still succeeds. + diff --git a/examples/firecrawl_web_monitor/analysis.py b/examples/firecrawl_web_monitor/analysis.py new file mode 100644 index 00000000000..c934499c368 --- /dev/null +++ b/examples/firecrawl_web_monitor/analysis.py @@ -0,0 +1,104 @@ +"""Analysis helpers kept separate from ZenML orchestration for testability.""" + +import json +from typing import Any, Dict, Optional + +from models import ChangeAnalysis, PageChange + +DEFAULT_MODEL = "gpt-5-mini" + + +def analyze_change( + change: PageChange, + goal: str, + api_key: Optional[str] = None, + model: str = DEFAULT_MODEL, +) -> ChangeAnalysis: + """Analyze a change with OpenAI or a deterministic local fallback. + + Args: + change: Normalized page change. + goal: Business goal that defines which changes matter. + api_key: Optional OpenAI API key loaded from a ZenML secret. + model: OpenAI model name. + + Returns: + A structured analysis suitable for alerting and later comparison. + """ + if api_key: + return _analyze_with_openai( + change=change, goal=goal, api_key=api_key, model=model + ) + return _analyze_without_llm(change=change) + + +def _analyze_with_openai( + change: PageChange, goal: str, api_key: str, model: str +) -> ChangeAnalysis: + """Ask OpenAI for a structured interpretation of the diff.""" + from openai import OpenAI + + prompt = ( + "Analyze this website change against the monitoring goal. " + "Return JSON with summary, impact, recommended_actions (a list), " + "meaningful (boolean), and confidence (low, medium, or high).\n\n" + f"Goal: {goal}\nURL: {change.url}\nStatus: {change.status}\n" + f"Unified diff:\n{change.unified_diff[:12000]}\n" + f"Structured diff:\n{json.dumps(change.structured_diff)[:6000]}" + ) + response = OpenAI(api_key=api_key).chat.completions.create( + model=model, + messages=[ + { + "role": "system", + "content": ( + "You are a precise web-change analyst. Base conclusions " + "only on the supplied evidence and return valid JSON." + ), + }, + {"role": "user", "content": prompt}, + ], + response_format={"type": "json_object"}, + ) + content = response.choices[0].message.content + if not content: + raise RuntimeError("The LLM returned an empty analysis.") + result: Dict[str, Any] = json.loads(content) + result.pop("analyzer", None) + return ChangeAnalysis( + **result, + analyzer=model, + ) + + +def _analyze_without_llm(change: PageChange) -> ChangeAnalysis: + """Produce a useful offline result from Firecrawl's own evidence.""" + judgment = change.firecrawl_judgment + if judgment: + actions = [item.reason for item in judgment.meaningful_changes] + return ChangeAnalysis( + summary=judgment.reason, + impact=( + "The change matches the configured monitoring goal." + if judgment.meaningful + else "The change appears to be noise for the monitoring goal." + ), + recommended_actions=actions or ["Review the recorded diff."], + meaningful=judgment.meaningful, + confidence=judgment.confidence, + analyzer="firecrawl-judgment-fallback", + ) + + changed_lines = [ + line + for line in change.unified_diff.splitlines() + if line.startswith(("+", "-")) and not line.startswith(("+++", "---")) + ] + return ChangeAnalysis( + summary=f"Firecrawl reported {change.status} with {len(changed_lines)} changed lines.", + impact="An LLM was not configured, so business impact was not inferred.", + recommended_actions=["Review the recorded diff."], + meaningful=change.status in {"changed", "new", "removed"}, + confidence="low", + analyzer="deterministic-fallback", + ) diff --git a/examples/firecrawl_web_monitor/configs/dev.yaml b/examples/firecrawl_web_monitor/configs/dev.yaml new file mode 100644 index 00000000000..52ea4ced17a --- /dev/null +++ b/examples/firecrawl_web_monitor/configs/dev.yaml @@ -0,0 +1,3 @@ +enable_cache: false +tags: + - local diff --git a/examples/firecrawl_web_monitor/configs/snapshot.yaml b/examples/firecrawl_web_monitor/configs/snapshot.yaml new file mode 100644 index 00000000000..82675b46f7e --- /dev/null +++ b/examples/firecrawl_web_monitor/configs/snapshot.yaml @@ -0,0 +1,34 @@ +parameters: + payload: + success: true + type: monitor.page + id: 019df960-5f2a-75fb-a98b-bd2d32ca67d4 + webhookId: f1e2d3c4-0000-0000-0000-000000000000 + data: + - monitorId: 019df960-06e7-7383-9d89-82c0113dc31a + checkId: 019df960-5f2a-75fb-a98b-bd2d32ca67d4 + url: https://example.com/pricing + status: changed + previousScrapeId: 019df94f-82c3-7e41-81f0-00c72b2d9c52 + currentScrapeId: 019df960-73ee-7ac2-97a9-fb0e442c21f1 + error: null + isMeaningful: true + judgment: + meaningful: true + confidence: high + reason: The Starter plan price increased from $19 to $24 per month. + meaningfulChanges: + - type: changed + before: "Starter \u2014 $19/mo" + after: "Starter \u2014 $24/mo" + reason: A competitor price change may require a positioning review. + diff: + text: "--- previous\n+++ current\n@@ -1,3 +1,3 @@\n # Pricing\n-Starter \u2014\ + \ $19/mo\n+Starter \u2014 $24/mo\n" + metadata: + environment: demo + goal: Identify product, pricing, positioning, or release changes that require + a response from our product or go-to-market teams. + notify_slack: false + llm_secret_name: null +enable_cache: false diff --git a/examples/firecrawl_web_monitor/constants.py b/examples/firecrawl_web_monitor/constants.py new file mode 100644 index 00000000000..aeb8152f2f6 --- /dev/null +++ b/examples/firecrawl_web_monitor/constants.py @@ -0,0 +1,6 @@ +"""Shared defaults for the Firecrawl monitoring example.""" + +DEFAULT_GOAL = ( + "Identify product, pricing, positioning, or release changes that require " + "a response from our product or go-to-market teams." +) diff --git a/examples/firecrawl_web_monitor/create_firecrawl_monitor.py b/examples/firecrawl_web_monitor/create_firecrawl_monitor.py new file mode 100644 index 00000000000..b3001f22c00 --- /dev/null +++ b/examples/firecrawl_web_monitor/create_firecrawl_monitor.py @@ -0,0 +1,102 @@ +"""Create a Firecrawl page monitor whose checks the pipeline can pull.""" + +import argparse +import os +from typing import Any, Dict, Optional + +import requests +from constants import DEFAULT_GOAL + + +def build_monitor_request( + target_url: str, + goal: str, + schedule: str, + webhook_url: Optional[str] = None, + webhook_token: Optional[str] = None, +) -> Dict[str, Any]: + """Build a Firecrawl v2 monitor request. + + Args: + target_url: Page that Firecrawl should monitor and scrape. + goal: Meaningful-change goal. + schedule: Firecrawl natural-language schedule. + webhook_url: Optional delivery endpoint. Without it, checks are + pulled via the API (``run.py --monitor-id``). + webhook_token: Optional bearer token for the webhook endpoint. + + Returns: + Firecrawl monitor request body. + """ + request: Dict[str, Any] = { + "name": f"ZenML monitor: {target_url}", + "schedule": {"text": schedule, "timezone": "UTC"}, + "goal": goal, + "targets": [ + { + "type": "scrape", + "urls": [target_url], + "scrapeOptions": { + "formats": ["markdown"], + "onlyMainContent": True, + }, + } + ], + } + if webhook_url: + headers = ( + {"Authorization": f"Bearer {webhook_token}"} + if webhook_token + else {} + ) + request["webhook"] = { + "url": webhook_url, + "headers": headers, + "metadata": {"source": "zenml-firecrawl-example"}, + "events": ["monitor.page"], + } + return request + + +def main() -> None: + """Create a monitor using credentials from the environment.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--target-url", required=True) + parser.add_argument("--goal", default=DEFAULT_GOAL) + parser.add_argument("--schedule", default="hourly") + parser.add_argument( + "--webhook-url", + default=None, + help="Optional endpoint Firecrawl should POST monitor.page events " + "to, e.g. a relay that triggers a ZenML pipeline snapshot. Omit it " + "to pull checks with `run.py --monitor-id` instead.", + ) + args = parser.parse_args() + + api_key = os.getenv("FIRECRAWL_API_KEY") + if not api_key: + raise RuntimeError("Set FIRECRAWL_API_KEY before creating a monitor.") + + response = requests.post( + "https://api.firecrawl.dev/v2/monitor", + headers={ + "Authorization": f"Bearer {api_key}", + "Content-Type": "application/json", + }, + json=build_monitor_request( + target_url=args.target_url, + goal=args.goal, + schedule=args.schedule, + webhook_url=args.webhook_url, + webhook_token=os.getenv("FIRECRAWL_WEBHOOK_TOKEN"), + ), + timeout=30, + ) + response.raise_for_status() + body = response.json() + monitor_id = body.get("id") or body.get("data", {}).get("id", "") + print(f"Created Firecrawl monitor: {monitor_id}") + + +if __name__ == "__main__": + main() diff --git a/examples/firecrawl_web_monitor/fetch_firecrawl_check.py b/examples/firecrawl_web_monitor/fetch_firecrawl_check.py new file mode 100644 index 00000000000..b42c0f50a3f --- /dev/null +++ b/examples/firecrawl_web_monitor/fetch_firecrawl_check.py @@ -0,0 +1,105 @@ +"""Pull Firecrawl monitor checks and convert them into pipeline payloads.""" + +from typing import Any, Dict, List, Optional + +import requests + +FIRECRAWL_API_URL = "https://api.firecrawl.dev/v2" + +# Firecrawl timestamps a completed check under one of these keys; used to pick +# the newest check without relying on the API's response ordering. +_CHECK_TIMESTAMP_KEYS = ("completedAt", "updatedAt", "createdAt", "startedAt") + + +def _latest_check(checks: List[Dict[str, Any]]) -> Dict[str, Any]: + """Return the most recent check by timestamp, ignoring response order. + + Args: + checks: Completed checks as returned by the checks API. + + Returns: + The check with the newest timestamp, or the first entry if none of + the checks carry a recognized timestamp field. + """ + for key in _CHECK_TIMESTAMP_KEYS: + if all(key in check for check in checks): + return max(checks, key=lambda check: check[key]) + return checks[0] + + +def build_page_payloads(check: Dict[str, Any]) -> List[Dict[str, Any]]: + """Convert a check-details response into per-page pipeline payloads. + + The per-page objects returned by the checks API match the ``data`` + entries of a ``monitor.page`` webhook, so each page becomes one payload + with the same envelope a webhook delivery would carry. ``monitorId`` and + ``checkId`` live on the parent check object in API responses, so they + are merged into each page. + + Args: + check: The ``data`` object of a check-details response. + + Returns: + One ``monitor.page``-style payload per page result. + """ + payloads: List[Dict[str, Any]] = [] + for page in check.get("pages", []): + page_data = { + "monitorId": check["monitorId"], + "checkId": check["id"], + **page, + } + payloads.append( + { + "success": True, + "type": "monitor.page", + "id": check["id"], + "webhookId": "api-pull", + "data": [page_data], + "metadata": {"source": "fetch_firecrawl_check"}, + } + ) + return payloads + + +def fetch_check( + api_key: str, monitor_id: str, check_id: Optional[str] = None +) -> Dict[str, Any]: + """Fetch a monitor check, defaulting to the latest completed one. + + Args: + api_key: Firecrawl API key. + monitor_id: Monitor to read checks from. + check_id: Specific check to fetch. Defaults to the most recent + completed check. + + Raises: + RuntimeError: If the monitor has no completed checks yet. + + Returns: + The ``data`` object of the check-details response. + """ + headers = {"Authorization": f"Bearer {api_key}"} + if check_id is None: + response = requests.get( + f"{FIRECRAWL_API_URL}/monitor/{monitor_id}/checks", + headers=headers, + params={"status": "completed"}, + timeout=30, + ) + response.raise_for_status() + checks = response.json().get("data", []) + if not checks: + raise RuntimeError( + f"Monitor {monitor_id} has no completed checks yet." + ) + check_id = _latest_check(checks)["id"] + + response = requests.get( + f"{FIRECRAWL_API_URL}/monitor/{monitor_id}/checks/{check_id}", + headers=headers, + timeout=30, + ) + response.raise_for_status() + data: Dict[str, Any] = response.json()["data"] + return data diff --git a/examples/firecrawl_web_monitor/history.py b/examples/firecrawl_web_monitor/history.py new file mode 100644 index 00000000000..ded0991ff5f --- /dev/null +++ b/examples/firecrawl_web_monitor/history.py @@ -0,0 +1,34 @@ +"""Print the recent version history of the monitoring report artifact.""" + +from models import MonitoringReport + +from zenml.client import Client + + +def main() -> None: + """Load and summarize recent report artifact versions.""" + versions = Client().list_artifact_versions( + name="web_monitoring_report", sort_by="desc:created", size=10 + ) + if not versions: + print("No monitoring reports found. Run `python run.py` first.") + return + + for version in versions: + try: + report: MonitoringReport = version.load() + except RuntimeError as error: + print( + f"v{version.version}: not loadable from this environment " + f"({error})" + ) + continue + print( + f"v{version.version}: {report.status} | " + f"meaningful={report.analysis.meaningful} | {report.url}" + ) + print(f" {report.analysis.summary}") + + +if __name__ == "__main__": + main() diff --git a/examples/firecrawl_web_monitor/models.py b/examples/firecrawl_web_monitor/models.py new file mode 100644 index 00000000000..16b8441bd04 --- /dev/null +++ b/examples/firecrawl_web_monitor/models.py @@ -0,0 +1,126 @@ +"""Typed data contracts for the Firecrawl monitoring example.""" + +from typing import Any, Dict, List, Literal, Optional + +from pydantic import BaseModel, ConfigDict, Field, field_validator + +ChangeStatus = Literal["same", "changed", "new", "removed", "error"] + + +class MeaningfulChange(BaseModel): + """A single meaningful change identified by Firecrawl.""" + + type: str + before: Optional[str] = None + after: Optional[str] = None + reason: str + + +class FirecrawlJudgment(BaseModel): + """Firecrawl's optional meaningful-change judgment.""" + + meaningful: bool + confidence: str + reason: str + meaningful_changes: List[MeaningfulChange] = Field( + default_factory=list, alias="meaningfulChanges" + ) + + model_config = ConfigDict(populate_by_name=True) + + +class FirecrawlDiff(BaseModel): + """Text and structured diffs emitted by Firecrawl.""" + + text: str = "" + structured: Dict[str, Any] = Field(default_factory=dict, alias="json") + + model_config = ConfigDict(populate_by_name=True) + + +class FirecrawlPageData(BaseModel): + """One page result from a ``monitor.page`` webhook.""" + + monitor_id: str = Field(alias="monitorId") + check_id: str = Field(alias="checkId") + url: str + status: ChangeStatus + previous_scrape_id: Optional[str] = Field( + default=None, alias="previousScrapeId" + ) + current_scrape_id: Optional[str] = Field( + default=None, alias="currentScrapeId" + ) + error: Optional[str] = None + is_meaningful: Optional[bool] = Field(default=None, alias="isMeaningful") + judgment: Optional[FirecrawlJudgment] = None + diff: FirecrawlDiff = Field(default_factory=FirecrawlDiff) + + model_config = ConfigDict(populate_by_name=True) + + @field_validator("diff", mode="before") + @classmethod + def _default_null_diff(cls, value: object) -> object: + """Firecrawl sends an explicit null diff for baseline 'new' pages. + + Args: + value: Raw ``diff`` value from the payload. + + Returns: + An empty diff object when the payload contains null. + """ + return value if value is not None else {} + + +class FirecrawlWebhook(BaseModel): + """Firecrawl webhook envelope used to start the pipeline.""" + + success: bool + type: str + id: str + webhook_id: str = Field(alias="webhookId") + data: List[Dict[str, Any]] + metadata: Dict[str, Any] = Field(default_factory=dict) + + model_config = ConfigDict(populate_by_name=True) + + +class PageChange(BaseModel): + """Normalized page change consumed by downstream analysis.""" + + event_id: str + monitor_id: str + check_id: str + url: str + status: ChangeStatus + previous_scrape_id: Optional[str] = None + current_scrape_id: Optional[str] = None + unified_diff: str = "" + structured_diff: Dict[str, Any] = Field(default_factory=dict) + is_meaningful: Optional[bool] = None + firecrawl_judgment: Optional[FirecrawlJudgment] = None + metadata: Dict[str, Any] = Field(default_factory=dict) + + +class ChangeAnalysis(BaseModel): + """Structured business analysis of a monitored page change.""" + + summary: str + impact: str + recommended_actions: List[str] + meaningful: bool + confidence: str + analyzer: str + + +class MonitoringReport(BaseModel): + """Final report joining source evidence with its analysis.""" + + url: str + status: ChangeStatus + goal: str + analysis: ChangeAnalysis + unified_diff: str + monitor_id: str + check_id: str + event_id: str diff --git a/examples/firecrawl_web_monitor/pipelines/__init__.py b/examples/firecrawl_web_monitor/pipelines/__init__.py new file mode 100644 index 00000000000..7f541babf16 --- /dev/null +++ b/examples/firecrawl_web_monitor/pipelines/__init__.py @@ -0,0 +1 @@ +"""Pipelines for the Firecrawl web monitoring example.""" diff --git a/examples/firecrawl_web_monitor/pipelines/monitoring.py b/examples/firecrawl_web_monitor/pipelines/monitoring.py new file mode 100644 index 00000000000..408b67205e5 --- /dev/null +++ b/examples/firecrawl_web_monitor/pipelines/monitoring.py @@ -0,0 +1,52 @@ +"""ZenML pipeline for analyzing Firecrawl monitoring events.""" + +from typing import Any, Dict, Optional + +from models import MonitoringReport +from steps.analyze_page_change import analyze_page_change +from steps.build_monitoring_report import build_monitoring_report +from steps.ingest_firecrawl_event import ingest_firecrawl_event +from steps.normalize_page_change import normalize_page_change +from steps.optionally_notify_slack import optionally_notify_slack + +from zenml import pipeline +from zenml.config import DockerSettings + +docker_settings = DockerSettings( + pyproject_path="pyproject.toml", + python_package_installer="uv", +) + + +@pipeline( + enable_cache=False, + tags=["web-monitoring", "firecrawl"], + settings={"docker": docker_settings}, +) +def firecrawl_web_monitor_pipeline( + payload: Dict[str, Any], + goal: str, + notify_slack: bool = False, + llm_secret_name: Optional[str] = None, +) -> MonitoringReport: + """Version, analyze, and optionally alert on a Firecrawl page event. + + Args: + payload: A ``monitor.page`` webhook payload. + goal: Business goal used to judge the diff. + notify_slack: Whether to use the active stack's alerter. + llm_secret_name: Optional ZenML secret containing an OpenAI key. + + Returns: + The final monitoring report. + """ + event = ingest_firecrawl_event(payload=payload) + change = normalize_page_change(event=event) + analysis = analyze_page_change( + change=change, goal=goal, llm_secret_name=llm_secret_name + ) + report = build_monitoring_report( + change=change, analysis=analysis, goal=goal + ) + optionally_notify_slack(report=report, notify_slack=notify_slack) + return report diff --git a/examples/firecrawl_web_monitor/pyproject.toml b/examples/firecrawl_web_monitor/pyproject.toml new file mode 100644 index 00000000000..f0a1beac5b9 --- /dev/null +++ b/examples/firecrawl_web_monitor/pyproject.toml @@ -0,0 +1,21 @@ +[build-system] +requires = ["setuptools>=68", "wheel"] +build-backend = "setuptools.build_meta" + +[project] +name = "zenml-firecrawl-web-monitor" +version = "0.1.0" +requires-python = ">=3.10" +dependencies = [ + "openai>=1.0.0,<3.0.0", + "requests>=2.27.11,<3.0.0", + "zenml>=0.93", +] +# Nothing is installed into site-packages: this project exists only to +# declare dependencies (also exported into Docker images via the pipeline's +# DockerSettings). All code runs from this directory, and installing +# modules with generic names like `constants` or `models` would shadow +# other projects and confuse supply-chain scanners. +[tool.setuptools] +py-modules = [] +packages = [] diff --git a/examples/firecrawl_web_monitor/run.py b/examples/firecrawl_web_monitor/run.py new file mode 100644 index 00000000000..760e2cfcc6c --- /dev/null +++ b/examples/firecrawl_web_monitor/run.py @@ -0,0 +1,102 @@ +"""Run the Firecrawl monitoring pipeline from a JSON payload or the API.""" + +import argparse +import json +import os +from pathlib import Path +from typing import Any, Dict, List + +from constants import DEFAULT_GOAL +from fetch_firecrawl_check import build_page_payloads, fetch_check +from pipelines.monitoring import firecrawl_web_monitor_pipeline + + +def load_payload(path: Path) -> Dict[str, Any]: + """Load a Firecrawl webhook payload. + + Args: + path: JSON payload path. + + Returns: + Parsed JSON object. + """ + with path.open(encoding="utf-8") as payload_file: + payload: Dict[str, Any] = json.load(payload_file) + return payload + + +def collect_payloads(args: argparse.Namespace) -> List[Dict[str, Any]]: + """Gather one payload per page, from the Firecrawl API or a local file. + + Args: + args: Parsed CLI arguments. + + Raises: + RuntimeError: If ``--monitor-id`` is used without FIRECRAWL_API_KEY. + + Returns: + Payloads to run the pipeline on, one per page result. + """ + if args.monitor_id: + api_key = os.getenv("FIRECRAWL_API_KEY") + if not api_key: + raise RuntimeError( + "Set FIRECRAWL_API_KEY to pull checks with --monitor-id." + ) + check = fetch_check( + api_key=api_key, + monitor_id=args.monitor_id, + check_id=args.check_id, + ) + return build_page_payloads(check) + return [load_payload(args.payload)] + + +def main() -> None: + """Parse CLI arguments and start the pipeline.""" + parser = argparse.ArgumentParser() + parser.add_argument( + "--config", + type=Path, + default=Path(__file__).parent / "configs/dev.yaml", + ) + parser.add_argument( + "--payload", + type=Path, + default=Path(__file__).with_name("sample_payload.json"), + ) + parser.add_argument( + "--monitor-id", + default=None, + help="Pull the latest completed check of this Firecrawl monitor " + "instead of reading --payload. Requires FIRECRAWL_API_KEY.", + ) + parser.add_argument( + "--check-id", + default=None, + help="Specific check to pull with --monitor-id. Defaults to the " + "latest completed check.", + ) + parser.add_argument("--goal", default=DEFAULT_GOAL) + parser.add_argument("--notify-slack", action="store_true") + parser.add_argument( + "--llm-secret", + default=None, + help="ZenML secret containing OPENAI_API_KEY and optional OPENAI_MODEL.", + ) + args = parser.parse_args() + + pipeline_instance = firecrawl_web_monitor_pipeline.with_options( + config_path=str(args.config), enable_cache=False + ) + for payload in collect_payloads(args): + pipeline_instance( + payload=payload, + goal=args.goal, + notify_slack=args.notify_slack, + llm_secret_name=args.llm_secret, + ) + + +if __name__ == "__main__": + main() diff --git a/examples/firecrawl_web_monitor/sample_payload.json b/examples/firecrawl_web_monitor/sample_payload.json new file mode 100644 index 00000000000..a3876417072 --- /dev/null +++ b/examples/firecrawl_web_monitor/sample_payload.json @@ -0,0 +1,38 @@ +{ + "success": true, + "type": "monitor.page", + "id": "019df960-5f2a-75fb-a98b-bd2d32ca67d4", + "webhookId": "f1e2d3c4-0000-0000-0000-000000000000", + "data": [ + { + "monitorId": "019df960-06e7-7383-9d89-82c0113dc31a", + "checkId": "019df960-5f2a-75fb-a98b-bd2d32ca67d4", + "url": "https://example.com/pricing", + "status": "changed", + "previousScrapeId": "019df94f-82c3-7e41-81f0-00c72b2d9c52", + "currentScrapeId": "019df960-73ee-7ac2-97a9-fb0e442c21f1", + "error": null, + "isMeaningful": true, + "judgment": { + "meaningful": true, + "confidence": "high", + "reason": "The Starter plan price increased from $19 to $24 per month.", + "meaningfulChanges": [ + { + "type": "changed", + "before": "Starter — $19/mo", + "after": "Starter — $24/mo", + "reason": "A competitor price change may require a positioning review." + } + ] + }, + "diff": { + "text": "--- previous\n+++ current\n@@ -1,3 +1,3 @@\n # Pricing\n-Starter — $19/mo\n+Starter — $24/mo\n" + } + } + ], + "metadata": { + "environment": "demo" + } +} + diff --git a/examples/firecrawl_web_monitor/steps/__init__.py b/examples/firecrawl_web_monitor/steps/__init__.py new file mode 100644 index 00000000000..1833fc71abc --- /dev/null +++ b/examples/firecrawl_web_monitor/steps/__init__.py @@ -0,0 +1 @@ +"""Steps for the Firecrawl web monitoring pipeline.""" diff --git a/examples/firecrawl_web_monitor/steps/analyze_page_change.py b/examples/firecrawl_web_monitor/steps/analyze_page_change.py new file mode 100644 index 00000000000..c24d0a56893 --- /dev/null +++ b/examples/firecrawl_web_monitor/steps/analyze_page_change.py @@ -0,0 +1,66 @@ +"""Analyze a normalized website change.""" + +from typing import Annotated, Optional + +from analysis import DEFAULT_MODEL, analyze_change +from models import ChangeAnalysis, PageChange + +from zenml import ArtifactConfig, log_metadata, step +from zenml.client import Client +from zenml.logger import get_logger + +logger = get_logger(__name__) + + +def _load_llm_credentials( + secret_name: Optional[str], +) -> tuple[Optional[str], str]: + """Load optional LLM credentials from the ZenML secret store.""" + if not secret_name: + return None, DEFAULT_MODEL + try: + values = Client().get_secret(secret_name).secret_values + except (KeyError, RuntimeError, ValueError): + logger.warning( + "ZenML secret '%s' was not found; using offline analysis.", + secret_name, + ) + return None, DEFAULT_MODEL + return values.get("OPENAI_API_KEY"), values.get( + "OPENAI_MODEL", DEFAULT_MODEL + ) + + +@step +def analyze_page_change( + change: PageChange, goal: str, llm_secret_name: Optional[str] +) -> Annotated[ + ChangeAnalysis, + ArtifactConfig(name="web_change_analysis", tags=["analysis", "llm"]), +]: + """Interpret the diff in terms of a business goal. + + Args: + change: Normalized web page change. + goal: Business goal for monitoring. + llm_secret_name: ZenML secret containing ``OPENAI_API_KEY``. + + Returns: + Structured change analysis. + """ + api_key, model = _load_llm_credentials(llm_secret_name) + analysis = analyze_change( + change=change, goal=goal, api_key=api_key, model=model + ) + log_metadata( + { + "change_analysis": { + "analyzer": analysis.analyzer, + "meaningful": analysis.meaningful, + "confidence": analysis.confidence, + "status": change.status, + "url": change.url, + } + } + ) + return analysis diff --git a/examples/firecrawl_web_monitor/steps/build_monitoring_report.py b/examples/firecrawl_web_monitor/steps/build_monitoring_report.py new file mode 100644 index 00000000000..5486591e8f4 --- /dev/null +++ b/examples/firecrawl_web_monitor/steps/build_monitoring_report.py @@ -0,0 +1,38 @@ +"""Build the final website monitoring report.""" + +from typing import Annotated + +from models import ChangeAnalysis, MonitoringReport, PageChange + +from zenml import ArtifactConfig, step + + +@step +def build_monitoring_report( + change: PageChange, analysis: ChangeAnalysis, goal: str +) -> Annotated[ + MonitoringReport, + ArtifactConfig( + name="web_monitoring_report", tags=["report", "monitoring"] + ), +]: + """Join evidence and analysis into a final versioned report. + + Args: + change: Source page change. + analysis: Interpretation of the change. + goal: Business monitoring goal. + + Returns: + Complete monitoring report. + """ + return MonitoringReport( + url=change.url, + status=change.status, + goal=goal, + analysis=analysis, + unified_diff=change.unified_diff, + monitor_id=change.monitor_id, + check_id=change.check_id, + event_id=change.event_id, + ) diff --git a/examples/firecrawl_web_monitor/steps/ingest_firecrawl_event.py b/examples/firecrawl_web_monitor/steps/ingest_firecrawl_event.py new file mode 100644 index 00000000000..db93b901ce7 --- /dev/null +++ b/examples/firecrawl_web_monitor/steps/ingest_firecrawl_event.py @@ -0,0 +1,41 @@ +"""Ingest and validate a Firecrawl webhook event.""" + +from typing import Annotated, Any, Dict + +from models import FirecrawlWebhook + +from zenml import ArtifactConfig, log_metadata, step + + +@step +def ingest_firecrawl_event( + payload: Dict[str, Any], +) -> Annotated[ + FirecrawlWebhook, + ArtifactConfig(name="firecrawl_monitor_event", tags=["firecrawl", "raw"]), +]: + """Validate and version the raw webhook envelope. + + Args: + payload: Firecrawl webhook JSON. + + Returns: + The validated webhook event. + """ + event = FirecrawlWebhook.model_validate(payload) + if event.type != "monitor.page": + raise ValueError( + f"Expected a 'monitor.page' event, received '{event.type}'." + ) + if not event.data: + raise ValueError("The Firecrawl event does not contain a page result.") + log_metadata( + { + "firecrawl_event": { + "event_id": event.id, + "event_type": event.type, + "page_count": len(event.data), + } + } + ) + return event diff --git a/examples/firecrawl_web_monitor/steps/normalize_page_change.py b/examples/firecrawl_web_monitor/steps/normalize_page_change.py new file mode 100644 index 00000000000..28b641e78e7 --- /dev/null +++ b/examples/firecrawl_web_monitor/steps/normalize_page_change.py @@ -0,0 +1,52 @@ +"""Normalize a Firecrawl page event.""" + +from typing import Annotated + +from models import FirecrawlPageData, FirecrawlWebhook, PageChange + +from zenml import ArtifactConfig, step +from zenml.logger import get_logger + +logger = get_logger(__name__) + + +@step +def normalize_page_change( + event: FirecrawlWebhook, +) -> Annotated[ + PageChange, + ArtifactConfig(name="web_page_change", tags=["firecrawl", "diff"]), +]: + """Convert a page result into a stable internal contract. + + Args: + event: Validated Firecrawl event. + + Returns: + A normalized page change. + """ + # Firecrawl delivers one monitor.page event per page, and the API-pull + # path emits one single-page payload per page. A multi-page event would + # therefore be a misconfiguration; warn so the dropped pages are visible. + if len(event.data) > 1: + logger.warning( + "Firecrawl event %s carries %d pages; only the first is " + "normalized.", + event.id, + len(event.data), + ) + page = FirecrawlPageData.model_validate(event.data[0]) + return PageChange( + event_id=event.id, + monitor_id=page.monitor_id, + check_id=page.check_id, + url=page.url, + status=page.status, + previous_scrape_id=page.previous_scrape_id, + current_scrape_id=page.current_scrape_id, + unified_diff=page.diff.text, + structured_diff=page.diff.structured, + is_meaningful=page.is_meaningful, + firecrawl_judgment=page.judgment, + metadata=event.metadata, + ) diff --git a/examples/firecrawl_web_monitor/steps/optionally_notify_slack.py b/examples/firecrawl_web_monitor/steps/optionally_notify_slack.py new file mode 100644 index 00000000000..af4e55e6234 --- /dev/null +++ b/examples/firecrawl_web_monitor/steps/optionally_notify_slack.py @@ -0,0 +1,42 @@ +"""Optionally send meaningful change reports to Slack.""" + +from models import MonitoringReport + +from zenml import step +from zenml.client import Client +from zenml.logger import get_logger + +logger = get_logger(__name__) + + +@step(enable_cache=False) +def optionally_notify_slack( + report: MonitoringReport, notify_slack: bool +) -> None: + """Post meaningful reports through the active ZenML alerter. + + Args: + report: Final monitoring report. + notify_slack: Whether notification is enabled for this run. + """ + if not notify_slack or not report.analysis.meaningful: + return + + alerter = Client().active_stack.alerter + if alerter is None: + logger.warning( + "Slack notification requested, but the active stack has no alerter." + ) + return + + actions = "\n".join( + f"- {action}" for action in report.analysis.recommended_actions + ) + alerter.post( + message=( + f"Website change detected: {report.url}\n" + f"*Summary:* {report.analysis.summary}\n" + f"*Impact:* {report.analysis.impact}\n" + f"*Recommended actions:*\n{actions}" + ) + )