On EC2 compute environments that use an instance profile for container credentials, running any @batch step fails immediately — before the job reaches AWS — with:
BatchJobException: Unable to launch AWS Batch job. No IAM role specified.
METAFLOW_ECS_S3_ACCESS_IAM_ROLE is not set; the container is expected to acquire credentials from the EC2 instance profile.
Metaflow version: master, 4871fb1
Deployment: AWS Batch on EC2 compute environment, no per-job IAM role
Steps to reproduce
- EC2 compute environment using instance-profile credentials
METAFLOW_ECS_S3_ACCESS_IAM_ROLE not set
- Run any
@batch-decorated flow
Expected
Job submits successfully; the AWS Batch ContainerProperties API documents jobRoleArn as optional.
Actual
The guard in batch_client.py (L82–85) raises unconditionally when iam_role is None, and _register_job_definition always passes "jobRoleArn": job_role to boto3 even when job_role is None, which boto3 rejects for string fields.
On EC2 compute environments that use an instance profile for container credentials, running any
@batchstep fails immediately — before the job reaches AWS — with:METAFLOW_ECS_S3_ACCESS_IAM_ROLEis not set; the container is expected to acquire credentials from the EC2 instance profile.Metaflow version: master,
4871fb1Deployment: AWS Batch on EC2 compute environment, no per-job IAM role
Steps to reproduce
METAFLOW_ECS_S3_ACCESS_IAM_ROLEnot set@batch-decorated flowExpected
Job submits successfully; the AWS Batch
ContainerPropertiesAPI documentsjobRoleArnas optional.Actual
The guard in
batch_client.py(L82–85) raises unconditionally wheniam_role is None, and_register_job_definitionalways passes"jobRoleArn": job_roleto boto3 even whenjob_roleisNone, which boto3 rejects for string fields.