Skip to content

Latest commit

 

History

History
48 lines (37 loc) · 1.88 KB

README.md

File metadata and controls

48 lines (37 loc) · 1.88 KB

Configs

Configs are organized as configs/<model>/<benchmark>/<prototype>.yaml:

configs/
  4omini/          # gpt-4o-mini
  qwen2.5-7b/      # Qwen2.5-7B-Instruct
  qwen27b/         # Qwen3.5-27B-AWQ-4bit
    alfworld/    sas.yaml  sas_contextflow.yaml  mas.yaml  sas_plan.yaml  sas_plan_fewshot.yaml  mas_relay_insufficient.yaml  env_config.yaml  env_config_hetero.yaml
    webshop/     sas.yaml  sas_contextflow.yaml  mas.yaml  sas_plan.yaml  mas_relay_insufficient.yaml
    workbench/   sas.yaml  sas_contextflow.yaml  mas.yaml
    widesearch/  sas.yaml  sas_contextflow.yaml  mas.yaml
    travelplanner/  sas.yaml  sas_contextflow.yaml  mas.yaml  sas_plan.yaml

sas / sas_contextflow / mas / sas_plan correspond to the four prototypes in the paper (Section 5.1); mas_relay_insufficient and sas_plan_fewshot are the two ablation studies (Section 6).

Before Running

Each config needs:

  1. API key — set llm.api_key, or export OPENAI_API_KEY / ANTHROPIC_API_KEY (picked up automatically if api_key is left unset).
  2. Data paths — configs reference data under ${MAS_SAS_DATA_DIR}; export that env var to point at your local copy of the ALFWorld/WebShop/WorkBench/TravelPlanner data (see repo root README for dataset setup).
  3. (Optional) --max-tasks, --experiment-id, --output-dir can be overridden on the command line without editing the YAML.

Using a local vLLM server (e.g. Qwen2.5-7B-Instruct, Qwen3.5-27B-AWQ-4bit)

Point llm.base_url at your running vLLM server (must end with /v1):

llm:
  model: Qwen/Qwen2.5-7B-Instruct   # or cyankiwi/Qwen3.5-27B-AWQ-4bit
  api_key: dummy
  base_url: http://HOST:PORT/v1
  default_temperature: 0.0
  max_retries: 1
  extra_body:
    chat_template_kwargs:
      enable_thinking: false

Example

python main.py run --config configs/qwen2.5-7b/alfworld/sas.yaml --experiment-id my_alfworld_run