Issues

• MegatronLMClient expects cache usage

  OpenGPTX/lm-evaluation-harness#106 · Filter by author janEbert opened on Jan 22, 2024

• Add MT-Bench-X to lm-evaluation-harness

  OpenGPTX/lm-evaluation-harness#104 · Filter by author lllAlexanderlll opened on Dec 12, 2023
  

• Inspect LAMBADA OpenAI fpr HF converted checkpoints

  OpenGPTX/lm-evaluation-harness#103 · Filter by author lllAlexanderlll opened on Dec 12, 2023

• Add Batching to Megatron-LM server

  OpenGPTX/lm-evaluation-harness#90 · Filter by author KlaudiaTH opened on Aug 4, 2023
  

• trust_remote_code is not passed to AutoTokenizer.from_pretrained

  OpenGPTX/lm-evaluation-harness#87 · Filter by author KlaudiaTH opened on Jun 20, 2023

• Upstreaming contributions?

  OpenGPTX/lm-evaluation-harness#81 · Filter by author StellaAthena opened on Apr 30, 2023
  1

• pawsx_de, pawsx_en, x_stance, gnad10 error message for metrics

  OpenGPTX/lm-evaluation-harness#76 · Filter by author katrinklug opened on Mar 27, 2023
  1

• xquad, mlqa and mlsum tasks are not correctly implemented

  OpenGPTX/lm-evaluation-harness#75 · Filter by author KlaudiaTH opened on Mar 16, 2023
  3

• Add evaluations for anli_de

  OpenGPTX/lm-evaluation-harness#71 · Filter by author katrinklug opened on Jan 26, 2023

• Add evaluations for miam

  OpenGPTX/lm-evaluation-harness#67 · Filter by author katrinklug opened on Jan 16, 2023

• Add evaluations for germandpr

  OpenGPTX/lm-evaluation-harness#65 · Filter by author katrinklug opened on Jan 16, 2023

• Add evaluation for fairlex

  OpenGPTX/lm-evaluation-harness#64 · Filter by author katrinklug opened on Jan 16, 2023

• Add evaluations for lextreme

  OpenGPTX/lm-evaluation-harness#63 · Filter by author katrinklug opened on Jan 16, 2023

• Add evaluation tasks with omp dataset

  OpenGPTX/lm-evaluation-harness#62 · Filter by author katrinklug opened on Jan 16, 2023

• Add evaluation for QG from xglue benchmark

  OpenGPTX/lm-evaluation-harness#60 · Filter by author katrinklug opened on Jan 16, 2023

• Add evaluation for germEval14

  OpenGPTX/lm-evaluation-harness#46 · Filter by author katrinklug opened on Nov 10, 2022

• Evaluation for German LER

  task
  OpenGPTX/lm-evaluation-harness#44 · Filter by author malteos opened on Nov 4, 2022
  

• XStannce Topic Classification

  task
  OpenGPTX/lm-evaluation-harness#43 · Filter by author malteos opened on Nov 4, 2022
  1

  

• Multi GPU Evaluation

  task
  OpenGPTX/lm-evaluation-harness#42 · Filter by author Lena-Jurkschat opened on Nov 3, 2022

• Results for xnli_de,pawsx_de,xquad_de

  OpenGPTX/lm-evaluation-harness#31 · Filter by author malteos opened on Aug 17, 2022

• Evaluation for XGLUE Task

  task
  OpenGPTX/lm-evaluation-harness#27 · Filter by author malteos opened on Aug 16, 2022

• Evaluation for STSb Multi MT task

  task
  OpenGPTX/lm-evaluation-harness#26 · Filter by author malteos opened on Aug 16, 2022

• GitHub actions: Fix out of disk space error

  OpenGPTX/lm-evaluation-harness#25 · Filter by author malteos opened on Aug 16, 2022

• Evaluation for GermEval 2015: LexSub

  task
  OpenGPTX/lm-evaluation-harness#21 · Filter by author malteos opened on Aug 16, 2022

• Evaluation for CoNLL-2003 task (NER)

  task
  OpenGPTX/lm-evaluation-harness#20 · Filter by author malteos opened on Aug 16, 2022