Dist ref kl#529

Open

Dahoas wants to merge 27 commits into

mainfrom

Commits on Jul 18, 2023

Implementing support for dense rewards
Dahoas
committed
added "num_return_sequences" param which corresponds to n in Best-of-N sampling

SharathRaparthy
authored and
Dahoas
committed
updates to "num_return_sequences" param

SharathRaparthy
authored and
Dahoas
committed
BoN implementation

SharathRaparthy
authored and
Dahoas
committed
Changed back to default.

SharathRaparthy
authored and
Dahoas
committed
TopK sampling instead of Top1

SharathRaparthy
authored and
Dahoas
committed
summed along dim=1

SharathRaparthy
authored and
Dahoas
committed
Generating samples in chunks

SharathRaparthy
authored and
Dahoas
committed
added gen_chunk_size parameter

SharathRaparthy
authored and
Dahoas
committed
chunking in forward prop

SharathRaparthy
authored and
Dahoas
committed
chunking generations in train and eval

SharathRaparthy
authored and
Dahoas
committed
Implementing support for dense rewards
Dahoas
committed
Fix distributed ref_mean, ref_var bug for dense rewards
Dahoas
committed
Make generation respect max seq length
Dahoas
committed
Make experience before first round of training
Dahoas
committed
Refactoring .generate/.generate_eval
Dahoas
committed
Fix BoN metric support
Dahoas
committed
Enforce chunk_size param for eval generation when present
Dahoas
committed
Fix: Don't shuffle prompt dataset
Dahoas
committed
Move inputs to device
Dahoas
committed
Fix style
Dahoas
committed
Fix: Do not shuffle empty experience dataloader
Dahoas
committed
Make experience before first round of training
Dahoas
committed
Refactoring .generate/.generate_eval
Dahoas
committed
Refactored decode, make_experience and added support for external reference models
Dahoas
committed
Fix BoN sampling after big refactor
Dahoas
committed
Fixing style
Dahoas
committed