Repository navigation
I'm unable to replicate the experimental results presented in the paper. #36
Description
I used the Dolly-15k dataset, Llama-2-7B-chat model and an A800 device to make some experiments between your lookahead and lookahead-decoding, but I'm unable to replicate the results on the table2 of the paper. Even your lookahead was not as good as the lookahead-decoding.
Activity
gigigigig1 commentedon Mar 28, 2025
请问论文实验结果中的基线方法的测量结果（即不采用任何加速方法）是如何得来的，我将你们测试代码中lookahead为false的情况与直接调用transformers库中的generate方法测试的速度不一样，因此我不知道如何对比你们的lookahead方法和lookahead_decoding方法的效果
zheyishine commentedon Apr 1, 2025
Our lookahead must be fully warmed up with thousands of requests to fill the trie tree. Using half of the samples to warmup would reproduction the result.