Skip to content

Pass in tokenizer to render in /message - #21

Open
steventsao wants to merge 1 commit into
anysphere:mainfrom
steventsao:update-model-to-tokenizer
Open

Pass in tokenizer to render in /message#21
steventsao wants to merge 1 commit into
anysphere:mainfrom
steventsao:update-model-to-tokenizer

Conversation

@steventsao

Copy link
Copy Markdown

Fixes curl failure in #19

➜  priompt git:(update-model-to-tokenizer) curl 'localhost:8008/message?message=what%20is%20the%20advantage%20of%20rust%20over%20c&name=a%20curious%20explorer'

{"prompt":{"type":"chat","messages":[{"role":"system","content":"The user's name is a curious explorer. Please always greet them in an extremely formal, medieval style, with lots of fanfare. Then seamlessly proceed to reply to their message in the most casual, 2010s, cool dude texting style. Please be over-the-top in both respects, and make the transition seem like it never happened."},{"role":"user","content":"what is the advantage of rust over c"}]},"tokenCount":88,"tokensReserved":1000,"tokenLimit":null,"tokenizer":{"name":"cl100k_base","shouldAddEosTokenToEachMessage":true},"durationMs":2.3981669999775477,"outputHandlers":[],"streamHandlers":[],"streamResponseObjectHandlers":[],"priorityCutoff":1000000000,"config":{}}%         

- Import `CL100K` tokenizer from `@anysphere/priompt/dist/tokenizer`.
- Update `/message` endpoint to accept `numTokens` query parameter.
- Parse `numTokens` and use it in the rendering process.
- Replace OpenAI chat completion logic with rendering using the tokenizer and token limit.
@hoomar91

Copy link
Copy Markdown

گربه روی شیروانی

2 similar comments
@hoomar91

Copy link
Copy Markdown

گربه روی شیروانی

@hoomar91

Copy link
Copy Markdown

گربه روی شیروانی

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants