Is your feature request related to a problem? Please describe.
For some repetitive, few-shot tasks, the model needs to consider only the most recent, or a few recent messages, not a long history. Think tasks like proofreading .
Describe the solution you'd like
In addition to topP, temperature, and other advanced settings, it would be beneficial to have custom context length, measured in # of messages. I.e. the model has only 4 most recent chat messages and (both requests and responses) as a context.
Describe alternatives you've considered
Using a new chat every time is the obvious alternative, but it produces clutter in chat history.
Is your feature request related to a problem? Please describe.
For some repetitive, few-shot tasks, the model needs to consider only the most recent, or a few recent messages, not a long history. Think tasks like proofreading .
Describe the solution you'd like
In addition to topP, temperature, and other advanced settings, it would be beneficial to have custom context length, measured in # of messages. I.e. the model has only 4 most recent chat messages and (both requests and responses) as a context.
Describe alternatives you've considered
Using a new chat every time is the obvious alternative, but it produces clutter in chat history.