Bug Report: Prompt caching not applied for single system message and streaming ignores cache tokens
Bug Report: Prompt caching not applied for single system message and streaming ignores cache tokens: a task in LegoFlow-SWE (Harbor dataset). Create a client and a chat request with one system message and a cache control (the message body must exceed the minimum cacheable size for the used model):
The task
Create a client and a chat request with one system message and a cache control (the message body must exceed the minimum cacheable size for the used model):
Part of Lego-X/LegoFlow-SWE.