Replies: 2 comments 2 replies
-
This was done intentionally because the service returns large chunks of text at a time, which renders in an unpleasant way. I can look into how this behaves more recently but I was already planning on making stream output rates configurable and may look at that then. |
Beta Was this translation helpful? Give feedback.
-
Yes. Gemini can sometimes return large chunks of text. However, I don't think the response stream should be buffered until the full response is received. This delay makes chatting with Gemini feel very slow. In my opinion, LibreChat could improve responsiveness by initially returning the original stream. Then, it could explore other ways to address the issue of large text chunks. |
Beta Was this translation helpful? Give feedback.
-
What happened?
When streaming of Gemini is enabled, the response seems to be returned in a constant speed. I have verified this through a self-written gemini proxy server. LibreChat always waits for the complete response from gemini then send to the user.
How to make it return the response in real-time? I can help submit a PR if I know how.
Steps to Reproduce
Just use Gemini
What browsers are you seeing the problem on?
Chrome
Relevant log output
No response
Screenshots
No response
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions