请问HN：如何在页面刷新时保持LLM令牌流？
和你们中的许多人一样，我正在通过服务器发送事件（SSE）逐个令牌地流式传输来自大型语言模型（LLMs）的响应。
保持SSE连接在页面刷新时不变的最佳方法是什么？
我没有看到很多关于这个的文档或示例。在我使用的大多数支持LLM的应用中，如果令牌正在流式传输而页面刷新或更改，流就会中断。
我想到的一个主意是将流式传输的令牌写入某种队列或Kafka主题，然后将我的用户界面连接到该队列，从那里流式传输令牌。但这似乎工作量很大。
大多数人是如何做到这一点的？
查看原文
Like many of you, I'm streaming responses token by token from LLMs using server sent events (SSEs).<p>What's the best way to maintain the SSE connection through a page refresh?<p>I haven't seen a lot of documentation or examples covering this. In most LLM enabled apps I've used, if tokens are currently streaming and the page refreshes/changes, the stream gets interrupted.<p>One idea I had was writing the streamed tokens into some sort of queue or kafka topic, then connecting my UI to the queue and streaming tokens from there instead. But that seems like a lot of work.<p>How are most folks doing this?