We ended up using JSONL for our implementation but it is also possible to switch using other "streamable" variants like SSE and even CSV if that is what you are into.
But, as the OP suggested, none of these are resumable. So we have created a pub/sub over HTTP that does exactly this.
The way it works is that you get a unique URL that is specific to the current authenticated session. You can request data as many time as you like from that URL and it will get streamed to your client from the last checkpoint.
It works well even with rudimentary tools like curl. There is practically no protocol of sorts required - the kind of implementation I really like because who wants more complications in their life, right?
The big issue build conversational and even agentic AI with off-the-shelf frameworks rather than using a platform such as ours is that there is a lot of plumbing involved to get these functionalities going which is makes even basic setup a lot more complicated depending on the kind of platform you are deploying to.
But, as the OP suggested, none of these are resumable. So we have created a pub/sub over HTTP that does exactly this.
The way it works is that you get a unique URL that is specific to the current authenticated session. You can request data as many time as you like from that URL and it will get streamed to your client from the last checkpoint.
It works well even with rudimentary tools like curl. There is practically no protocol of sorts required - the kind of implementation I really like because who wants more complications in their life, right?
The big issue build conversational and even agentic AI with off-the-shelf frameworks rather than using a platform such as ours is that there is a lot of plumbing involved to get these functionalities going which is makes even basic setup a lot more complicated depending on the kind of platform you are deploying to.