How does watch API scale w/ PG? If there
# spicedb
a
How does watch API scale w/ PG? If there are 10000 RPCs to the watch endpoint simultaneously streaming events, are there 10000 queries running periodically?
e
Right now each watch opens its own connection on the backend, so you wouldn't want to open that many. The recommendation is to open a watch for all possible consumers and then distribute to individual clients on the application side. But we've definitely floated ideas to make this simpler / easier, especially with Spanner where there are a limited number of watch slots. Might be worth writing up a GH issue to discuss some future options to simplify this.
y
there's also been discussion of providing easier mechanisms to move events from the watch API to something like kafka, such as through a connector
a
Thank you both! Is there a GH issue around a "connector"? We may be interested in contributing to something like that
y
not yet - my understanding is that there's been a couple of starts on this idea (one using Bento and one that i started at my previous company but haven't pushed up yet) but nothing written down yet
i can create one, though
j
@Alec we'd also welcome collab on it too
y
@Alec I added a tracking issue here: https://github.com/authzed/spicedb/issues/2015 and we'd welcome your input on the topic
9 Views