TL;DR there is a datastructure called a roaring bitmap that some databases have support for through extensions, and it would be possible to pass that in as a filter on your queries, so then the flow would be:
1) get roaring bitmap of all things that the user has access to
2) query the database/search elasticsearch with my other parameters/filters
3) return the results to the user