As for intelligently - but probabilistically - determining confidentiality (if I read that correctly), that does sound pretty interesting in scenarios where metadata is just simply insufficient. Also tricky. Sounds like you thought about these problems pretty deeply.
Congratulations on the launch Rauf & Lennert! Always great to have more innovation in the open source AI space :D. It looks like Airweave works well with Cursor, something we don't have nailed down yet!
On permissioning: we default to per-user syncs that adopt the permissions of the syncing user and mirror source ACLs (e.g., Drive items a user owns or that are sharedWithMe). In practice, founders avoid leaking private docs by either (a) having each user sync their own corpus, or (b) using a centrally-scoped token limited to Shared Drives/team folders and excluding personal “My Drive.” You can also keep separate collections and only expose cross-user search behind your own checks. We’re exploring richer org-level RBAC mapping on a per-customer basis (e.g., mapping Drive/SharePoint groups to index ACLs), but the above works today.
@Weves: Thanks, appreciate it!
I see in another comment that you encourage each user to build their own dataset with their own permissions, but often this breaks for founders. If I have a Super Secret Personnel Planning Google Doc at a founder level, how can I be the one to set up the system for our company, but ensure that only files that I've explicitly shared with the company are ingested? What if a file needs to be made anyone-with-link-can-access for sharing with a strategic partner, but that shouldn't be indexed for the entire company?
Far too much of the world relies on the security-by-obscurity of public-but-unindexed links, and communications that might look public from a metadata perspective but were carefully designed for a very specific group of people who have verbal/mental context about confidentiality expectations. Being able to categorize by likely confidentiality, and allowing an administrator to partition access on a project and sub-project basis based on that, might be crucial for growth.
My recollection is that Onyx had limited support for some security use cases, but very rudimentary. Hoping you can solve this in a thoughtful way!
Onyx links for comparison:
https://www.onyx.app/
https://docs.onyx.app/developers/guides/chat_guide
https://docs.onyx.app/admin/connectors/official/