But seems like this may be less about the absolute price but more about the way the 100/month of credit feels?
What do you think could be better? The intention of the 250/month scale tier was intended for companies scaling up that want BYOC for data residency etc. etc. and give them enough to test things internally without worrying about an overage bill before running it directly on prod but this might be able to be implemented better.
It clones the entire DB so it's not quite 6s per TB but actually the entire thing takes <6s independent of size
Probably something to adjust on the site to make it clearer!
It's called branch hooks and lets you register SQL to be run against the branch after it's created but before it's handed to you (or an agent)
So you can retain production shapes but manipulate the data however you want to make it safe
https://docs.tryardent.com/architecture
But essentially we get around the restrictions of the original DB by replicating into a different postgres compatible DB that essentially serves as a read replica. That DB is the one that branches but since it mirrors the original DB you get effective clones
By doing this we get a lot more control over what we can do to create the clones. The read replica clones using copy on write + isolated autoscaling compute to clone in 6s. We use neon to do this since we think they've implemented those two properties well.
Since it's default postgres logical replication + DDL triggers you can technically point it at any "branching enabled" db on the other end in order to achieve the same effect
Yep! Agreed. We've tried to combat this with the "branch_hooks" being team/org level policy objects so we can do enforcement of any kind on the branches before they're ever actually handed to users. This would be things like access control + defined anonymization rules. The broader hope with this class of objects/policies is they can serve as enforcement barriers and essentially allow scoped access at the org level across branches.
The proxy we run in the middle also helps a lot here. Since the URL is minted by our control plane and is not the "real" DB url we can authenticate each user from the URL they're using and enforce RBAC controls.
for example:
User 1's API key is 1234
The CLI can auto-construct urls like: postgresql://{APIKEY}:{ANYTHING}@{IDENTIFIER}--postgres.routing.tryardent.com:5432/DB_NAME?{params}
Your API key is something that can be scoped per user
This is an off the cuff example but essentially we have a way of knowing who is calling the host and thus can enforce if APIKEY = You can't access this DB based on whatever rules.
Curious to understand what additional pieces would be helpful here because this is 100% very important to get right.
On the real world data impact I absolutely agree. We added something called "branch hooks" which essentially let you define SQL to run against the branch before it's returned
This lets you essentially anonymize and modify the branch to scrub unintended external side effects.
It's something that we're still working on though and trying to design the right abstractions around because we want to get that part right.
We'd let you instantly clone prod + user defined auto-anonymization so you can test writes. The architecture also somewhat takes the place of an existing read replica if you want to use it like that to make it more cost efficient.
Also since we're using copy on write for the clones they're incredibly storage efficient and the autoscaling compute helps minimize cost on clones by minimizing excess compute uptime
Neon does support copy on write branching natively and autoscaling compute but you make certain performance tradeoffs. A lot of the folks we've talked to that use RDS or Planetscale are reliant on things like query latencies supported by that platform's specific architecture but also want the ability to test on branches. We let you get the best of both worlds (branch but leave your DB where it is and freely choose your production environment based on prod concerns)
Supabase does have branching but they do not branch the data so you can't test any interactions that rely on the data. You can restore from backup as an option but this slows down based on data size since you're actually moving data as opposed to copy on write.
Longer term we want to be the place you branch all your data infra. So expanding to S3, Snowflake, MySQL etc.
For now though we're focusing on just postgres and getting it right!
Hey HN! We’re Vikram and Evan from Ardent (https://tryardent.com). We're building database sandboxes for you and your coding agents.In the last two years coding agents have gotten dramatically more capable at handling complex engineering tasks. But without access to a realistic sandbox at the DB layer for testing, they ship garbage that can take down production databases. I spent over a year...
Llms are just too creative. They will explore the search space of probable paths to get to their answer. There's no way you can patch all paths
We had to build isolation at the infra level (literally clone the DB) to make it safe enough otherwise there was no way we wouldn't randomly see the DB get deleted at some point
Look into copy on write branching. We built this natively into our AI Data Engineer (https://tryardent.com) so it could make modifications to databases with 0 blast radius pretty much because yes it's impossible to make an LLM 100% safe if it has no proper guard rails preventing destructive actions
email me -> [email protected]
We're building support for snowflake too if that's something you use
You can't trust any agent to be perfect with a real db so unless you find an infra level way to isolate it, you can't get rid of the problem
So we built a system that creates copy on write copies of your DB and allocates a copy for each agent run. This means a completely isolated copy of your DB with all your data that loads in under a second but zero blast radius risk to your actual system for the agent to operate on. When you're okay with the changes we have a "quick apply" to replay those changes onto your real db
Website is a little behind since we just launched our db sandboxing feature to existing customers and are making it public next week :)
If you want to try it email me -> [email protected]
Hey HN!We’re Vikram and John, founders of ArdentAI (https://www.ardentai.io/).We built ArdentAI to tackle the pain points of data engineering—time-consuming pipelines, manual transformations, and error-prone debugging. With ArdentAI, you can automate these tasks up to 100x faster.What is it?ArdentAI is an AI Agent that connects directly to your databases and handles all the heavy lifting. Think...
You can use Ardent to sync and transform data, autofix airflow pipelines, modify schemas and more directly on your own existing stack
It's designed to run completely independently. You just tell it to do a task and let it do the work for you
https://ardentai.io <- to check it out
While not dealing with that kind of scale yet, our application (An AI Data Engineer that has done migration work for users) needs to do before and after comparisons and find diffs. We use a branchable DB to compute those changes efficiently (DoltGres)
Could be an interesting thing to consider since it's worked well for us for that part.
Our build if u wanna check that out too -> https://Ardentai.io
It drops directly into your stack, no new configuration needed
It has its own compute engine and will soon support spark to be able to dynamically perform large scale ETLs and data manipulation.
I also am working towards supporting automatic data pipeline building and data quality checks.
It's live right now @ https://Ardentai.io
Check it out :)