javierluraschi

Just added a website for the same benchmark, easier read: https://aideploybench.com

Hi HN. I built a benchmark for myself that measures not just coding an app, but rather the end-to-end loop: Build, deploy, troubleshoot. Then an independent verifier (currently opus 5.5) checks the live URL and performs basic tests.

The benchmark runs five web apps: (1) static website for say a company page or the like (2) the usual CRUD with auth for say SaaS product (3) an API to be consumed by other services (4) a dashboard with data to mimic internal company dashboards and the like (5) a realtime chat application with support for web sockets.

It currently compares four platforms but can be easily extended to more, feel free to open GitHub issues or send me a PR.

I was specially interested in comparing human intervention (Human Time). As in, how much time is spent for a human to deploy to particular platforms the first time?

The benchmark tracks build/deploy time, tokens, agent cost, security, compliance review of the deployment (secrets handling, least privilege, terms fit, attack surface, etc.). Methodology and reports (including screenshots) are in the repo.

Repo: https://github.com/cloudishai/ai-deploy-bench

Full disclosure: I'm working on Cloudish. Happy to answer questions about the methodology or results.

Edit: Removed table due to formatting issues.

> Amongst the two beliefs, where do I fall? Machine consciousness implies “we created consciousness”, which is equal to saying “we are gods”, which sounds like megalomaniacal hubris to me - therefore no.

What if everything has consciousness already? As in panpsychism. In such case, we are not gods - therefore yes.

Carbon tax is the only way, you need to pay for picking up the garbage from the road or the air, the cost to clean op pollution and trash needs to be accounted for.

Model cards won't save us: https://x.com/javierluraschi/status/2077147606791430372?s=20

For context, last year I generated Pacman 2D with Grok and ChatGPT, mediocre result see:

https://www.linkedin.com/feed/update/urn:li:activity:7298354...

Puts in perspective how far we've come in just one year.

Huge opportunity for engineering teams to optimize costs and migrate projects out of Lovable Cloud: https://news.ycombinator.com/item?id=48460946

Worth sharing Starcloud’s paper in this post:

https://starcloudinc.github.io/wp.pdf

It will benefit all people but it will disproportionally benefit more rich people.

No you can’t, the system is setup to make you work that hard. Their compensation policy is based on rewarding disproportionally the top performers…

So if you can work 10% more than your peers, you get not 10% bonus but rather 30%-100% more. So it makes business sense to put the extra 10%, until everyone is working at 110% and then again, adding an extra 10% pays off, rinse release, death spiral.

The compensation model is pure evil.

I remember being so stressed on an immigration visa, if you don’t perform well and get fired, you also get deported.

That would be nice, but if you want to immigrate, that’s the current deal.

Any chance we can get a high-level breakdown of the AWS services they are paying for?

Same and thanks!

Scheduled time in your website to troubleshoot and try it out

Like the concept! After signing up and defining my criteria clicking “Start Search” returns no results

What is the max number of dimensions supported for a vector?

My bad, thanks!

https://slack-status.com/ -- no incidents reported at 3:33PM PST

Yes, getting a bunch of retry and messages failing to send.