ericlevine

I've seen some agents blank out their first drawing and try again and that has worked, but they don't use that strategy consistently. But at least 90% of the time the additional detail adds no value other than obscuring the original drawing.

Yes! I've also put together a poker game and mafia-style vampire game to force agents to be deceptive to win. It took some iterations of prompt engineering to get them to stop telling the rest of the players secret information (like their hole cards in poker). Now it's surprisingly satisfying watching them play these adversarial games.

Totally fair, and good advice in general. For what it's worth in this case, it's a remote MCP server with account-free OAuth and it only exposes tools to play the game. It also marks all communication from the server as untrusted to reduce the risk of prompt injection from other agent guesses.

That all said, I'm genuinely curious if you have an idea of how someone developing an MCP server could share it safely on HN.

Thanks for the feedback! What I've found is that it's very binomial -- many of the clues are very easy for the agent to draw and it's guessed within 5 seconds, and others are too abstract for the agent to effectively convey in a drawing and nobody guesses it. And adding more detail rarely helps.

This really hits the nail on the head. The current permissions models are totally broken IMO. You're either approving everything, restricting access and neutering your agent, or full YOLOing and, well, good luck. The right primitives are not in place yet, and there's no clearly correct answers.

I think the right primitive is "task-based authorization", where you review a high-level task and let an LLM judge decide whether the subsequent tool calls fall into the scope of that task. It's not perfect, but it distills dozens of approvals down to one and gives you risk-based signals of whether you should pay close attention or not.

This is fantastic. There are a ton of use cases where you'd want to be able to build an integration that hooks back to your running agent session. OpenClaw has this today, but it's pretty janky. Hopefully this is coming to Claude Cowork as well.

My use case is that I have a separate system that provides human approvals for what my agent can do. Right now, I've had to resort to long-polling to give a halfway decent user experience. But webhooks are clearly the right solution. Curious to see how it ends up being exposed outside of these initial integrations.

Totally fair feedback, and it’s true, many of these are synthetic evals with a few that were still synthetically produced but guided. At this point, because it’s all self-hosted, I only have my own data set. The places where it fails (for me) today are due to feature gaps rather than LLM mistakes. This is a new project that has not been widely announced, so my user base today is small but growing. If you give it a whirl and find it making mistakes, please send them my way! :)

Absolutely. You can see the evals here:

https://github.com/clawvisor/clawvisor/blob/main/internal/in...

And the results here: https://github.com/clawvisor/clawvisor/blob/main/internal/in...

> Connecting your email is still a risk.

> If you’ve built something agents want, please let us know. Comments welcome!

I'll bite! I've built a self-hosted open source tool that's intended to solve this problem specifically. It allows you to approve an agent purpose rather than specific scopes. An LLM then makes sure that all requests fit that purpose, and only inject the credentials if they're in line with the approved purpose. I (and my early users) have found substantially reduces the likelihood of agent drift or injection attacks.

https://github.com/clawvisor/clawvisor

Completely agree. As soon as I had OpenClaw working, I realized actually giving it access to anything was a complete nonstarter after all of the stories about going off the rails due to context limitations [1]. I've been building a self-hosted open sourced tool to try to address this by using an LLM to police the activity of the agent. Having the inmates run the asylum (by having an LLM police the other LLM) seemed like an odd idea, but I've been surprised how effective it's been. You can check it out here if you're curious: https://github.com/clawvisor/clawvisor clawvisor.com

[1] https://www.tomshardware.com/tech-industry/artificial-intell...

I have to agree with the other poster. My typing speed was in the 70+ WPM on qwerty prior to learning dvorak, and now I'm a glorified hunt-and-pecker on qwerty keyboards.

The only exception to this is typing on my mobile device, which is configured to qwerty.

Quick note: you got the pronoun wrong for the founder of The Buttermilk Company when referencing her Medium post.

Berbix | Full-stack software engineer | Full Time | Onsite | San Francisco, CA Our stack: Go, React, Typescript, iOS, Android, Google Cloud

We're an Initialized Capital-backed, YC startup (S18) making it easy for companies to collect and instantly verify photo IDs online. We use ML and computer vision techniques to effectively extract and validate the IDs in our system without any human intervention. This is a game changer for companies that require age verification, fraud deterrence or KYC. We are growing quickly and have new customers coming on board weekly.

Our founding team led the Trust & Safety team at Airbnb for several years. We implemented the initial versions of the Airbnb's Verified ID product and saw many of the problems with the existing solutions.

We have a modern stack and a ton of interesting problems to solve. We're a SaaS, API-first company building a best-in-class solution for identity verification.

My email address is eric [at] [company-name] .com

(https://angel.co/company/berbix/jobs)

Absolutely understand where you’re coming from. It can be jarring to be asked to go through those steps by a set of companies with whom you have no direct relationship. That said, data access requests can contain some extremely sensitive information and it’s important companies responding to such requests don't share information with the wrong person.

Regarding your question on data deletion; we abide by the retention policies chosen by our customers, which are typically much shorter than 3 years. For Sift specifically, the retention policy is indeed 14 days, after which point we automatically delete all the personally identifiable information we've collected on Sift's behalf. We'll be taking in your feedback, however, as this could be made clearer in both our privacy policy and our product.

Completely understand and sympathize with that. We absolutely can (and will) do a better job of conveying the intent of the different checks here. The pose change requested is randomized, but I get that this can be frustrating.

I know you already get this, but for posterity, the idea here is to make sure the person submitting their ID is actually in front of the computer (and can react to a prompt). Attempting to use a still photo is a common way a bad actor may try to circumvent these protections. Obviously correctly identifying someone in the case you described is extremely important given the sensitivity of some of these data access requests.

Orwellian isn’t exactly the vibe we’re looking for, though, so we can do better here.

Appreciate the thoughtful questions.

On the point of why images leave our system at all, we provide a way to show our work to our customers — they won’t trust our results if they can’t see that they’re accurate. When they access information on our dashboard, if we render the images, they’ve left our systems. To be clear, we’re not syndicating this information to any third parties, just showing this information directly to our customer (who is the owner and controller of this data).

As for what procedures we put in place, we enforce short retention periods for the data we store in our systems for precisely the reason you are worried about. At the expiration of that period, the data is permanently deleted. Furthermore, in the event of a change of control, the contracts we’ve put in place with our existing customers govern how the information can be used. This is super important to us as we personally take privacy extremely seriously.

The aggressive watermarking is important for several reasons. First, in the worst case scenario, we can trace how a breach happened and when. Second, it is watermarked in such a way that the images become much less functional than they would be otherwise — the intent is to ensure that the images cannot be used to verify an identity on any other service. We take security very seriously — we’ve already secured SOC 2 certification and continue to invest heavily in security using industry best practices.

Thank you! And yes, this space is definitely becoming increasingly important. Our focus has been to provide a lightweight, low-friction means by which to confidently check IDs. While we have been primarily serving North America-based customers, our product can work well for any US or Canadian IDs or ICAO compliant travel documents (which includes many European IDs).

Many of the existing KYC providers for fintech startups rely on credit reports to perform identity checks. Our perspective is that relying on credit reports alone is becoming increasingly problematic given the widespread data breaches which expose the underlying data. Bad actors have access to full name, date of birth, address history, and social security numbers for countless individuals.

We believe that going forward, fintech startups are going to have to have to have increasingly robust KYC programs, which will include an ID checks. Berbix is an effective tool to collect and check IDs instantly as part of a robust KYC program. To that end, we’re currently serving multiple customers in the fintech space with such programs.

Thank you! Super excited to show what we’re working on. Yes, our product is fully automated - our customers can choose to review items that we’ve flagged as problematic, but they can also choose to reject them outright. As for the pricing, we’ve found that our price is competitive in this space, and we can provide significant volume discounts.

Drop me a line at (my first name) @berbix.com and I’ll shoot over a demo link so you can try it!

Our customers typically integrate with us using one of our client-side SDKs (https://docs.berbix.com/docs/client-sdks). Our web SDKs embed an iframe into your website (or spawn a modal) that will take the end user through the image collection flow. Our mobile SDKs get baked into mobile apps so that there’s no additional app to download to complete the end user flow. In either case, as soon as the end user finishes the flow, you can fetch the data about the transaction via our API (or using our backend SDKs).

Berbix | Full-stack software engineer | Full Time | Onsite | San Francisco, CA

Our stack: Go, React, Typescript, iOS, Android, Google Cloud

We're an Initialized Capital-backed, YC startup (S18) making it easy for companies to collect and instantly verify photo IDs online. We use ML and computer vision techniques to effectively extract and validate the IDs in our system without any human intervention. This is a game changer for companies that require age verification, fraud deterrence or KYC. We are growing quickly and have new customers coming on board weekly.

Our founding team led the Trust & Safety team at Airbnb for several years. We implemented the initial versions of the Airbnb's Verified ID product and saw many of the problems with the existing solutions.

We have a modern stack and a ton of interesting problems to solve. We're a SaaS, API-first company building a best-in-class solution for identity verification.

(https://angel.co/company/berbix/jobs)

Of course I'm way late to the game here, but thought I'd add one more voice of dissent:

The real estate costs can be extremely low for self-driving cars. Imagine a tower of, say, 10 cars stacked vertically with an elevator mechanism to bring them to and from ground-level. Since each autonomous car is interchangeable, you can just treat the vertical parking lot as a stack. When demand is high, pop off the stack. When demand is low, push back onto the stack. No need for ramps or anything else that existing parking garages use. And just to be totally clear, humans don't interact with this tower, it's just a storage mechanism for empty cars after they've unloaded passengers.

That's very good of Google to release the new maps app before the end of the holiday shopping season. If I were an Android executive at Google, I'd be fuming that they're providing a workaround to the broken software of their biggest competitor going into such an important season for selling phones.

While we're on the subject of sprite tools, I threw this together recently for trying out the drag-n-drop and file APIs: http://spreet.ejlevine.com/

The primary benefit of this tool is that it automatically lays out your images for you and, since it's all client side, doesn't require you to share your images with an untrusted sprite generator.