mlenhard

I don't have exact numbers here, and it varies by company, but a rough guess is around 10-20%. This might be a bit biased because the companies willing to talk to me are probably seeing the worst of it.

Also, outside the labs, most of the companies I've talked with have shut off free tiers and free credits entirely because the abuse is so bad.

This was poorly worded on my part; I meant in terms of dollars, not tokens.

From talking with some of the companies experiencing this, I can confirm that a portion of it is actual credit card fraud. Tokens have become a pseudo-currency, making them a prime target for abuse.

Some of the abuse is more benign, but there is also real fraud through chargebacks, account takeovers, and stolen credit cards.

This was specifically meant to be about the ai credit resellers, not the relays themselves. I put together another piece here: https://vectoral.com/blog/token-relay-market that tackles that side of the market.

I don't think device fingerprinting is the right approach here.

Client-side detection can always be sidestepped, and you need to intermediate the actual inference to get enough signals to make an accurate prediction. There are hundreds of listings for cursor tokens/credits right now.

We use canary values to detect the resellers, and I believe that's the only approach that will actually work at scale.

I was going to cover this in a follow-up article, but yeah, there are network of token brokers who buy unused credits from startups and then resell them.

Yeah, I should probably clean this up. The sentence is a bit hard to understand. What I was trying to show was the steep discounts offered by resellers.

I actually came across Plandex the other day. I haven't had the chance to play around with it yet, but it looked really cool.

But agree that even basic prompts can be a struggle. You often need to name the tool in the prompt to get things to work reliably, but that's an awful user experience. Tool call descriptions play a pretty vital role, but most MCP servers are severely lacking in this regard.

I hope this a result of everything being so new and the tooling and models will evolve to solve these issues over time.

Agree on the unpredictability of results issue. Tool call selection is still sort of a black box.

How do you know what variations of a prompt trigger a given tool to be called or how many tools is too many before you start seeing degradation issues because of the context window. If you are building a client and not a server the issue becomes even more pronounced.

I even extracted the Claude electron source to see if I could figure out how they were doing it, but it's abstracted behind a network request. I'm guessing the system prompt handles tool call selection.

PS: I released an open source evals package if you're curious. Still a WIP, but does the basics https://github.com/mclenhard/mcp-evals

This is pretty cool. You should also attempt to scan resources if possible. Similar to the tool injection attack Invariant Labs discovered, I achieved the same result via resource injection [1].

The three things I want solved to improve local MCP server security are file system access, version pinning, and restricted outbound network access.

I've been running my MCP servers in a Docker container and mounting only the necessary files for the server itself, but this isn't foolproof. I know some others have been experimenting with WASI and Firecracker VMs. I've also been experimenting with setting up a squid proxy in my docker container to restrict outbound access for the MCP servers. All of this being said, it would be nice if there was a standard that was set up to make these things easier.

[1] https://www.bernardiq.com/blog/resource-poisoning/

Yeah, you aren't far off with SQL injection comparison. That being said it's not really a fault of the MCP spec, more so with current client implementations of it.

One of the biggest issues I see, briefly discussed here, is how one MCP server tool's output can affect other tools later in the same message thread. To prevent this, there really needs to be sandboxing between tools. Invariant labs did this with tool descriptions [1], but I also achieved the same via MCP resource attachments[2]. It's a pretty major flaw exacerbated by the type of privilege and systems people are giving MCP servers access to.

This isn't necessarily the fault of the spec itself, but how most clients have implemented it allows for some pretty major prompt injections.

[1] https://invariantlabs.ai/blog/mcp-security-notification-tool... [2] https://www.bernardiq.com/blog/resource-poisoning/

Yeah, I plan on improving the formatting and adding a few more examples. There were even still some typos in the piece. To be honest, I didn't plan on sharing it yet; I just figured it might be helpful for the OP, so I shared it early.

I also think the docs are pretty good. There's just something about seeing the actual network requests that helps clarify things for me.

Yeah, at its core it's just a proxy, so there are a lot of other tools out there that would do the job. It does have a nice UI and I try to support projects like it when I can.

I'll check out your proxy as well, I enjoy looking at anything built around networking.

I was in the same boat in regards to trying to find the actual JSON that was going over the wire. I ended up using Charles to capture all the network requests. I haven't finished the post yet, but if you want to see the actual JSON I have all of the request and responses here https://www.catiemcp.com/blog/mcp-transport-layer/

super cool. Debugging locally across multiple microservices can be a huge pain.

A small feature request/idea - packaging this as a helm chart to make local development on something like Minkikube easier.

Congrats on the 1.0 Release, big milestone.

I'm personally really excited about all of the recent tooling for postgres aggregates. Definitely a pain point for a lot of developers and its easy to fall in trap where things work fine in the beginning and then query times explode as requirements change and the dataset grows. Nice to not have to spin up another DB in order to solve the problem as well.

Congrats on the launch!

For those who have not experimented with columnar based databases, I would highly recommend toying around with them.

The performance improvements can be substantial. Obviously there are drawbacks involved with integrating a new database into your infrastructure, so it is exciting to see columnar format introduced to Postgres. Removes the hurdle of learning, deploying and monitoring another database.