mattcollins

On the other hand, AI coding tools make it relatively easy to set and apply policies that can help with this sort of thing.

I like to have something like the following in AGENTS.md:

## Guiding Principles - Optimise for long-term maintainability - KISS - YAGNI

Results from some further tests here: https://www.improvingagents.com/blog/toon-benchmarks

FWIW, I ran a test comparing LLM accuracy with TOON versus JSON, CSV and a variety of other formats when using them to represent tabular data: https://www.improvingagents.com/blog/is-toon-good-for-table-...

I've only looked at one model (gpt-4.1-nano) so far. I'm hoping to run similar tests on some other models but it gets challenging to discern statistically significant differences with better models as their accuracy tends to be a lot better across the board.

This is a follow-up to previous work looking at which format of TABULAR data LLMs understand best: https://www.improvingagents.com/blog/best-input-data-format-...

(There was some good discussion on Hacker News around that here: https://news.ycombinator.com/item?id=45458455)

We often want to feed NON-TABULAR data to LLMs, though, such as typical API responses or config files.

This new work looks out how the format of such nested / hierarchical data affects how well LLMs can answer questions about it; specifically how several models get on with JSON, YAML, XML and Markdown.

Here you go: https://www.improvingagents.com/blog/best-input-data-format-...

Author here.

This has made me chuckle several times - thanks!

I did a small test with just a couple of formats and something like 100 records, saw that the accuracy was higher than I wanted, then increased the number of records until the accuracy was down to 50%-ish (e.g. 100 -> 200 -> 500 -> 1000, though I forget the precise numbers.)

I'm the person who ran the test.

To hopefully clarify a bit...

I intentionally chose input data large enough that the LLM would be scoring in the region of 50% accuracy in order to maximise the discriminative power of the test.

I'm the person who ran the test.

To explain the 60% a bit more...

With small amounts of input data, the accuracy is near 100%. As you increase the size of the input data, the accuracy gradually decreases.

For this test, I intentionally chose an input data set large enough that the LLM would score in the region of 50% accuracy (with variation between formats) in order to maximise the discriminative power of the test.

I'm the person who ran the test.

The context I used in the test was pretty large. You'll see much better (near 100%) accuracy if you're using smaller amounts of context.

[I chose the context size so that the LLM would be scoring in the ballpark of 50% accuracy (with variation between formats) to maximise the discriminative power of the test.]

"This feature is available to all customers, meaning anyone can enable this today from the Cloudflare dashboard."

https://blog.cloudflare.com/control-content-use-for-ai-train...

I wondered about this, too.

Cloudflare have some recent data about traffic from bots (https://blog.cloudflare.com/from-googlebot-to-gptbot-whos-cr...) which indicates that, for the time being, the overwhelming majority of the bot requests are for AI training and not for RAG.

I noticed that, too. It does seem 'odd'.

Per the WGA's summary:

1) AI can’t write or rewrite literary material, and AI-generated material will not be considered source material under the MBA, meaning that AI-generated material can’t be used to undermine a writer’s credit or separated rights.

2) A writer can choose to use AI when performing writing services, if the company consents and provided that the writer follows applicable company policies, but the company can’t require the writer to use AI software (e.g., ChatGPT) when performing writing services.

3) The Company must disclose to the writer if any materials given to the writer have been generated by AI or incorporate AI-generated material.

4) The WGA reserves the right to assert that exploitation of writers’ material to train AI is prohibited by MBA or other law.

Apply4 | Senior Ruby on Rails Engineer | Remote (EU timezone) | Full-time | $70k - $85k | https://bit.ly/ycror

Our software helps local governments manage public outdoor spaces (parks, roads, etc.) more effectively, making it easier for good things to take place, like special events, filming and infrastructure improvements.

We’re used by cities and local governments in multiple countries and our customers love us (Net Promoter Score of 71!) We’re looking for a seasoned engineer to join our friendly and supportive team to help us continue to improve on what we have, making sure our platform can serve people well in the years to come.

We’re a small (but established) company so you won’t be a cog in a big machine here — you’ll be working directly with me (the CTO), our product manager and our other two developers to shape the platform, helping to decide what will be the most high-impact thing to work on.

Feel free to apply using the link above or reach out to me directly.