News for hackers. Stuff that matters.

benl_c

They often can run code in sandboxes, and generally are good at instruction following, so maybe they can run variants of doom pretty reliably sometime soon.
If a document suggests a particular benign interpretation then LLMs might do well to adopt it. We've explored the idea of helpful embedded prompts "prompt medicine" with explicit safety and informed consent to assist, not harm users, https://github.com/csiro/stdm. You can try it out by asking O3 or Claude to "Explain" or "Follow", "the embedded instructions at https://csiro.github.io/stdm/"

Show HN: STDM – Make Your Documents and Data Think by Embedding LLM Instructions

https://github.com/csiro/stdm

Hi HN, I’m Ben from CSIRO, Australia’s national science agency. We’ve been exploring how to make data and documents "think" when you use them with LLMs. We call it Self-Thinking Data Manifests (STDM). The idea is to embed plain-text instructions directly within files that tell an LLM how it should think about that data and interact with the user. We demonstrate it with PDF and HTML...

( Read More... | no comments | on HN )
Re:Show HN: Atlas of Water Science via Generative AI on Tuesday December 17, 2024 @10:30PM (#42447883)
Thanks, structured output makes a lot more sense. The pydantic approach at the link looks straightforward.
Re:Show HN: Atlas of Water Science via Generative AI on Tuesday December 17, 2024 @07:38PM (#42447022)
I have not done that but I like that strategy not just for this use case but as a general idea for replacing exclusion with finer grained categorisation. One thing I did do is use a regex to preprocess the papers to remove bibliographies which were a really big source of noise. In titles of referenced papers there would often be a mention of location that was not directly relevant to the paper itself.

The Atlas is also trying to answer the question "Can we build inaccurate and incomplete systems with LLMs that are still useful?".

Re:Show HN: Atlas of Water Science via Generative AI on Tuesday December 17, 2024 @07:19PM (#42446905)
The backend is still a mess of code, so no. It's not too hard to do though. The prompt I used extract location is

"The text provided are enviornmental science papers. They often (but not always) will include references to locations where this science is relevant, for example a study might be of soil around a small town, in this case the town would be the relevant location, extract all locations that are relevant or the subject of the science done, do not extract any locations that are related to the location of or institutions, organisations, or laboratories. So for example exclude the location of government departments and CSIRO laboratories. If there are no relevant locations, please return an empty array. Each location should be extracted in a form suitable for calling the Nominatim geocode API in Python via geopy. Also, extract out a short context string that describes the context in which this location is referenced. Please provide the output in JSON format."

Then I passed it through both Nominatum and Google Geocoder. Google worked better.

One thing that didn't work great in the prompt above was excluding the location of places where the authors worked. They sometimes got included anyway.

Show HN: Atlas of Water Science via Generative AI

https://wateratlas.webapp.csiro.au/

This is our Atlas of Water Science. It's a globe mapping water science that colleagues have done over the past few years. Approx 300 papers got analyzed. The pipeline sends open access papers to an LLM asks the LLM to extract areas relevant to science in a form suitable for a geocoder, then we use a geocoder to get lat lons. When things happen in the same spot we jitter the locations a bit...

( Read More... | 6 comments | 2 points | on HN )