prats226

A good experiment would be to also try giving it access to latency traces so it can identify issues? Wrt coding agents, giving access to observability tools often improve coding/debugging ability for me

Try https://docstrange.nanonets.com/ once, 10k docs you can use for free. Strong table performance. Do give feedback if any. Powered by bigger model compared to our open source one which is quiet popular on HF.

If with LLM's you can deanonymize at scale, on a personal level, you should also be able to figure out what posts are leading to this deanonymization and remove them or modify them.

Instead of markdown -> LLM to get JSON, you can just train a slightly bigger model which you can constrain decode to give JSON rightaway. https://huggingface.co/nanonets/Nanonets-OCR2-3B

We recently published a cookbook for constrained decoding here: https://nanonets.com/cookbooks/structured-llm-outputs/

Nice, it would be good idea to develop CFG for this as well so can embed it into all these constrained decoding libraries

One of the authors here, will checkout the diagram link.

Every commercial model provider is adding structured outputs so will keep updating the guide.

https://docstrange.nanonets.com/ as well, wrapper on top of 7B version of https://huggingface.co/nanonets/Nanonets-OCR2-3B

Then you can just download finetuned version of same multi-modal foundation model that's trained on documents?

Top 3 models on huggingface are all OCR models. Most automation projects involve documents where you need a model finetuned to understand all elements inside documents and provide grounding and confidence scores etc which is why these subset of models are gaining popularity

Would be intersting to see where funding goes to fix these issues. News would heavily impact public opinion and hence political influence and public funding.

Yes, and its not just OCR (Optical Character Recognition), it understands layouts, captures signatures, charts, watermarks etc so way beyond just characters

Here is link to open source model: https://huggingface.co/nanonets/Nanonets-OCR-s

And hosted model: https://docstrange.nanonets.com/

It boils down to information loss in compaction driven by LLM's. Either you could carefully design tools that only give compacted output with high information density so models have to auto-compact or organize information only once in a while which eventually is going to be lossy.

Or you just give loads of information without thinking much about it, assuming models will have to do frequent compaction and memory organization and hope its not super lossy.

Reason I felt like they are closely connected are because for designing tools for lets say coding agents, you have to be thoughful of context engineering.

Eg linear MCP is notorious for giving large JSONs which quickly fill up context and hard for model to understand. So tools need to be designed slightly differently for agents keeping context engineering in mind compared to how you design them for humans.

Context engineering feels like more central and first-principle approach of designing tools, agent loops.

Context engineering is another name people have given to same skill?

You can always put automation for your google home to blast music at full volume at right time. And if you don't wake up from sound of music yourself, your neighbour will knock on your door for sure!

With google serving AI overviews, now an average search query should cost more? Compute is getting cheaper but also algorithms getting more and more complex, increasing compute?

Read long time ago that even SFT for conversations vs base model for autocomplete reduces intelligence, increases perplexity

> But here’s the important part: LLMs don’t know how to use tools. They don’t have native tool calling support. They just generate text that represents a function call.

Its not a completely true statement. Eg openAI uses libraries like llguidance to get LLM to produce structured output, its not completely unguided free form text that happens to mimic a function call with parameters, truly.

This is super interesting to think about in LLM world where lot of software is getting replaced with LLM calls.

In terms of output of an LLM, there is no clear promise in the contract, only observable behaviour. Also the observable behaviour is subject to change with every update in LLM. So all the downstream systems have to have evals to counter this.

One good example is claude code where now people have started complaining them switching models effecting their downstream coding workflows.

Can you give a specific example where at certain depth it has stopped becoming useful?

I see that in tool calling, we usually specify just the inputs to functions and not what typed output is expected from function.

In DSL style agents, giving LLMs info about what structured inputs are needed to call functions as well as what are outputs expected would probably result in better planning?

https://www.jonathansapps.com/ Someone recently suggesting this and first thought I came to my mind is prevailing advice is to not show kids screen as long as possible

Interestingly, deepseek paper mentions RL with process reward model. However they mentioned it failed to align model correctly due to subjectivity involved in defining if the intermediate step in process is right or wrong

Isn't it because there is a difference in your field and other fields?

1) Scope - Other fields like law, medicine atleast are impacting one unit at a time, vs software which is impacting large number of users through your work. I am sure research interviews will go through similar process?

2) Feedback - Just basis past work, you would get a good sense of their aptitude. Very hard to do it in programming without you spending a lot of time going through their work?

3) Subjectivity - Wrt coding, very good way to get objective output in interview by getting other person to write code, can't do that in medicine for example?