zora_goron

Why might this be, does anyone know?

This difference between medical board examinations and real world practice is something that mirrors my real-world experience too, having finished med school and started residency a year ago.

I’ve heard others say before that real clinical education starts after medical school and once residency starts.

Anyone have any anecdotes on how expensive this is to operate, ie compared to performing the same task via Claude Code?

I wrote about some similar observations in the clinical domain -- I call it the "human -> AI reasoning shunt" [0]. Explicitly requesting an AI tool to perform reasoning is one thing, but a concern I have is that, with the increasing prevalence of these AI tools, even tasks that theoretically are not reasoning-based (ie helping write clinical notes or answer simple questions) can surreptitiously offload some degree of reasoning away from humans by allowing these systems to determine what bits of information are important or not.

[0] https://samrawal.substack.com/p/the-human-ai-reasoning-shunt

This doesn’t necessarily apply to this particular offering, but having working in clinical AI previously from a CS POV and currently from as a resident physician, something I’m a little wary of is the “shunting” of reasoning away from physicians to these tools (implicitly). One can argue that it’s not always a bad thing, but I think the danger can lie in this happening surreptitiously by these tools deciding what’s important and what’s not.

I wrote a little bit more of my thoughts here, in case it’s of interest to anyone: [0]

On that same vein, I recently made a tool I wrote for myself public [1] - it’s a “copilot” for writing medical notes that’s heavily focused on letting the clinician do the clinical reasoning, with the tool exclusively augmenting the flow rather than attempting to replace even a little bit of it.

[0] https://samrawal.substack.com/p/the-human-ai-reasoning-shunt

[1] https://x.com/samarthrawal/status/1894779710258733330

Does anyone know how this “user decides how much compute” is implemented architecturally? I assume it’s the same underlying model, so what factor pushes the model to <think> for longer or shorter? Just a prompt-time modification or something else?

Does anyone know, how "reasoning effort" is implemented technically - does this involve differences in the pre-training, RL, or prompting phases (or all)?

“A programming language is for thinking of programs, not for expressing programs you’ve already thought of. It should be a pencil, not a pen.” - from PG’s “Hackers & Painters”

Thanks for trying it out! Honestly, no current idea of business model - this was just something I thought would be helpful in my daily workflow so I built it out.

The buttons are stored to localstorage, as is the editor text - everything stays locally (besides what is sent to the LLM). I'm planning on a simple import/export mechanism for transferring buttons across different browsers/computers!

Hi there, thanks for trying it out and the kind words! These are great suggestions - thank you!

I'm still trying this out, but I've been using Superlist [0] (no affiliation) for different projects. For me, having a single app that is both a Markdown document <-> todo app has been helpful. I'm able to jot down notes in greater structure and depth than in another todo app and interweave todo items throughout, which helps contextualize tasks a lot better (especially when working across different projects).

Reminds me of org-mode in Emacs - overall, I'm really liking the cross-platform sync and Markdown interface out-of-the-box.

[0] https://www.superlist.com/

The article mentions,

>> “you've tried response_format: "json" and function calling and been disappointed by the results”

Can anyone share any examples of disappointments or issues with these techniques? Overall I’ve been pretty happy with JSON mode via OpenAI API so I’m curious to hear about any drawbacks with it.

I see a couple comments comparing llama.cpp and Ollama, and I think both have utility for different purposes. Having used both llama.cpp (which is fantastic) and Ollama, a couple things that I find valuable about Ollama out-of-the-box --

- Automatically loading/unloading models from memory - just running the Ollama server is a relatively small footprint; every time a particular model is called it is loaded into memory, and then unloaded after 5 mins of no further usage. It makes it very convenient to spin up different models for different use-cases without having to worry about memory management or manually shutting down those tools when not in use.

- OpenAI API compatibility - I run Ollama on a headless machine that has better hardware and connect via SSH port forwarding from my laptop, and with a 1 line change I can reroute any scripts on my laptop from GPT to Llama-3 (or anything else).

Overall, at least for tinkering with multiple local models and building small, personal tools, I've found the utility:maintenance ratio of Ollama to be very positive -- thanks to the team for building something so valuable! :)