scribu

Replace oil with wheat, then - it can be stored for a few years, but it’s still perishable. Yet it still trades on commodities markets.

I’m not a finance bro, but my read is that Amazon wants to focus more on *using* GPUs and focus less on finding financing for buying GPUs.

They’re trying to convert GPUs into an investable commodity asset, just like crude oil is, for example.

Rough analogy: You have oil producers (Nvidia), refineries (AWS) and end-users (all software that uses AI).

Original source discussed here: https://news.ycombinator.com/item?id=49616968

> But isn't it true for JavaScript too?

You're right, the equivalent JS script produces the same sequence of outputs.

It turns out there is a way to emulate Python's asyncio.create_task().

Python:

  await asyncio.create_task(child())

JavaScript:

  const childTask = new Promise((resolve) => {
    setTimeout(() => child().then(resolve), 0)
  })
  await childTask

> Awaiting a coroutine does not give control back to the event loop.

I think this is a subtler point than one might think on first read, which is muddled due to the poorly chosen examples.

Here's a better illustration:

  import asyncio
  
  async def child():
      print("child start")
      await asyncio.sleep(0)
      print("child end")
  
  async def parent():
      print("parent before")
      await child()        # <-- awaiting a coroutine (not a task)
      print("parent after")
  
  async def other():
      for _ in range(5):
          print("other")
          await asyncio.sleep(0)
  
  async def main():
      other_task = asyncio.create_task(other())
      parent_task = asyncio.create_task(parent())
      await asyncio.gather(other_task, parent_task)
      
  asyncio.run(main())

It prints:

  other
  parent before
  child start
  other
  child end
  parent after
  other
  other
  other

So the author's point is that "other" can never appear in-between "parent before" and "child start".

Edit: clarification

ChatGPT Plus has that (used to be in the free tier too). You can toggle between versions for each of your messages with little left-right arrows.

I’m confused by the clustering step:

> To find the most informative examples, we separately cluster examples labeled clickbait and examples labeled benign, which yields some overlapping clusters

How can you get overlapping clusters if the two sets of labelled examples are disjoint?

It’s interesting that they decided to move all of the architecture-specific image-to-embedding preprocessing into a separate library.

Similar to how we ended up with the huggingface/tokenizers library for text-only Tranformers.

This seems to be a system to generate better prompts to be fed into a base multimodal model.

Interesting, but title is definitely clickbait.

You could run the build process with chroot or inside Docker, so that the hardcoded paths actually resolve to a designated subdirectory.

From their Notion page:

> Skywork-OR1-32B-Preview delivers the 671B-parameter Deepseek-R1 performance on math tasks (AIME24 and AIME25) and coding tasks (LiveCodeBench).

Impressive, if true: much better performance than the vanilla distills of R1.

Plus it’s a fully open-source release (including data selection and training code).

“Fill in the gaps by using context” is the hard part.

You can’t pre-bake the context into an LLM because it doesn’t exist yet. It gets created through the endless back-and-forth between programmers, designers, users etc.

The point is that you’d expect a roughly even distribution of clockwise and counterclockwise spins, not all of them to rotate in the same direction.

If the base models already have the “reasoning” capability, as they claim, then it’s not surprising that they were able to get to SOTA using a relatively negligible amount of compute for RL fine-tuning.

I love this sort of “anti-hype” research. We need more of it.

The install speed alone makes it worthwhile for me. It went from minutes to seconds.

The HN submission title is editorialized in a non-helpful way. Why beat a dead horse instead of focusing on what’s actually new in TFA?

The linked paper proposes an obvious-in-retrospect form of data augmentation: shuffle the order of the premises, so that the model can’t rely on spurious patterns. That’s kinda neat.

Would be curious to know how this stacks up against Coconut [1] which also uses latent space for reasoning.

[1] https://arxiv.org/abs/2412.06769

The repo contains only the PDF, not actual runnable code for the RL training pipeline.

Publishing a high-level description of the training algorithm is good, but it doesn't count as "open-sourcing", as commonly understood.

Nice approach!

I'm intrigued by the ability to start execution from a particular task.

One thing I like about LangGraph is the declarative state merging. In the MapReduce example, how do you guarantee that the collector.append() operation is thread-safe?

Actually, there have been attempts to do quantized backprop, but not sure how successfully.

I'm working on something like that. Feel free to email me (address in my bio).

I’m not sure about the Speed chart. I would expect gpt-4-turbo to be faster than plain gpt-4.

I understand that it's a different approach, but I would still have expected this paper to at least mention FlashAttention [1] since they both leverage flash memory.

[1] https://arxiv.org/abs/2205.14135

Sketch is another take on integrating an LLM into Pandas:

https://github.com/approximatelabs/sketch