Powdering7082

> I turned it on briefly to receive a confirmation code that I needed in order to maintain that phone plan itself, and once spiritually but not literally - when going on a remote and slightly dangerous hike, friends insisted that I bring along a few communication devices that they had for emergency comms and use them to check in once every few hours.

It is funny to have witnessed the conversion from the default state of people is being not connected digitally to being connected digitally and once people start living with the 24/h connection they start the perceive simply being out of cell network as carrying inherent risk.

I've noticed people reacting to going hiking beyond cell networks with a "but that's dangerous!" response. Being outside of cell networks was the natural state of being for the vast majority of human existence!

https://github.com/jepsen-io/particledb is currently 404 (probably still private?)

> ParticleDB’s home page advertised that ParticleDB was “formally proven” with a TLA+ proof.

Perhaps they meant formerly, probably just a speech to text problem

This was using llama-serve on my local which by default gave it current time & unhelpfully "get_info" which is specific to the browser

The full request was

``` {"messages":[{"content":"What animal lays the biggest egg?","role":"user"}],"return_progress":true,"sse_ping_interval":1,"stream":true,"tools":[{"function":{"description":"Returns the current local date and time in ISO 8601 format, with the IANA time zone name","name":"get_datetime","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"},{"function":{"description":"Get runtime info (OS name), may call when user asks about local files or shell commands","name":"get_info","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"}],"reasoning_format":"auto","reasoning_control":true,"backend_sampling":false,"timings_per_token":true} ```

Disabling tools removes the bit about mentioning instructions.

Just an interesting detail I noticed.

Do y'all have any penalty/regularization w.r.t. reasoning traces or is it pure SFT -> RL?

I am seeing the following in the reasoning trace:

> However, the instructions say I must use available tools when answering questions.

On "What animal lays the biggest egg?" with no system prompt

I assume this is something baked into the model through lots of RL with tool use?

Do you use API for google & anthropic or subscription? I was under the impression those two are apt to ban you for using their models in a non-approved way

What exactly would be the tragedy of the commons here?

Publishing a crap app on the app store doesn't make anyone else's app worse

Number of git pushes in GH is way up. 320M in Q1 this year vs 80M in Q1 or 2020.

https://innovationgraph.github.com/global-metrics/git-pushes

Just because you haven't installed new software doesn't mean that new software doesn't exist.

Nice work thanks for doing it, a couple of notes:

It looks like you are referencing Engram [1], but aren't actually gathering a n-gram (e.g. n=1) but rather individual token_ids.

use_cache=True in ``` with torch.no_grad(): outputs = model.generate(*inputs, max_new_tokens=150, do_sample=False, pad_token_id=tokenizer.eos_token_id, use_cache=True)

    return tokenizer.decode(outputs[0][prompt_len:], skip_special_tokens=True).strip()

```

I think there's a bug around not updating current_train_input_ids as more tokens are updated and processed. To be honest I don't fully understand the code so I could be wrong. Happy to chat more if you're interested I'll shoot you an email!

Lastly just for my sake, please correct me if I am wrong, but my reading is that you are learning an additional gate on top of a select number of layers that modifies locally & dynamically for one particular token to better match the training set that was filtered to not include any refusals.

[1]: https://github.com/deepseek-ai/Engram/blob/main/Engram_paper...

The author's talks on nix go back to 2020: https://fzakaria.com/talks

I suspect they know and care

Me too! (although I am hoping to release it, but it was primarily for me)

I was thinking about writing something and hopefully starting a community/resource area around hyper-personalized software, it's so easy to make now. I was thinking about how it'd be useful to have a place to share these, for ideation, sharing techniques and the ability for LLMs to riff on something already existing. It's not quite like open source's advantage of having many people contribute to the same project, but rather something that is closer to evolution, giving the next generation a place to start modifying.

Would you have any interest in sharing what you've made?

Is your claude.md, skills or other settings that you have honed public?

So by default it pulls out recent PRs, grabs only the tests that were commited and then checks to see how well an agent with different claude.mds can finish the test suite?

This excerpt is well worth reading IMO

Back when I first started doing these clean-up projects, I started by just picking up litter that was in my own neighborhood. (Because that was where I lived, and because I had never been to a lot of the other neighborhoods in my area.) But I found that the more that I did this kind of work, the more that I wanted to do it, and I eventually found myself going beyond my own neighborhood and into neighborhoods that I had never been to before. (Including the ones that I had always heard were "bad neighborhoods".)

Then to make things more interesting, I started using the city bus system for the first time, and I started making it a point to go someplace new that I had never been to before whenever I picked up litter. And after going through a big stack of monthly bus passes, and walking down just about every street in the city (and doing it alone and without a phone) I want to say that not only has nothing bad ever happened to me, but I've encountered a lot of strangers who were almost "too nice" to me...

Because these clean-up projects involve a lot of walking and lugging around heavy stuff, it seems that no matter where I go, strangers will keep pulling over to offer me a ride. And because I do these projects even during extreme weather, the more intense the weather gets, the nicer people will become. (During the summer on really hot days, strangers will keep pulling over just to ask if I'm going to be OK working outside in the heat and if they can go and buy some cold water for me, and sometimes people will even try to give me an umbrella or an extra coat on days when it's raining or snowing.)

And there were times when I would pick up a penny that was in the middle of road or stuck in a crack in the sidewalk, and I guess that it would give strangers walking by the impression that I must need money, and sometimes people would actually pull out their wallet and start trying to give me money!

Strangers will also come up and thank me for what I'm doing, and sometimes they will end up talking to me for a long time, and I've ended up meeting a lot of friendly people this way.

I have been shown such a good side of people, that it simply wouldn't make sense for me to go back to being fearful of strangers and automatically imagining the worst-case scenarios about them. (Like I tended to do back when I didn't get out much and my view of the outside world was being shaped by watching the News.)

I don't doubt that there is crime in my area. (After all, "littering" itself is a crime, and there are MILLIONS of examples of this crime in plain sight where I live.)

But because I have been doing these clean-up projects, I've spent more time outside and less time looking at a screen in the past few years than I have at any other time in my life. And I know that what I am about to say will probably sound crazy to anyone who did the exact opposite of that and who spent the past few years locked in their homes and being bombarded all day long by the media with stories about crime, riots, racism, sickness, and war, but I honestly have never felt safer going outside than I do today.

I started picking up litter in my neighborhood because I wanted to help make the world a better place, and because it got me to get out more and start to base my view of the outside world on my actual experience in the outside world, the world is a much better place to me now, and that is the priceless treasure that I found while picking up a zillion pieces of litter.

I mean they are being sued for insider trading, not exactly a bad headline maybe it could say alleged?

It seems extremely unlikely to me, a casual observer of the shit show that was Luna/Terra, that the suit would be successful

Yeah I loved the guests he had, but eventually had to stop listening to him

No erasure coding seems like a pretty big loss in terms of how much resources do you need to get good resiliency & efficiency

Except that you don't own the things you buy on steam

With respect to winning a game of chess?

Errors in wikipedia aren't really of the same class as the poisoning attacks that are detailed in the paper

Very reasonable to push back on your experience and that creating an account to post a baseless claim is not great, but none of those points serve as examples that rebut the claim.

> As of 2024, SpaceX's internal costs for a Falcon 9 launch are estimated between $15 million[186] and $28 million,[185] factoring in workforce expenses, refurbishment, assembly, operations, and facility depreciation.[187] These efficiencies are primarily due to the reuse of first-stage boosters and payload fairings.[188] The second stage, which is not reused, is believed to be the largest expense per launch, with the company's COO stating that each costs $12 million to produce.[189]

From wikipedia: https://en.wikipedia.org/wiki/Falcon_9#Pricing

Do yourself a favor and skip what Gary Marcus has to say on this and read the METR study it's self

Really concerning that what appears to be the top model is in the family of models that inadvertently starting calling it's self mechahitler

From my minor testing I agree that it's crazy fast and not that good at being correct

Yeah it's been a good learning lesson. I only put in what I was comfortable losing (and fast!). The way the stock moved after the absolutely abysmal earnings will certainly stick with me lol

I've been buying Tesla puts at a small scale for the past 6 months, usually puts about a month out.

It's been a great way to lose money so far.

Did the reporter reach out to Anthropic for public comment on this? They list a "source familiar" with some details about what the intended purpose was for, but no mention on the why