jtbaker
jtbaker on Hacker News
I love the magic mouse, had one going since 2016 and it is still great. Not a fan of their keyboards but the swipe + scrolling exp. on the magic mouse is great IMO.
Yeah looks like it. Was just trying to search up a writeup of it, was pretty sure I remembered reading that a while ago.
Write a log processing/aggregation tool!
highest impact is when there is network overhead, like accessing them over S3/HTTP/S, and you're accessing many files as part of a query, like using a glob `-*.parquet` pattern. I imagine the same applies when you're doing local I/O, but not as pronounced of an improvement due to lower latency.
Being able to put it behind an HTTP service / load balancer and implement horizontal scaling based on resource utilization, or even scale to zero via Cloud Run etc. Drool!
S4E07, e.g. “407 Proxy Authentication Required” was a work of art. Appreciated the whole show, but particularly that episode.
> They are out here selling the shovels
just wish their containerization / linux compat wasn't so nerfed. Just moved my homelab's inference engine over to a GB10 from a Mac MTP setup to have first class k8s support, as well as open up access to the broader CUDA ecosystem (cuOpt etc.).
Oh hell yes. Throw us a bone @bcantrill
Have you been using an MTP setup? I've been having pretty good luck with the Qwen models with the built in MTP heads via
https://mtplx.com at around Q4. My main driver rig is an M5 Max MBP work got for me a few weeks ago. Hitting 1000-1200 TPS prefill on Qwen 3.8-Flash-Next there.
having a 64GB mac mini m4 pro the last few years with some increasingly capable usefulness has kept me interested in this stuff in a way that using a paid platform wouldn't have. Similar to running K8s in a homelab, something about interacting with the hardware makes it more engaging/interesting, for me at least.
In general, I'm a big believer in doing more with fewer resources, within reason, and think having local setups really helps me be mindful with what's happening under the hood with these systems and managing context efficiently to get high quality results.
Same! Recently ported some code from httpx -> niquests and the experience has been good, although admittedly the governance of the project seems a little unclear.
I (ok, Qwen 3.7 27B) wrote some benchmarking code to compare throughput, and niquests seems to be substantially more performant for traffic that doesn't need to traverse the public web, e.g. intra cluster service to service communication:
https://gist.github.com/jtbaker/61061d27949ef48ac31e85ff28c2...
uv run scripts/bench_http.py --count 1000
Benchmarking 1000 requests to http://localhost:8000/health (concurrency=16, timeout=30.0s)
Date: 2026-08-28T10:18:34.480111
Running httpx...
Running httpx2...
Running niquests...
Metric httpx httpx2 niquests
-------------------------------------------------------
mean 17.8ms 4.6ms 3.3ms
min 3.2ms 1.5ms 1.2ms
max 632.4ms 11.1ms 10.8ms
p50 15.9ms 4.2ms 3.0ms
p95 32.4ms 7.4ms 5.6ms
p99 52.1ms 8.5ms 8.4ms
total 1.15s 315.5ms 239.7ms
throughput 869.2 req/s 3170.0 req/s 4171.1 req/s
Winner: niquests (4.8x faster by wall clock)
Qwen3.8 27B doing a lot of lifting right now, and people seem to run it pretty well on 1-2x 3090 setups...
You appear to be replying to a model.
> the only big gotcha I encountered was finding, somewhat counter-intuitively, that it's often best minimize partitioning.
For parquet, I think with partitioning, it's really important to be mindful of the ordering of the data within the parquet file and also the query patterns of the main use cases. A little hard to generalize well to every pattern I guess.
found the twitter street team
I don't think DuckDB itself can coordinate work across multiple nodes. But you could put it behind an HTTP layer and scale horizontally based on resource utilization?
I've got a couple of different use cases:
- ETL pipelines running on K8s nodes. Using their streaming processing engine means I can run smaller pods/nodes if needed, for datasets that may have required large dataframe-like transformations that may have buffered a big dataset into memory previously.
- A CLI distributed to an internal team to do a postprocessing step on a large modeling dataset - to get it into a consumable format and upload it to a bucket as a .db file.
- A SvelteKit app that used the node duckdb bindings to attach to the .db on the bucket and explore the results through a suite of BI tools. These tables have millions of rows, and would be pretty heavy to store in PG. The DuckDB version works really, really well.
-
DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable.
Thanks to the team for everything!
https://github.com/antirez/ds4 coming out at the same time I started a new job and they gave me an m5 max a few months ago was the lightbulb moment for me.
M5 max is 614 GB/s in the 40gpu variant