giyanani

It depends on what model you’re running, and for what workload. For personal use (one or two convos at a time) with models that fit in gpu memory, pcie bandwidth doesn't really matter. Just try and be on gen 3 x8 or higher.

Llama is decent at auto optimizing it if you let it use both gpus. It’ll split the workload so the contiguous layers are all on one gpu. Once the model is loaded, you only transfer weights between gpus once (per token?), at the layer boundary.

I run on an 8gb 3070 and 12 gb 3060, and the only weird thing is that the weaker card gets more layers (and therefore work) because it has more ram.

Oh, if you’re barely fitting the models into your vram, you may need to explicitly adjust the layer balance between cards — sometimes it fails to realize it should have put certain things (like draft models) on the other card so you can fit one more layer in.

> Given the lack of forecasting signal, the obvious next step then is to seek out external (exogenous) features in the real world that can help prediction models. After all many real world time series are event-driven (FX, bitcoin) and hence they are exposed to shocks and drifts which can also be measured or accounted for in the data generating process.

This is the approach I'm using at my job, which is incident detection with customer metrics. We're tagging our time series data with common features -- such as country, customer type, etc -- with the idea that we can do a graph-like search to find exogenous variables. We can also use this to identify time series that have a similar "data generating process" and are simply different "realizations" of each other.

We don't need great time series forecasts, just something that detects large deviations quickly. We can then add in an existing dataset of _known_ incidents, indexed by the same common features, as a training/validation set.

To be fair, the article calls that out specifically at the end:

> What makes this especially frustrating is the lack of transparency. Every other ADS company in the NHTSA database, Waymo, Zoox, Aurora, Nuro, provides detailed narratives explaining what happened in each crash. Tesla redacts everything. We cannot independently assess whether Tesla’s system was at fault, whether the safety monitor failed to intervene in time, or *whether these were unavoidable situations caused by other road users*. Tesla wants us to trust its safety record while making it impossible to verify.

“Lacking” implies it’s an unintentional deficiency.

Rust has deliberately taken a “batteries not included” approach due to learned experiences in python, where many std-lib packages are basically abandoned/not recommended anymore.

This gives the ecosystem time to develop better approaches to things, rather than ossifying around the initial implementation.

Yes, it does lead to a larger set of dependencies, but some of the most common packages (including anyhow) were written and are maintained by members of the rust lang organization/team itself.

> But it is a problem, because somewhere along the line one of those indirect dependencies is going to start conflicting with something else in the graph; the bigger the graph, the more conflicts.

Rust is statically compiled, so it’s possible to have different versions of the same dependency in your dependency graph without build issues. Yes, that can lead to binary bloat and slower build times and some related problems, but in practice the compiler is good at dealing with those issues.

Why do you say shuffle is “SIMD’s killer app”? I’ve only dabbled in vector instructions from a learning perspective, and seen others mention it’s important too, but have yet to understand why.

Those numbers aren’t directly comparable — caches usually trade off size for performance, and Apple’s L1 sits in performance somewhere between Ryzen’s L1 and L2. Ditto for Apple’s L2 versus Ryzen’s L2 and L3.

I can’t remember the exact article I read that in, but it’s pretty obvious when looking at Anandtech’s latency charts.

M1 latency: https://www.anandtech.com/show/17024/apple-m1-max-performanc...

Ryzen latency: https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-di...

Road/car dominance wasn’t exactly established in a free market either.

There’s a whole host of subsidies for building/maintaining roads paid for by everyone [1]. There’s the regulatory advantages driving has (ex: the common zoning requirement that new construction ensure enough parking space [2]). There’s the negative externalities of highways/roadways. (Ex: the health impacts of pollution[3], and the health impacts of sitting in a car rather than walking/biking (part of the way) to their destination [4])

[1] https://usa.streetsblog.org/2013/01/23/drivers-cover-just-51... [2] https://www.planning.org/planning/2018/oct/peopleoverparking... [3] https://theicct.org/new-study-quantifies-the-global-health-i... [4] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8744747/

Jane street had a puzzle related to this, called eldrow [1], a few months ago.

The gist is you’re trying to find the worst sequence of (hard mode) guesses possible for any word in the wordle set.

Their site shows submissions of up to 16 words and claims 17 is impossible [2]. I’ve been trying to do the same, but have yet to cut the search space enough[3].

Ex: calculating the worst chain for jazzy (with a 1000 word subset of the full word list) took an m1 core 1 hour and resulted in the ten-word chain:

civic, buggy, woody, array, leaky, mammy, fanny, happy, tasty, jazzy

[1] https://www.janestreet.com/puzzles/eldrow-index/ [2] https://www.janestreet.com/puzzles/eldrow-solution/ [3] https://github.com/roshangiyanani/wordle/blob/main/wordle/el...

I’m not sure how fuschia does it, or how feature-based capabilities work, but Cheri[1] uses capabilities for memory management and isolation.

It uses a couple of techniques, like wide/tagged pointers, object ids, and a special hardware managed bit to track illegal modifications.

If I had to hazard a guess, those object ids are probably useful for general capability systems.

I think apple (maybe as just an arm feature?) can do encrypted pointers, with a per application key tracked by the kernel.

[1] https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/

The memory ordering is different, too. Apple gets around it by adding support for intel ordering in the m1 cpus, and then changing to that mode on a per-process basis.

https://twitter.com/ErrataRob/status/1331735383193903104

If you mix arm and x86 code inside a process, you can't tell which ordering to use.

The typing (>= python 3.6?) and collections (>= python 3.8?) packages have definitions for a bunch of protocols (basically interfaces for structural typing).

So for that List[int] example, you probably want to take a(n) Iterable[int], Iterator[int], or Collection[int] instead, depending on exactly how you use it.

I would argue the M1 Pro/Max are targeted at much higher end workloads than an i3. Light workloads (web browsing, launching apps), don’t need many high performance cores, since they’re mostly single-threaded.

What is interesting is how intel has 4-8 e cores even on the high end.