suresk
Questions like "What do these lines of code do?" or "How does this fit into the big picture?" or "Wait, this doesn't make sense?" are rarely answered by the source.
This is a bit fresher in my mind in the math domain, but I don't think it is any different in any number of other domains, including coding. I've been working through a math textbook, gotten confused about how the author gets from step 2 to step 3, taken a picture of the text, and had AI explain it to me - it almost always gives me a much better understanding of what is going on and helps make things so much clearer. There is a level of interactivity that can't exist in a book or other "source" of information.
I think there is a bit of tension when it comes to learning and sometimes the struggle itself is informative, but there is a reason people hire tutors and go to classes taught by teachers vs just reading a textbook, and cutting yourself off from a tool because you've seen it be wrong about something seems like a silly mistake.
I will.
Can these be wrong? Certainly. So can humans. Many of your examples are of humans being wrong. That doesn't make LLMs - or humans - useless. The fact that they are not infallible is not a reason to avoid using them and I'm not going to throw out a tool that has been incredibly valuable to me because someone on the internet got some bad CORS advice.
Kinda the same with language/technology stuff - it can be a great tutor and it can scaffold other parts of a project for you. It can give you feedback and let you focus on the interesting parts.
I guess the motivation itself may be hard because of the fear of it taking over much of our jobs, but having this kind of help/feedback is pretty cool for the sake of learning things just because they are interesting!
You may also want to look at other sorting algorithms - common CPU sorting algorithms are hard to maximize GPU hardware with - a network sort like bitonic sorting involves more work (and you have to pad to a power of 2) but often runs much faster on parallel hardware.
I had a fairly naive implementation that would sort 10M in around 10ms on an H100. I'm sure with more work they can get quite a bit faster, but they need to be fairly big to make up for the kernel launch overhead.
At ~$600/kWh for capacity, the ROI isn't great. I have a pretty big differential on my rates because I have an EV, and even then I'd need over a decade to make the $1,000 back assuming I fully discharged it every day.
Their paper [1] only mentions using PTX in a few areas to optimize data transfer operations so they don't blow up the L2 cache. This makes intuitive sense to me, since the main limitation of the H800 vs H100 is reduced nvlink bandwidth, which would necessitate doing stuff like this that may not be a common thing for others who have access to H100s.
I'd think things like optimizing for occupancy/memory throughput, ensuring coalesced memory accesses, tuning block sizes, using fast math alternatives, writing parallel algorithms, working with profiling tools like nsight, and things like that are fairly transferable?
The register file size makes sense, I didn't think they were that much of the die on those processors but I guess they had to be pretty aggressive to meet power goals?
Optimizing something for SIMD execution isn't often straightforward and it isn't something a lot of developers encounter outside a few small areas. There are also a lot of hardware architecture considerations you have to work with (memory transfer speed is a big one) to even come close to saturating the compute units.
Being able to redact a few values would be really important for us (I just wrote Python scripts to clean the har files up), but I’m going to play around with it this weekend.
Two questions -
1) Any way to customize which headers/cookies get scrubbed?
2) Is there a way to get something like the lower-level export you can get by going to chrome://net-export/?
I've actually had some success with getting ChatGPT to create Redshift queries based on user text and then I can run them and render results, which has some interesting use-cases.
Max calls out the biggest problem with using something like ChatGPT in a search flow - it is way too slow. I've talked to a lot of people wondering if we can just shove a catalog at ChatGPT and have it magically do a really good job of search, and token limits + latency are two pretty hard stops there (plus I think it would be generally a worse experience in many cases).
What I'm trying to look at now is how LLMs can be used to make documents better suited for search by pulling out useful metadata, summarizing related content, etc. Things that can be done at index time instead of search time, so the latency requirements are less of an issue.
Around the time that change was made to Go, Andrew and I were looking at this and wondering how big of a performance hit it was and if there were a better way to structure that. I had a hunch that the compiler should be smart enough to not compile that as a switch/giant if block, and a quick trip to a disassembler showed it using binary search. This commit: https://github.com/golang/go/commit/1ba96d8c0909eca59e28c048... added the jump table and has some nice analysis on where it makes sense to do binary search vs jump tables.
As far as I can tell, with certain restrictions (that are fine in this case), it is pretty reliable at optimizing giant if/else blocks and switches.
> E.g. Haskell has an awesome concurrent GC that’d work like crap for Java, because it assumes tons of really short-lived, really small garbage and almost no mutation. The other way around is also bound to be problematic—I don’t know how the Scala people do it
I don't know a ton about Haskell's GC, but at surface level it seems very similar to several of the JVM GC implementations - a generational GC with a concept of a nursery. Java GC is very heavily designed around the weak generational hypothesis (ie, most objects don't live long) and very much optimizes for short-lived object lifecycles, so most GC implementations have at least a few nursery-type areas before anything gets to the main heap where GC is incredibly cheap, plus some stuff ends up getting allocated on the stack in some cases.
The only big difference is that in Haskell there are probably some optimizations you can do if most of your structures are immutable since nothing in an older generation can refer to something in the nursery. But it isn't super clear to me that alone makes a big enough difference?
One of the things I really like about Go is that a lot of the designs and decisions, along with their rationales, are pretty well documented. Here are the GC docs, for example - https://go.dev/doc/gc-guide.
For example, Go doesn't move data on the heap around so to combat fragmentation, it breaks the heap up into a bunch of different arenas based on fixed sizes. So a 900 byte object goes into the 1K arena, etc. This wastes some heap space, but saves the overhead and complexity with moving data around.
In our case, it wasn't a single hard-coded number - the input data gave us the upper bound, and the difference between the upper bound and the median case was so small that going with the upper bound worked out best.
One in particular - I was profiling an application with low-latency needs and GC was taking up a ton of time. Mission control showed tons of allocations of arrays - at one point it was creating a bunch of lists in a loop and adding stuff to them, which triggered creating a new underlying array. We found that a) Many of the arrays were just over the first resizing size, and b) There was a good heuristic that we could use to give them an initial size that would never have to be expanded and wouldn't result in huge amounts of waste.
This had a pretty dramatic effect on our GC times and the overall latency. I think this is where the JVM really shines - tons of tooling to help you profile and observe these kinds of details to help you figure out when you actually need to care about stuff like the initial array capacity.
It has been a fun project to play around with for someone like me who thinks this kind of stuff is fun and interesting but will probably never get a chance to work on it full time. Cool to see it noticed, though!
This actually does seem like a great icebreaker-type question for a C developer. I'm less convinced every developer should be able to go into that level of detail if they are doing Javascript or Python or whatever. Ideally they'd be able to at least reason through some of this and demonstrate some understanding of some of this, but I'm not sure it is a great icebreaker question for all developer roles.
Of course there is a trivial solution in pretty much any language, for example something like str(Decimal(x) + 1) in python. That's dead-simple and anyone should be able to do at least that.
The later addendum that they want someone to know how data is store by the computer seems to imply that they want an answer that doesn't use these conversions, which gets a lot trickier depending on the context. I think you'd still expect someone who does C/C++ work to get there pretty quickly, but it is more than 15 seconds - you have to think about some corner cases and how you handle the string having to be resized in some situations.
But in some languages, it isn't obvious to me how you'd do it (especially since strings are immutable in many languages) and I think it is maybe a bit unfair to ask someone who does Java or Python or whatever to do it off the top of their head.
I think the hard part is that "Equality of Opportunity" is either so strictly defined that it is pointless, or it very quickly becomes really squishy and feels like "Equality of Outcome".
Most college applicants today are going to be something like 2 - 4 generations removed from official, legally sanctioned segregation (a situation I think most people would agree doesn't count as equality of opportunity). Would you argue that the average white student and average black student have equality of opportunity today?