News for hackers. Stuff that matters.

wilhelm____

Re:ChatGPT Can Determine Your Age Indirectly on Thursday March 07, 2024 @09:14PM (#39637071)
An example of the current state of LLM reasoning.
Re:What We Know About LLMs (A Primer) on Wednesday July 26, 2023 @08:44AM (#36876725)
^^ yes
Re:What We Know About LLMs (A Primer) on Wednesday July 26, 2023 @08:44AM (#36876720)
On the contrary, I agree - while there is certainly hype being generated around AI, particularly generated by the "VC hype cycle", the fundamental advancements we've made with LLMs are quite real.

Part of the reason I wrote this is to separate the signal from the noise and why one should be {cautiously, more tempered} optimistic in the medium term.

Re:What We Know About LLMs (A Primer) on Wednesday July 26, 2023 @08:40AM (#36876675)
thank you, Yi!
Re:What We Know About LLMs (A Primer) on Wednesday July 26, 2023 @08:39AM (#36876667)
pre-training is developing the language model's base understanding of conditional word probabilities.

SFT and RLHF is attempting to further guide the model in terms of steerability + alignment of output.

In fact, the InstructGPT authors were worried about losing the pre-trained model's underlying probability distribution, so they try a version where it penalizes the model deviating too significantly from the original distribution (using KL). I don't remember them seeing a significant difference in performance.

Re:What We Know About LLMs (A Primer) on Wednesday July 26, 2023 @08:36AM (#36876624)
haha ty ty!
Re:What We Know About LLMs (A Primer) on Wednesday July 26, 2023 @08:36AM (#36876622)
I mention that above this sentence in the image - glad someone was "paying attention to" the text ;)
Re:What We Know About LLMs (A Primer) on Wednesday July 26, 2023 @08:35AM (#36876605)
thank you!
Re:Trading algo visual debugging tool on Sunday January 15, 2023 @01:02PM (#34391435)
Released this open source tool for debugging trading algos. Useful for those doing crypto trading.
Re:"TL;DR" on Recent Transformers Papers on Sunday August 15, 2021 @02:18PM (#28190969)
yep that is a good one. The goal of this repo to have a bird's eye view of different papers as I read them and to distill that info into one table.

"TL;DR" on Recent Transformers Papers

With the explosion in research on all things transformers, it seemed there was a need to have a single table to distill the "tl;dr" of each paper's contributions relative to each other. Here is what I got so far: https://github.com/will-thompson-k/tldr-transformers . Would love feedback - and feel free to contribute too :)

( Read More... | 2 comments | 4 points | on HN )