psaccounts
Re:The Q, K, V Matrices
on Thursday January 08, 2026 @02:40PM (#46545455)
I published a video that explains Self-Attention and Multi-head attention in a different way -- going from intuition, to math, to code starting from the end-result and walking backward to the actual method.
Hopefully this sheds light on this important topic in a way that is different than other approaches and provides the clarity needed to understand Transformer architecture. It starts at 41:22 in the below video.
Re:LLMs Generate Text – A Clear and Comprehensive Step-by-Step Guide [...
on Wednesday September 24, 2025 @02:42PM (#45364388)
This video tutorial provides an intuitive, in-depth breakdown of how an LLM learns language and uses that learning to generate text. Key concepts shown below are covered in a way that is both broad and deep, while still keeping the material accessible without losing technical rigor:
* Historical context for LLMs and GenAI
* Training an LLM -- 100K overview
* What does an LLM learn during training?
* Inferencing an LLM -- 100K overview
* 3 steps in the LLM journey from pre-training to serving
* Word Embeddings -- representing text in numeric format
* RMS Normalization -- the sound engineer of the Transformer
* Benefits of RMS Normalization over Layer Normalization
* Rotary Position Encoding (RoPE) -- making the Transformer aware of token position
* Masked Self-Attention -- making the Transformer understand context
* How RoPE generalizes well making long-context LLMs possible
* Understanding what Causal Masking is (intuition and benefit)
* Multi-Head Attention -- improving stability of Self Attention
* Residual Connections -- improving stability of learning
* Feed Forward Network
* SwiGLU Activation Function
* Stacking
* Projection Layer -- Next Token Prediction
* Inferencing a Large Language Model
* Step by Step next token generation to form sentences
* Perplexity Score -- how well did the model does
* Next Token Selector -- Greedy Sampling
* Next Token Selector -- Top-k Sampling
* Next Token Selector -- Top-p/Nucleus Sampling
* Temperature -- making an LLM's generation more creative
* Instruction finetuning -- aligning an LLM's response
* Learning going forward
Show HN: Comprehensive and Intuitive Introduction to Deep Learning
Dear HN -- I am doing a multi-part web series on teaching AI, specifically Deep Learning. Learning AI is not easy. The topics are highly technical and cover a broad range of areas without offering any intuitive or clear entry point for someone new to the field. I find that most tutorials focus on heavy math making them inaccessible to most people; or treat topics only superficially; or cover...
Re:How SQLite Is Tested
on Monday June 20, 2016 @10:21AM (#11938189)
Could you share which 3 static analysis tools you use?
We found Coverity to be prohibitively expensive; but unfortunately there isn't any real alternative!
Re:Ritalin Gone Wrong
on Sunday January 29, 2012 @05:25PM (#3526470)
You are saying this out of the research you have done yourself, right? And yes, I have done the research and also have experienced it first hand -- something that Stanford doctors could not do (and an army of 3 different specializations nonetheless) was cured using Homeopathy. Is this placebo effect?
Re:Ritalin Gone Wrong
on Sunday January 29, 2012 @02:08PM (#3525847)
It is a shame that most people are unaware of the efficacy of Homeopathy to treat ADD, ADHD and autism-spectrum disorders. Do some research and you'll see how hundreds of thousands of people have been cured of these ailments using Homeopathy alone.
Re:LevelDB: a fast and lightweight key/value database library
on Monday May 09, 2011 @01:06PM (#2529395)
Thanks for the pointer. Are there any known open-source implementations of the same?
Re:LevelDB: a fast and lightweight key/value database library
on Sunday May 08, 2011 @10:38PM (#2527337)
But it doesn't really describe the LSM data structure!
Re:LevelDB: a fast and lightweight key/value database library
on Sunday May 08, 2011 @08:01PM (#2527046)
The implementation seems to use Log-Structured Merge Trees.
The only paper on this data structure seems to be: http://goo.gl/CVF1l
This paper is poorly written and quite honestly not useful to implement an LSM tree. Does anyone know of a better paper than this one?
Re:LevelDB: a fast and lightweight key/value database library
on Sunday May 08, 2011 @04:34PM (#2526670)
Is there a more useful technical paper on Fractal Trees? Better yet is there a open source implementation of the same?
Book on Parallel Programming
Re:Oracle embraces SQLite; wraps it around BerkeleyDB as SQL API
on Wednesday March 31, 2010 @11:22PM (#1233749)
What would be the licensing model if the product is open source, but is NOT free (i.e., customers are charged for an open-source product in which BDB is used)?
Re:What programming language to teach first
on Wednesday February 03, 2010 @02:16PM (#1098562)
Perl followed by C
Ask HN: License fee for BDB (Berkeley Database)
Does anyone know how much Oracle charges for using BDB (Berkeley Database) in a dual-licensed open-source+commercial application?While Tokyo Cabinet (http://1978th.net/tokyocabinet) provides a similar key-value DB like BDB, it lacks some features like replication, custom recovery logs and checkpointing which BDB has. Oracle on its part has made it very difficult to determine BDB licensing (we...
Ask HN: Power and Cooling Cost compared to Server Cost
Has anyone done, or is aware of any, calculations to compare the cost of power and cooling compared to the cost of hardware (servers) in a typical data center? This is to compute a true total cost of ownership of self-hosting servers. Of course real TCO includes:hardware_cost + power + cooling + rental + human_cost + maintenanceIs there any study that says something like (TCO - hardware_cost) =...
Ask HN: Web 2.0 and The Four Steps to the Epiphany
Folks,Those of you who have read Steven Gary Blank's excellent book "The Four Steps to the Epiphany", do you think these principles apply to Web 2.0 companies. In general every commercial business entity must make profits. And most successful enterprise software/hardware companies surprisingly seem to fit the model described in the book. But the majority of the Web 2.0 companies seem to defy...
Do you know of any good filetype/mimetype icons?
If your startup is using filetype/mimetype icons, where did you get them from? Some sites use simple, yet elegant, icons (e.g.,: http://www.basecamphq.com/images/screenshots/files.png). The FamFam silk icon set has some, but they are small (16x16 pixels). Is there any other place from where good, yet elegant, filetype/mimetype icons can be obtained (either free or for a nominal price)?
Ask HN: What software do you use to make screencasts for you product?
What software do you use to make screencasts for your products? Are there any good tools on Windows or Mac that are available for free?
Web Startups with a subscription based business model
Are there any founders here who are building Web Startups with a business model to sell subscriptions (or at least a free+premium model and not an advertising driven business model)?* How difficult was is to acquire paying customers? * What type of customers are these -- individuals or small organizations? * What kind of objections did you typically face before they signed the check? * And...
Does Amazon S3 really save money?
With a price tag of $0.150/GB/month, storing 1TB of data costs around $150/month on Amazon S3. But this is a recurring amount. So, for the same amount of data it would cost $1800/year and $3600/2-years. And this doesn't even include the data transfer costs.Consider the alternative, with colocation the hardware cost of storing 1TB of data on two machines (for redundancy) would be around...
Re:Ask HN: If you were starting from scratch again, what language woul...
on Wednesday December 17, 2008 @03:00PM (#401366)
C
Ask YC: Has anyone raised VC/Angel funding recently?
Has anyone raised VC/Angel funding recently (i.e., in the last two months)? Specifically has anyone raised Seed/series A funding? If so, what has your experience been? Is it significantly more difficult now or is it still somewhat doable? Do you know of any VCs/Angels that are still investing in Seed/Series A rounds?
Ask YC: Which custom icon designers do you use for your startup?
Which custom icon design company/freelancer do you use for your startup? Not logo designs but icons.I personally like clean icons like the Fugue set (www.pinvoke.com) or the Silk icon set (www.famfamfam.com). But neither of these designers is available for freelancing projects!
Ask YC: nginx or apache for PHP?
Since I did not find authoritative information elsewhere about which one is more preferred in production environments, I thought I'd ask here.Based on your experience would you go with nginx (+fastcgi) or apache to run a high traffic website that is written using PHP?It would also benefit the community if someone could post information about any HOWTO/tutorial resources describing the best ways...