News for hackers. Stuff that matters.

A comment on Stealing Part of a Production Language Model.

Re:Stealing Part of a Production Language Model by smaddox on Tuesday March 12, 2024 @03:45AM (#39677101)
They don't disclose the embedding dimension for gpt-3.5, but based on table 4, comparing the Size and # Queries columns, gpt-3.5-turbo presumably has an embedding dimension of roughly 20,000? Interesting...
[ Parent | on HN ]
Re:Stealing Part of a Production Language Model by HarHarVeryFunny on Tuesday March 12, 2024 @09:43AM (#39679384)
How does embedding dimension size relate to maximum context length ?
[ Parent | on HN ]
Re:Stealing Part of a Production Language Model by mfscholar on Tuesday March 12, 2024 @10:04AM (#39679652)
Embedding size and maximum context length are not related. The maximum context length of gpt-3.5-turbo is known though: 2^14
[ Parent | on HN ]