Re:ChatGPT's API Is So Good and Cheap, It Makes Most Text Ge...
by mark_l_watson on Saturday March 11, 2023 @09:31PM
(#35115788)
I agree with your optimism, although I can understand if people want to argue with this optimism.
I think that we are seeing exponential improvements in deep learning, specifically in LLMs. A few months ago, I would see about one new demo or great paper a day that really impresses me. Now I see many great new demos, or products, or papers a day. In the 1980s we would build commercially useful models using a few layers with backdrop with small layer sizes. We built our own Harvard Architecture hardware for forward feeds and backdrop, a whopping 5 million flops/second. That was fun, but I prefer modern tools, thank everyone very much!
BTW, I am working on a short book that uses LangChain, Llama-Index (used to be called GPT-Index), and a few other libraries to solve a few problems that I find interesting.
I think that we are seeing exponential improvements in deep learning, specifically in LLMs. A few months ago, I would see about one new demo or great paper a day that really impresses me. Now I see many great new demos, or products, or papers a day. In the 1980s we would build commercially useful models using a few layers with backdrop with small layer sizes. We built our own Harvard Architecture hardware for forward feeds and backdrop, a whopping 5 million flops/second. That was fun, but I prefer modern tools, thank everyone very much!
BTW, I am working on a short book that uses LangChain, Llama-Index (used to be called GPT-Index), and a few other libraries to solve a few problems that I find interesting.