matanyal
matanyal on Hacker News
When you have a pile of hardware and silicon Intellectual Property, patents, etc, IMO it's pretty clever. However, I'm a Groq Engineer, and I'm mega-biased.
We intend to be very competitive on cost, power, hardware, TCO, whatever it is. Custom-built silicon+hardware has the advantage in this space.
We are great for image embedding (and audio, with more to come!) There is no reason you should be forced to use graphics cards intended for gaming for any AI workload.
Groq Engineer here as well; we actually built our compiler to compile pytorch, TensorFlow, and Onnx natively, so a lot of the amazing work being done by y'all isn't building much of a moat. We got LLama2 working on our hardware in just a couple of days!
Groq Engineer here, I'm not seeing why being able to scale compute outside of a single card/node is somehow a problem. My preferred analogy is to a car factory: Yes, you could build a car with say only one or two drills, but a modern automated factory has hundreds of drills! With a single drill, you could probably build all sorts of cars, but a factory assembly line is only able to make specific cars in that configuration. Does that mean that factories are inefficient?
You also say that H200's work reasonably well, and that's reasonable (but debatable) for synchronous, human interaction use cases. Show me a 30b+ parameter model doing RAG as part of a conversation with voice responses in less than a second, running on Nvidia.
Posted by
matanyal on Wednesday January 24, 2024 @10:13AM
from the ask-hn dept.
A few years back, my options for dealing with manifest where either templated yaml (helm) or overlayed yaml (kustomize). I've been playing around with Jsonnet, tanka, and cdk8s, and I realized there has to be something out there that works better.I recently discovered poetry for python, and it blew my mind with how it dealt with many of the pain points with python, I guess I'm wondering if...
Reading is one thing, but think about stuff like website generation, searching for information in massive datasets, real-time audio chats that don't sound like the AI misheard everything with a pause, and stuff like that.
It told me "Yeehaw Ridem Cowboy" was potentially problematic, which is news to me living out west.
Interesting, no mention of Groq for number 6.
Hey Jeff,
Good to see that y'all are still pushing in the right direction! I left between the layoffs to pursue some innovation in the semiconductor space, but I look back fondly on my time at Twilio (Sendgrid)!
Building APIs into businesses is no easy task, but I'm glad that there are those still fighting the good fight, even if the field changes. Best of luck to the Twilio of the future!