News for hackers. Stuff that matters.
Re:Mamba Explained: The State Space Model Taking On Transfor... by landryraccoon on Sunday February 25, 2024 @01:26PM (#39503327)
Why would Meta, Microsoft, Amazon and Google want Nvidia to remain dominant in hardware? Are you treating “big tech” like they all have one hive mind?
Re:Mamba Explained: The State Space Model Taking On Transfor... by behnamoh on Sunday February 25, 2024 @01:58PM (#39503626)
For MSFT, AMZN, GOOG, the competitive advantage comes from having huge datasets (that Nvidia doesn't have). It's a symbiosis that benefits the data-rich and GPU-rich players.
Re:Mamba Explained: The State Space Model Taking On Transfor... by pixl97 on Sunday February 25, 2024 @02:49PM (#39504121)
This still makes little sense as that scale will always matter. If you can drop the compute cost of a model by 10x it means you can increase model integrity/intelligence/speed etc beyond what your compute bound competitors have.

Simply put, for the time being huge datasets are going to be needed and those with bigger (cleaner?) datasets will have a better behaving model.

Re:Mamba Explained: The State Space Model Taking On Transfor... by fauigerzigerk on Sunday February 25, 2024 @03:03PM (#39504232)
Where is the symbiosis? If data is the differentiator, how do the data owners benefit from Nvidia eating into their margins?
Re:Mamba Explained: The State Space Model Taking On Transfor... by coldtea on Sunday February 25, 2024 @08:50PM (#39506845)
It's sumbiosis based on a common factor they both appreciate:

- the data/processing having to be large means the data-owners have a benefit

- the data/processing having to be large means NVIDIA also has a benefit (sells more GPUs to handle all that load)

Re:Mamba Explained: The State Space Model Taking On Transfor... by fauigerzigerk on Monday February 26, 2024 @03:08AM (#39508690)
Data owners are benefitting from having access to data while others don't.

High processing cost is not a benefit to them at all. It's just a cost eating into their margins.