Re:Ultra Fast Bert
by Y_Y on Wednesday November 22, 2023 @11:44AM
(#38381576)
You can't get flops on a Hailo-8, they're fixed-point only. As much as these specialised inference chips are cool, we're a long way from just being able to drop them in where a GPU was. Not to mention the memory is hugely constrained. The Hailo chips I've worked with were all limited to 20MiB for the weights which is a squeeze even at 4-bit.
[ Parent
| on HN ]
There are no comments on this comment yet.