News for hackers. Stuff that matters.

masc98

Show HN: Bertina – 3M Bert (ITA)

https://huggingface.co/mascIT/bertina-3M

Enough with LLM news. it may sound boring but I've pretrained a tiny version of BERT, approx 3M params, from scratch on a bunch of italian data (wiki mostly).According to benchmarks it performs just 5-8% worse than a fully fledged 100M vanilla ita BERT.I've used a 15k vocab. size and tweaked the hidden size. Nothing too fancy, it's been a good excercise though.

( Read More... | no comments | on HN )
it's time this project gains the visibility it deserves.
Re:Hello on Sunday November 06, 2022 @03:25AM (#33490419)
Thanks for the post. Anyways, I'm a data scientist by day and geek by night, atm I am learning Rust and I just love it, it provides me all the good stuff that make me piss off about Python. Actually, I have found out that a 30-minutes Rust session before I start working in the morning gives me a good vibe boost for the remaining 8 hours of work.
Thank you for your support guys! 5 of you have already bought the premium plan.. this is insane! I'll do my best.
Thank you for your comment. It may sound a coincidence but I've updated the pricing and bundle limits (there was a typo in the pro 30 instead of 130..) just some minutes ago.

Have a good day!