joaogante

> Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior?

It can be done -- it is the basis for assisted generation and related work. It does require full access to the model, to be time and money-efficient. See https://huggingface.co/blog/assisted-generation

Disclaimer: I'm the author of the blog post linked above.

vs code: https://marketplace.visualstudio.com/items?itemName=HuggingF... vim: https://github.com/huggingface/hfcc.nvim

A 3090 (or any GPU with >=20GB VRAM) can run StarCoder with int8 quantization at about 12 tokens per second, 33 with assisted generation -- which will come out for StarCoder in the coming days.

When 4-bit quantization comes out, I would expect a GPU with 12GB VRAM to be able to run it.

Disclaimer: I work at Hugging Face

The models are on huggingface now -- https://huggingface.co/models?other=galactica

Portuguese living in Lisbon here -- I deeply agree with the parent comment. Prior to covid, it was very common to hear stories where families were to put all their savings into new mortgages to buy more AirBnBs.

I also see a bright future in Portugal. Many young people that moved out after university want to come back, after making a killing in their career. They might earn less with the move, but you don't need a six digit salary to live very comfortably here.

> Pick your 10 favorite GitHub project with more than 10k stars and see who the primary contributors are.

Hugging Face transformers has >60k stars and fewer than 30 employed maintainers, many sharing other responsibilities. Arguably, part its success comes external model contributions from FAANG companies (among others), but the key ingredient was the creation of an open platform.

Disclaimer: I work there