You can just get the GGUF off hugging-face. Usually googling for "<modelname> gguf" gets you to the site, then choose a quantization that works for you. But as I said, not working on Pixel 4. Should get a new screen tomorrow and can't wait to try on my Pixel 6 Pro.

Eg "phi-3 gguf" eventually take you here [1]. and you can download the Q4 quantized model on that page.

[1] https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf...