On Android you can simply run vanilla llama.cpp inside a terminal, or indeed any stack that you would run on a Linux desktop that doesn't involve a native GUI.

Yep, termux is a good way to do this. Llama.cpp has Android example as well, I forked it here GitHub.com/iakashpaul/portal you can try it with any supported GGUF/Q4+Q8 models

There's an app called Private AI that will let you run models locally on Android. It has a few smaller models available for free to try it out, but the larger models like Llama 3 (or the option to use your own downloaded models) require a $10 unlock purchase.

You can either modify the Android example inside llama.cpp or my fork of it at GitHub.com/iakashpaul/portal

Increase the Ctx to more than 100 & link to any Q4 GGUF of 7B