Yes, though how you do it depends on what you're doing.

I do a lot of training of encoders, multimodal, and vision models, which are typically small enough to fit on a single GPU; multiple GPUs enables data parallelism, where the data is spread to an independent copy of each model.

Occasionally fine-tuning large models and need to use model-parallelism, where the model is split across GPUs. This is also necessary for inference of the really big models, as well.

But most tooling for training/inference of all kinds of models supports using multiple cards pretty easily.

Yes, multi-GPU on the same machine is pretty straightforward. For example ollama uses all GPUs out of the box. If you are into training, the huggingface ecosystem supports it and you can always go the manual route to put tensors on their own GPUs with toolkits like pytorch.

Yes. Depends what software you're using. Some will use more than one (e.g. llama.cpp), some commercial software won't bother.

I just made a video on this very thing: https://youtu.be/JtbyA94gffc