It might help you to dive into SD and comfyui. Loras, finetunes, embeddings, etc. Are easier to understand from a practical standpoint when you can visually compare their output. That will give you intuition for how similar layers work for LLLMs.