I think the prompt does add value, but its about wrapping the LLM + Prompt into a journey that works well for the intended use case. I think the most elegant implementation so far of this is: - notionai - blenderGPT

Both are embedded right where you need them, and they are tuned to provide great results. Another ChatGPT with a better prompt probably isn't much of a moat.