I believe Carl Sagan played with the idea for a fictional story, but never got beyond what he explored as non-fiction (primarily, to my mind in the book The Dragons of Eden). Also, you can argue some of Stephen Baxter's books explore the ideas, though Stephen Baxter has tried to be both historically accurate enough that he generally doesn't use fantasy terms like dwarves/elves, and smart enough to not fall into stereotype traps that you could easily point to in his descriptions of other hominids. (It's a running theme in among other things his Manifold books, and probably most interesting in hints and tertiary themes in the Long Earth young adult fiction he cowrote with Terry Pratchett, where the term elves is actually used, though not entirely based on hominids known to this Earth.)

My very lay understanding is that in the cross-over period where early Sapiens lived with other hominids you could draw some parallels to Neanderthals as dwarves (or sometimes perhaps trolls) to Sapiens viewpoints at the time and in that case Denisovans might be elves.

If you want to throw in bad pop-genetics, there'd be more "half-dwarves" among European descendants and more "half-elves" among Asian descendants, from that viewpoint.

"Malazan Book of the Fallen" [1] has multiple non-standard races inspired from ancient hominid.

[1] https://en.wikipedia.org/wiki/Malazan_Book_of_the_Fallen

The closest I can think of is William Golding's novel The Inheritors:

https://en.wikipedia.org/wiki/The_Inheritors_(Golding_novel)