I think something is wrong with your Spotlight index if it takes that long. Recently something went a bit funny with my Spotlight that made it take as long as you're describing, but after a reboot it's pretty instantaneous as usual. I work with large numbers of files (tens of thousands per project, with about 40-50 projects) so I don't think it can be that. I'm pretty sure there are ways to rebuild your Spotlight index.

I haven't come across issues with ordering, but my use cases are usually pretty simple, so that might be down to a difference in our workflows.