The way we do things today is basically guesswork. Which suits programmers just fine, as it lets them write a lot of code. But that's not the best of all possible worlds, only the default world.
This is really only an issue if you appoint programmers to serve as systems analysts. Sadly, this is all too common among businesses today.
Remember, programmers are detailists who must speak the language of the computer, and do not see the big picture, only one small part of a much bigger puzzle. A good systems analyst is a generalist who understands the business and can empathize with the users. They have strong communication skills, and can take a big-picture perspective and articulate their perspective and solutions clearly to management. They most certainly would not dismiss user requests and complaints as "incoherent ramblings". Programming and systems analysis are separate professions that require vastly different personality types, with the consequence that programmers make very poor systems analysts.
There's a reason we don't have them anymore.
Again, gathering requirements is a time-consuming and confusing process, making the software late and often wrong, because most companies assign programmers to the systems analysis process, not people skilled in actual systems analysis. A good systems analyst has the empathy and high-level insight it takes to understand the existing business systems and the true needs of the users, and design a new system to better meet those needs -- traits sorely lacking in programmers. Note that a business system is not the same as software. It incorporates all of the procedures, whether performed by humans or machine, necessary for business operations.
As a consequence, we tend to declare that nothing can be done about gathering requirements and implement "Agile" methodologies, which are nothing more than institutionalized guesswork: in the words of Milt Bryce, "do a superficial feasibility study, do some quick and dirty systems design, spend a lot of time in programming, install prematurely so you can irritate the users sooner, and then keep working on it till you get something accomplished." And the software is still late and wrong!
- go crazy in understanding the problem
- break it in small words
- repeat above steps until it's easy to write in forthI have used a similar approach when I needed to get a PoC out in 6 weeks, we made it, but not without drastically modifying "the spec". In one instance the PM wanted continuous queries over geospatial data, millions of points and arbitrary locations along with a time dimension. I changed the spec to use 3 buckets, near, medium and far. That feature took 2 hours, the continuous query itself would have taken weeks or longer.
Successful projects have a malleability to them.
This philosophy is often described by 80s lispers. They're not top-down nor bottom-up.. they lie in the middle. Trying to make a comfy intermediate layer so they can experiment various upper layers to fit the problem and adjust if rapidly in case of changes. Similar to IR in compilers I guess.