Mostly I don't consider that I have great talent for language design.(OT, I just learned that that Eric Schmidt was a co-author of the 1975 version of lex.)
Yacc and lex (and even more modern versions of the parser generators) have good "educational" value, but are far from being the silver bullets in practice.
Any compiler I write these days would do what you are saying--recursive descent. There is a nice technique called "chart" that helps with this.
The crux of it is that it's a macro-expansion language, which means that you pretty much need to do the kind of stream-processing approach TeX uses, since you don't know the expansion of a macro up-front, which in turn can very well change the meaning of tokens downstream of that again. For an (admittedly extreme) example of this, consider TikZ; it's basically an entire new programming language, implemented purely by TeX macros.
I sometimes compare it to programming in Common Lisp, but you all you have available are the primitives (except lambda, I supppose) and defmacro.
First, you have to learn the DSL's of these tools (which are non-trivial in detail). Second you have to integrate them into your toolchain. Third - and that is my biggest complaint - you have to connect the Parser combinator's AST to your domain AST. The last thing is pretty amusing as I consider it the main purpose of a parser.
In practice (hopefully after evaluating the need for a custom data format as it might be possible to hijack existing languages and standards) I either use a combinator-parser (if FP available) or an ad-hoc recursive descent parser. Maybe I consider a lexer but the representation of tokens is also a non-trivial decision.
One example: shell aliases. The shell syntax in POSIX is defined in the standard lexer/parser divide (which is already a bit of a pain, as a significant of the parsing logic is needed to correctly tokenize command expansions). When you encounter an alias expansion (which since it has to happen in the command position, doesn't happen until parse time) you expand the alias, and then need to re-lex the results.
In practice, many mature compilers and interpreters use a recursive-descent parser rather than any of the LL/LR/LALR things I learned in school.
Common Lisp actually defines its syntax in terms of a recursive-descent algorithm, and that's a language that largely came out of academia.
It was, for example, where I learned what an "em" is, the difference between a hyphen and an em-dash, what a kern is, what italic correction is, what leading is, and on and on.
But I found the language description not very clear. There are passages in the TeXbook that refer to different stages in processing as the stomach, mouth, gullet, etc. I later read an offhand comment that Knuth did not use a conventional lexer/parser setup in implementing TeX, and that decision made TeX more ad hoc from a language point of view. This, in turn, may have made the macro expansion setup more complex, with its expandable vs. not expandable tokens, restricted modes, etc.
I wonder if others who have more experience with language design had the same reaction?