By contrast, Python is more dynamic, and so the refactoring in PyCharm is pretty weak. It usually catches some of what it needs, but I often need to find the rest. It does appear to be the case that PyCharm is doing flow analysis to infer types. I.e., even PyCharm is doing more than regex matching.
The former is easy, the latter is impossible with regexes only. Which tells you that they are very different things. For refactoring you need not only a proper parse, but also the ability to "look things up in symbol tables". Which is what I said: There are good reasons to run a proper language frontend from the IDE. But syntax highlighting isn't one of these reasons. And refactoring isn't possible with only a simple parse without using symbol tables.
> Statically typed languages require semantic analysis and symbol table lookups, so they are out.
Isn't that the typechecker, not the parser?
Anyway I'm just posting now as I first read this comment when you posted it and tried to recall this blog post by the infamous Steve Yegge, and just finally did: http://steve-yegge.blogspot.com/2007/06/rich-programmer-food...
IntelliJ highlights variables etc. differently if they're unused and can highlight different identifiers in different colors. Not sure about other current IDEs.
That said I never felt like I needed such features, normal syntax highlighting does the job pretty well for me.
Variable renaming respecting scope, function renaming that updates the name at the call sights but ignoring overloads, function extraction... and so on. You may not use it or indeed use eclipse at all.
I can't tell you about the details, clearly something going on beyond syntax highlighting via keyword regex matching. With a beefy enough box it all works well enough.
Now, there are valid reasons for running a proper language frontend from the IDE: Error reporting of all kinds, not just syntax but also type errors and whatever else the compiler likes to complain about. But there the IDE should not try to replicate the parser. Parsing only a context-free surface language will not catch type errors, exactly because enforcing a static type system requires one to "look things up in a symbol table". So the actual compiler should provide an IDE-friendly mode where it runs its frontend and reports errors back, and the IDE should not try to roll its own version of this.
In other words, the IDE is not relevant either way.
> Any programming language that is massively adopted is context free?
Statically typed languages require semantic analysis and symbol table lookups, so they are out.
And yet we have context-free grammars for all widely adopted statically typed programming languages. And we also have additional semantic checks for all of them. The notion of a "context free programming language" being one that has context free syntax and no semantic checks at all is not a useful notion. Don't worry about it while designing your language.