1. This is what the DOM is, to some extent. All those components on the page are “native” components, we have declarative markup for them (HTML) and we can script them. They are “rendered” by native code in browsers which are some of the most highly optimised codebases around.
2. The API design for JS and for higher performance systems is quite different. Take UITableViewDataSource for example, it does view prefetching, caching, object reuse, etc. As of iOS 12, this also feeds into the CPU throttling control to ensure enough compute is available on a cell fetch that it will meet the 8-16ms deadline so that no frames are dropped. This is just a totally different way of thinking about APIs and performance that the JS ecosystem is not designed to work with.
It’s possible that a total rewrite of that stack could provide significant benefits, but I think that would be more from removing legacy cruft in the DOM, than from it being a paradigm shift.
As for the DOM, again I think that while it's true that native HTML elements are highly optimized, many applications have much larger performance goals in mind and HTML/DOM has proven to be an ineffective system when trying to do very specific performance optimizations. Check out flipboard.com on your phone for an example of how much smoother a web app can feel when you bypass the DOM and let React take over the draw calls.
For one, there's a lot to be learned from immediate-mode UIs (I'm thinking Dear IMGUI, not Unity3D's GUI); ie state inside components generates complexity, which is a bad thing. Your views should be nothing more than a pure FSM transforming app state to UI directives. IMGUIs have a ton of tricks to simplify UI development. With these in mind, FP becomes a much more interesting paradigm than OOP for this. The same logic applies whether you render the entire UI 60 times a second or you cache the previous render to only do the minimum amount of updates.
A good abstraction is one where features can emerge from its design. For example you rarely have to test your views when they're just a simple pure transform. You don't need an entire framework and command-patterns to implement undo when your app state is immutable and modelled as a sequence of actions. With each emergent feature you remove the need for tons of code and tooling; that yields agility.
I still believe most web application are much more complex than they should be, especially on the UI level, regardless of the framework used. And until we solve that problem, I don't think the web stack is the way to go for native. We're just replacing one set of problems for another one.
I strongly believe we shouldn't make things easier, but simpler.
So you believe such a UI API should be immutable (whatever that means in this context), and modelled as a sequence of actions? I think that's fine, but probably not practical in reality. I don't see the point of having a debate about the merits of functional programming vs. OOP in the middle of your post. People are going to build tools around the API that will abstract away whatever semantics you feel are best in order to fit their own needs. So the goal is to build an API that's simple at it's core to implement and use.
Inevitably someone actually has to go and code the adapter layer between the protocol receiver and the actual low level rendering layer (be it OpenGL, UIKit, etc.) and unless you build a system that's easy to understand on that end as well, you're gonna end up with an API no one actually likes or wants to use.
I don't think it's unreasonable to expect that the low-level rendering engine (ie. an implementation in UIKit) maintains some level of UI state. That is - after all - where most of the optimizations will come from. And that's how UIKit, OpenGL and most other rendering systems work. Things generally aren't immutable because it's a.) not efficient, and b.) a nightmare to work with at the lowest levels. It seems to me the appropriate abstraction is build the protocol around a UI state tree, because that's what everyone else is already working with. This is incredibly easy to maintain on either end, and higher level libraries can easily remove the pain of UI state management (a la React, Vue, etc.)
Thats how I've been building UIs for the last few years. `re-frame` in ClojureScript for one is a fantastic framework built on top of these concepts.
> I don't see the point of having a debate about the merits of functional programming vs. OOP in the middle of your post
I made that distinction to highlight the differences between retained-mode UIs versus immediate-mode UIs. There are more similarities to OOP vs FP than differences here.
Think of it this way; the application itself already contains all the state it needs. A retained-mode UI will deep copy that state into the UI components while an immediate-mode UI will make the UI state a function of the application state. There is huge value in doing that because it dramatically simplifies the UI stack, this translates into shorter iteration times, simpler reasoning and less bugs. Its no good thinking about higher level tooling if the fundamental API is overly complex.
> unless you build a system that's easy to understand on that end as well
We shouldn't care about whether something is easy or difficult; we're only going to learn it once after all. We should instead care about whether its simple or complex, because that is a direct function of our ability to reason about things once they are learned. Most things that are easy to learn generate more complexity than simplicity and I believe this is harmful in the long term.
> I don't think it's unreasonable to expect that the low-level rendering engine (ie. an implementation in UIKit) maintains some level of UI state.
You're right, its perfectly reasonable and precisely what React does under the hood. Where I disagree is that I don't want the cached UI state to leak into the components. Just like the current state of the DOM is irrelevant when rendering the virtual DOM. Its purely an optimization mechanism and one which should be allowed to evolve independently from the higher level components.
> Things generally aren't immutable
Then why are so many things moving towards immutability? Modern video game renderers push really hard to be stateless, to be completely decoupled from the game threads, to not leak cached state into entities and a bunch of other FP-inspired concepts.
The end result is incredibly more parallelism. The rendering pipelines are much simpler and performant. There are quite a few advantages to this approach. It also generates optimal memory layouts and accesses, these are simply emerging from such a design. Its much easier to have linear memory accesses if you constantly recreate the display lists in fresh memory. Updating a state tree is much more complex than creating a display list and it greatly cripples the potential optimizations.
> higher level libraries can easily remove the pain of UI state management
That is not the feeling I get when I look at the average React project; most components use getState/setState, need to touch the virtual DOM for almost every operation, are needlessly maintaining two views of the same state (app and UI) and whatnot. Or they go full Functional Components and the result is a huge stack of component layers from hell.
> It seems to me the appropriate abstraction is build the protocol around a UI state tree
I believe this is fundamentally wrong; the appropriate abstraction should be built around the application state. You can derive UI state from that, and doing so liberates you from having to maintain the UI state independently.
I think Cordova could be leveraged at the very least.
A hashmap of nodes on screen is still very high level; you're basically replacing the DOM with data, yet keeping the exact same structure. But now you lose static types and everything is slower.
Redrawing doesn't happen at the node level either; that would be terribly inefficient. Things are instead batched together.
You don't have to compile your styles to CSS currently; just write them as JS objects and call the CSS constructors yourself from code. You'll quickly find its not productive for most styling.
I simply don't believe the vast majority of developers to be able to properly handle rendering at that level. No offence but all your points about low-level rendering is not how it actually works. Its incredibly easy to fool yourself into thinking you wrote a well-designed, efficient piece of code and learn years later every single piece of it was far from optimal.
Thats why we have layers of abstractions.
This would allow developers to get native performance out of a web-based rendering context. It would also be theoretically easy to build new renderers around the standard UI API. So you could swap out your backend between browser-native and a mobile/desktop library backed by OpenGL/WebGL, or another headless solution, etc.
Importantly, I think you need to limit the scope of this new UI standard (perhaps just to basic shapes, like how SVG is similarly limited), that way you leave UI implementations like dropdown boxes to ecosystem programmers (so you'd have a dropdown box for each React, Vue, etc. -- this sounds bad but people do this already.) Limiting the API scope also means making it easier for new renderers to enter the scene. Compare such a world to the current hellscape that is building your own HTML renderer. The API is so insane it's really only possible for big companies to take it on (Google, Microsoft, Apple, Mozilla.)
How this ties back into this post: you could create an _____ Native, which would really just implement everything you want from React Native, but without React. Then you can just plug Vue or React into the renderer and boom you're done.
Edit: Observant people will notice my ideal system resembles X11. Indeed, I think X11 is an interesting proposition in "shell" (browser, native) contexts, as shell applications are inherently window-managed and sandboxed. This allows for applications to reinvent the wheel without disrupting the fundamental UX of window management on the system OS.