News for hackers. Stuff that matters.
Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Tuesday April 16, 2019 @07:47PM (#19678362)
Wow, it seems to work really well. After giving it a small test drive, I'm impressed. Solid work from Mozilla, and a pretty informative article too!

That said, between this, iodide and ObservableHQ, I guess I give up. The browser is the new OS.

My question now is - how can we make browser environment to be more like Emacs (bear with me)? My main complaints are:

- Browser ergonomy absolutely sucks, and there's nothing you can do about it. You're at the mercy of each service's UI, which usually means you'll be clicking a lot and maybe sometimes get a keyboard shortcut here and there. Forget about advanced features, or consistency between sites.

- You have near-zero control over your computing environment. Between sandboxing and half the code being obfuscated blob of transpiled JS (the other half of the code is on the server), you can maybe automate the UI a little with userscripts and fix it up with userstyles.

- There's near-zero interoperability. Unless service authors agree on and implement some APIs, things can't talk to each other. Forget about making them talk. Whatever little wiring you can sometimes do (thanks to services like Zapier and IFTTT), you never have control over the process, and it always involves communication through third-party servers, even if all you'd like is to transfer some local data between browser tabs.

If the browser is the next OS, can we make it suck less than desktop OSes in terms of ergonomy / productivity? Desktop OSes already suck in this regard compared to the promise of old Smalltalk and Lisp systems (that's why I live in Emacs most of the time), so all I see is a downwards trend.

Current state of things is fine for casual use and casual users, but if I'm to spend 8+ hours a day doing serious work in a web browser, the browser needs to be better.

Re:Pyodide: Bringing the scientific Python stack to the browser by pkghost on Wednesday April 17, 2019 @06:59AM (#19681433)
Hypothesis: it would be better to live in a world where UI is mostly user-authored, and B2C is mostly an API marketplace.

In other words, imagine that instead of being forced to digest a plurality of static and unforgiving view hierarchies from designers on high, you have with a single, long-lived, personalized UI that is flexible, changes with your needs, and allows you to integrate new data and services alongside or into existing workflows. Think of it as a personal UI agent.

I like this model of software primarily because of how much creative power and agency it gives end users. "B2C" could end in "creator" instead of "consumer". It's unix-y in its composable nature, and more in line with the hopes we had for the web's ability to unlock creativity in its infancy.

It also seems like a step towards a world where we see fewer ads, because there's more of a culture of paying for digital products by usage or subscription (though ad-supported options would remain; I don't see this as a revolution against ad-supported models).

Obviously, lots of challenges to execution, but would love to hear of anyone else thinking along these lines!

Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @08:19AM (#19681848)
> Hypothesis: it would be better to live in a world where UI is mostly user-authored, and B2C is mostly an API marketplace.

I tentatively believe in this hypothesis. That's part of the reason I moved most of my computing into Emacs - I get greater control over the UI there, and reap the benefits of deep interoperability.

I think this would be a working system. You'd have a separate market for services, and a separate market for software consuming these services. I doubt most regular people would become full-blown creators, but they would happily shop for more ergonomic tools and customize them to the limit of their needs. It's kind of similar to the "right to repair", where nobody honestly expects that everyone will be fixing their appliances themselves - the point is that those who can, would, and they would also offer their services to those who can't or don't want.

I'm not sure how to get to this world. That's kind of the question in my original post - I see fixing the browser for productive use to be a potential stepping stone involving the same or similar changes.

I have one idea I'm hesitant about: it would be nice if we could force all services on the Internet to communicate with open and documented protocols and APIs, while simultaneously banning any kind of "you can only use our official app" clauses in ToS. That is, force the decoupling between a service and client software that consumes it.

Re:Pyodide: Bringing the scientific Python stack to the browser by bglusman on Wednesday April 17, 2019 @08:09AM (#19681801)
I like the idea, but I think most users never would/could author their own UI's, and many who could would only want to a small percentage of the time.... but maybe your idea could work if there was essentially an API marketplace AND a UI marketplace, and services competed on both of them seperately... v1 from a service would (probably) always offer both to be usable but if it published some contract for how they talk to each other so they were decoupled and anyone could author a new UI (and, I suppose if it were a marketplace, sell it, but maybe given constraints only selling the API is realistic and UI is more FOSS/enthusiast driven/fork based?) and make it discoverable as an alternative to try out.
Re:Pyodide: Bringing the scientific Python stack to the browser by solatic on Thursday April 18, 2019 @09:21AM (#19690769)
I think having a distinct separation between back-ends and swappable front-ends would be a good thing, but by this point, it would require government intervention to make it happen. Twitter, Facebook, etc. froze out third-party clients to protect their advertising revenue. They're not going to let third-party clients back in unless they're forced to, and such a move by the government would be tantamount to declaring war on the advertising industry, so, it's not likely to happen.
Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @08:23AM (#19681874)
I agree with your points here, and I think two markets - service API market and client software market - would be a great thing to have. One of the things that makes a lot of services suck (social media platforms in particular) is the control they have on APIs they provide. To pick one example, I can sort of use the Facebook API for some of the things, after jumping through enough contractual hoops, but I can't really go and build a Facebook client that would have feature parity with the official one. Not without risking my account getting banned. I especially couldn't distribute such a client either.
Re:Pyodide: Bringing the scientific Python stack to the browser by jonrgrover on Tuesday April 23, 2019 @04:47PM (#19732675)
To do this we need to build technologies that can work with information and meaning in code in addition to data. Users think in terms of information and meaning much more than in terms of data. These technologies would allow us to build code in the language that users use. This will give us the right tools to build meaningful user interfaces easily. In other words, we need the right tools for the job. I am building one of the many tools needed for this. It is called the endeme. It is a library for c# that can be found here: https://github.com/jonrgroverlib/InfoLib. Discussion can be found here: https://thisstack.wordpress.com.
Re:Pyodide: Bringing the scientific Python stack to the browser by Pete_D on Wednesday April 17, 2019 @03:16PM (#19685553)
I think it's impossible. We lost. The incentives aren't there for interoperability and support for client-side customization to meaningfully improve - big names will resist, either because it makes it easier for users to sidestep ads and tracking, or because the kinds of features it would require aren't within reach of the minimum viable user (https://www.reddit.com/r/dredmorbius/comments/69wk8y/the_tyr...).

Legally-enforced open APIs and client-agnosticity like you mention downthread sounds wonderful (maybe we could start by deprecating the User-Agent header) but for similar reasons, I can't see it ever taking off.

Pessimism notwithstanding, here's an idea: imagine a browser extension where you could implement privileged pages/tabs that had access to other tabs' script environments/DOM, and could inject elements from other pages into itself. So you could write a workflow like:

1. open a Wikipedia page in a background tab (`const wiki = new Tab(etc...);`)

2. run some JS to pick out values from a <table> in it

3. open a Google Docs spreadsheet in a background tab

4. fill out the spreadsheet using the previously obtained data

5. as a bonus, inject the table + spreadsheet elements into the extension page for side-by-side reference

Not sure what the emacs analogy would be - maybe something like org-mode and Babel being able to generate things using the contents of other buffers.

Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @06:43PM (#19687082)
> The incentives aren't there for interoperability and support for client-side customization to meaningfully improve - big names will resist, either because it makes it easier for users to sidestep ads and tracking, or because the kinds of features it would require aren't within reach of the minimum viable user

Yup. Also, because of security - that, I imagine, will be the argument legitimizing the whole thing. I'm acutely aware how interoperability conflicts with security. Every security compromise you make to make the software be more useful is something that can be a way for users to self-pwn when subjected to social engineering attacks. My Emacs is a monster of productivity, but then again it's so niche that nobody is writing elisp malware.

> Not sure what the emacs analogy would be - maybe something like org-mode and Babel being able to generate things using the contents of other buffers.

Your example is actually something one would do in Emacs - (1) open a buffer with some data in the background, (2) copy the interesting parts, (3) paste it into a new buffer in some specific major mode that helps you (4) restructure the data, and (5) show the work; all in a single step you could bind to a key or button.

Different example of Emacs-style integration I'd like to do: run all my IMs in the browser (FB Messenger, Slack, Skype, Telegram) as background tabs, and have a resident piece of code that reacts to new messages in each and aggregates them, displaying a foreground tab with "event stream" ordered by time (and possibly a "reply" button). Kind of like IRC channel unifying all your browser IMs, with ability for you to add new ones without having to deal with OAuth and API calls to IM servers.

Elsewhere in the thread[0], 'zapzupnz says that "data can be available online without necessarily being web first. Heck, most things are — data provided to mobile apps and desktop clients via services that might have a web frontend.". Well, so how come I can't easily do automated queries to my bank to fetch my balance, put it in whatever accounting app I want to use, and display it on my personal dashboard for extra fun? The only way I can do this today is through scripting around bank's authentication and scrapping their page - which risks termination of account for ToS violations. This is less about the browsers and more about API control and problematic ToS - but it would be mighty easier to do if I could easily shuttle arbitrary data between browser tabs myself.

BTW. this thread made me realize that I'm not very clear on what a browser extension can or can't do; maybe the situation is slightly better than I think. Need to check it out.

--

[0] - https://news.ycombinator.com/item?id=19680964

Re:Pyodide: Bringing the scientific Python stack to the browser by spurgu on Wednesday April 17, 2019 @05:50PM (#19686780)
Puppeteer?
Re:Pyodide: Bringing the scientific Python stack to the browser by mdboom on Tuesday April 16, 2019 @08:26PM (#19678573)
These are some good points.

One small way in which Iodide advances this is that by having the editor being a single text editing widget (rather than multiple cells as you see in Jupyter and others), it should be easier to replace that widget with an alternative editor, or (using an extension) link to a native editor on the machine.

But all of these other issues, I agree, are things that would be nice to improve upon. I think in part this comes from so much of the "productivity on the web" stack is dominated by big players (Google Docs, Microsoft Office 365) there hasn't been a big push for interoperability and customizability. I'd love to see a movement around that (but definitely out-of-scope for what the Pyodide team can currently take on!)

Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @08:44AM (#19682012)
Sure, I'm not saying Pyodide team should take that on! You've already done an awesome job, and the whining in my comments isn't aimed at you - it's aimed at the direction computing is heading, of which your project is but a symptom. Right now there isn't much you could do; the browser platform isn't geared towards this kind of user-driven interop.
Re:Pyodide: Bringing the scientific Python stack to the browser by scribu on Tuesday April 16, 2019 @08:25PM (#19678567)
> If I'm to spend 8+ hours a day doing serious work in a web browser, the browser needs to be better.

Agreed! Jupyter notebooks are great, but I miss my Vim keybindings when editing code in cells.

(Plugins like Vimperator always collide with some native keybindings.)

Re:Pyodide: Bringing the scientific Python stack to the browser by hipplec on Tuesday April 16, 2019 @09:03PM (#19678756)
If you run EIN (Emacs Ipython Notebook module) in emacs with vim keybindings (via spacemacs or evil mode, or some other distribution/mode) you can get some pretty great results. Would definitely look into it if you're either already using emacs or willing to try it out.
Re:Pyodide: Bringing the scientific Python stack to the browser by jaster on Wednesday April 17, 2019 @03:24AM (#19680489)
I tried it at some point, but a minor inconvenience was that it kept adding superfluous metadata to the cells I was editing.

Since some of the notebooks I work on are collaborative and versioned with git, this was painful for reading diffs (even more so with the ipynb format) so I dropped it to my regret.

Re:Pyodide: Bringing the scientific Python stack to the browser by jaster on Wednesday April 17, 2019 @03:18AM (#19680466)
It's far from perfect, but I use jupyterlab with the jupyterlab-vim extension for this: https://github.com/jwkvam/jupyterlab-vim
Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @03:02AM (#19680417)
Not just keybindings. I don't know how your Vim setup looks, but my Emacs setup gives me some extra "non-default" tools that are consistent across the whole program, whether I'm editing Lisp, JS or managing files in a directory. Two that I use daily are semantic select (you press a key to expand your selection to the nearest encompassing semantic unit) and multiple cursors.

I did spend several days in a row typing code 10+ hours a day in Observable notebook (I had some idea I needed to validate and demonstrate to other people in the company). While the tool itself was amazing, the coding aspect was not.

Re:Pyodide: Bringing the scientific Python stack to the browser by esclerofilo on Tuesday April 16, 2019 @08:34PM (#19678613)
qutebrowser had a keybinding for opening things in vim IIRC
Re:Pyodide: Bringing the scientific Python stack to the browser by smartmic on Wednesday April 17, 2019 @02:50AM (#19680355)
Fully valid points. And I see no reason why we should accept the urge to use the browser for many tasks which are way better accomplished with existing solutions. After all, aren't we increasing the complexity while limiting functionality and ease of use?

Sure, if your core business is in the Cloud and browser market, this may look different, but I believe the terminal will still continue to serve the scientific computing community well…

Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @03:43AM (#19680575)
I'm getting tired of fighting an uphill battle here. Web technologies have overwhelming dev mindshare; all the cool things happen either as web pages or Electron apps now (which are even worse than web pages, because raw browsers we can upfix a bit with userscripts and plugins).

> terminal will still continue to serve the scientific computing community well

I hope so, but note how data is on the web, the output is expected to be on the web (so others can consume it), collaboration is expected to be on the web, and now you can code things on the web... Everything is moving into the browser.

Re:Pyodide: Bringing the scientific Python stack to the browser by zapzupnz on Wednesday April 17, 2019 @05:16AM (#19680964)
> the output is expected to be on the web

That's no more true now than it has been for the past 10 to 15 years. However, output is just one view; something that automatically generates HTML to display things, like scientific data, is great, but that's far from the only, or even primary, way that many things are consumed from the internet.

Other views exist that can visualise the same data in other ways. That is, data can be available online without necessarily being web first. Heck, most things are — data provided to mobile apps and desktop clients via services that might have a web frontend.

My personal thinking is that people really are too hooked on this narrative about the browser as an OS, and maybe have gotten tunnel vision from it.

The internet, as it is today, is still mainly services. Many things may be more and more accessible in the web by preference, yet remain freely available and consumable through other forms (REST APIs, direct database access, etc) and delivered through apps and integrations within operating systems. Some examples might include:

- Dropbox: files sync with the various apps and utilities - iCloud: mail, contact, calendar are mostly consumed by the native apps. Pages, Keynote, Numbers documents are mostly edited in the Mac apps. - Office 365: the Windows apps remain the dominant way to access and edit Word, Excel, and PowerPoint documents. - Github: the most popular source code repository today, but still very much consumed at the command line. The website is strongest at wiki editing, forums, and so on, but the bulk of code moving back and forth done with the `git` command or IDE integration.

Let us not underestimate how much of the internet is not primarily consumed in web browsers and likely never will be through the vast majority of certain markets (for instance, developers), where internet-located data will be consumed as services that interact with apps much more than as web sites.

Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @08:40AM (#19681977)
> That's no more true now than it has been for the past 10 to 15 years.

It is, though. 10 years ago I'd expect to get a PDF or an .XLS or a bundle of Matlab code. Maybe a static page. Today, if you can't interactively explore the data in the browser, it's considered subpar.

> The internet, as it is today, is still mainly services. Many things may be more and more accessible in the web by preference, yet remain freely available and consumable through other forms (REST APIs, direct database access, etc) and delivered through apps and integrations within operating systems

Disagree. The Internet may be mostly services, but they're services with default UIs you're forced to use, and are not freely consumable through other forms. REST APIs are restricted both in terms of features and what ToS allows you, and more often than not you couldn't build an alternative UI with feature parity to the original one.

> Dropbox: files sync with the various apps and utilities

Sorta, kinda. Can I have an alternative implementation of the Dropbox client? The problem isn't bad here though - Dropbox does one thing and does it well, i.e. syncing files with real OS-es and their filesystems. You can work with that to the extent your OS lets you.

> Pages, Keynote, Numbers ... Office 365

All being increasingly replaced by Google Docs, because it's free and you already have an account. You can't edit those outside the browser.

> Github

Exception, not the rule, and it's an artifact of the fact that developers still mostly work on desktop OSes with real filesystems. I'm worried about the future in which we'll all be using some future evolution of VS Code in the browser, communicating with future Github in the background over some APIs you can't hook into.

I'm not saying everything is in the browser now. But it sure as hell looks like in 10 years it'll all be.

Re:Pyodide: Bringing the scientific Python stack to the browser by zapzupnz on Thursday April 18, 2019 @08:08AM (#19690364)
> Today, if you can't interactively explore the data in the browser, it's considered subpar.

PDFs still remain a primary form of interchange for academic documentation. People who really want to crunch the data will want raw data files or some sort of common interchange format.

> but they're services with default UIs you're forced to use

I gave a bunch of examples where you're not forced to use them. Most of the popular web services are also available through apps on mobile.

> REST APIs are restricted both in terms of features and what ToS allows you

For public REST APIs, sure. For private use, it's still things like REST and whatnot that power desktop and mobile apps, bringing data from those internet-based services to local clients, completely bypassing the web.

> Can I have an alternative implementation of the Dropbox client?

You can, but that isn't the point. The point is that the Dropbox client, the executable program or mobile app, is how people tend to use Dropbox. If not that, people also connect to Dropbox through alternative third party apps.

Completely possibly, and I wager mostly the case, to bypass the web.

> All being increasingly replaced by Google Docs

Citation needed.

> You can't edit those outside the browser

You can, using Google's apps. They're not fantastic, but that's not the app's fault, that's Google's. Meanwhile, iWork and Office 365 let one easily edit documents with the desktop and mobile apps — and Office remains king of the hill in enterprise, whatever people may wish to believe.

> Exception, not the rule

But still highly relevant.

> I'm not saying everything is in the browser now. But it sure as hell looks like in 10 years it'll all be

It really doesn't. There's no pattern from which to base this. Apps are far more popular, especially thanks to the rise of smartphones; the web remains a convoluted mess of inconsistent looks and feels that increasingly assume a high-quality broadband connection even though there are still vast portions of the developed world that still don't have adequate internet to completely shift away from local apps; and plenty of enterprises still run on older versions of software and standards.

When one examines the whole picture, the web is far from being as ubiquitous as you imagine it to be or shall be — it's merely the next biggest alternative that happens to be huge.

Re:Pyodide: Bringing the scientific Python stack to the browser by pjmlp on Wednesday April 17, 2019 @05:29AM (#19681042)
> all the cool things happen either as web pages or Electron apps now

There are also plenty of cool things on mobile OSes.

Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @06:10AM (#19681222)
Most of which are webpages in a webview these days; mobile software is also much more skewed towards casual use due to device form factor. Here I'm focusing on more professional or even prosumer use cases.
Re:Pyodide: Bringing the scientific Python stack to the browser by pjmlp on Wednesday April 17, 2019 @06:53AM (#19681399)
Not really, plenty of native code as well.

Also both iOS and Android have very interesting architecture features, still not widespread on desktop OSes.

Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @08:25AM (#19681887)
> Also both iOS and Android have very interesting architecture features, still not widespread on desktop OSes.

Most of those are heavily favouring security at the expense of interoperability and user control. Not exactly the direction I'd like things to see heading on the desktop. Compared to laptops and PCs, mobile devices are essentially interactive TVs.

Re:Pyodide: Bringing the scientific Python stack to the browser by pjmlp on Wednesday April 17, 2019 @10:43AM (#19682986)
From architecture point of view there are plenty of interesting things to explore beyond security.

But yeah, I want to see sandboxes everywhere and also enjoy the direction that OS X and Windows are going into that regard.

Re:Pyodide: Bringing the scientific Python stack to the browser by Vinnl on Wednesday April 17, 2019 @05:09AM (#19680938)
> the other half of the code is on the server

Somewhat of a side note, but the same holds true for many equivalent native apps as well.

> There's near-zero interoperability.

This is the problem that Tim Berners-Lee's SOLID is trying to solve: https://solid.inrupt.com/

Re:Pyodide: Bringing the scientific Python stack to the browser by pjmlp on Wednesday April 17, 2019 @05:27AM (#19681026)
Well, I guess we could built up on this old idea.

https://www.lively-kernel.org/

Re:Pyodide: Bringing the scientific Python stack to the browser by TeMPOraL on Wednesday April 17, 2019 @08:28AM (#19681900)
Interesting, but that's still something to be used within a web application. Here, I'm talking about fixing up the browser. I don't see how I could use Lively Kernel to e.g. force consistent advanced autocomplete UI on every text field in every tab, or to shuttle data between two tabs harbouring applications that were not explicitly designed to talk to each other.
Re:Pyodide: Bringing the scientific Python stack to the browser by dekhn on Wednesday April 17, 2019 @09:20AM (#19682262)
The browser developers have some hard constraints that prevent them from making the browser better for desktop applications. There were a lot of battles at Google over making ssh work in the browser while supporting standard terminal keyboard shortcuts. Even today I hate working in Jupyter because I can't alt-tab-navigate the tabs (it switches browser tabs).

Everything you described is an expected consequence of inner platform syndrome. It would have IMO been better if libraries like qt made cross-platform, network-driven desktop application development the preferred software delivery mechanism.