News for hackers. Stuff that matters.
Re:Jupyter Notebook 5.0 by minimaxir on Tuesday April 04, 2017 @01:21PM (#14035092)
For those who use R, I strongly recommend looking into R Notebooks (http://rmarkdown.rstudio.com/r_notebooks.html), as there is a lot more versatility involved, especially over the Jupyter/IRKernel approach. Although it's R only, unfortunately (you can run Python code in it but not the way you expect)

I'd like to see some things ported into Jupyter from R Notebooks, like JavaScript data tables and the separation of code and output, making it easy to version control only the code. (Atleast this 5.0 release makes tables nonugly)

Re:Jupyter Notebook 5.0 by carreau on Tuesday April 04, 2017 @01:26PM (#14035158)
It would be great to sit with the Rstudio-notebooks developers and user and discuss how we could get the two platform to converge and be more interoperable. I guess both team have a lot on their plate, and we would need more manpower, but understanding and collaborating with other project for the good of user is always something the Jupyter team is happy to do.
Re:Jupyter Notebook 5.0 by karthik_ram on Tuesday April 04, 2017 @10:15PM (#14039122)
I'm happy to connect you anytime! Just come by the west wing of the office. :)
Re:Jupyter Notebook 5.0 by imcoconut on Tuesday April 04, 2017 @06:11PM (#14037714)
The jupyter team has been working on jupyter lab which is a more r studio style environment. It incorperates notebooks, terminals, file editing, traditional ipython console, plus some other cool stuff in a web ide. You should be able to use an R jupyter kernel with it.

It's in alpha right now but they're making crazy fast progress on it. I've been using it and it's awesome

Re:Jupyter Notebook 5.0 by carreau on Tuesday April 04, 2017 @07:17PM (#14038183)
Yes I know I'm one of the Jupyter dev :-) there is still a difference in notebook format and protocols. In that sens JupyterLab is not more compatible with Rstudio than the normal notebook.
Re:Jupyter Notebook 5.0 by imcoconut on Tuesday April 04, 2017 @09:06PM (#14038810)
Lol, sorry - well hopefully someone else reading this thread can find out about jupyter lab. it really is amazing.

Thanks so much for all your hard work!

Re:Jupyter Notebook 5.0 by nerdponx on Tuesday April 04, 2017 @07:18PM (#14038191)
Didn't Yhat already try this with Rodeo?
Re:Jupyter Notebook 5.0 by cercatrova on Tuesday April 04, 2017 @07:57PM (#14038427)
Something like Beaker? (www.beakernotebook.com)
Re:Jupyter Notebook 5.0 by robbles on Tuesday April 04, 2017 @01:53PM (#14035431)
What does "a lot more versatility" mean? Just wondering what the advantages are.

Or do you just mean that R Notebooks are specifically better than the R support in Jupyter?

Re:Jupyter Notebook 5.0 by minimaxir on Tuesday April 04, 2017 @01:58PM (#14035482)
In addition to what I've already said, R Notebooks have native support for widgets, autogenerated TOC, and themes (although all of those can be added to Jupyter I believe). There's also the meta-support from being embedded in an IDE. (package management/variable explorer/etc)

Here's an example of one of my notebooks with all 3 things: http://minimaxir.com/notebooks/breach-network/

It's definitely a better fit for R.

Re:Jupyter Notebook 5.0 by halflings on Tuesday April 04, 2017 @02:49PM (#14035988)
Thanks for the example! That definitely includes a bunch of things I think Jupyter should have, but "purists" might consider out of scope (theming, ToC, something as simple as being able to hide cells interactively).

Currently, Jupyter works well for development, but the result is often hard to read because code gets in the way. (unless you use nbconvert, but that sometimes defeats the purpose)

Re:Jupyter Notebook 5.0 by ciupicri on Tuesday April 04, 2017 @02:10PM (#14035622)
Jupyter notebooks have widgets too, created with combination of Python and JavaScript.
Re:Jupyter Notebook 5.0 by jonathanpoulter on Tuesday April 04, 2017 @03:38PM (#14036423)
Some widgets are pretty advanced, I saw a presentation by a developer from Bloomberg, in NYC, about their bqplot widget library.

https://github.com/bloomberg/bqplot

They even have gamepad/controller support, which allows dynamic interactions with widgets.

Edit: Live demo of the project from PyData London 2016: https://youtu.be/eVET9IYgbao?t=27m45s

Re:Jupyter Notebook 5.0 by skierscott on Wednesday April 05, 2017 @08:36AM (#14041613)
And this depends on ipywidgets (or at least similar - Syvian Corlayhas developed both).

These widgets support embedding in other contexts like static HTML pages or Sphinx docs [1]. An example can be seen on the docs [2]

[1]:http://ipywidgets.readthedocs.io/en/latest/embedding.html

[2]:http://ipywidgets.readthedocs.io/en/latest/examples/Widget%2...

Re:Jupyter Notebook 5.0 by cgriswald on Tuesday April 04, 2017 @03:01PM (#14036072)
I believe you can also embed one inside an IDE. I am pretty sure you can do this with PyCharm.
Re:Jupyter Notebook 5.0 by peatmoss on Tuesday April 04, 2017 @03:28PM (#14036335)
Well, for me the big one is that R Notebooks work cleanly with revision control systems like git.

That said, both Jupyter and R Markdown Notebooks are but a pale shadow of the support offered by Org-mode (seriously!).

Re:Jupyter Notebook 5.0 by jacquesm on Tuesday April 04, 2017 @04:48PM (#14037046)
> That said, both Jupyter and R Markdown Notebooks are but a pale shadow of the support offered by Org-mode (seriously!).

Yes, but the large number of excellent notebooks available for Jupyter (and R as well) all over the web as well as support for CUDA and all kinds of extremely powerful libraries such as tensorflow) give those a serious edge over Org-mode, even though Org-mode is super powerful by itself.

Re:Jupyter Notebook 5.0 by dragandj on Tuesday April 04, 2017 @05:26PM (#14037363)
I run my Clojure + OpenCL + CUDA code from Org-mode without problems.
Re:Jupyter Notebook 5.0 by jacquesm on Tuesday April 04, 2017 @06:56PM (#14038040)
On a remote machine or on your local machine?
Re:Jupyter Notebook 5.0 by taeric on Tuesday April 04, 2017 @07:42PM (#14038331)
Either. :)

If you setup a source block, you can use TRAMP to actually execute the command in a source block on a remote machine. So:

    #+BEGIN_SRC sh :dir /[email protected]:~/remotedir 
      ls
    #+END_SRC
The above will run ls on "remotemachine.com" and put the results in an output block below it.

Edit: Meant to add that you can do the same with docker. Just use "/docker:dockerId:" as the dir, and it will execute in a docker instance locally. Using multihop addresses, this can get extreme.

Re:Jupyter Notebook 5.0 by jacquesm on Tuesday April 04, 2017 @07:44PM (#14038343)
Oh, that's very cool. Almost QNX like!

Org mode is one of the few real literate programming tools that I'm aware of, the other one is 'Leo'.

Re:Jupyter Notebook 5.0 by gpderetta on Wednesday April 05, 2017 @06:57AM (#14041158)
that's it. I'm learning org-mode.
Re:Jupyter Notebook 5.0 by jacquesm on Wednesday April 05, 2017 @07:54AM (#14041377)
Please do a write-up.
Re:Jupyter Notebook 5.0 by taeric on Wednesday April 05, 2017 @12:06PM (#14043111)
If there are examples you'd like to see written up, I'd be interested in helping out.

I used to try complicated examples, which led to pages such as http://taeric.github.io/Sudoku.html. I'm now much more into writeups such as http://www.howardism.org/Technical/Emacs/literate-devops.htm.... (Note, I did not write the second one.)

Re:Jupyter Notebook 5.0 by taeric on Tuesday April 04, 2017 @04:45PM (#14037028)
I always feel guilty pointing out that org-mode can do this really well. :) Thanks for taking that hit!

Know of any good resources that show this for folks that don't know what we are talking about?

Re:Jupyter Notebook 5.0 by txru on Tuesday April 04, 2017 @05:36PM (#14037441)
Are there any public examples of really good Jupyter-like org-mode projects with their source?

I'm really intrigued by it, but Jupyter is much more clearly documented (org docs are downright sprawling), so I've always gone that path.

Re:Jupyter Notebook 5.0 by carreau on Tuesday April 04, 2017 @07:18PM (#14038192)
I would look at atom's Hydrogen https://atom.io/packages/hydrogen
Re:Jupyter Notebook 5.0 by txru on Tuesday April 04, 2017 @07:27PM (#14038245)
Looks like a good demo of Jupyter, but I was looking for demos like this for org-mode.
Re:Jupyter Notebook 5.0 by scottkuo on Tuesday April 04, 2017 @02:27PM (#14035792)
Re:Jupyter Notebook 5.0 by jimmcslim on Tuesday April 04, 2017 @06:58PM (#14038053)
It would be great if Microsoft would open up a bit about what their plans are for this service, to the extent that it could be something I can rely on in a commercial context.
Re:Jupyter Notebook 5.0 by smortaz on Tuesday April 04, 2017 @08:50PM (#14038716)
Hi, fair question. Lots of people are using the service in edu as well as commercial contexts. We haven't put any restrictions in that regard on it.

Other than that the plan is to keep this (or something similar) running as long as there is sufficient interest and so far there seems to be quite a bit!

Note that it does run on docker which means ultimately it's not fully "secure", but we hope to switch to hyperv-linux when it's available.

[smortaz at msft]

Re:Jupyter Notebook 5.0 by andendau on Tuesday April 04, 2017 @10:16PM (#14039127)
I hope you all can style the notebook to look as good as the way kaggle does their version. I understand if you leave it alone too.
Re:Jupyter Notebook 5.0 by rocho on Wednesday April 05, 2017 @03:43AM (#14040446)
Just the other day I discovered that there exist a kernel (SOS kernel) that allows to use Python and R simultaneously in the same notebook (on a per-cell basis) and even has primitives for data exchange between the two.

I'm definitely using this for my next project.

Re:Jupyter Notebook 5.0 by a_bonobo on Wednesday April 05, 2017 @03:53AM (#14040483)
You can also use R inside 'normal' Python notebooks via rpy2:

from rpy2 import robjects

my_dictionary_results = some_method()

names_dict = robjects.ListVector(my_dictionary_results)

%load_ext rpy2.ipython

%R library(some_lib)

%R -i my_dictionary_results doStuff(my_dictionary_results)

Image display etc. works fine

Re:Jupyter Notebook 5.0 by isolli on Wednesday April 05, 2017 @07:56AM (#14041388)
In a similar vein: Beaker notebooks (http://beakernotebook.com/)
Re:Jupyter Notebook 5.0 by jacquesm on Tuesday April 04, 2017 @02:25PM (#14035773)
Does it work in the same client-server fashion? (With Jupyter you can run the notebook on a server or the head machine of a cluster and then access the notebook remotely from your laptop or desktop.)
Re:Jupyter Notebook 5.0 by minimaxir on Tuesday April 04, 2017 @02:32PM (#14035839)
No. All local, although you could set up an Rstudio Server in theory and run the Notebook there, although I have less experience with that.

There are R packages which allow communicating with a server, mostly big data packages. (sparklyr allows you to connect to a remote Spark cluster)

Re:Jupyter Notebook 5.0 by dandermotj on Tuesday April 04, 2017 @03:02PM (#14036087)
It's very easy to set up an RStudio server. I used to run one on Azure. Took minutes to set up and you get the whole RStudio suite, including R notebooks.
Re:Jupyter Notebook 5.0 by WeDontNeedH1Bs on Tuesday April 04, 2017 @02:32PM (#14035828)
Myself and all the Data Scientists I know use Python, while the Data Analysts in my company use R. Does R have anything like the Natural Language Tool Kit that Python has? Or SpaCy for fast NLP? Also, between PYKE, ProbLOG, and python-constraint, a lot of my logic programming needs are satisfied in Python. Does R have anything like that for logic programming?
Re:Jupyter Notebook 5.0 by minimaxir on Tuesday April 04, 2017 @02:35PM (#14035870)
Not as effectively. Which is why I use R for tabular data and Python for text/image/nontabular data.

And there's nothing wrong with knowing and using both languages.

Re:Jupyter Notebook 5.0 by WeDontNeedH1Bs on Tuesday April 04, 2017 @03:48PM (#14036511)
I think that makes sense and is why our Data Analysts use R.