skierscott

> a low confidence score

Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1].

This happens as the final operation of a neural net, and is required for training.

Softmax is a tool to make an array of positive numbers look like a probability distribution:

    out = x / x.sum()

x[i] is a class prediction, but x.sum() != 1. Say if the network was uncertain, x[cat, dog] = [0.03, 0.01]. These are small values that do not imply great confidence (the network was trained on vectors with out.sum() = 1. The network would predict “dog” using softmax because out[dog] = 0.75 > 0.25 = out[cat].

But then in inference/prediction, the confidence is ignored. What if x.sum() is small? That would imply that the network is uncertain.

[1]: https://en.m.wikipedia.org/wiki/Softmax_function

mpi4py is a solid implementation, but the docs aren’t the greatest, especially if you’re not familiar with MPI.

On iOS this isn’t functional. I can’t advance past the first slide.

>> Going forward, GitHub will remain an open platform, which any developer can plug into and extend.

Does an “open platform” mean “open source” or “a plugin system”? I think making it open source would alleviate a lot of dev concerns.

> Some states destroy the blood spots after a year, 12 states store them for at least 21 years.

What are the other 12 states?

My favorite is Alan Turings quote:

> Mathpix's AI definitely passes THIS Turing test!

Isn’t this what JITs do?

> “If you understand and agree, Apple and GCBD have the right to access your data stored on its servers. This includes permission sharing, exchange, and disclosure of all user data (including content) according to the application of the law.”

> In other words, once the agreement is signed, GCBD — a company solely owned by the state — would get a key that can access all iCloud user data in China, legally.

What user data will this decrypt? Are iMessage and FaceTime still safe?

There’s also a closed form for the Fibonacci numbers. It runs in O(1).

http://stsievert.com/blog/2015/01/31/the-mysterious-eigenval...

Not true. Infinite sums are not defined for series that don’t converge, and sum(1..n) does not converge as n => infinity.

Yes, you can play a nice trick with that sum if you ignore the fact that inf-inf is meaningless.

Where does Apple Notes store it’s files? What format are they in?

Wow, great demo. Even works on mobile!

> 2% is HUGE at this point

Hell, on MNIST 0.14% is huge. Geff Hinton created an entire new architecture to get 0.25% error, which is far better than the baseline 0.39% error [1].

To be fair, he did reduce the error by 35%.

[1]:https://arxiv.org/pdf/1710.09829.pdf

> what has changed in Python 3.6 that they're happy to do this now when they were pissed off about it a couple years ago?

Most of mine have to do with developer productovity and I didn’t find until Python 3.5.

– @, the matmul operator

– fstrings (f”{foo}”)

– parameter typing (foo: int = 0) which has exciting work with Cython

– other async features. I forget what library it was (tensorflow?) but for Python 3 it had better async support

This adds support for new compilers.

This makes Dask (pure Python paralleization) run 20% faster https://twitter.com/mrocklin/status/923581208482721794 according to Matthew Rocklin, the main developer

...I read the article in Safaris Reader mode. The font is too big for mobile.

But this is a great article.

> Was this was the anxiety that had been mentioned in the medical literature?

I don’t know. I’ve had a difference expierence with brain injury and anxiety. My recovery resulted in classic anxiety; I experienced social anxiety and depression.

Turns out generating a ML model is pretty easy too, even after training in Python. There's a simple 2 or 3 line conversion from keras/sklearn to an ML model.

http://stsievert.com/blog/2017/06/11/coreml/

I verified this because I'm a graduate student studying ML and I'll likely use this.

D’oh. I meant anonyomized (I would edit comment but can’t).

I’m doing research (not deployment) and have the same feeling. PyTorch has inspired a blog post [1], Tensorflow didn’t.

Briefly, the benefits of PyTorch are

* easy conversion to NumPy arrays (meaning rest of Python can be used!). This is a bottleneck in Tensorflow; for reasonable sizes, PyTorch is 1000x faster.

* trackbacks are easy to follow (because defines graph by running)

* it’s as fast as tensorflow [2] (or at least torch is, which calls the same C functions as PyTorch, and there’s a tweet [4] by a core dev saying to expect the same speeds). Plus on the web I’ve only found anecdotes that support PyTorch faster than tensorflow.

* it’s easy to extend; everything is a simple Python class. e.g., see their different optimizers [3]

[1]:http://stsievert.com/blog/2017/09/07/pytorch/

[2]:https://github.com/soumith/convnet-benchmarks

[3]:https://github.com/pytorch/pytorch/tree/master/torch/optim

[4]:https://twitter.com/soumithchintala/status/83545486710789734...

PyTorch does have ONNX, a tool to convert from PyTorch models to Caffe2 models: http://pytorch.org/tutorials/advanced/super_resolution_with_...

I work with data scientists and it’s scary how much they have access to.

One first hand story: this person searched a database to find tax records for a specific individual and their income. The records were de-anonymized but there’s enough data there he could figure it out.

We should design systems to preserve individual privacy. The person I heard the story from had no business in this persons tax records.

iCloud security overview: https://support.apple.com/en-us/HT202303

I'm excited for self-driving cars and ML/AI tools that help professionals (e.g., drug discovery, treatment).

The Keras lead developer wrote a post called "The future of deep learning"[1]. The podcast Partially Derivative has a good summary of it[2].

On the implementation side, I'm looking forward to distribution. Computers are all around us, sitting by idly – how can we put them to use? How can we make them secure? We're starting to see integration of ML models into mobile systems (with Apple's .mlmodel and ARKit).

[1]:https://blog.keras.io/the-future-of-deep-learning.html

[2]:http://partiallyderivative.com/podcast/2017/07/25/the_future...

[3]:http://papers.nips.cc/paper/4390-hogwild-a-lock-free-approac...

Same here.

Whoops -- thanks for that! I forgot to give that link but meant to.

It appears the largest benefits is embedding Lua code into TeX (found at [1] in "Programmability").

The example they present from that page:

    \documentclass{article}
    \usepackage{luacode}
    \begin{document}
    \pagestyle{empty}
    \begin{luacode*}
      function esn (n)
        return (1 + 1/n)^n
      end
      function etn (n)
        tex.print(string.format('%5d & %1.8f \\\\', n, esn(n)))
      end
    \end{luacode*}

    Convergence to $e$:

    \begin{tabular}{ll}
    \rule[-2mm]{0pt}{4mm}$n$ & $(1 + \frac{1}{n})^n$ \\
    \hline
    \luadirect{
      for n = 10, 110, 10 do
        tex.print(etn(n))
      end
    }
    \hline
    \end{tabular}

    \end{document}

which allows a pretty table to be printed in the document.

The details: https://puri.sm/shop/librem-5/

> that can also become a full desktop computer with an option for a compatible keyboard, mouse, and monitor ... It can be a desktop computer and phone all-in-one.

I'm interested to see their solution. How useable will the interface be?