apw

I found these slides very helpful:

http://fodava.gatech.edu/files/uploaded/DLS/Vapnik.pdf

In particular, I was unsure after reading the original article whether the additional information--for example, the poetry--was available to the learner on test inputs. The above slides explicitly state that it is not.

One possibility is to use the neuromorphic chips as souped-up branch predictors -- instead of predicting one bit, as in a branch predictor, predict all bits relevant for speculative execution. This can effect large-scale automatic parallelization.

See this paper at ASPLOS '14 for details:

http://hips.seas.harvard.edu/content/asc-automatically-scala...

The IBM Blue Gene/Q cannot in any reasonable way be described as "racks of PCs running Linux with souped up network cards".

I thought the Owhadi et al. paper was about model mispecification; i.e., the true model is not in the hypothesis space. That's pretty fundamentally different--and far less of a problem--than gradient descent's "sensitivity to initial conditions".

It's not clear that a nicely formatted error message is preferable to a core dump.

With a core dump one can explore the execution environment at the time of the crash.

A nice compromise is a macro that prints an error message and then calls __builtin_trap().

I don't think that "it can be emulated by computers" follows from "intelligence is a purely physical phenomenon".

Even fairly simple quantum systems (which are "purely physical phenomena") cannot be emulated by any classical computer in any meaningful sense, since the computational complexity of integrating the dynamical equations is exponential. Even if we could recruit all the atoms in the known Universe, we still couldn't build a classical computer capable of emulating many simple quantum systems.

It's quite likely that he was using `perf':

    $ perf stat make
    ...
    116,222 page-faults               #    0.046 M/sec
    ...

https://perf.wiki.kernel.org/index.php/Tutorial

I'm surprised someone hasn't written a Markov Chain-based Yik Yak post generator.

It would learn all the worst insults at a particular school, then apply them to the entire student body at random intervals. After a while, nobody could tell the actual malicious posts from the random posts, so all would be ignored.

Or so I would hope.

While the visualization is attractive, my fear is that it cannot convey the deeper reason why those sine and cosine waves magically sum to the desired function.

Leaving rigour aside for the moment: think of functions f : R -> R as infinite-dimensional vectors. The integer harmonics of sine and cosine comprise a set of orthonormal "vectors" that form a basis for all functions on R (some fine print goes here).

Now compute the inner product of your desired function with every element of that basis. Each such inner product is a real number which we will call a coefficient. The list of nonzero coefficients, once you have computed them, is a complete description of your function.

Now it is clear why those sine and cosine functions "magically" add up to your desired function, since we are simply multiplying each of them by their corresponding coefficient that we computed above.

That visualization is no more (or less!) amazing than the fact that (1, 2, 3) = 1(1, 0, 0) + 2(0, 1, 0) + 3(0, 0, 1).

If you can express your workflow as a set of dependencies (granted, not all workflows are easily expressed this way), make gives you parallel and incremental computation "for free".

Imagine that you needed to download a tar archive, unpack it, then run several simulations followed by regressions followed by figure plotting on the data. You could write a shell script to do this, but it would be hard to make the shell script simulate the capabilities of `make -j', and you'd have to do a lot of timestamping and file existence checking to simulate the incremental computation capabilities of make.

For those looking for some details behind how Crazy Stone actually works, this set of slides may help:

http://remi.coulom.free.fr/JFFoS/JFFoS.pdf

The codebook of a lossless predictive coding system is exactly "say[ing] the same things in different words".

The dead forests you mentioned may have became massive coal beds via the "charcoal route"; i.e. a global firestorm may have incinerated essentially the entire biosphere.

    The global debris layer created by the end-Cretaceous 
    impact at Chicxulub contained enough soot to indicate
    that the entire terrestrial biosphere had burned.

Source:

K-Pg extinction: Reevaluation of the heat-fire hypothesis; Robertson, Lewis, Sheehan & Toon, 2013; Journal of Geophysical Research: Biogeosciences

http://onlinelibrary.wiley.com/doi/10.1002/jgrg.20018/abstra...

I checked the FAQ, but didn't see an answer -- how do you prevent malicious actors from returning bogus data?

What prevents some enterprising high school student from purchasing some Atrazine off Amazon.com, catching a few hundred tadpoles from their local pond, and doing their own study?

Its worth nothing that as this arms race proceeds, the historical record of Bitcoin transactions becomes harder to undo, and 51% attacks become too costly for even some nation-states.

Could you please elaborate on "and surprisingly it needs to be intrinsically worthless"?

Bitcoin is far more than "just" a solution to the Byzantine Generals problem.

Inside every Bitcoin transaction is a small program that contains no loops. If that program when executed returns true, the transaction is valid.

The simplest, most boring kind of transaction--but the one that most people assume "is Bitcoin"--is a transfer from address P to address Q. Where things get more interesting is in using those little programs in more creative ways; e.g., k-of-n escrow, attestation of ownership, etc.

A whole generation of kids is going to grow up soon routinely using small Bitcoin escrows to hedge everyday scenarios. They won't quite able to believe that their parents operated in a world based on centralized authority subject to rent-seeking corruption.

It's possible that someone has found a way to speed up scrypt. We could solve for a lower bound on what that speedup must be.

Since most transactions are on the order of 1/1000 of a Bitcoin, it's useful to rephrase what you said as: around 150 million mBTC are being transacted (on the blockchain) per day.

This is only true if you use the simplest Bitcoin transaction.

However, Bitcoin supports programmable m-of-n transactions, which enable highly nuanced levels of risk hedging.

I suspect that it is only a matter of time before Silicon Valley (and elsewhere) innovators realize this and start providing smooth "select your desired level of risk" services.

You make a good point. It's easy for me to criticize Bitcoin from the standpoint of assuming it is the official currency of some country, or the world's only currency.

However, in so doing I am unintentionally ignoring the much more relevant and likely possibility: that Bitcoin doesn't take over the world, but is simply so easy and useful for the cases you described that it "just" becomes the "Internet's currency".

This softens my criticisms of how a deflationary currency carries risks, since in this scenario Bitcoin is continually being converted to and from other currencies.

Will you need to change that definition if I show you a machine learning algorithm capable of significantly outperforming the best human algorithms on the resume classification problem?

Thank you for your comment. Very interesting points.

Isn't it wrong to say "compiled their C code at the lowest optimization level" though? As I read the paper, they compiled their C code at -O2. It's hard to work backwards in time to know exactly what that meant when they published their paper, but I suspect it meant something very similar to the current gcc definition:

-O2 Perform nearly all supported optimizations that do not involve a space-speed tradeoff.

It's not clear to me why this author, and many others, seem start with the assumption that Bitcoin has become the only currency in a country (or the world), then proceed to list all the bad things that would occur.

It seems much more useful, and realistic, to list all the bad things (or good things) that will occur should Bitcoin become, say, 1% of the world's money supply.

You might want to read about Dynamo:

http://www.hpl.hp.com/techreports/1999/HPL-1999-78.pdf

"Contrary to intuition, we demonstrate that it is possible to use a piece of software to improve the performance of a native, statically optimized program binary, while it is executing. Dynamo not only speeds up real application programs, its performance improvement is often quite significant."

Good point, the minimum security of Bitcoin is that of gold; i.e. a safe guarded by people with weapons.

But since Bitcoin is digital, its maximum security is far, far higher. Your private keys can be encrypted and safely stored with any desired level of redundancy. Paper wallets are a big step backwards.

Also, if secret sharing is used, Bitcoins cannot be stolen unless the attacker gets to k of the n involved parties.

MPI has a reduce operator. On some supercomputers, the operator is implemented in an ASIC.

It is quite easy to implement map/reduce style programs in MPI. It is quite difficult or impossible to implement the parallel algorithms that win the Gordon Bell prize, almost all of which are implemented in MPI, using Hadoop.