News for hackers. Stuff that matters.
Re:Do Deep Nets Really Need to be Deep? [pdf] by bainsfather on Sunday October 05, 2014 @05:15AM (#8411835)
The TIMIT (speech) benchmark they use is not freely available - you have to pay for it. It is a great shame that papers use this benchmark - how do I reproduce their results? How do I compare my methods to theirs?

Meanwhile the image benchmarks MNIST, NORB, CIFAR-10, CIFAR-100 are freely available. Kudos to the people who made them.

Re:Do Deep Nets Really Need to be Deep? [pdf] by espadrine on Sunday October 05, 2014 @09:07AM (#8412184)
Could we crowdsource a speech benchmark?

We start with one individual with a trust level of 1 (a probability of correct work in the bayesian sense). All other contributors start with a trust level of 0.

Anyone with a trust level above MIN_TRUST (say, 0.6), called a trustee, can validate others' work. This status is dynamic, such that a trustee can stop being one, invalidating all its verifications.

Valid work is work that has a score above MIN_TRUST. Such work is included in the benchmark (with a possible added check, such as a lower bound for the number of votes received).

The score of a work is the lower bound of the Wilson score confidence interval for a Bernoulli parameter with a confidence level of 95%. Given `total` the number of votes from trustees and `valid` the number of votes that claimed this was a valid work:

    z = 1.96
    z2 = z * z
    positive = valid / total
    score(work) = positive + z2 / (2*total)
      - z * sqrt((positive*(1-positive) + z2/(4*total)) / total)) / (1 + z2/n)
The trust level of each contributor is computed as the proportion of validated work (by a trustee) amongst their work, minus the proportion of invalid work. In math:

    trust(contributor) = max(0, (valid_work - invalid_work) / total_work)
Each trustee may verify as many pieces of work as they have produced themselves. They receive work randomly amongst work that is not yet valid.

A piece of work can also be discarded if it reached a certain number of votes and has a certain low score.

In this case, each piece of work would be some text read out loud by the contributor.

Re:Do Deep Nets Really Need to be Deep? [pdf] by bainsfather on Sunday October 05, 2014 @10:05AM (#8412289)
There is this: http://www.voxforge.org/home

"VoxForge was set up to collect transcribed speech for use with Free and Open Source Speech Recognition Engines (on Linux, Windows and Mac)."

I also wondered about using librivox audiobooks+text for training, maybe.

Re:Do Deep Nets Really Need to be Deep? [pdf] by jwatte on Sunday October 05, 2014 @03:23PM (#8413165)
Research costs money. You need machines, a place to work, employees need to eat and pay the rent (even if you're the only employee.) Cost of database is just another cost of research. Most scientific fields have significantly higher costs than computer science. Just be glad you're not in high energy physics...
Re:Do Deep Nets Really Need to be Deep? [pdf] by bainsfather on Monday October 06, 2014 @05:04PM (#8418294)
I'm not asking for access to their hardware (computers or particle accelerators).

I am asking for access to their data - something that costs them nothing more whether I have it or not.

Science and the free sharing of information go together.