News for hackers. Stuff that matters.
Re:How to build your own neural network from scratch in Python by a_bonobo on Friday October 05, 2018 @12:23AM (#18145744)
Too often you see articles like this and they start with

$ import a_whole_bunch_of_stuff

Good to see that this is not the case here :)

The fast.ai course has a similar exercise in the beginning, but you'll still import the weights from somewhere else.

Their fast.ai v1 library has a very short implementation too (loading the MINIST example dataset and then using Resnet18):

    from fastai import *  
    from fastai.data import *  
    untar_data(MNIST_PATH)  
    data = image_data_from_folder(MNIST_PATH)  
    learn = ConvLearner(data, tvm.resnet18, metrics=accuracy)  
    learn.fit(1)
Done!

Source: http://docs.fast.ai/

Re:How to build your own neural network from scratch in Python by sprobertson on Friday October 05, 2018 @01:33AM (#18145939)
How is your example not "$ import a_whole_bunch_of_stuff"?
Re:How to build your own neural network from scratch in Python by ScoutOrgo on Friday October 05, 2018 @12:23PM (#18149388)
This notebook from fast.ai is more in line with what op mentioned. It starts from nothing but Python and slowly swaps out chunks of code with functions/classes from Torch/fastai:

https://github.com/fastai/fastai/tree/master/courses/dl1

Re:How to build your own neural network from scratch in Python by a_bonobo on Friday October 05, 2018 @02:21AM (#18146069)
should've been more explicit - deleted that part accidentally in an early edit, but yeah, that's what I would've expected, an 'import from' at the beginning
Re:How to build your own neural network from scratch in Python by MrUnderBridje on Friday October 05, 2018 @01:24PM (#18149946)
You always have to start somewhere.
Re:How to build your own neural network from scratch in Python by partycoder on Friday October 05, 2018 @01:18AM (#18145893)
How to make cake, by Russell Peters

First, you get cake. Then you make it for 20 minutes. Then you have cake.

Re:How to build your own neural network from scratch in Python by amelius on Friday October 05, 2018 @04:59AM (#18146696)
Step 1. Write function that evaluates cake. Step 2. Use genetic algorithm to bake perfect cake.
Re:How to build your own neural network from scratch in Python by zodPod on Thursday October 11, 2018 @02:59PM (#18196125)
There's a guy who makes youtube videos like this a lot. He's a good guy and a smart guy but seems to either not care or not realize he isn't helping. I've seen videos like "Computer Vision In 5 lines" where the first line is "import helperclass.py" or something like that and that helper class has like 1500 lines of code that he wrote to implement. Sure people need to be aware of what they're learning but if someone has no programming experience and finds that video they're going to just think it's magic.
Re:How to build your own neural network from scratch in Python by ShorsHammer on Friday October 05, 2018 @01:39AM (#18145955)
It's not mentioned in the article but he is importing numpy.

It has around 100k LOC.

Re:How to build your own neural network from scratch in Python by halflings on Friday October 05, 2018 @02:30AM (#18146112)
numpy is considered a low-level backbone of data science on Python; re-writing it would be equivalent to re-writing the "print" function.

OP is probably referring to somebody importing a high-level class that does something complex (auto-differentiation etc.)

Re:How to build your own neural network from scratch in Python by MrUnderBridje on Friday October 05, 2018 @04:41AM (#18146645)
You always have to start somewhere. Do you want to code all the OS from the ground up, or do you have minimal expectations about the environment your software will run on? Do you want to look at all the tiny details, or do you wish to focus on a specific aspect of the problem? The article author could also give an explanation of linear algebra, but I guess he expects the reader to be familiar with this part of the problem.

I guess that there's always a software layer which may be considered "backbone". For people designing networks, all the tedious work of building the network is just plumbing, and they probably expect it to be automated from a formal description of the network.

Re:How to build your own neural network from scratch in Python by FartyMcFarter on Friday October 05, 2018 @07:36AM (#18147278)
> I guess that there's always a software layer which may be considered "backbone".

The point is that you don't need any software layers at all to code up a basic neural network implementation. A programming language with basic floating-point operations is all you need. The algorithms are not complicated so even x86 Assembly is practical for this purpose if you're already experienced with it. So the "backbone" can simply be your favorite compiler.

> Do you want to code all the OS from the ground up

If it's a "make your OS" course then yes - a simple OS of course. If you want to become experienced in compilers, then writing a compiler from scratch for a simple language is mandatory for people who want to have solid fundamentals. It's not a coincidence that projects like these are common in Computer Science and Software Engineering courses.

Re:How to build your own neural network from scratch in Python by MrUnderBridje on Friday October 05, 2018 @01:08PM (#18149816)
> The point is that you don't need any software layers at all to code up a basic neural network implementation.

I grafted my answer to the wrong post, I meant to respond to the guy talking about the project implicitely using Numpy.

Offtopic remark: It would be nice to be able to move a post to a different thread, or make a single answer to several messages.

Re:How to build your own neural network from scratch in Python by mos_basik on Friday October 05, 2018 @01:43PM (#18150105)
>single answer to several messages

Always surprises me that out of all the different kinds of internet communities, it's only the oft-maligned imageboards that consistently provide that feature.

I was kind of expecting Discourse to play with the idea, since they seem to be the modern-reboot-of-forums with the most traction + willingness to experiment, but they haven't so far, afaik.

Re:How to build your own neural network from scratch in Python by spicytunacone on Friday October 05, 2018 @05:16PM (#18151806)
How would you represent that? On HN and similar every response is its own thread. This is true with email as well, but I don't have experience with any significant threads or mailing lists.

The naïve solution would be to just limit nesting and order nested comments chronologically. Another solution may be to CC @postid in your messages and have posts display "responding to: id1, id4" at the top and "responses: id91" at the bottom with appropriate links. There's a question here about which post to write or display your response under. Finally, an extravagant solution would be to really represent discussion as a graph and I think a lot of time would need to be spent creating an interface with minimal frustration as I feel it can get really involved (i.e., a lot of clicking).

Curious what you or anyone else thinks.

Re:How to build your own neural network from scratch in Python by maurits on Friday October 05, 2018 @04:46AM (#18146659)
Almost obligatory, but Stanford cs231n [1] lets you implement a complete (deep) neural network from scratch [2] before venturing into pytorch/tf. Its super fun.

[1] http://cs231n.stanford.edu/

[2] http://cs231n.github.io/assignments2017/assignment1/

Re:How to build your own neural network from scratch in Python by drej on Friday October 05, 2018 @01:15AM (#18145883)
I can highly recommend Joel Grus’ live coding video. He creates a deep learning library only using numpy in an hour and it’s really fun to watch it all come together. https://youtu.be/o64FV-ez6Gw
Re:How to build your own neural network from scratch in Python by lunchladydoris on Friday October 05, 2018 @03:45AM (#18146412)
That was really good! Thanks for the link.
Re:How to build your own neural network from scratch in Python by tomglynch on Friday October 05, 2018 @12:37AM (#18145770)
Towards Data Science is definitely one of the best Medium Publications. I hope they don't begin to monetise with Medium's member-only content. It's such a drawback to Medium.
Re:How to build your own neural network from scratch in Python by vesche on Friday October 05, 2018 @01:16AM (#18145886)
Nah, let's make that 10 lines: https://gist.github.com/vesche/72dafed33d614710f03f1b75cff1c...

I got a beer for anyone who can golf it under 5 lines

Re:How to build your own neural network from scratch in Python by applecrazy on Friday October 05, 2018 @01:34AM (#18145944)
Naive solution (1 line):

https://gist.github.com/applecrazy/deda2fac6e83c07b93e001731...

Edit: I literally took newlines, converted to \n in a string, and then exec()ed the whole thing. Here's a repl of it working: https://repl.it/@applecrazy/Code-Golfing-a-Neural-Net

Re:How to build your own neural network from scratch in Python by vesche on Friday October 05, 2018 @02:36AM (#18146135)
Oh dear, I got played.

PM address if you want the beer, you sly dog. Obligatory, I'm a normal dude: http://vesche.github.io/

Edit: Wait, you can't PM on here... PM on reddit /u/vesche

Re:How to build your own neural network from scratch in Python by applecrazy on Friday October 05, 2018 @09:20AM (#18147850)
Well, I don't drink (see my bio), but thanks for the offer. :P
Re:How to build your own neural network from scratch in Python by rjplatte on Friday October 05, 2018 @12:32PM (#18149480)
I will accept the beer in his stead.
Re:How to build your own neural network from scratch in Python by earenndil on Friday October 05, 2018 @01:36AM (#18145948)
Sure! Optimizing for number of lines (instead of number of characters):

* Remove 'import numpy as n' and use __import__('numpy') in place of it everywhere

* Remove s and d functions; inline them where they're called

* Get rid of 'class N', as it's unnecessary. If adding globals is cheating, then you can do '__import__('math').x = x' instead of 'self.x = x' (yes this will work and persist).

* Technically, you don't have to print the result at the end

Where do I go for my beer?

Re:How to build your own neural network from scratch in Python by vesche on Friday October 05, 2018 @02:39AM (#18146152)
Re:How to build your own neural network from scratch in Python by anonytrary on Friday October 05, 2018 @01:19AM (#18145897)
Not readable, doesn't count ;)
Re:How to build your own neural network from scratch in Python by plg on Friday October 05, 2018 @09:23AM (#18147880)
Honest question: what are the reasons to code this up using OOP, creating a neural network object plus methods, instead of data structures and functions that operate on them?

If it’s just personal preference, I’m fine with that, I’m not trying to start a flame war.

Re:How to build your own neural network from scratch in Python by claytonjy on Friday October 05, 2018 @01:13PM (#18149851)
Python, both the language and community, are very strong proponents of OOP. While you can do a lot of more functional stuff, esp. w/ functools, the community at large tends to discourage that. "Never use map/filter" is a weirdly common phrase among pythonistas. So this, like most python-driven examples, is doing things in a pythonic way.

Coming from the R side, I tend to prefer structures & functions as well, but if I tried to write Python that way I'd be wary about showing that code to anyone more entrenched in the Pythonic way of thinking.

Re:How to build your own neural network from scratch in Python by jononor on Saturday October 06, 2018 @06:47AM (#18154470)
The never use map/filter is in favor of list comprehensions, which is not really a OOP construct.

But sure, it's a conventional thing. And for a long time the built-in alternatives to a class for such a datastructure tuples and dicts, neither which are very nice for functions to operate on (dict values have to be addressed with d['key'] instead of d.key). With a class and method there is also no doubt as to what the function operates on, which is convenient when there type hints and IDE support is missing. This is changing since Python 3.5 and type checking tools like MyPy.

Since Python 3.7 there are also data classes, a primitive for classes which just hold values. https://www.linuxjournal.com/content/introducing-python-37s-... But it will take a while before programming conventions change.

Re:How to build your own neural network from scratch in Python by derekmcloughlin on Friday October 05, 2018 @03:35AM (#18146388)
I found this very useful:

https://makeyourownneuralnetwork.blogspot.com/ https://www.amazon.com/dp/1530826608/ref=cm_sw_r_tw_dp_U_x_x...

First part goes into what a NN is and how backprop works, second part is an implementation in Python.

Re:How to build your own neural network from scratch in Python by bluemania on Friday October 05, 2018 @12:43AM (#18145780)
Thanks for submitting, this is going to come in handy for my education! I have an assignment next week which requires creating a neural network, then substituting various optimizers in place of backpropagation and comparing performance over iterations. Have found that simple numpy based NN's are easier to examine and connect the changes with the theory, this guide looks to help further with this understanding!
Re:How to build your own neural network from scratch in Python by dnautics on Friday October 05, 2018 @04:36AM (#18146628)
That sounds really fun! Do you get to pick optimizers? Obviously direct gradient is a thing (calculate the partial for every parameter) but also particle swarm optimization might be cool to profile.
Re:How to build your own neural network from scratch in Python by peter303 on Friday October 05, 2018 @05:28AM (#18146794)
I suggest a remarkably useful device: Four circular disks when attached to a cart make it easy to move the cart from location to another. Furthermore you could put goods in that cart and move them too. A lot easier than carrying the goods on you back or an animal. A brief search of the internet finds this is a new idea.
Re:How to build your own neural network from scratch in Python by qbrass on Friday October 05, 2018 @01:12PM (#18149843)
What do the disks do? A cart already moves pretty easily on it's wheels, so how much improvement have you made by attaching these disks to it?
Re:How to build your own neural network from scratch in Python by amasad on Friday October 05, 2018 @01:36AM (#18145950)
Very cool! Here's the full version if you want fo run it/play with it

https://repl.it/@turbio/neural-network

Re:How to build your own neural network from scratch in Python by starpilot on Friday October 05, 2018 @12:47AM (#18145794)
Another thing to try is calculating backpropagation by hand, on paper, with a small NN. This is what my mentor said his final exam on NN in college involved.
Re:How to build your own neural network from scratch in Python by anonytrary on Friday October 05, 2018 @01:21AM (#18145904)
I would have thought calculus 1 was a prerequisite for that course.
Re:How to build your own neural network from scratch in Python by wheresvic1 on Friday October 05, 2018 @04:04AM (#18146498)
If you're looking for a similar from scratch tutorial in Java, check this one out: https://smalldata.tech/blog/2016/05/03/building-a-simple-neu...
Re:How to build your own neural network from scratch in Python by master_yoda_1 on Friday October 05, 2018 @10:57AM (#18148638)
This is not as glorified as it sound. 100 of thousands of students around the world build neural net from scratch as their homework. People please move on, spend your time somewhere else which is more productive. I coded neural network using c++ in 2009 as homework. But I never though about showing this to people.
Re:How to build your own neural network from scratch in Python by tobr on Friday October 05, 2018 @12:16PM (#18149311)
> People please move on

Strange reaction to a “beginner’s guide”, how can someone move on before they learn the basics?

Re:How to build your own neural network from scratch in Python by deytempo on Friday October 05, 2018 @12:39PM (#18149551)
Programming != math