News for hackers. Stuff that matters.
Re:Simple example of machine learning in TensorFlow by dbcurtis on Monday March 13, 2017 @12:54PM (#13859640)
I like these kind of "Hello, world!" examples for TensorFlow. As a TensorFlow beginner, I need all the references I can get. Here is what I need right now: "Hello, we meet again!". I can build a neural net model, and train (albeit, often badly) a model, but saving and restoring the trained weights so that I can run the model again is giving me fits. I am clearly missing something fundamental about how to restore a TensorFlow NN model.

For your next tutorial, may I suggest: 1) a list of do's and don'ts for constructing a savable/restorable model, and 2) a wee bit of example code.

Of course, now that I have discovered Keras I'm moving away from low-level direct TensorFlow. But I suspect I'm not the only one a bit foggy about the whole save/restore work flow.

Re:Simple example of machine learning in TensorFlow by pmalynin on Monday March 13, 2017 @01:05PM (#13859804)
As a person that uses TensorFlow for his day job:

I find that saving and restoring are of the weirder things with TensorFlow, you can either go all out an decide to save out all the variables, or only the ones needed for the model.

You usually don't want to save out gradients (which are also variables) since they take up a bunch of space and aren't actually that useful to restore. Now on the other, what are model variables -- do you want to save model variables + the moving averages ... or just the averages. But then when you're loading you'll have to "shadow" the moving averages to the real variables that actually run in your model.

Good news though, most of the scaffolding code you can write once and re-use it over and over again.

Re:Simple example of machine learning in TensorFlow by minimaxir on Monday March 13, 2017 @01:17PM (#13859942)
In Keras, it's just a simple model.save() [to a hdf5 file] and load_model(). This includes both the weights and the architecture.

Models with a few million parameters result in a file around ~50MB, which is still reasonable for modern production use cases.

Re:Simple example of machine learning in TensorFlow by glial on Monday March 13, 2017 @09:24PM (#13864176)
Keras makes using deep learning for simple-ish use cases sooooo easy.
Re:Simple example of machine learning in TensorFlow by matheweis on Monday March 13, 2017 @11:02PM (#13864621)
I second this - I'm really excited about Keras being integrated into the core of Tensorflow (other than the chance it might lose the Torch compatibility).
Re:Simple example of machine learning in TensorFlow by jostmey on Monday March 13, 2017 @01:14PM (#13859906)
+1
Re:Simple example of machine learning in TensorFlow by jostmey on Monday March 13, 2017 @03:18PM (#13861351)
I would put together a tutorial on how models are saved and restored if I understood it, which is to say it still confuses me
Re:Simple example of machine learning in TensorFlow by hcrisp on Monday March 13, 2017 @02:04PM (#13860547)
This save/restore tutorial was shared early by someone who is writing similar TensorFlow primers:

https://blog.metaflow.fr/tensorflow-saving-restoring-and-mix...

Re:Simple example of machine learning in TensorFlow by dbcurtis on Monday March 13, 2017 @04:02PM (#13861803)
Great! Thanks, bookmarked.
Re:Simple example of machine learning in TensorFlow by maaaats on Monday March 13, 2017 @02:51PM (#13861050)
> As a TensorFlow beginner, I need all the references I can get.

I think that having to use a duck typed language makes it so much harder than it has to be. The API is huge, and you get really no help from the IDE. If I could do tf.train.GradientDescentOptimizer as in the example, and then get autocomplete to what I can do next with that object it would really help. Or list functions that takes that kind of object as input. Now, one is searching in the dark.

Re:Simple example of machine learning in TensorFlow by Jach on Monday March 13, 2017 @03:41PM (#13861586)
This sounds like a problem with your IDE than with the language. Vim doesn't have this problem. See https://www.youtube.com/watch?v=TNMjbaimk9g and I could probably find older systems that predate the plugins mentioned in the video.
Re:Simple example of machine learning in TensorFlow by allenlavoie on Monday March 13, 2017 @09:50PM (#13864327)
A lot of the best practices for initializing/saving/restoring/etc. are handled automatically if you wrap your model in an Estimator[1] using a model_fn. It also enforces a "clean" model specification (in a new Graph) and decoupling of the input pipeline from the model.

However, Estimators are not in TensorFlow core, which means the API isn't fixed quite yet.

[1] https://www.tensorflow.org/extend/estimators

Re:Simple example of machine learning in TensorFlow by pred_ on Monday March 13, 2017 @02:49PM (#13861027)
That's nifty; I was looking for something like that just a few weeks ago for a work demonstration! Ended up doing https://gist.github.com/fuglede/ad04ce38e80887ddcbeb6b81e97b... instead.
Re:Simple example of machine learning in TensorFlow by rhcom2 on Monday March 13, 2017 @03:16PM (#13861321)
Thank you to you and OP for both sharing these resources. Really helpful.
Re:Simple example of machine learning in TensorFlow by nemo1618 on Monday March 13, 2017 @01:21PM (#13859990)
I wish there were more TensorFlow examples written in Go. I made the mistake of checking out TensorFlow as my first intro to ML and it flew about 10 miles over my head. Slowly learning now, but most of the documentation and tutorials are written in Python.

This blog series was also helpful on a conceptual level: https://medium.com/emergent-future/simple-reinforcement-lear...

Re:Simple example of machine learning in TensorFlow by make3 on Monday March 13, 2017 @01:48PM (#13860330)
as a deep learning professional, the deep learning community is something like 99% Python. You'd probably better learn Python at least well enough to recreate the corresponding Go code in your mind instantly.
Re:Simple example of machine learning in TensorFlow by djhworld on Monday March 13, 2017 @05:59PM (#13862847)
tbh you'll be better off just sticking with python for now, until you understand what's going on.
Re:Simple example of machine learning in TensorFlow by dnautics on Monday March 13, 2017 @06:19PM (#13863025)
I think it's a really good exercise to take TensorFlow examples and algorithms and reimplement them in another language.
Re:Simple example of machine learning in TensorFlow by calebm on Monday March 13, 2017 @01:26PM (#13860050)
>>> You are one buzzword away from being a professional. Instead of fitting a line to just eight datapoints, we will now fit a line to 8-million datapoints. Welcome to big data.

LOL :) (Side-note: 8 million is still not big data)

Re:Simple example of machine learning in TensorFlow by mcrad on Monday March 13, 2017 @02:30PM (#13860842)
Big Data is a reference to complexity of the data & underlying system that data represents, NOT the number of datapoints.

lol

Re:Simple example of machine learning in TensorFlow by pyromine on Tuesday March 14, 2017 @12:52AM (#13865101)
Big data is really just a buzzword that no one knows what it really means, because everyone's definition is different

lol

Re:Simple example of machine learning in TensorFlow by happycube on Tuesday March 14, 2017 @01:55AM (#13865341)
I always think in terms of Munchkin: "any data that is not Big is small"
Re:Simple example of machine learning in TensorFlow by JonathonCwik on Monday March 13, 2017 @03:34PM (#13861502)
So I'm still wrapping my head around some of the math (I haven't had a math class in a handful of years)...

I get the output of the model (y_model = m*xs[i]+b), it's the y = mx + b where we know x (from the dataset) and have y be a variable.

The error is where I start to lose it, so I get the idea of the first part (ys[i]-y_model). It's basically the difference between the actual y value (from the dataset). I get that we want this number to be as small as possible as the closer to zero it is for the entire dataset that means we get closer to the line going through (or near) all the points and the closest fit will be when this total_error is nearest to zero.

What I don't get is the squaring of the difference. Is it just to make the difference a larger number so that it's a little more normalized? How do you get to the conclusion that it needs to be normalized? Same thing with the learning rate? I believe these to be correlated but I can't tell you how...

Re:Simple example of machine learning in TensorFlow by cowabungabruce on Monday March 13, 2017 @03:42PM (#13861594)
Squaring gets you guaranteed positive numbers. Remember, we are adding all the errors together to optimize the model: If we get

sum_errors_A = 4 + -3 + -1

sum_errors_B = 1 + 1 + 1

B is obviously the better model, but it has a higher error than A when comparing. If we squared all the terms and then added, B would be the stronger model.

Re:Simple example of machine learning in TensorFlow by JonathonCwik on Monday March 13, 2017 @04:04PM (#13861829)
Ah, gotcha! Makes a lot more sense now.
Re:Simple example of machine learning in TensorFlow by jasode on Monday March 13, 2017 @03:40PM (#13861575)
>What I don't get is the squaring of the difference.

The following answer is about standard deviation but the desired properties from squaring is also relevant to your question:

http://stats.stackexchange.com/questions/118/why-square-the-...

Re:Simple example of machine learning in TensorFlow by flor1s on Tuesday March 14, 2017 @03:21AM (#13865676)
I don't think there is much to gain from this tutorial, then again it doesn't pretend to offer you much either. For example in the code you are discussing, defining a model as variables and operations, instead of as a function, was confusing to me. Probably Tensorflow overloads the operations, but normally when you read "y = mx + b" you expect y to be computed directly and not to be stored as a model. "f = lambda m, x, b: m x + b" seems much more clear to me.
Re:Simple example of machine learning in TensorFlow by gkjohns on Monday March 13, 2017 @03:41PM (#13861589)
a) it ensures that they're positive so they don't just cancel each other out and b) like you mentioned, it penalizes huge errors more heavily.

There are also historical reasons for using squared error. The square function is smooth and differentiable to you can analytically solve for the gradient. Before fast computers this was crucial for solving regression problems as a closed for makes everything easier.

Re:Simple example of machine learning in TensorFlow by Kiro on Monday March 13, 2017 @01:48PM (#13860336)
This is awesome!

I currently have a small pet project where I think some simple ML would be cool but I don't know where to start so these things are great.

Basically my use case is that I have a bunch of 64x64 images (16 colors) which I manually label as "good", "neutral" or "bad". I want to input this dataset and train the network to categorize new 64x64 images of the same type.

The closest I've found is this: https://gist.github.com/sono-bfio/89a91da65a12175fb1169240cd...

But it's still too hard to understand exactly how I can create my own dataset and how to set it up efficiently (the example is using 32x32 but I also want to factor in that it's only 16 colors; will that give it some performance advantages?).

Re:Simple example of machine learning in TensorFlow by nl on Monday March 13, 2017 @04:51PM (#13862224)
Re:Simple example of machine learning in TensorFlow by hermiti on Monday March 13, 2017 @04:56PM (#13862266)
Kiro,

Here's how I solved a similar problem in my deep learning class. Instead of classifying between 10 animals you would just have 3 possible labels.

https://github.com/hermiti/deep_learning_project_2/blob/mast...

Re:Simple example of machine learning in TensorFlow by cosmicexplorer on Tuesday March 14, 2017 @03:58AM (#13865811)
What is the meaning of the "?bare" query string in the url? I googled around for the meaning of query strings on the github site but only found rnandom repos on github (not sure how to narrow the search). The first time I tried removing it I saw another folder named "to_do", but this is gone now so it might give a version which is cached for longer somehow?
Re:Simple example of machine learning in TensorFlow by cosmicexplorer on Tuesday March 14, 2017 @12:34PM (#13868989)
OK, found out what a bare repository means and pretty sure that's what it refers to. Still can't find any documentation for the query string parameter and don't know how that makes sense for github's repository view page.
Re:Simple example of machine learning in TensorFlow by blauditore on Monday March 13, 2017 @02:19PM (#13860738)
I'm not sure about the rules, but shouldn't posts linking to own, personal projects be prefixed with "Show HN:"? I've seen a lot such posts lately where the poster was clearly the author as well.
Re:Simple example of machine learning in TensorFlow by pvg on Monday March 13, 2017 @02:28PM (#13860824)
No, Show HN is a different thing with its own (generally stricter) rules. You don't have to add it to things just because you happen to be the author.

https://news.ycombinator.com/showhn.html

Re:Simple example of machine learning in TensorFlow by kyleschiller on Monday March 13, 2017 @03:28PM (#13861448)
I think you meant bare bones.
Re:Simple example of machine learning in TensorFlow by mediocrejoker on Monday March 13, 2017 @01:49PM (#13860349)
I'm guessing english is not your first language, so I just wanted to point out that "bare bottom" is generally synonymous with "uncovered buttocks" ie. in the context of changing an infant's diaper.

Perhaps you were meaning to put "bare bones"? Google's definition of the latter is "reduced to or comprising only the basic or essential elements of something."

Don't want to detract from your point but I think your title is throwing some people off. I know I would be hesitant to click something at work that sounds like it could contain nudity.

Re:Simple example of machine learning in TensorFlow by dekhn on Monday March 13, 2017 @01:56PM (#13860449)
I think the last example was a clASSifier, so it makes sense.
Re:Simple example of machine learning in TensorFlow by fanpuns on Monday March 13, 2017 @02:00PM (#13860504)
But it works sooo perfectly in the current vote count...

https://t.co/5cLU9F5MCh

Re:Simple example of machine learning in TensorFlow by hughes on Monday March 13, 2017 @01:53PM (#13860409)
Well, the title of the tutorial is "Naked Tensor"...
Re:Simple example of machine learning in TensorFlow by sctb on Monday March 13, 2017 @03:13PM (#13861291)
Thanks, we've removed that from the submission title.
Re:Simple example of machine learning in TensorFlow by bencollier49 on Monday March 13, 2017 @12:55PM (#13859653)
"Bare bottom"? I'm not clicking on this.
Re:Simple example of machine learning in TensorFlow by bencollier49 on Tuesday March 14, 2017 @04:45AM (#13865982)
Downvoted! The title might have changed now, but the original one was completely indecipherable. As far as I could tell it was genuinely some sort of image recognition algorithm for naked buttocks.
Re:Simple example of machine learning in TensorFlow by BonoboBoner on Monday March 13, 2017 @02:01PM (#13860516)
Simple example? Before finishing the first paragraph, it says

"The slope and y-intercept of the line are determined using gradient descent."

What on earth does that mean? Maybe they should teach mathematics in english at universities outside of english speaking countries. German mathematics does not help here.

I wish there was a 4GL like SQL for machine learning using dynamic programming for algorithm selection and model synthesis like a dbms query planner.

PREDICT s as revenue LEARN FROM company.sales as s GROUP BY MONTH ORDER BY company.region

Re:Simple example of machine learning in TensorFlow by sampo on Monday March 13, 2017 @02:16PM (#13860695)
> "The slope and y-intercept of the line are determined using gradient descent."

Slope and intercept are very standard names for the parameters of a linear regression model. Gradient descent is the name of the algorithm used.

Re:Simple example of machine learning in TensorFlow by spitfire on Monday March 13, 2017 @02:48PM (#13861018)
This has been done before, though by who eludes me. I've seen it here on HN a few years ago.

and it is absolutely a good idea, as long as you include validation and QA abilities right along side train/predict.

Re:Simple example of machine learning in TensorFlow by ice109 on Monday March 13, 2017 @02:03PM (#13860540)
an article written in english assumes familiarity with english vernacular? who would've thought!