News for hackers. Stuff that matters.
Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @10:08AM (#25254477)
CreateML is getting really impressive. I just went through the process of using it to train an Object Detection model to detect sharks and fish at an aquarium (and deploying the model to an iPhone) last week.

Final result: https://twitter.com/braddwyer/status/1330654868839784451

The full walkthrough video: https://www.youtube.com/watch?v=kBn7Cd8Z8yE

Re:Apple Create ML: Creating an Image Classifier Model by wil421 on Monday November 30, 2020 @10:35AM (#25254802)
Do you think it would be possible to detect the species of shark? I was thinking about trying to detect what breed of dog or maybe species of fish someone caught.
Re:Apple Create ML: Creating an Image Classifier Model by nkristoffersen on Monday November 30, 2020 @11:37AM (#25255629)
Could probably detect if it was a hotdog or not hotdog.
Re:Apple Create ML: Creating an Image Classifier Model by ceejayoz on Monday November 30, 2020 @08:00PM (#25261042)
Train it, then test it on https://www.reddit.com/r/hotdogsorlegs/.
Re:Apple Create ML: Creating an Image Classifier Model by tosh on Monday November 30, 2020 @12:27PM (#25256310)
this might be a vc-case!
Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @01:10PM (#25256892)
With the right training data, definitely.

Check out the Oxford Pets dataset as a starting point (you can download in Create ML format from here): https://public.roboflow.com/object-detection/oxford-pets

Re:Apple Create ML: Creating an Image Classifier Model by dagmx on Monday November 30, 2020 @11:02AM (#25255134)
It's possible but you'd need more training data IMHO the more similar things get.
Re:Apple Create ML: Creating an Image Classifier Model by criddell on Monday November 30, 2020 @11:54AM (#25255861)
That's really neat. Once your model is complete, does it work offline?
Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @12:50PM (#25256600)
Yep, in this example the weights are compiled into the app and inference is done locally on the iPhone processor.
Re:Apple Create ML: Creating an Image Classifier Model by criddell on Monday November 30, 2020 @12:56PM (#25256697)
Is the app available anywhere? I'd love to see if it works on my aquarium.
Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @01:15PM (#25256952)
I didn't clean it up and submit for app review but if you have XCode you can compile and run it by tweaking the sample app (start at 22:25 in the YouTube video above).

Here are the weights: https://www.dropbox.com/s/c49a5zqqt1ml89o/Aquarium%202.mlmod...

Re:Apple Create ML: Creating an Image Classifier Model by crb002 on Monday November 30, 2020 @12:25PM (#25256280)
What apps do you use for drawing the labeling/bounding boxes?
Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @12:52PM (#25256634)
There are a handful of free tools you can use. We have write-ups of most of them in the "Labeling Your Images Yourself" section here: https://blog.roboflow.com/object-detection/

Roboflow (our startup) can import the annotation files from all of those tools and convert them to Create ML's format for you.

Re:Apple Create ML: Creating an Image Classifier Model by Unicode007 on Monday November 30, 2020 @11:07AM (#25255204)
Create ML is awesome and very user friendly. I achieved impressive accurate results to classify Among Us game states on Apple Silicon Macs to support keyboard controls.

https://news.ycombinator.com/item?id=25254954

I open sourced it today: https://github.com/andresavic/AmongKey

Re:Apple Create ML: Creating an Image Classifier Model by tosh on Monday November 30, 2020 @11:25AM (#25255444)
love this, I can already see how something like this could add keyboard shortcuts to apps that don't have them (or not the ones I want)

thanks for making this open source!

Re:Apple Create ML: Creating an Image Classifier Model by codetrotter on Monday November 30, 2020 @06:14PM (#25260275)
Next step will be to train the computer to detect who is acting suspiciously in the game, and then make the computer play the game for you so you don’t have to use any controls at all to play the game :p
Re:Apple Create ML: Creating an Image Classifier Model by hokumguru on Monday November 30, 2020 @11:11AM (#25255246)
holy crap this is awesome.
Re:Apple Create ML: Creating an Image Classifier Model by LeicaLatte on Monday November 30, 2020 @10:27AM (#25254720)
If you are looking to automate training on your new M1 Macs, I have published some Create ML examples here which might help you get started - https://github.com/krishkumar/createml-playgrounds

These new chips run Create ML really well, no surprises there.

Re:Apple Create ML: Creating an Image Classifier Model by dan1234 on Monday November 30, 2020 @10:50AM (#25254989)
Where did you get the movie reviews dataset? I'm interested in doing something similar for music.
Re:Apple Create ML: Creating an Image Classifier Model by LeicaLatte on Monday November 30, 2020 @11:07AM (#25255197)
That datasets in the repo are fairly old. I suggest using newer datasets.
Re:Apple Create ML: Creating an Image Classifier Model by elpakal on Monday November 30, 2020 @11:52AM (#25255837)
+1 on CreateML ease of use. Paired with some of the other models they provide out of the box, it can be very easy to get up and going quickly. I used it to create a mustache detector and before running an image through my model I needed to detect if an image had a face (images of hot dogs, for eg, were leading to weird results). I was delighted to find the ability to detect faces in a few lines of code right out of the box.

Now I just wish I could convert my model to Tensorflow! (or serve it over iCloud somehow?)

Re:Apple Create ML: Creating an Image Classifier Model by my123 on Monday November 30, 2020 @12:21PM (#25256231)
Use onnxtools at https://github.com/onnx/onnxmltools to convert your model from Core ML to ONNX. You'll then be able to use it elsewhere.
Re:Apple Create ML: Creating an Image Classifier Model by zitterbewegung on Monday November 30, 2020 @11:35AM (#25255594)
Using CreateML is a vastly better experience on ease of use using Machine Learning models. Even CoreML which underpins CreateML allows you to use using GPT2 and Bert from from Huggingface (see https://github.com/huggingface/swift-coreml-transformers).

Shameless plug: I made a Youtube video on how to use Style Transfer in CreateML see https://www.youtube.com/watch?v=adfCMup9YnM

Re:Apple Create ML: Creating an Image Classifier Model by douk on Monday November 30, 2020 @10:03AM (#25254436)
I'm really curious if this training process is optimized for the new 8-core neural engine on the M1 SoC. I'd like to see a comparison between the performance on M1 vs Intel macs.

I supposed this might serve as a good demonstration of how the neural engine performs?

edit: Perhaps running inference in the iOS simulator on the macs would also be a comparison point?

Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @10:10AM (#25254497)
We're actually running a benchmark of that very thing this week! I'll follow up with what we find.
Re:Apple Create ML: Creating an Image Classifier Model by jagged-chisel on Monday November 30, 2020 @09:44AM (#25254209)
For the ML-impaired, how portable is this process? Does it work on other ML platforms? Are the digital assets (specifically this MLMODEL file/bundle) easily transmuted to work with, say, TF?
Re:Apple Create ML: Creating an Image Classifier Model by mumblemumble on Monday November 30, 2020 @10:28AM (#25254723)
It's not a pre-existing standard, but it is a documented format. It's based on protocol buffers, so I would expect it's fairly easy to write your own reader if necessary.

https://apple.github.io/coremltools/coremlspecification/

Re:Apple Create ML: Creating an Image Classifier Model by Someone on Monday November 30, 2020 @11:47AM (#25255767)
That project also has code for converting from quite a few formats (https://coremltools.readme.io/docs) to CoreML.

Having that (3-clause BSD-licensed) code will help in writing the reverse converters.

Re:Apple Create ML: Creating an Image Classifier Model by thebruce87m on Monday November 30, 2020 @09:48AM (#25254252)
I’m also ML impaired, but would people really train the model on a Mac? Surely you would want to train in the cloud or on something with a proper discrete GPU. It seems that a better article would be how to take e.g. yolo and convert it.
Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @10:25AM (#25254696)
Apple's main goal with Create ML is to make something app developers can use to add machine learning powered features to their apps. So it makes sense that they'd want something that you can train with the hardware you already have as an app developer.

It's actually pretty performant. I trained an object detector (from scratch) on my 2016 MBP in about 6 hours[1].

As of macOS 11 also support transfer learning for object detection which starts from pre-trained weights and should converge more quickly if your dataset is similar to COCO[2] but I haven't tested it out yet.

[1] https://blog.roboflow.com/createml/ [2] https://blog.roboflow.com/coco-dataset/

Re:Apple Create ML: Creating an Image Classifier Model by seek3r on Monday November 30, 2020 @10:00AM (#25254387)
The article is not explicit about it but I think they’re using transfer learning. They take a general model trained, for instance, on ImageNet and use it to train a more specialised model to classify whatever you want. The training of the specialised model is way faster than the training of the general model.
Re:Apple Create ML: Creating an Image Classifier Model by bfieidhbrjr on Monday November 30, 2020 @09:58AM (#25254366)
Typically the model is pre-trained and you're just training the last layer, or adding a new layer, which is much much less computationally expensive.
Re:Apple Create ML: Creating an Image Classifier Model by fastball on Monday November 30, 2020 @09:58AM (#25254360)
Presumably this can be trained on the neural engines on iPhone/iPad chips and the new M1s.

Also my Macbook Pro has an 8GB GPU.

Re:Apple Create ML: Creating an Image Classifier Model by stephenr on Monday November 30, 2020 @09:57AM (#25254348)
people do run Macs in datacenters, so if you wanted to have it run in a dc I’m sure you could. Secondly all pro level Macs have discrete graphics, so I’m not sure what you mean by “proper discrete gpu”
Re:Apple Create ML: Creating an Image Classifier Model by sliken on Monday November 30, 2020 @02:38PM (#25257954)
Not sure what you mean by "all pro level macs". But the macbook pro M1 has no discrete GPU. It does however have a "neural processor" to accelerate ML.
Re:Apple Create ML: Creating an Image Classifier Model by stephenr on Monday November 30, 2020 @03:05PM (#25258266)
Ah right I always forget about the 13” Ugly step sister - it was igpu only for the intel models too.

However the other pro lines (MBP15/16 iMac Pro; Mac Pro) are all equipped with dGPUs

Re:Apple Create ML: Creating an Image Classifier Model by junipertea on Monday November 30, 2020 @10:27AM (#25254722)
Discrete nvidia GPU with CUDA. While there is Tensorflow compiled for radeon GPUs, majority of libraries are based on CUDA
Re:Apple Create ML: Creating an Image Classifier Model by dagmx on Monday November 30, 2020 @11:06AM (#25255181)
Tensorflow was recently released for Metal
Re:Apple Create ML: Creating an Image Classifier Model by skwb on Monday November 30, 2020 @05:56PM (#25260113)
Has there been any major updates to using TF for AMD GPUs recently? Or is it still too much of a headache to do it?
Re:Apple Create ML: Creating an Image Classifier Model by stephenr on Monday November 30, 2020 @10:36AM (#25254819)
... The models created by this are designed to be used in apps for on-device ML, on Macs, iPhones, iPads, etc.
Re:Apple Create ML: Creating an Image Classifier Model by seek3r on Monday November 30, 2020 @10:02AM (#25254417)
I did a quick Google search (so maybe I’m missing something) but there is no official support for converting Core ML — which is the foundation for Create ML — models to Tensorflow or PyTorch. You need to look for third-party libraries. Although there exist libraries to convert Tensorflow models to Core ML.
Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @10:20AM (#25254637)
You can convert CoreML models to ONNX which is widely supported cross-platform for doing inference. https://github.com/onnx/onnxmltools
Re:Apple Create ML: Creating an Image Classifier Model by therealmarv on Monday November 30, 2020 @02:16PM (#25257656)
sorry my lack of understanding: So you think an image classifier trained on Create ML can be exported to another framework of not?
Re:Apple Create ML: Creating an Image Classifier Model by yeldarb on Monday November 30, 2020 @03:17PM (#25258402)
It’s not entirely clear to me exactly what was meant by the grandparent post by converting to PyTorch or Tensorflow.

There are two discrete concepts: training and inference. You can think of training a bit like the source code and inference a bit like running the compiled binary. The different frameworks have their own serialization formats for their model weights.

If the goal is to do inference using CoreML weights trained in CreateML on non-Apple platforms (eg on a server or android device), converting them to ONNX is a way to do that.

You probably won’t be able to pick up training on another framework though.

Re:Apple Create ML: Creating an Image Classifier Model by uberneo on Monday November 30, 2020 @03:57PM (#25258873)
Wondering if Create ML has a Sound Classifier as well.. Will make it easy to classify different bird's sounds or to identify music notes from different instruments.. something like piano/guitar notes classifier
Re:Apple Create ML: Creating an Image Classifier Model by mlsoundanalysis on Monday November 30, 2020 @11:18PM (#25262261)
This is a nice introduction to sound classification with Create ML:

https://developer.apple.com/videos/play/wwdc2019/425/

Re:Apple Create ML: Creating an Image Classifier Model by poyu on Monday November 30, 2020 @08:44PM (#25261319)
It does. Not sure how well it works though.

https://developer.apple.com/documentation/createml

Re:Apple Create ML: Creating an Image Classifier Model by petargyurov on Monday November 30, 2020 @02:24PM (#25257767)
For anyone looking into alternatives, I am currently compiling a list here [0]. The focus of the research is software that aids with the entire flow of a typical ecologist who wishes to classify/detect different species.

There are a lot of great pieces of software out there for non-technical users.

[0] https://www.notion.so/pgyurov/Camera-Trap-Pipeline-Solution-...

Re:Apple Create ML: Creating an Image Classifier Model by iamgopal on Monday November 30, 2020 @09:01PM (#25261451)
Is there a image classifier that use my Dropbox, scan all images sort by tag, and save all neural and tag data on folder itself ? That I can use to access all the images ?
Re:Apple Create ML: Creating an Image Classifier Model by interestednml on Monday November 30, 2020 @11:23PM (#25262278)
Apple added action classification for video:

https://developer.apple.com/videos/play/wwdc2020/10043/

and style transfer to Create ML this year:

https://developer.apple.com/videos/play/wwdc2020/10642/

Re:Apple Create ML: Creating an Image Classifier Model by amelius on Monday November 30, 2020 @01:14PM (#25256944)
That's cool. How can I transfer the learned model to an RPi so I can use it in my home automation project?
Re:Apple Create ML: Creating an Image Classifier Model by petargyurov on Monday November 30, 2020 @02:28PM (#25257817)
I am not too familiar with CreateML but take a look at Lobe. It allows you to export models and it also comes with the option of using MobileNet (which is more suitable for running on edge devices).

You can also find examples on how to use the exported model on Lobe's Github.

Re:Apple Create ML: Creating an Image Classifier Model by nsriv on Monday November 30, 2020 @02:28PM (#25257815)
This seems like Apple's answer to the recently Microsoft-acquired lobe.ai
Re:Apple Create ML: Creating an Image Classifier Model by judge2020 on Monday November 30, 2020 @03:28PM (#25258541)
Create ML has been generally available since 10.14 (2018)[0], while Lobe's product had a preview in 2018 and was bought by Microsoft the same year[1]. Apple has been pushing on-device ML for 4 years or so now[1], so, if anything, Lobe is Microsoft's answer to Apple.

0: https://developer.apple.com/documentation/createml

1: https://blogs.microsoft.com/blog/2018/09/13/microsoft-acquir...

1: https://www.zdnet.com/article/ai-big-data-and-the-iphone-her....

Re:Apple Create ML: Creating an Image Classifier Model by meesterdude on Monday November 30, 2020 @01:54PM (#25257422)
Is there an upper limit to the number of classes you can train it to detect?
Re:Apple Create ML: Creating an Image Classifier Model by momokoko on Monday November 30, 2020 @04:17PM (#25259095)
It’s so sad to watch the success of Apple bring upon the impending death of open source.
Re:Apple Create ML: Creating an Image Classifier Model by rodgerd on Monday November 30, 2020 @07:15PM (#25260761)
Given pretty much every conversation on the topic of ML here advises people to use the proprietary NVidia ML tools with proprietary NVidia graphics drivers for NVidia cards, I'm failing to see this as any better or worse.