News for hackers. Stuff that matters.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by darepublic on Wednesday January 05, 2022 @11:52AM (#29811411)
Amazing, I was looking to do something very similar for myself-- dig into pytorch library to see how everything was working.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by rg111 on Wednesday January 05, 2022 @12:27PM (#29811955)
There is also the book Deep Learning from Scratch.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by hex9893628af on Wednesday January 05, 2022 @01:55PM (#29813277)
Are there any resources that use languages other than python? I would prefer to work with code examples in a language that I might actually use.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by asnt on Wednesday January 05, 2022 @02:47PM (#29813946)
Maybe not as comprehensive or approachable:

Julia: https://juliadiff.org/ChainRulesCore.jl/stable/FAQ.html#Wher...? (bottom)

OCaml: https://ocaml.xyz/book/

Re:Minitorch: Implement the Torch API from Scratch, from Cor... by kbutler on Wednesday January 05, 2022 @02:09PM (#29813447)
If you're doing machine learning, python is a very viable language to actually use.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by hex9893628af on Wednesday January 05, 2022 @02:30PM (#29813715)
Not to me it isn’t.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by cinntaile on Wednesday January 05, 2022 @04:05PM (#29814928)
Almost all neural network frameworks are Python based, so you're severely limiting yourself here.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by systemsignal on Wednesday January 05, 2022 @04:25PM (#29815147)
What language do you use?
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by cbm-vic-20 on Wednesday January 05, 2022 @02:50PM (#29813980)
No! You must respect the Python ML monoculture! Now hand me my set of MNIST digit images!
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by rg111 on Thursday January 06, 2022 @01:49AM (#29820246)
> that I might actually use

The only language that you might actually use is Python.

Re:Minitorch: Implement the Torch API from Scratch, from Cor... by p1esk on Wednesday January 05, 2022 @03:10PM (#29814274)
“from scratch” == “using numpy and numba”
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by shard on Thursday January 06, 2022 @12:09AM (#29819686)
If you wish to implement the Torch API from scratch, you must first invent the universe. (Apologies to Sagan)
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by p1esk on Thursday January 06, 2022 @02:36AM (#29820523)
"from scratch" in this context does not make any sense - Pytorch is already implemented "from scratch" using Python, C++, and CUDA. Implementing Pytorch using Numpy is the opposite of "from scratch".
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by shard on Thursday January 06, 2022 @04:32AM (#29821306)
I didn't dig too deeply, however on the pytorch installation page, it says "Please ensure that you have met the prerequisites below (e.g., numpy)"

Does pytorch already require numpy?

Re:Minitorch: Implement the Torch API from Scratch, from Cor... by p1esk on Thursday January 06, 2022 @11:29AM (#29825659)
It does, because it can convert a numpy array to a pytorch tensor and back, but I don’t think numpy is used for anything else.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by lostmsu on Tuesday January 11, 2022 @01:01PM (#29895115)
They must be doing something else with numpy as Python has a generic API, that allows direct access to data of numpy array-like structures: Buffer Protocol https://docs.python.org/3/c-api/buffer.html
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by p1esk on Tuesday January 11, 2022 @03:10PM (#29897121)
It’s not just about access to underlying data - users might want to import from, or export to numpy array (e.g. for visualization, etc).
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by lostmsu on Tuesday January 11, 2022 @11:33PM (#29902342)
Buffer protocol means PyTorch can have a torch.tensor method, that can create a tensor of anything implementing this protocol (which NumPy arrays do), and likewise numpy.array can be constructed out of PyTorch array because PyTorch Tensor implements this protocol. E.g. they don't need to know about each other (so no dependency), only about the protocol.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by rg111 on Thursday January 06, 2022 @02:13AM (#29820382)
This material is focused on teaching how to develop a Deep Learning library, and not teaching Deep Learning through implementing things from scratch.

If that's what you want, then there are:

- Data Science from Scratch

- Deep Learning from Scratch

Re:Minitorch: Implement the Torch API from Scratch, from Cor... by p1esk on Thursday January 06, 2022 @02:33AM (#29820506)
It's totally fine if you want to implement a DL library using Numpy. Just don't call it "from scratch". Especially if you say "in pure Python". Numpy is just as "pure Python" as Pytorch, so this does not make much sense.
Re:Minitorch: Implement the Torch API from Scratch, from Cor... by rg111 on Thursday January 06, 2022 @12:03PM (#29826181)
This is focused on API Design. Not on developing a DL library from scratch.