Re:Transformers as Support Vector Machines
by adamnemecek on Sunday September 03, 2023 @02:40AM
(#37368221)
All machine learning is about finding hyperplanes.
[ Parent
| on HN ]
Re:Transformers as Support Vector Machines
by nologic01 on Sunday September 03, 2023 @02:54AM
(#37368269)
The large dimensionality seems to be what creates the need for heuristic designs rather than a generic approach
Re:Transformers as Support Vector Machines
by adamnemecek on Sunday September 03, 2023 @02:56AM
(#37368276)
Hyperplanes are the heuristics.
Re:Transformers as Support Vector Machines
by alexmolas on Sunday September 03, 2023 @03:26AM
(#37368415)
Hyperplanes is all you need
Re:Transformers as Support Vector Machines
by quickthrower2 on Sunday September 03, 2023 @08:48AM
(#37369999)
Hyperplanation is all you need
Re:Transformers as Support Vector Machines
by jpfed on Sunday September 03, 2023 @02:48PM
(#37373122)
Someday I'm going to write a paper that achieves SOTA results with a nigh-incomprehensible mishmash of diverse techniques and title it "All You Need Considered Harmful".
Re:Transformers as Support Vector Machines
by tudorw on Sunday September 03, 2023 @04:16AM
(#37368667)
and multi-dimensional topological manifolds, maybe :)
Re:Transformers as Support Vector Machines
by revskill on Sunday September 03, 2023 @04:40AM
(#37368776)
What is hyperplane ?
Re:Transformers as Support Vector Machines
by noduerme on Sunday September 03, 2023 @04:46AM
(#37368808)
it's a word (a made up word)
Re:Transformers as Support Vector Machines
by MattPalmer1086 on Sunday September 03, 2023 @04:56AM
(#37368856)
All words are made up!
Re:Transformers as Support Vector Machines
by noduerme on Sunday September 03, 2023 @07:54AM
(#37369671)
Yeah, but only a few are made up to seem like terms of art designed to obfuscate their actual meaning; and usually prepending "hyper-" to something is a signal that a more clear description of the thing doesn't yet exist.
Downvote away, fellas.
Re:Transformers as Support Vector Machines
by adamnemecek on Sunday September 03, 2023 @12:02PM
(#37371467)
Re:Transformers as Support Vector Machines
by adamnemecek on Sunday September 03, 2023 @04:54AM
(#37368845)
For 2d, a line, for 3d a plane, for nd a hyperplane.
Re:Transformers as Support Vector Machines
by shakow on Sunday September 03, 2023 @05:42AM
(#37369032)
A subspace of dimension n-1 of a n-dimensional vector space. It is an extension of the well-known concept of
a 2d-plane in a 3d-space to nd-spaces.
You could also describe a hyperplane as the set of solutions of a system of linear equations.
Re:Transformers as Support Vector Machines
by shakow on Sunday September 03, 2023 @04:53PM
(#37374452)
Or as the subspace of all the vectors are orthogonal to a given single vector, or as the subspace generated by any orthogonal basis with one base vector removed, or as the kernel of a linear form, ... – but a more visual explanation is probably better as a first foray in the question.
I agree that a more visual explanation is better in general.
I was trying to hint how the visual explanation relates to the long vectors of numbers we actually feed our machine learning contraptions with. Not sure I was successful.
Re:Transformers as Support Vector Machines
by revskill on Monday September 04, 2023 @01:54AM
(#37377541)
This is really good !
Re:Transformers as Support Vector Machines
by loehnsberg on Sunday September 03, 2023 @06:36AM
(#37369264)
A hyperplane is a multi-dimensional linear function that splits space into two distinct regions. In the context of a classifier, it splits feature space into disjunct sub-spaces (one for each class). SVMs effectively place a hyperplane with maximum margin, thereby separating classes in an optimal way.
Re:Transformers as Support Vector Machines
by im3w1l on Sunday September 03, 2023 @12:17PM
(#37371603)
Worth keeping in mind that though it may be optimal according to some mathematical criterion, that is no guarantee that it's the best for the purposes you have in mind.
a partition of Euclidean space into two convex sets ;)
Re:Transformers as Support Vector Machines
by mafribe on Sunday September 03, 2023 @08:06AM
(#37369729)
This is wrong!
The term hyperplane already assumes that the hypothesis space that your learning algorithm searches has some kind of dimension and is some variant of an Euclidean / vector space (and its generalisations). This is not the case for many forms of ML, for example grammar induction (where the hypothesis space is Chomsky-style grammars) or inductive logic programming (hypothesis space are Prolog (or similar) programs), or, more generally, program synthesis (where programs form the hypothesis space).
Re:Transformers as Support Vector Machines
by adamnemecek on Sunday September 03, 2023 @01:03PM
(#37371993)
It can also just be some sort of partitioning. I would be really surprised if there was no partitioning of some space.
Re:Transformers as Support Vector Machines
by mafribe on Sunday September 03, 2023 @01:32PM
(#37372243)
Note that "some sort of partitioning" isn't a hyperplane. A partition is a set-theoretic concept. A hyperplane is (a generalisation of) a geometric concept, so has much more structure.
Re:Transformers as Support Vector Machines
by adamnemecek on Sunday September 03, 2023 @03:11PM
(#37373363)
Alright how about coalgebra.