News for hackers. Stuff that matters.
Re:Transformers as Support Vector Machines by adamnemecek on Sunday September 03, 2023 @02:40AM (#37368221)
All machine learning is about finding hyperplanes.
Re:Transformers as Support Vector Machines by nologic01 on Sunday September 03, 2023 @02:54AM (#37368269)
The large dimensionality seems to be what creates the need for heuristic designs rather than a generic approach
Re:Transformers as Support Vector Machines by adamnemecek on Sunday September 03, 2023 @02:56AM (#37368276)
Hyperplanes are the heuristics.
Re:Transformers as Support Vector Machines by alexmolas on Sunday September 03, 2023 @03:26AM (#37368415)
Hyperplanes is all you need
Re:Transformers as Support Vector Machines by quickthrower2 on Sunday September 03, 2023 @08:48AM (#37369999)
Hyperplanation is all you need
Re:Transformers as Support Vector Machines by jpfed on Sunday September 03, 2023 @02:48PM (#37373122)
Someday I'm going to write a paper that achieves SOTA results with a nigh-incomprehensible mishmash of diverse techniques and title it "All You Need Considered Harmful".
Re:Transformers as Support Vector Machines by tudorw on Sunday September 03, 2023 @04:16AM (#37368667)
and multi-dimensional topological manifolds, maybe :)
Re:Transformers as Support Vector Machines by revskill on Sunday September 03, 2023 @04:40AM (#37368776)
What is hyperplane ?
Re:Transformers as Support Vector Machines by noduerme on Sunday September 03, 2023 @04:46AM (#37368808)
it's a word (a made up word)
Re:Transformers as Support Vector Machines by MattPalmer1086 on Sunday September 03, 2023 @04:56AM (#37368856)
All words are made up!
Re:Transformers as Support Vector Machines by noduerme on Sunday September 03, 2023 @07:54AM (#37369671)
Yeah, but only a few are made up to seem like terms of art designed to obfuscate their actual meaning; and usually prepending "hyper-" to something is a signal that a more clear description of the thing doesn't yet exist.

Downvote away, fellas.

Re:Transformers as Support Vector Machines by adamnemecek on Sunday September 03, 2023 @12:02PM (#37371467)
Re:Transformers as Support Vector Machines by adamnemecek on Sunday September 03, 2023 @04:54AM (#37368845)
For 2d, a line, for 3d a plane, for nd a hyperplane.

https://en.wikipedia.org/wiki/Hyperplane

Re:Transformers as Support Vector Machines by shakow on Sunday September 03, 2023 @05:42AM (#37369032)
A subspace of dimension n-1 of a n-dimensional vector space. It is an extension of the well-known concept of a 2d-plane in a 3d-space to nd-spaces.
Re:Transformers as Support Vector Machines by eru on Sunday September 03, 2023 @06:17AM (#37369185)
You could also describe a hyperplane as the set of solutions of a system of linear equations.
Re:Transformers as Support Vector Machines by shakow on Sunday September 03, 2023 @04:53PM (#37374452)
Or as the subspace of all the vectors are orthogonal to a given single vector, or as the subspace generated by any orthogonal basis with one base vector removed, or as the kernel of a linear form, ... – but a more visual explanation is probably better as a first foray in the question.
Re:Transformers as Support Vector Machines by eru on Sunday September 03, 2023 @09:19PM (#37376237)
I agree that a more visual explanation is better in general.

I was trying to hint how the visual explanation relates to the long vectors of numbers we actually feed our machine learning contraptions with. Not sure I was successful.

Re:Transformers as Support Vector Machines by revskill on Monday September 04, 2023 @01:54AM (#37377541)
This is really good !
Re:Transformers as Support Vector Machines by loehnsberg on Sunday September 03, 2023 @06:36AM (#37369264)
A hyperplane is a multi-dimensional linear function that splits space into two distinct regions. In the context of a classifier, it splits feature space into disjunct sub-spaces (one for each class). SVMs effectively place a hyperplane with maximum margin, thereby separating classes in an optimal way.
Re:Transformers as Support Vector Machines by im3w1l on Sunday September 03, 2023 @12:17PM (#37371603)
Worth keeping in mind that though it may be optimal according to some mathematical criterion, that is no guarantee that it's the best for the purposes you have in mind.
Re:Transformers as Support Vector Machines by blt on Sunday September 03, 2023 @03:32PM (#37373572)
a partition of Euclidean space into two convex sets ;)
Re:Transformers as Support Vector Machines by mafribe on Sunday September 03, 2023 @08:06AM (#37369729)
This is wrong!

The term hyperplane already assumes that the hypothesis space that your learning algorithm searches has some kind of dimension and is some variant of an Euclidean / vector space (and its generalisations). This is not the case for many forms of ML, for example grammar induction (where the hypothesis space is Chomsky-style grammars) or inductive logic programming (hypothesis space are Prolog (or similar) programs), or, more generally, program synthesis (where programs form the hypothesis space).

Re:Transformers as Support Vector Machines by adamnemecek on Sunday September 03, 2023 @01:03PM (#37371993)
It can also just be some sort of partitioning. I would be really surprised if there was no partitioning of some space.
Re:Transformers as Support Vector Machines by mafribe on Sunday September 03, 2023 @01:32PM (#37372243)
Note that "some sort of partitioning" isn't a hyperplane. A partition is a set-theoretic concept. A hyperplane is (a generalisation of) a geometric concept, so has much more structure.
Re:Transformers as Support Vector Machines by adamnemecek on Sunday September 03, 2023 @03:11PM (#37373363)
Alright how about coalgebra.