News for hackers. Stuff that matters.
Re:Python 3.5 to Ship with Matrix Multiplication Operator by zurn on Tuesday April 08, 2014 @03:42AM (#7551956)
To everyone arguing against special-casing matrix multiplication, please base your arguments on the PEP's "Motivation" section to avoid too much of rehashing the obvious. It even has a subsection "But isn't matrix multiplication a pretty niche requirement?"
Re:Python 3.5 to Ship with Matrix Multiplication Operator by cabinpark on Tuesday April 08, 2014 @04:47AM (#7552183)
I would argue that almost every single linear algebra routine can be interpreted in some form as a matrix-matrix or matrix-vector multiplication. Considering that matrix-matrix and matrix-vector are fundamental numerical operations (they are part of BLAS which are the basic API functions called by every single linear algebra routine I know of) having a concise notation is key.

And since formulas can get complicated quickly, having a closer 1-1 correspondence with the mathematics is critical for understanding the meaning of the code. The PEP contains a nice example from statistics. And trust me, when you are writing numerical code, being able to read the mathematical formula clearly is essential, especially when you have a lot of formulas and need to figure out why your code is giving you numerical non-sense.

Re:Python 3.5 to Ship with Matrix Multiplication Operator by scythe on Tuesday April 08, 2014 @11:35AM (#7554013)
>I would argue that almost every single linear algebra routine can be interpreted in some form as a matrix-matrix or matrix-vector multiplication.

It's not really necessary to argue. Cayley's theorem guarantees that every group is a subgroup of a symmetric group:

http://en.wikipedia.org/wiki/Cayley's_theorem

A symmetric group is then a subgroup of the general linear group in any field, where the general linear group GL(K, N) is the set of invertible NxN matrices with entries from a "field" (just think "numbers") and therefore any [finitely generated] group is isomorphic to a group of matrices under matrix multiplication; this is the underlying concept of representation theory.

If we back up a little, a group is a very general sort of algebraic structure; many important concepts have an underlying group structure, such as rotations, permutations, and any sort of reversible computation. This latter case implies that matrix multiplication is Turing complete; the simplest such set of matrices is generated by the Toffoli gate matrix. The relationship of groups to geometry is due to the underlying correspondence between the axioms of a group and those of geometrical transformations. A set of reversible geometric transformations includes an identity element -- do nothing -- and obeys associativity (sorta complicated, but it makes sense if you think about it) and inverse operations (by assumption): this makes it a group, and it can be represented by matrices. If we remove "reversible", we get exceptions -- like the cross product -- but these are usually related to groups (cross product -> quaternion algebra).

So matrix multiplication is actually really, really fundamental in a lot of mathematics. It's also a special case of tensor contraction, which could justify another tower post (but won't).

>And trust me, when you are writing numerical code, being able to read the mathematical formula clearly is essential, especially when you have a lot of formulas and need to figure out why your code is giving you numerical non-sense.

(spent three months chasing a bug where two commands were out of order)

Re:Python 3.5 to Ship with Matrix Multiplication Operator by JadeNB on Tuesday April 08, 2014 @05:20PM (#7556090)
> A symmetric group is then a subgroup of the general linear group in any field, where the general linear group GL(K, N) is the set of invertible NxN matrices with entries from a "field" (just think "numbers") and therefore any [finitely generated] group is isomorphic to a group of matrices under matrix multiplication; this is the underlying concept of representation theory.

As a representation theorist, I am very sympathetic to this point of view, but I'm not sure that it proves that "[almost] every single linear algebra routine can be interpreted as a matrix-matrix multiplication"—unless one first has some reduction from an arbitrary linear-algebra routine to a group action.

> A symmetric group is then a subgroup of the general linear group in any field, where the general linear group GL(K, N) is the set of invertible NxN matrices with entries from a "field" (just think "numbers") and therefore any [finitely generated] group is isomorphic to a group of matrices under matrix multiplication; this is the underlying concept of representation theory.

Also, as a very minor nitpick, I think that you want 'finite' instead of 'finitely generated'; even infinite groups embed in (infinite) symmetric groups, but it's not obvious to me that infinite symmetric groups embed in (finite-dimensional) matrix groups, and it's certainly not true (just by counting cardinality) that infinite but finitely generated groups embed in finite symmetric groups.

Re:Python 3.5 to Ship with Matrix Multiplication Operator by omaranto on Tuesday April 08, 2014 @09:27AM (#7553218)
What you say about almost all linear algebra routines being multiplication sounds crazy to me. How are the following supposed to be multiplication: (1) computing the transpose, (2) finding a basis for the kernel of a matrix, (3) factorizations, such as QR, LU or SVD (3) computing the Jordan normal form of a matrix, etc.?
Re:Python 3.5 to Ship with Matrix Multiplication Operator by j2kun on Tuesday April 08, 2014 @10:38AM (#7553655)
It's not as crazy as it sounds, though I wouldn't make such a blanket statement. Things can be phrased in terms of matrix multiplication (maybe not a single matrix multiplication), it's just not the most efficient way to go about it.

1. Transpose is a linear map on matrices (vectors in R^nm), so in a very concrete sense it is precisely a matrix multiplication. And it's not hard to figure out the matrix, because it's analogous to the matrix that swaps entries in vectors.

2. Finding the first eigenvalue can be approximated via matrix multiplication, and for sparse matrices with good spectral gap this extends to all of them.

3. Row reduction can be phrased as matrix multiplication, and hence finding the basis for the kernel of a matrix is a byproduct of it, as is computing eigenvectors when given eigenvalues.

4. Computing orthonormal basis is a sequence of projection operators (and all linear maps are matrix multiplications)

5. I'm pretty sure nobody computes the Jordan canonical form on computers.

The point is that the connection between matrices and linear maps is the spirit of linear algebra.

Re:Python 3.5 to Ship with Matrix Multiplication Operator by jjoonathan on Tuesday April 08, 2014 @12:18PM (#7554335)
It is as crazy as it sounds because we were talking about implementing an actual language/library, not about doing the linear-algebra equivalent of restating Turing's thesis.

> not the most efficient way to go about it

understatement of the century.

Re:Python 3.5 to Ship with Matrix Multiplication Operator by j2kun on Tuesday April 08, 2014 @03:59PM (#7555622)
> we were talking about implementing an actual language/library

No, we're talking about language-level syntactic standardization of the most fundamental computational concept in linear algebra (and I would argue all of mathematics). My list just gives evidence to how unifying the concept is.

If you want supercomputing-grade efficiency you won't stop using numpy/scipy just because of this update, or you've already rewritten your Python prototype in C/Fortran anyway.

Re:Python 3.5 to Ship with Matrix Multiplication Operator by dragonwriter on Tuesday April 08, 2014 @07:08PM (#7556606)
> If you want supercomputing-grade efficiency you won't stop using numpy/scipy just because of this update

Especially since the update is just creating a language feature for numpy and similar libraries to leverage -- there is still no matrix implementation (and thus no matrix multiplication) in the standard library, just an overloadable operator so that numerical libraries can consistently use it for matrix multiplication and use * for elementwise multiplication, providing a standard API for those libraries.

Re:Python 3.5 to Ship with Matrix Multiplication Operator by GFK_of_xmaspast on Tuesday April 08, 2014 @11:29AM (#7553981)
Lots of factorizations such as the NMF can be implemented as a hill climb where you just multiply things against other things (and normalize elementwise)
Re:Python 3.5 to Ship with Matrix Multiplication Operator by wyuenho on Tuesday April 08, 2014 @08:48AM (#7553018)
It seems like I can boil down the rationale as:

1. Numpy people overload * for element-wise mul and matmul inconsistently and confusing people 2. The prefix func calls convention for matmul makes the formulas difficult to read 3. Python's precedence for splitting / into / and // can't apply to matmul because * * is already taken. 4. ` is banned, ?! lend unrelated meanings to context, $ is Perl and PHP baggage, so @ is the only thing left?

I got lost on the choice for @. If ?! lends unrelated meaings, and $ is Perlism, why doesn't @ suggest some kind of concat ops or Bashish/Perlish array sigils?

I'd personally much prefer >< . It looks like x, so it's much clearer. I don't understand the PEP's reason for not using ><.

Re:Python 3.5 to Ship with Matrix Multiplication Operator by kzrdude on Tuesday April 08, 2014 @09:33AM (#7553251)
Yes `backticks` are banned since Python 3 because 1) They are hard to type many on common keyboard layouts 2) They are hard to read, especially in Python books.
Re:Python 3.5 to Ship with Matrix Multiplication Operator by tomp on Tuesday April 08, 2014 @12:32PM (#7554441)
I don't understand the hate for backticks either. I used to dislike them too, when I was just a hobby programmer, and used a Slovenian keyboard layout (even for programming). Needless to say, after switching to English layout, my programming skills quadrupled - ` isn't the only symbol barely accessible (without finger acrobatics) on international keyboards (other examples include: {}][|~\^ ).
Re:Python 3.5 to Ship with Matrix Multiplication Operator by kzrdude on Tuesday April 08, 2014 @08:30PM (#7556971)
I've stuck to Swedish layout but I've remapped all of `{}[]()\ in fact! :-)
Re:Python 3.5 to Ship with Matrix Multiplication Operator by Dorian-Marie on Tuesday April 08, 2014 @11:10AM (#7553861)
I use backticks a lot when writing Markdown and Ruby code and it's easy to type (on a Mac keyboard at least) and is as easy to print as any other character.
Re:Python 3.5 to Ship with Matrix Multiplication Operator by kzrdude on Tuesday April 08, 2014 @11:32AM (#7553996)
Mac keyboard layout varies by both country and language though.
Re:Python 3.5 to Ship with Matrix Multiplication Operator by ak217 on Tuesday April 08, 2014 @10:29AM (#7553598)
I don't get why they don't just let * do matrix multiplication on matrices, and introduce .* for element-wise multiplication.
Re:Python 3.5 to Ship with Matrix Multiplication Operator by dagw on Tuesday April 08, 2014 @10:52AM (#7553741)
Because * has been element-wise multiplication in python numeric/numpy since the python 1.0 days. Introducing such a massive and fundamental backwards incompatible syntax change just as python 3 is starting to settle down and slowly gaining acceptance is probably not a good idea
Re:Python 3.5 to Ship with Matrix Multiplication Operator by ak217 on Tuesday April 08, 2014 @12:10PM (#7554278)
Great point, I forgot about that. In that case, I wish they'd used * * * or something. @ just doesn't seem right to me.
Re:Python 3.5 to Ship with Matrix Multiplication Operator by pekk on Tuesday April 08, 2014 @12:34PM (#7554456)
This is a problem for python numeric/numpy, not Python.

Let them fix their own problems

Re:Python 3.5 to Ship with Matrix Multiplication Operator by dragonwriter on Tuesday April 08, 2014 @12:51PM (#7554559)
The change being made to python exists to enable the fix preferred by much of the community to the API fragmentation problem that exists among python numeric libraries given the need for both convenient matrix multiplicationa and convenient elementwise multiplication.

Since its a fairly dominant application area that is pretty key for Python (there's a reason there are so many bundled python distributions that include the common scientific/numeric libraries, and that those environments are often chosen as pedagogical tools even for general-purpose programming), making a fairly modest language-level change to enable a clean resolution to this fragmentation is a sensible thing to do.

Re:Python 3.5 to Ship with Matrix Multiplication Operator by sieisteinmodel on Tuesday April 08, 2014 @02:52PM (#7555222)
Also, A x B is not used for dot product in the math world. That is mostly cross product, and might trigger wrong associations.
Re:Python 3.5 to Ship with Matrix Multiplication Operator by adamtj on Tuesday April 08, 2014 @08:07PM (#7556872)
Speaking of the obvious, I'm somewhat surprised that the PEP didn't discuss dropping the "dot" function and using the existing function call operator directly. It seems like a natural idea to me. After all, a matrix represents a linear transformation, and a linear transformation is ... a function.

If you add some whitespace and squint just right, it even kinda sorta looks like math. If you further wrap all matrices and vectors in parentheses, you can pretend it's a whole new operator that lets you do matrix multiplication by juxtaposition.

  S = ((H) (beta) - r).T (inv((H) (V) (H.T))) ((H) (beta) - r)
Still, I think this is more readable:

  S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)