Look at the "working with images" section. If you manually touch every pixel in an image, you're doing something wrong. You code will be 5X as long, and 100X as slow.
Re:Scipy - the embarrassing way to code
by brent on Tuesday May 06, 2008 @03:41PM
(#182745)
This is exactly why I have urged people to learn Matlab (on this forum and others). Specifically, I'm sick of seeing hundreds of lines of code written in Java that are essentially a poor (slow) implementation of a one liner in Matlab that uses a 20 year old fortran library. I have to go back all of 8 days to find the following quote in response to my plea:
"...and I question the usefulness of learning MATLAB unless you actually need it."
The problem is, most people don't realize they need it and just plod along writing hundreds of lines of python, perl, ruby, java, etc. implementing a decade old numerical computation.
Re:Scipy - the embarrassing way to code
by ejs on Tuesday May 06, 2008 @03:51PM
(#182753)
When you write a lot of matlab it becomes a different way of thinking. Instead of the for loops multiple levels deep you can write it all as operations to multi-dimensional matrices.
I would agree that many problems people spend time on can be found as single commands in matlab.
Re:Scipy - the embarrassing way to code
by gruseom on Tuesday May 06, 2008 @04:29PM
(#182806)
The comments here on Matlab have me intrigued. I'm ignorant and have a few newb questions. How does it compare to the APL family of vector languages (J/K/Q)? How does it compare to Mathematica? What roughly are the licensing terms for commercial use (i.e. if you want to build a product with it)?
Re:Scipy - the embarrassing way to code
by randomhack on Tuesday May 06, 2008 @09:06PM
(#183026)
I would suggest looking into NumPy/SciPy instead of Matlab. Matlab has a very good library for numerical computation but in my experience, Matlab is absolutely terrible as a language for anything except matrix stuff.
NumPy gives you a good matrix syntax while building on an excellent general purpose language. Personally I felt NumPy is a lot cleaner and Python a lot more fun than Matlab though Matlab has many many "toolboxes"/libraries not in NumPy.
Re:Scipy - the embarrassing way to code
by tokipin on Tuesday May 06, 2008 @06:55PM
(#182940)
it's an algol language with array manipulation. APL languages start with array manipulation as a primitive in the design of the language, so they are much better at these sorts of things
Matlab is for engineers, Mathematica is for scientists. Matlab is designed for speedy matrixy stuff. Mathematica uses a lisp-like structure 'replacement' pattern matching evaluation scheme, and intrinsically supports symbolic mathematics and is the the only language i would really call multiparadigmatic
Re:Scipy - the embarrassing way to code
by gruseom on Tuesday May 06, 2008 @07:36PM
(#182971)
Thanks. My project is calculation-intensive and performance is important. I would love to use kdb. Arthur Whitney's work is stunning. Unfortunately, their licenses are draconian. Can you offer any insight about these various options from the standpoint of building a commercial product on top of them?
Re:Scipy - the embarrassing way to code
by gruseom on Wednesday May 07, 2008 @04:33PM
(#183825)
Think of it as similar to a spreadsheet. The calculations are mostly linear algebra and are represented visually in 2d arrays.
What language is it currently written in
Common Lisp (at least, the calculation part). It's a great language for developing the product, but we don't want to greenspun our own vector-based calculation engine if we don't have to.
Re:Scipy - the embarrassing way to code
by tokipin on Tuesday May 06, 2008 @08:22PM
(#182989)
truthfully, i have no idea. it's certainly unfortunate that these languages/environments are not widely usable. i don't know of a more powerful scratchpad than Mathematica
have you looked at R? it's an open source matlab. though i don't know what kind of database options it might have
Re:Scipy - the embarrassing way to code
by brent on Wednesday May 07, 2008 @08:07AM
(#183310)
R isn't "an open source matlab". R is an open source implementation of S-plus. If anything is the open source Matlab it is octave. However, I agree they can be used interchangeably for solving many problems.
Re:Scipy - the embarrassing way to code
by maximilian on Wednesday May 07, 2008 @12:20AM
(#183155)
matlab costs boatloads of money for commercial stuff. At least from my perspective.
I wish there were an open alternative to matlab that was fast. Matlab is kinda fast if you do exactly the right thing, but its interpretedness gets in the way. Most of the people I know doing computational stuff write everything in c using some outside libraries to speed things up. Often they just code up all their own algorithms from scratch. Its lame, but they usually halve the computation time. I could imagine that SciPy and NumPy is much much faster than Matlab.
Matlab is crazy easy to code for though. Its really handy for prototyping.
Re:Scipy - the embarrassing way to code
by llimllib on Wednesday May 07, 2008 @02:02PM
(#183595)
> I could imagine that SciPy and NumPy is much much faster than Matlab.
SciPy and NumPy tend to be just about as fast as Matlab, when used in their default manner. When you drop into inline Blitz/C for your inner loops using Scipy though, you can achieve orders of magnitude improvements.
Re:Scipy - the embarrassing way to code
by carterschonwald on Tuesday May 06, 2008 @04:51PM
(#182822)
in terms of interface, it has more in common with mathematica (both good and bad) than do the APL style languages, basically the language semantics are C style with dynamic shape checking for array operations and a nice syntax for many different linear algebraic operations.
It'd be nice if there was a nice type system to aid in getting comfortable with matlab's semantics, in my opinion
Re:Scipy - the embarrassing way to code
by carterschonwald on Thursday May 08, 2008 @12:50AM
(#184213)
if you don't know any of the theory behind a problem, you don't even necessarily know if theres a solution thats useful in practice.
A good example in Computer Science is the various reductions (efficient or not )between different problems, where you relate the difficulty of the problem to any other computational problem which you find you're able to encode within the structure of the orginal problem.
Re:Scipy - the embarrassing way to code
by ola on Tuesday May 06, 2008 @04:24PM
(#182794)
The article seems to have two messages: one that it's better to fully comprehend the problem before you start coding; the other that learning a powerful library will save you pages of code. Both well rehearsed truths of computer programming.
As for myself I often find that I have to fully explore the problem on my own terms before I can maturely optimize my source.
Re:Scipy - the embarrassing way to code
by ola on Tuesday May 06, 2008 @09:00PM
(#183015)
That's a very true observation mhartl and I try to touch on it in my last sentence although it is enigmatically worded as I was trying to sneak a "Premature optimization is the root of all evil" reference into it. Alas not all wordplay is destined for upmods!
Re:Scipy - the embarrassing way to code
by plinkplonk on Tuesday May 06, 2008 @11:24PM
(#183115)
> give J (an APL descendant) a try,
J is not open source. Might be a factor to consider.
From the J site,
"Jsoftware licenses source for all the Jsoftware binaries. There are several factors in determining the cost of source licenses. Primary is the source itself: J Engine (portable C); Windows GUI support (C++); and Unix GUI support (Java). Secondary is the scope of use: internal; distributed products; and competitive (to Jsoftware) products. And finally the platforms: Windows; Unix; PocketPC; etc. The price range is from $10,000 to $400,000. Regular updates to our current source levels are available for separate update fees."
Re:Scipy - the embarrassing way to code
by bsaunder on Wednesday May 07, 2008 @12:25AM
(#183156)
I've noticed a similar conceptual pattern while investigating meta-programming. I think there's something fundamental going on here. Something that conventional programming constantly misses. Something remarkably simple, consistent and powerful. It's where code is data along a continuous path of transformations.
Python is going to replace Matlab, but the pattern in the same.
People aren't really used to thinking about matrices and vectors as primitive types and operations.
By the way, this is an excellent resource for learning Matlab. http://www.andrew.cmu.edu/course/16-720/handouts.html
Look at the "working with images" section. If you manually touch every pixel in an image, you're doing something wrong. You code will be 5X as long, and 100X as slow.
Good observation, succinctly stated. Most of my days are spent trying to figure out what I need to do, not how to do it.
"...and I question the usefulness of learning MATLAB unless you actually need it."
The problem is, most people don't realize they need it and just plod along writing hundreds of lines of python, perl, ruby, java, etc. implementing a decade old numerical computation.
I would agree that many problems people spend time on can be found as single commands in matlab.
NumPy gives you a good matrix syntax while building on an excellent general purpose language. Personally I felt NumPy is a lot cleaner and Python a lot more fun than Matlab though Matlab has many many "toolboxes"/libraries not in NumPy.
Matlab is for engineers, Mathematica is for scientists. Matlab is designed for speedy matrixy stuff. Mathematica uses a lisp-like structure 'replacement' pattern matching evaluation scheme, and intrinsically supports symbolic mathematics and is the the only language i would really call multiparadigmatic
What language is it currently written in
Common Lisp (at least, the calculation part). It's a great language for developing the product, but we don't want to greenspun our own vector-based calculation engine if we don't have to.
have you looked at R? it's an open source matlab. though i don't know what kind of database options it might have
I wish there were an open alternative to matlab that was fast. Matlab is kinda fast if you do exactly the right thing, but its interpretedness gets in the way. Most of the people I know doing computational stuff write everything in c using some outside libraries to speed things up. Often they just code up all their own algorithms from scratch. Its lame, but they usually halve the computation time. I could imagine that SciPy and NumPy is much much faster than Matlab.
Matlab is crazy easy to code for though. Its really handy for prototyping.
SciPy and NumPy tend to be just about as fast as Matlab, when used in their default manner. When you drop into inline Blitz/C for your inner loops using Scipy though, you can achieve orders of magnitude improvements.
Check it out: http://www.scipy.org/PerformancePython
It'd be nice if there was a nice type system to aid in getting comfortable with matlab's semantics, in my opinion
A good example in Computer Science is the various reductions (efficient or not )between different problems, where you relate the difficulty of the problem to any other computational problem which you find you're able to encode within the structure of the orginal problem.
As for myself I often find that I have to fully explore the problem on my own terms before I can maturely optimize my source.
Often the best way to understand a problem is to try to implement a solution.
http://www.jsoftware.com/stable.htm
and here's a post about it:
http://www.squidoo.com/jprogramming
J is not open source. Might be a factor to consider.
From the J site,
"Jsoftware licenses source for all the Jsoftware binaries. There are several factors in determining the cost of source licenses. Primary is the source itself: J Engine (portable C); Windows GUI support (C++); and Unix GUI support (Java). Secondary is the scope of use: internal; distributed products; and competitive (to Jsoftware) products. And finally the platforms: Windows; Unix; PocketPC; etc. The price range is from $10,000 to $400,000. Regular updates to our current source levels are available for separate update fees."
Haven't done anything with it yet, but it's on my radar.
It's sort of a set union of every good free math program.
The more you work, the less you appear to have.
The efficiency paradox.