News for hackers. Stuff that matters.
Re:Scipy - the embarrassing way to code by ivankirigin on Tuesday May 06, 2008 @04:01PM (#182763)
I've done lots of Matlab development, and this is too true.

Python is going to replace Matlab, but the pattern in the same.

People aren't really used to thinking about matrices and vectors as primitive types and operations.

By the way, this is an excellent resource for learning Matlab. http://www.andrew.cmu.edu/course/16-720/handouts.html

Look at the "working with images" section. If you manually touch every pixel in an image, you're doing something wrong. You code will be 5X as long, and 100X as slow.

Re:Scipy - the embarrassing way to code by scott_s on Tuesday May 06, 2008 @02:43PM (#182690)
The bottleneck in writing code isn’t in the writing of the code, it’s in understanding and conceptualising what needs to be done.

Good observation, succinctly stated. Most of my days are spent trying to figure out what I need to do, not how to do it.

Re:Scipy - the embarrassing way to code by noahlt on Wednesday May 07, 2008 @10:20PM (#184114)
Exception: interfacing with 3rd-party libraries.
Re:Scipy - the embarrassing way to code by brent on Tuesday May 06, 2008 @03:41PM (#182745)
This is exactly why I have urged people to learn Matlab (on this forum and others). Specifically, I'm sick of seeing hundreds of lines of code written in Java that are essentially a poor (slow) implementation of a one liner in Matlab that uses a 20 year old fortran library. I have to go back all of 8 days to find the following quote in response to my plea:

"...and I question the usefulness of learning MATLAB unless you actually need it."

The problem is, most people don't realize they need it and just plod along writing hundreds of lines of python, perl, ruby, java, etc. implementing a decade old numerical computation.

Re:Scipy - the embarrassing way to code by ejs on Tuesday May 06, 2008 @03:51PM (#182753)
When you write a lot of matlab it becomes a different way of thinking. Instead of the for loops multiple levels deep you can write it all as operations to multi-dimensional matrices.

I would agree that many problems people spend time on can be found as single commands in matlab.

Re:Scipy - the embarrassing way to code by gruseom on Tuesday May 06, 2008 @04:29PM (#182806)
The comments here on Matlab have me intrigued. I'm ignorant and have a few newb questions. How does it compare to the APL family of vector languages (J/K/Q)? How does it compare to Mathematica? What roughly are the licensing terms for commercial use (i.e. if you want to build a product with it)?
Re:Scipy - the embarrassing way to code by randomhack on Tuesday May 06, 2008 @09:06PM (#183026)
I would suggest looking into NumPy/SciPy instead of Matlab. Matlab has a very good library for numerical computation but in my experience, Matlab is absolutely terrible as a language for anything except matrix stuff.

NumPy gives you a good matrix syntax while building on an excellent general purpose language. Personally I felt NumPy is a lot cleaner and Python a lot more fun than Matlab though Matlab has many many "toolboxes"/libraries not in NumPy.

Re:Scipy - the embarrassing way to code by carterschonwald on Tuesday May 06, 2008 @10:40PM (#183090)
I second this sentiment. The difference in natural expressiveness is almost painful
Re:Scipy - the embarrassing way to code by tokipin on Tuesday May 06, 2008 @06:55PM (#182940)
it's an algol language with array manipulation. APL languages start with array manipulation as a primitive in the design of the language, so they are much better at these sorts of things

Matlab is for engineers, Mathematica is for scientists. Matlab is designed for speedy matrixy stuff. Mathematica uses a lisp-like structure 'replacement' pattern matching evaluation scheme, and intrinsically supports symbolic mathematics and is the the only language i would really call multiparadigmatic

Re:Scipy - the embarrassing way to code by gruseom on Tuesday May 06, 2008 @07:36PM (#182971)
Thanks. My project is calculation-intensive and performance is important. I would love to use kdb. Arthur Whitney's work is stunning. Unfortunately, their licenses are draconian. Can you offer any insight about these various options from the standpoint of building a commercial product on top of them?
Re:Scipy - the embarrassing way to code by brent on Wednesday May 07, 2008 @08:08AM (#183313)
Perhaps I can help, but first. What calculations are you doing (e.g. are they all linear algebra?) What language is it currently written in?
Re:Scipy - the embarrassing way to code by gruseom on Wednesday May 07, 2008 @04:33PM (#183825)
Think of it as similar to a spreadsheet. The calculations are mostly linear algebra and are represented visually in 2d arrays.

What language is it currently written in

Common Lisp (at least, the calculation part). It's a great language for developing the product, but we don't want to greenspun our own vector-based calculation engine if we don't have to.

Re:Scipy - the embarrassing way to code by tokipin on Tuesday May 06, 2008 @08:22PM (#182989)
truthfully, i have no idea. it's certainly unfortunate that these languages/environments are not widely usable. i don't know of a more powerful scratchpad than Mathematica

have you looked at R? it's an open source matlab. though i don't know what kind of database options it might have

Re:Scipy - the embarrassing way to code by brent on Wednesday May 07, 2008 @08:07AM (#183310)
R isn't "an open source matlab". R is an open source implementation of S-plus. If anything is the open source Matlab it is octave. However, I agree they can be used interchangeably for solving many problems.
Re:Scipy - the embarrassing way to code by maximilian on Wednesday May 07, 2008 @12:20AM (#183155)
matlab costs boatloads of money for commercial stuff. At least from my perspective.

I wish there were an open alternative to matlab that was fast. Matlab is kinda fast if you do exactly the right thing, but its interpretedness gets in the way. Most of the people I know doing computational stuff write everything in c using some outside libraries to speed things up. Often they just code up all their own algorithms from scratch. Its lame, but they usually halve the computation time. I could imagine that SciPy and NumPy is much much faster than Matlab.

Matlab is crazy easy to code for though. Its really handy for prototyping.

Re:Scipy - the embarrassing way to code by llimllib on Wednesday May 07, 2008 @02:02PM (#183595)
> I could imagine that SciPy and NumPy is much much faster than Matlab.

SciPy and NumPy tend to be just about as fast as Matlab, when used in their default manner. When you drop into inline Blitz/C for your inner loops using Scipy though, you can achieve orders of magnitude improvements.

Check it out: http://www.scipy.org/PerformancePython

Re:Scipy - the embarrassing way to code by carterschonwald on Tuesday May 06, 2008 @04:51PM (#182822)
in terms of interface, it has more in common with mathematica (both good and bad) than do the APL style languages, basically the language semantics are C style with dynamic shape checking for array operations and a nice syntax for many different linear algebraic operations.

It'd be nice if there was a nice type system to aid in getting comfortable with matlab's semantics, in my opinion

Re:Scipy - the embarrassing way to code by carterschonwald on Tuesday May 06, 2008 @04:25PM (#182800)
I think the deeper idea to take away from this is that knowing more theory is always a good thing
Re:Scipy - the embarrassing way to code by carterschonwald on Wednesday May 07, 2008 @03:47PM (#183778)
which begs the question of why it seems that even in pretty technical domains, theres still an aversion to aggressive use of theory tools?
Re:Scipy - the embarrassing way to code by brent on Wednesday May 07, 2008 @10:58PM (#184149)
What does this have to do with theory?
Re:Scipy - the embarrassing way to code by carterschonwald on Thursday May 08, 2008 @12:50AM (#184213)
if you don't know any of the theory behind a problem, you don't even necessarily know if theres a solution thats useful in practice.

A good example in Computer Science is the various reductions (efficient or not )between different problems, where you relate the difficulty of the problem to any other computational problem which you find you're able to encode within the structure of the orginal problem.

Re:Scipy - the embarrassing way to code by ola on Tuesday May 06, 2008 @04:24PM (#182794)
The article seems to have two messages: one that it's better to fully comprehend the problem before you start coding; the other that learning a powerful library will save you pages of code. Both well rehearsed truths of computer programming.

As for myself I often find that I have to fully explore the problem on my own terms before I can maturely optimize my source.

Re:Scipy - the embarrassing way to code by mhartl on Tuesday May 06, 2008 @08:08PM (#182981)
it's better to fully comprehend the problem before you start coding

Often the best way to understand a problem is to try to implement a solution.

Re:Scipy - the embarrassing way to code by ola on Tuesday May 06, 2008 @09:00PM (#183015)
That's a very true observation mhartl and I try to touch on it in my last sentence although it is enigmatically worded as I was trying to sneak a "Premature optimization is the root of all evil" reference into it. Alas not all wordplay is destined for upmods!
Re:Scipy - the embarrassing way to code by Prrometheus on Wednesday May 07, 2008 @12:14AM (#183150)
Would it be advisable to try to implement a solution on a chalkboard first? Mathematical notation and diagrams can be amazingly succinct.
Re:Scipy - the embarrassing way to code by tokipin on Tuesday May 06, 2008 @06:58PM (#182942)
give J (an APL descendant) a try, it has some good tutorial thingies that appear right in the console thingie:

http://www.jsoftware.com/stable.htm

and here's a post about it:

http://www.squidoo.com/jprogramming

Re:Scipy - the embarrassing way to code by plinkplonk on Tuesday May 06, 2008 @11:24PM (#183115)
> give J (an APL descendant) a try,

J is not open source. Might be a factor to consider.

From the J site,

"Jsoftware licenses source for all the Jsoftware binaries. There are several factors in determining the cost of source licenses. Primary is the source itself: J Engine (portable C); Windows GUI support (C++); and Unix GUI support (Java). Secondary is the scope of use: internal; distributed products; and competitive (to Jsoftware) products. And finally the platforms: Windows; Unix; PocketPC; etc. The price range is from $10,000 to $400,000. Regular updates to our current source levels are available for separate update fees."

Re:Scipy - the embarrassing way to code by scott_s on Tuesday May 06, 2008 @07:33PM (#182970)
The introduction to J that peaked my interest: http://prog21.dadgum.com/14.html

Haven't done anything with it yet, but it's on my radar.

Re:Scipy - the embarrassing way to code by mooneater on Tuesday May 06, 2008 @11:26PM (#183116)
Re:Scipy - the embarrassing way to code by JulianMorrison on Wednesday May 07, 2008 @05:22PM (#183880)
I can't believe nobody has mentioned Sage Math (http://www.sagemath.org/).

It's sort of a set union of every good free math program.

Re:Scipy - the embarrassing way to code by bsaunder on Wednesday May 07, 2008 @12:25AM (#183156)
I've noticed a similar conceptual pattern while investigating meta-programming. I think there's something fundamental going on here. Something that conventional programming constantly misses. Something remarkably simple, consistent and powerful. It's where code is data along a continuous path of transformations.
Re:Scipy - the embarrassing way to code by ambition on Wednesday May 07, 2008 @10:18AM (#183373)
Read Godel Escher Bach. Given the comment above, you'll like it.
Re:Scipy - the embarrassing way to code by edw519 on Tuesday May 06, 2008 @04:13PM (#182783)
This is a familiar story, regardless of technology.

The more you work, the less you appear to have.

The efficiency paradox.

Re:Scipy - the embarrassing way to code by bridgetroll on Tuesday May 06, 2008 @04:18PM (#182788)
"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." -Antoine de Saint Exupéry
Re:Scipy - the embarrassing way to code by edw519 on Tuesday May 06, 2008 @04:20PM (#182791)
Bulletin board material.
Re:Scipy - the embarrassing way to code by bridgetroll on Tuesday May 06, 2008 @04:27PM (#182804)
I have it posted at my desk in a nameplate frame. I try to adhere to it. I often fail.
Re:Scipy - the embarrassing way to code by rapind on Tuesday May 06, 2008 @08:27PM (#182995)
Great article. Learn the hard way and it'll stick. Plus it's fun.