News for hackers. Stuff that matters.
Re:Gcc compile error with 2GB of data by joeyh on Thursday June 09, 2011 @08:14PM (#2639262)
What a great technical problem, except for this bit:

> Btw, I didn't try to hide behind "this is scientific computing -- no way to optimize". It's just that the basis for this code is something that comes out of a "black box" where I have no real access to

Black box != science.

Re:Gcc compile error with 2GB of data by esrauch on Thursday June 09, 2011 @08:56PM (#2639355)
A research assistantship I held was based on classified data; all of the published work had to be approved by the DoD and the actual data we used wasn't allowed to be published which made our results entirely unreproducable.
Re:Gcc compile error with 2GB of data by jedbrown on Friday June 10, 2011 @06:19AM (#2640331)
I'm certain the methods being used here are just wrong. It looks like it's code generated by a symbolic algebra system. Chances are that using a more scalable methodology would eliminate the code size problem. For example, symbolic manipulations often have exponential space complexity in the depth of the expression tree, but automatic differentiation can do the required operation in polynomial time and space (often linear or log-linear).

Unfortunately, "no way to optimize" is basically a statement that the questioner is not interested in an algorithmic way to make the problem go away.

Re:Gcc compile error with 2GB of data by roel_v on Friday June 10, 2011 @05:38AM (#2640264)
It seems you don't understand the context of research projects. When you're assigned to write a program that processes data x coming group another group, it's valid to say that that input data is a 'black box' to you. That still doesn't mean that the whole research is 'black box' or somehow not 'real science'.
Re:Gcc compile error with 2GB of data by eru on Thursday June 09, 2011 @08:50PM (#2639339)
I wish that was true.
Re:Gcc compile error with 2GB of data by sevenproxies on Thursday June 09, 2011 @10:02PM (#2639501)
Science is a blackbox. We make theories about the world we live in based upon the results of experiments. Richard Feynman commented method of thinking about science on one of his videos.

http://www.youtube.com/watch?v=o1dgrvlWML4

Re:Gcc compile error with 2GB of data by jerf on Thursday June 09, 2011 @10:06PM (#2639510)
Science studies a black box. The science itself shouldn't be a black box. What is the added value of a black box representation of a black box?
Re:Gcc compile error with 2GB of data by cbr on Thursday June 09, 2011 @10:52PM (#2639590)
> What is the added value of a black box representation of a black box?

The ability to make predictions.

Re:Gcc compile error with 2GB of data by jerf on Friday June 10, 2011 @12:20AM (#2639789)
How? You handed me a black box. You wrote some code, and you fed it some data, and you made some prediction, and all I really have is your word for it that your prediction worked, but I sure can't do anything with it. I don't know what it is or how it works, how to extend it, how to build on it, anything.

That's not science by any useful definition of the term.

Re:Gcc compile error with 2GB of data by shareme on Friday June 10, 2011 @03:15AM (#2640077)
wht do you call a machine so complicated that we do not know its source code?

Its called the human body and the black box is known as Medical Science..:)

Re:Gcc compile error with 2GB of data by robin_reala on Friday June 10, 2011 @05:13AM (#2640230)
We know its source code. We just don’t understand it.

http://www.ornl.gov/sci/techresources/Human_Genome/home.shtm...