News for hackers. Stuff that matters.
Re:The Excel Depression by gtani on Friday April 19, 2013 @12:10AM (#5574764)
I thought it was kind of well known: don't use Excel for floating point math, statistical analysis, anything moderately involved that would have to be reproducible and auditable

http://www.pages.drexel.edu/~bdm25/excel2007.pdf

http://support.microsoft.com/kb/78113

http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_...

Re:The Excel Depression by achal on Friday April 19, 2013 @12:33AM (#5574841)
From what I can tell, the actual issue in this case was that they simply summed the wrong thing, rather than ran into some floating point error.

Also, I haven't gone through the first paper, but that second link just seems to show how they generally conform to IEEE 754, except for three cases (div/0 errors instead of infinity, NaN errors immediately instead of allowing further computations, and [this one is a bit more important] they don't implement denormalized numbers). I'm not sure why that would mean Excel is terrible at floating point math.

Re:The Excel Depression by podperson on Friday April 19, 2013 @12:28AM (#5574820)
The errors in question has nothing to do with inherent flaws in Excel (beyond, perhaps, the difficulty of reviewing Excel code). The problems were in decisions made to exclude data, arbitrary "analysis" with no basis in statistics or common sense (19 - IIRC - years of data from England were given the same weight as one year of data from New Zealand), and an egregious formula error that simply ignored several rows of data.
Re:The Excel Depression by Toenex on Friday April 19, 2013 @05:08AM (#5575626)
beyond, perhaps, the difficulty of reviewing Excel code

This has always been one of my concerns when Excel is used much beyond 'electronic squared paper'. The interwoven data and code nature of the tool makes testing and verification more difficult - certainly not impossible but going against the grain of the tool. Excel is used by a diverse range of people, many of whom have limited software skills and could do with a tool that enforced the right behaviour.

Also I think this shows the need for openness when people make such important claims based on data. Particularly in this case where all the data and calculations could easily be zipped up and put on a website.

Re:The Excel Depression by james1071 on Saturday April 20, 2013 @05:06AM (#5580843)
Agree-the fundamental problem is that formulae are copied from cell to cell.

It is so easy to make a mistake on a big spreadsheet.

A much better approach would be to write user defined functions.

Re:The Excel Depression by tadfisher on Friday April 19, 2013 @01:46AM (#5575096)
To be fair, Excel's concept of a weighted average has no basis in statistics or common sense, as I have seen in a couple of published works.

Here's a tip: if your results are reproducible only in Excel, something is terribly wrong.

Re:The Excel Depression by podperson on Friday April 19, 2013 @08:23AM (#5576204)
I don't think they were using Excel's weighted average function in this case but simply averaging filtered figures per country and then averaging the averages. This has no statistical basis, but it's not Excel's fault
Re:The Excel Depression by mbetter on Friday April 19, 2013 @01:10AM (#5574990)
I would suggest that in a situation like this, "the difficulty of reviewing Excel code" is even overstated. The direct dependencies of any cell are discoverable with trivial effort.
Re:The Excel Depression by jabbernotty on Friday April 19, 2013 @12:38AM (#5574858)
It is not well known. I'm guessing that what you are saying isn't really the reality, based on what little I know of the oil and gas industry.
Re:The Excel Depression by klipt on Friday April 19, 2013 @04:36AM (#5575534)
I don't know if it's still the case, but when I was an undergrad a professor pointed out that Excel's Gaussian tail probabilities disagreed with everything else (Mathematica, Matlab, R, etc.) for large values.
Re:The Excel Depression by nandemo on Friday April 19, 2013 @02:34AM (#5575221)
The financial industry uses a lot of Excel too.
Re:The Excel Depression by jeff303 on Friday April 19, 2013 @09:32AM (#5576528)
Yeah but usually not as the system of record/origin for any data. Mostly for consuming data from elsewhere, and often in conjunction with various plugins (some of which can allow Excel to act as the "front end" to a production server that actually crunches/stores the numbers).
Re:The Excel Depression by mbetter on Friday April 19, 2013 @01:05AM (#5574972)
Your citations don't even remotely suggest your conclusion.