News for hackers. Stuff that matters.
Re:Government Open Source Office deal set to provide major s... by lifeisstillgood on Tuesday October 20, 2015 @07:40AM (#10418565)
Do we still care about anything other than excel? And we care about excel because it is far and away the best fastest and most robust spreadsheet out there.

This focus on office applications as the way in for Open Source software seems to me like transforming 19th Century transport by building faster horses.

Re:Government Open Source Office deal set to provide major s... by vidarh on Tuesday October 20, 2015 @07:44AM (#10418580)
Who are "we"?

It's been a decade since I've worked anywhere were anyone cared about Excel. I realize that I'm working in a "bubble" given the type of companies I tend to work with, but it does clearly illustrate that, yes, there are certainly people and companies that care about spreadsheets other than Excel.

Re:Government Open Source Office deal set to provide major s... by lifeisstillgood on Tuesday October 20, 2015 @03:35PM (#10421254)
Excel is cared about a lot in some industries - finance it is still running huge amounts of the workflow (from hedge funds to major banks whose auditors pull their hair out)

The main point I'm making is that of the troika of office applications, only the spreadsheet still has value. I suspect I could teach everyone Markdown far faster and with better outcomes than teach Word users to move to (open office equivalent)

It's something about software literacy - you just don't need Word anymore.

Re:Government Open Source Office deal set to provide major s... by chris_wot on Tuesday October 20, 2015 @09:07AM (#10418865)
Actually, there are a whole raft of issues you are conflating here. You need to ask why Excel is so widely used. It is used as:

Use Case 1: A basic spreadsheet for data manipulation, with no extensive macros or VBA involved. People use it for vlookups and other formulas they find useful.

Use Case 2: As a statistical analysis tool to crunch data.

Use Case 3: As a software platform, making extensive use of Visual Basic for Applications (VBA).

So here are the issues I have with your statement that Excel "is far and away the best, fastest and most robust spreadsheet out there"...

Fallacy 1: Excel is the best technical option

Excel is not necessarily the fastest spreadsheet out there for all operations. In many cases I believe it is still very fast, and beats LibreOffice under certain document loads, but this is often not by much and in fact there is ongoing work towards locating problem documents and profiling of LibreOffice code to remove bottlenecks. In fact, in LO 4.2 many performance bottlenecks were largely solved. [1]

At this point, I do think it's largely a myth that Calc is dreadfully slow. Even if it should be found that there are performance issues, however, I know that the LO community, and in particular Collabora, have a good story around their ability to use valgrind to benchmark and analyze results. As a normal part of development work, Collabora are often provided with documents that they analyse and contribute performance improvements into the core LO code base.

There are many, many errors in even basic statistical functions in most spreadsheet applications, but Microsoft Excel is particularly egregious. However, IMO the LO devs are more thorough when it comes to this [2][3]. So for use case 2 (statistical work) then I think it's not suitable (actually, for any stats work, why not just use R?)

So if Excel is not necessarily better than Calc in terms of performance, accuracy or overall robustness (you might want to look at a feature comparison here: https://wiki.documentfoundation.org/Feature_Comparison:_Libr... - quite informative I think!) then really the issue is that it's being used as a platform for VBA development.

---

Fallacy 2: Excel is the best integrated option for office application development work

So now here is the rub: this has been something causing Microsoft vendor lockin for years. Expertise in financial firms around Excel is very high, and often quite extensive custom "solutions" are based on Excel spreadsheets, but almost always via VBA.

But... Microsoft's VBA diverged from Visual Basic in VB6, and Microsoft have been struggling to know what to about it ever since. I point out that they removed support for VBA in Office 2008 for OS X, but then had to put it back in due to market demands in Office 2012, and I also note that the only update to VBA7 is to introduce actual pointers.

So basically, this will only be a competitive advantage for so long for Microsoft. As the marketplace starts to open up - and it is! - firms will be looking for solutions that either do NOT rely on Excel or VBA and that can integrate with better scripting languages, like Python.

In fact, I've been hearing that a lot of financial institutions are getting quite concerned with data silos: these are usually in Excel, haven't been audited, aren't integrated into the business' DW, are isolated and thus cause problems with transparency and decision making, and are almost NEVER documented meaning that should the maintainer leave then that spreadsheet will either be used and unmaintained, or will need to be redeveloped anyway.

So really, the key advantage Excel has is ubiquity, and really only then because of VBA, but that's a double-edged sword and eventually businesses will migrate away from it.

1. https://wiki.documentfoundation.org/ReleaseNotes/4.2#Perform...

2. http://lists.freedesktop.org/archives/libreoffice/2013-April...

3. So far as I'm aware, Excel's RNG is now using The Mersenne Twister algorithm in Excel 2010 and beyond, but that's not officially confirmed from Microsoft. It's very hard to be certain of what is being used without such a confirmation, and given that Microsoft has form in releasing a flawed RNG update already (which they never got around to fixing for years and years!) I don't know how you could consider Excel to be "more robust" than LibreOffice.

Re:Government Open Source Office deal set to provide major s... by greggyb on Tuesday October 20, 2015 @10:16AM (#10419254)
I'm not disagreeing with what you've included above, but I'd like to add a fourth, new use case.

Excel is increasingly being used as Microsoft's gateway drug to an excellent self-service BI platform, Power BI. There is incredible power in providing a smooth (and approaching seamless) upgrade path from workbook with embedded queries, to a proper dimensional modelling tool (Power Pivot, AKA SSAS Tabular columnstore engine), to a proper dashboard visualization tool (Power View), to a hosted, centralized team solution (Power BI portal - mobile app batteries included), to an on prem enterprise-grade analysis engine (SSAS Tabular).

Again, this is not a refutation, but a new use case and powerful new functionality to address the concerns you raised in Fallacy 2.

Re:Government Open Source Office deal set to provide major s... by chris_wot on Tuesday October 20, 2015 @10:30AM (#10419351)
That's quite cool - didn't really know about that!

The only thing is that the reason VBA is so widely used is because it's built in to every single version of Excel (except Excel 2008 on OS X), which is why it's ubiquitous. Are Power BI and PowerView things that can be used within Excel without using a server? Are there additional licensing costs?

Re:Government Open Source Office deal set to provide major s... by greggyb on Tuesday October 20, 2015 @10:44AM (#10419459)
Power BI is a series of free add-ins for Excel starting in 2010. Depending on your year and version they are either included or are downloadable. They are available in the standalone products and Pro Plus. Additionally there is a standalone, free program Power BI Desktop Pro which includes all the pieces without a dependency on Excel at all.

The desktop and mobile apps are completely free and the desktop pieces can be used without a server. The web portal is cloud-hosted and freemium to anyone with a corporate email address and the mobile apps require at least the free web portal.

Feel free to email me if you're curious about more details. Gregory.Baldini at gmail dot com.

Re:Government Open Source Office deal set to provide major s... by chris_wot on Tuesday October 20, 2015 @10:50AM (#10419512)
Oh! I think I will tomorrow - I just finished a stint developing some Qlikview reports so I'm interested in other BI solutions.
Re:Government Open Source Office deal set to provide major s... by frik on Tuesday October 20, 2015 @12:45PM (#10420240)
And yet this BI platform shares the same limitation as Excel. It uses Excel WebApp hosted as SharePoint WebPart/"App". The Excel row and column limits are also true for BI. If you want to show millions of data entries, better choose a more mature BI platform.

On the one side companies want to ged rid of their Excel data silos with many unverified formulas, on the other side some choose a BI platform based on Excel (data is loaded from MSSQL DB).

Re:Government Open Source Office deal set to provide major s... by greggyb on Tuesday October 20, 2015 @02:44PM (#10420952)
This is not at all true. The Power Pivot add-in provides a mature in-memory columnstore database engine with no hard row limits - this can easily manage 10s of millions of rows, and with a decent model architecture can grow to 100s of millions. When upgrading to SSAS Tabular on a server with enough RAM, billions of rows are not unreasonable.

Additionally, the hosted solution allows Excel Services to be the presentation layer, but the real push is toward Power View.

I would suggest you do more research before spreading this disinformation. There has been significant development on this front and what you are saying is very much out of date.

Re:Government Open Source Office deal set to provide major s... by frik on Thursday October 22, 2015 @12:50AM (#10430460)
What I wrote is true for MS BI on Sharepoint 2013 with MS SQL Cluster 2012 in real world. It's probably the co consultants fault to use SQL2012 instead of SQL2014. MS mentions the new 2016 release on the upcoming MS SQL version will do away some of the current limitations, as you mentioned.
Re:Government Open Source Office deal set to provide major s... by greggyb on Thursday October 22, 2015 @09:37AM (#10432100)
SharePoint 2010 and SQL Server 2008R2 introduced support for hosted Power Pivot workbooks.[0]

Power Pivot is the Excel add-in (available for Excel 2010+) which enables dimensional modelling and >1M row analysis in Excel, and which can be transitioned to a hosted environment via SharePoint and SQL Server.

Starting with SQL Server 2012, SSAS includes Tabular mode, which allows a seamless upgrade path from an Excel Power Pivot model to an SSAS instance.

This is seamless because the same DB engine is used in both products. Power Pivot runs a private, anonymous instance of SSAS Tabular behind the Excel process, with a few features disabled (e.g. row-level security).

Power View is a high level visualization and dashboarding tool introduced for SharePoint 2010 and 2013 as a part of the SQL Server 2012 SSRS module.[1] This is the visualization tool designed to pair with Power Pivot and the Tabular model. It is also available as an add-in for Excel 2013+[2], and as part of the standalone Power BI Desktop[3] which has been available since Power BI portal's general availability in July of 2015.

I do not know what you are referring to by saying that there is a 1M row limit in practice with SP 2013 and SQL Server 2012, because this is simply not the case. If your organization is not using the features and these are pain points for you, I'd love to have further discussion with you offline - my email is gregory.baldini at gmail dot com.

[0]https://technet.microsoft.com/en-us/library/ee210682(v=sql.1...

[1]https://support.office.com/en-us/article/Power-View-in-Share...

[2]https://support.office.com/en-us/article/Power-View-Explore-...

[3]https://powerbi.microsoft.com/en-us/desktop