News for hackers. Stuff that matters.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by guelo on Friday October 05, 2018 @04:48PM (#18151621)
People say "use a real database", fine, but where is the visual data entry? Where are the easy adhoc reports? Where are the forms? I guess if you're on Windows you can use Microsoft Access, but that's the only accesible tool I know of that doesn't require a team of programmers or an enterprise license for some Oracle/SAP type monstrosity.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by foobarrio on Friday October 05, 2018 @05:33PM (#18151915)
This was my reaction to Flash. It's not a "real" programming environment is what everyone told me. Yea but it was a great authoring tool for non hard-core programmers. I helped a designer write a few simple puzzle games in Flash and it was overall a productive experience. Had I attempted to do that in html/JS I'm not sure it would have gone so smoothly. I haven't tried to write web games in the last 10 years so maybe there's some collection of html/JS tools that lets me develop in a similarly. I'm willing to bet though they're 10x harder to set up and require command-line knowledge (fine for me, not so much my designer friend).
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by derefr on Friday October 05, 2018 @06:43PM (#18152316)
Flash (aka Adobe Animate), at this point, is an HTML5 authoring tool.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by gowld on Friday October 05, 2018 @06:32PM (#18152267)
Has anything replaced Flash as a productive way to build animated games?
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by TeMPOraL on Saturday October 06, 2018 @04:10AM (#18154122)
Construct[0].

I used to sneer at such tools, considering them "not real gamedev". That was until, on one local edition of Global Game Jam, a designer in our team did in Construct in 15 minutes what took three of us coders a couple hours. This drove home the point that we were really overcomplicating things by coding a simple game from scratch (in XNA, back then).

--

[0] - https://www.scirra.com/construct2 is the one I played with, but they also seem to have a web version now, for better or worse, which is available at https://www.scirra.com/

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by mch82 on Friday October 05, 2018 @11:56PM (#18153438)
Unity / Unreal (which can be quite accessible for projects that constrain themselves to built in features & asset store components)
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by sin7 on Friday October 05, 2018 @05:04PM (#18151744)
I use PostgreSQL, which is a real database. I use DBeaver as the database manager. It allows me to enter data by typing it in. For reports, I query the data. It has autocomplete. No forms.

For presentation, I toss the data on Excel. Bold the headers. Change the number type (pct, $, etc) to the correct one.

I would say that it has made my life a lot easier. There is one repository where the data is held. If I mess up, I go back to the master tables. In Excel, I would usually start over. Joins are easy. Excel can do some things better. When it can, then I use it. It's not a one or the other. It's ok to use both.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by gerdesj on Friday October 05, 2018 @05:24PM (#18151854)
Me too but throw in MariaDB as well (I'm not exactly bi-lingual but can speak both with a dodgy accent and know some words better in one and some in the other.) I use Libre Office because Excel doesn't work very well on Arch Linux out of the box.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by ryanmonroe on Friday October 05, 2018 @08:16PM (#18152748)
Not having a good data entry gui is the biggest benefit of not-excel in my experience. 90% of the time your data has a real source that's not "Jim's head" and you should find a way to impute the data from that source directly, avoiding the ever-present inaccuracies that come from human-imputed data.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by acct1771 on Saturday October 06, 2018 @10:40AM (#18155358)
Was the typo to drive home your point?
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by mclehman on Saturday October 06, 2018 @07:13PM (#18157877)
My guess is that they meant this definition of impute, and are advocating for removing Jim's inferences entirely.

Impute (finance):

assign (a value) to something by inference from the value of the products or processes to which it contributes.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by paultopia on Monday October 08, 2018 @10:46AM (#18167926)
Also, even when you need human entered data, a spreadsheet is waaaaay too easy to make typos with, like putting things in the wrong row/column, overwriting previous entries, etc. Use a form.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by mch82 on Friday October 05, 2018 @11:56PM (#18153434)
Also, Excel is installed everywhere by default & not blocked by IT. Try getting approval to use Anaconda...
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by PhasmaFelis on Friday October 05, 2018 @05:57PM (#18152070)
I read a thing a few months back about some Western game developers collaborating with Japanese counterparts for the first time, and their biggest surprise was that a whole lot of game data is stored in an Excel file during development. If you want to change an enemy's stats or a physics parameter, you don't dig through config files with a text editor, you just pull up the spreadsheet. Reportedly it made a lot of tweak-test-repeat work much quicker.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by DCoder on Saturday October 06, 2018 @12:58AM (#18153609)
Trivia: Diablo 2 stores its data as tab-separated value files. Some of those files have more columns than Excel could support at the time… so the modding community made their own editors.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by pmiller2 on Friday October 05, 2018 @06:50PM (#18152355)
I don’t understand how editing a number in an excel file is any easier than editing a number in a text file. What was it that made the difference?
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by vbezhenar on Friday October 05, 2018 @07:20PM (#18152515)
Probably table representation. It's relatively hard to implement something similar using text format. You can have row for every unit in the game with columns corresponding to various characteristics. It looks very natural in Excel, but it would require quite a lot of formatting and errors with text file. Also Excel could have formulas which might be useful somewhere and supporting formulas in config file is another level of complexity and almost certainly it'll be worse than Excel.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by Nullabillity on Friday October 05, 2018 @07:59PM (#18152691)
Then use a real database. You probably have a SQLite dependency anyway, so take advantage of it.

Or don't, since SCMs are much better at diffing plain text than either.

> Also Excel could have formulas which might be useful somewhere and supporting formulas in config file is another level of complexity and almost certainly it'll be worse than Excel.

You'll have an easier time reimplementing Excel's formula system than basic arithmetic? I find that hard to believe.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by PhasmaFelis on Saturday October 06, 2018 @01:14AM (#18153661)
> You'll have an easier time reimplementing Excel's formula system than basic arithmetic? I find that hard to believe.

I'm not sure you understood what the parent was saying? This seems like a non sequitur.

In a spreadsheet, derived values can be updated instantly. Say you want to upgrade a bandit from leather armor to chainmail (6 or so equipment items changed), and then verify that his total defense and modified movement speed are still within reasonable ranges for the part of the game where he appears. With a text file, you've got some arithmetic to do. With a spreadsheet, it's there at a glance.

Or say your enemies are missing too much, and you want to give them +10% accuracy across the board, for all of 50+ enemies. In a text file, that's a lot of cursoring around. In a database, you could knock together a query to do it, but that still might take a couple of minutes, depending. In a spreadsheet, I could do it in 15 seconds. (And if I screw it up, a couple of Ctrl-Zs puts everything back where it started. That's not so easy in a database.)

And, y'know, a lot of programmers really underestimate the value of good formatting. A database can spit out nice columnar tables easily, but a spreadsheet can do things like put borders between stat clusters related to different categories; automatically color-code data so you can see at a glance what the highest and lowest values in a given column are, or spot out-of-range derived values; add alternate-row shading to make it easier to scan across a long row; and on and on. That is useful. It lets you find and correlate the data you want faster with fewer errors. Not everything has to be viewed in flat text.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by shabble on Saturday October 06, 2018 @03:50AM (#18154067)
Your first couple of points reminds me of a talk[1] about using a graph database for tracking all sorts of entity and quest dependency relationships to assist in game balancing and content creation.

[1] https://www.airpair.com/neo4j/posts/modelling-game-economy-w...

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by accatyyc on Saturday October 06, 2018 @03:29AM (#18154006)
All this sounds very useful! If I ever decide to make a game I will definitely use a spreadsheet as database.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by DCoder on Saturday October 06, 2018 @01:16AM (#18153668)
I have tinkered with modding Diablo II (which uses tab-separated values for parameters) and the classic Command & Conquer series (which uses INI files).

Both approaches have tradeoffs:

- TSV files can have so many columns that you can't really edit them in a text editor, you'll never find your place. Some of these in D2 have more than 256 columns.

- INI files don't give you a full list of configurable parameters, so you might not even know what flags the engine supports if they're mentioned only once in the entire file or not mentioned at all. Some object types in Red Alert 2 support over 700 flags, not all of them are actually used in the default config…

- two items described in a TSV file are easier to compare than two INI sections, with INI you need a second editor window and some ordering of the keys to make sense of things.

- on the other hand, diffing a TSV file for changes is a lot more challenging.

At the time, I remember thinking how a proper database would make this easier.

---

I have also seen a game where a \n was used as a record separator and \r was used to insert newlines into the text string within a record. Fun times when your text editor tries to "intelligently" handle and normalize line endings.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by Too on Saturday October 06, 2018 @03:00AM (#18153939)
You could have json and a schema. VSCode for example does this, all settings is just a settings.json but it still has auto-complete inside it so you always know what settings are available.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by philwelch on Friday October 05, 2018 @08:23PM (#18152770)
Aside from the UI advantages others mentioned, another possible benefit is that, with the right formulas set up, you can instantly eyeball the combined effect changing one factor has on every other factor that’s derived from it. I don’t know how important that normally is, but I play a lot of Paradox games, and those games are almost entirely governed by numbers that have combinatoric effects on gameplay.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by celticninja on Friday October 05, 2018 @07:43PM (#18152622)
Quickly changing a few stats would probably be easier in an excel file, every stat is directly below the next, one key press to move to the stat and change it.is faster than scrolling 3 or 10 lines to the next one then placing the cursor on the correct character etc. I can see how it would be faster for some use cases. If someone is doing it then it is probably because they found it to be effective.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by watwut on Saturday October 06, 2018 @03:22AM (#18153992)
Because non developer is able to find it at glance. And then he is able to see how related values chaged - at glance with zero effort and delay.

And non developer can easily write own formula to next cell whenever he feels like and don't need to wait for programmer, don't need to explain and don't need to fight if programmer happen to be stubborn.

Even seeing what depends on what is easier with right excel plugin then debugging scripting code.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by just_myles on Friday October 05, 2018 @07:05PM (#18152450)
I agree. Text file edit seems much quicker.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by regularfry on Sunday October 07, 2018 @08:11AM (#18160185)
Back when I worked on cars, the configuration parameters the firmware devs used for the control units were all in excel spreadsheets, with some plugin or other and a bunch of macros to flash the hardware. As far as I could tell, the main benefit of this setup was that the devs didn't have to build their own UI that looked good in a table, and allowed searching and sorting.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by pitay on Saturday October 06, 2018 @07:49AM (#18154649)
Text files or Excel for the game data?

Pardon my ignorance, but why aren't they using some sort of database for the data? Or at least storing the data in some sort of master database and exporting it to text files for the game to read? It would be lovely to hear the techniques and reasons from anyone in the know.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by PhasmaFelis on Sunday October 07, 2018 @01:53AM (#18159039)
I don't know exactly, but I wrote a post about some hypothetical advantages here: https://news.ycombinator.com/item?id=18153661

(IIRC it's for convenience in development. I doubt the game ships with an embedded Excel file.)

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by p_l on Sunday October 07, 2018 @10:42AM (#18160777)
Modding tools (supposedly based on real SDK) for Homeworld (don't recall if 1 or 2, I think 2 since I recall switching some crucial values to make game easier for last mission) used Excel with extensions.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by Nullabillity on Friday October 05, 2018 @08:02PM (#18152705)
> People say "use a real database", fine, but where is the visual data entry?

Any visual DB client?

> Where are the easy adhoc reports?

Isn't that just queries? Then throw it into a view to "save".

> Where are the forms?

No idea what you're looking for here, but this sounds a lot like your first question.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by yawaramin on Friday October 05, 2018 @08:51PM (#18152850)
Ad-hoc reports are not just saved queries, they're saved queries + something like Word's mail merge feature so that you get a data-driven, print-quality document, like an invoice.

Forms are data entry views that save to the backing tables. Nowadays the most well-known forms are probably https://docs.google.com/forms/

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by Kagerjay on Friday October 05, 2018 @05:03PM (#18151731)
Just use airtable, its like microsoft access but easier to use and pickup. Or just use google forms + google spreadsheets
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by schwinn on Friday October 05, 2018 @06:58PM (#18152410)
Actually, I struggled making the transition to airtable from excel. Do you have recommendations for training? I do want to learn it, maybe no to a mastery level, but proficient
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by Kagerjay on Saturday October 06, 2018 @10:44AM (#18155376)
I actually didn't find any specific "airtable" training, I just used the docs and forums. I had prior knowledge working with microsoft access and database design, so airtable was an easy transition.

If I were to learn it without any prior experience, I would just look through every template they have. See which industries I'm most familiar with. And look at how those databases are organized.

https://airtable.com/templates

They have a really good 12 minute overview though, hitting every major concept though. https://vimeo.com/165624533

I would also learn a little bit about relational database design. This 5 minute video is a good starting point https://www.youtube.com/watch?v=NvrpuBAMddw

I like using databaseanswers.org as a reference resource. This is a very nice 5 minute introduction to designing a database. http://www.databaseanswers.org/data_models/5_minute_tutorial...

Basically, if you come from an excel background, it would help if you knew a little bit about database design, and relational databases. Its not necessary, but Airtable is practically excel + microsoft-access.

Start with a template, and play around with it. I would get familiar with this page in particular though. Formula and lookups are something you'll use often in any airtable base. https://support.airtable.com/hc/en-us/articles/202576519-Gui.... This was perhaps the least obvious concept for me to pickup naturally.

If you need any help though feel free to reach out to me I have contact information on my username

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by MisterOctober on Friday October 05, 2018 @08:25PM (#18152780)
There is iron in these words. I love using SQL and hate using Excel, but when it comes to common statistical calculations such as standard deviation or a Pearson function, or if you want to generate a graph and can't [or don't want to] use a tool such as Tableau or Crystal, Excel really is easier [or, at minimum, quicker] for generating output sometimes.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by Something1234 on Friday October 05, 2018 @10:28PM (#18153163)
I absolutely hate plotting anything excel. I feel so stupid trying to get something to plot. GGPlot works so much better.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by interlocutor on Friday October 05, 2018 @08:18PM (#18152752)
You can use a real database and have Excel like UI, plus data entry, with good reporting tools, here's one: http://pebblereports.com/
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by kevinpet on Saturday October 06, 2018 @12:35AM (#18153555)
Filemaker was great for this. Around 2000, you could get an amazing level of productivity for building small custom relatively complicated apps without writing any real code.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by gthtjtkt on Friday October 05, 2018 @08:56PM (#18152865)
> but where is the visual data entry?

That's where the "Power" suite enters the picture (Power Query, Power Pivot, etc.). Great way for non-technical people to get database data into Excel for reporting/analysis. I've trained plenty of people with only basic Excel skills to use it. The best part (for them) is that it's like a macro, so you can rerun it with new data any time.

You can even use those to pull an existing SSRS report directly into Excel.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by daveslash on Friday October 05, 2018 @05:30PM (#18151893)
Obligatory Comment: "Excel as a Database" by Rory Blyth, circa early 2000's.

"As a developer, you've probably, at some unfortunate point in your life (possibly several points, actually), been handed an Excel file that has been crammed full of 'data' by someone in marketing and told to 'do something with it.' " http://wyorock.com/excelasadatabase.htm

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by PhasmaFelis on Friday October 05, 2018 @05:59PM (#18152079)
As a developer, what should I do if I actually want to do that? Because I get a lot of satisfaction out of massaging messy wads of data into a clean, uniform, accessible form. Is there a job title or something I should look for?
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by codazoda on Friday October 05, 2018 @06:55PM (#18152391)
The Analytics team at my company does this. ETL's we call them (Export, Transform, Load). Some of us are app devs and some are data analysts; there's cross-over. We do other things too, but moving data between systems is a big part of it.

As for job titles; Software Engineer (or Developer) of Analytics, Data Analyst, Data Scientist, or something along those lines. Probably varies by company.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by zer00eyz on Friday October 05, 2018 @07:40PM (#18152607)
I have a story that may amuse you.

Recently I worked on a contract doing some data ingestion that the resident team didn't want to deal with.

They had been provided a JSON api to get what should have been a batch file. The owner/api creator said "this is good enough we won't accommodate you." So much for sensible data transmission and batch processing.

Because I have had to deal with these sorts of things before I have a fairly robust tool chain to hammer api's with requests to get the data in a reasonable time frame. In this case my client went full tilt - and simply hammered the API till its owner gave in and started sending batch files.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by sk5t on Saturday October 06, 2018 @12:01AM (#18153455)
What do you mean by "batch file"? Surely not a .bat.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by zer00eyz on Saturday October 06, 2018 @06:11PM (#18157566)
Yes the end result in most cases is good old CSV -

repeated variable names are bloat when getting lots of data, extra formatting is the same... CSV's are great for getting these things down.

As another poster pointed out (S)FTP is the way to go when sending CSV's, and they are compressed in most cases to save storage and data transmission on both ends.

I am old enough to remember when mailing a hard disc or a tape might be a faster way to move a LOT Of data. The practice is still alive and well, but the option is now by the "truckload"

https://aws.amazon.com/snowmobile/

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by intended on Saturday October 06, 2018 @01:19AM (#18153672)
Probably means a single report with all the data sent at a regular interval. (All transactions between 00:00 hours and 24:00 hours the previous day, sent by 9:00 am to these email inboxes).

Instead the client was given an api to query and no batch processing of information was allowed.

I’ve encountered this idea somewhere, and fortunately have never had to deal with such a broken scenario. Getting piecemeal information is infuriating.

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by strken on Saturday October 06, 2018 @02:06AM (#18153785)
He probably means the output of a batch processing pipeline. Clients will calculate a set of data once a day and shove it into some kind of storage[0] for you to download.

[0] probably an FTP server with no TLS and with the same weak username and password for everyone who connects, a setup which they won't change no matter how many times you ask

Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by thenipper on Friday October 05, 2018 @07:58PM (#18152681)
What the other child poster said. This is 90% of my job. It’s really fun a lot of the time and absolutely brutal the rest of it. Be prepared for things like writing code that checks the color or text styling of cells etc. people encode data in whacky ways
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by MisterOctober on Friday October 05, 2018 @08:19PM (#18152755)
Seriously. I frequently get handed spreadsheets with integers typed as strings... people using strings for things that, heaven's sake, should undoubtedly be booleans... Getting the thing into usable condition takes up more time and energy than the actual analysis they desire
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by synotna on Saturday October 06, 2018 @02:27AM (#18153837)
I've seen that before, to "work around" locale issues, bring a European company recieving data from US companies
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by VHRanger on Friday October 05, 2018 @08:16PM (#18152745)
"Data engineer" (more Dev) or "Data scientist" (more analytics).
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by snakeboy on Friday October 05, 2018 @06:11PM (#18152149)
"Data Scientist" should do the trick.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by merb on Friday October 05, 2018 @06:19PM (#18152193)
basically you can use excel to query postgresql. either with Power Query and NPGSQL or for the more non developer experience you can buy an addin: https://www.devart.com/excel-addins/postgresql.html
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by LaszloKv on Friday October 05, 2018 @08:38PM (#18152806)
I wondered the same thing for a while. Just this week I came across https://www.knack.com/. It looks pretty good so far but I haven't really explored it yet to see how well it works out.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by Klathmon on Friday October 05, 2018 @09:41PM (#18153016)
I can't speak to the product, but I did interview and worked a trial week at knack about a year ago and they are a very bright group of people that are hell-bent on making and maintaining a great product.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by CivBase on Saturday October 06, 2018 @03:07PM (#18156694)
Good manual data entry support is nice, but Excel sacrifices good automated data management to do it. Automating basic read and writes isn't terribly hard if you can find a good library, but it's still far more complicated than it needs to be.
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by rabidrat on Saturday October 06, 2018 @03:55AM (#18154080)
Isn't this Salesforce?
Re:The First Rule of Microsoft Excel: Don’t Tell Anyone You’... by dustingetz on Friday October 05, 2018 @05:50PM (#18152025)