if you set up a gas station near the off ramp of some major interstate, say I-65 North, you will see cars pulling in to fill up on gas. maybe buying a coffee. now, these aren’t your customers in the traditional sense of a Target or Walmart customer. Because you will never see them again. They were driving from town A to town B via the interstate- they started running out of gas and needed to refuel, so they are in your gas station now. Once they gas up, off they go. They aren’t going to come back to you and establish a customer relationship or something. We’ve all been to tons of gas stations on the interstate and we’ll probably never go back to the same one twice - unless we are plying the same route everyday like a truck driver. So the task is to find and convert these truck drivers, who are the true repeat customers.
I was working on an android app which had like millions of unique cookies. When they hired me they said we have million of users. No you don’t. If you put out an android app in some popular domain, say news, entertainment, tax accounting etc- people will download and “use” your app. they are checking it out. they aren’t users, in the sense they aren’t using it everyday or want to have a relationship with you, pay subscription etc. conversion stats are minuscule, like 0.01%. So maybe 1 out of 10000 users is the truck driver. The vast majority will never ever use your app again. To do data science with these millions of rows of user interactions and find some nuggets just because you know your way around pandas or sklearn is a fool’s pursuit. To ask foolish questions of your data, like why are all these people churning, is silly - they aren’t your users, they haven’t converted, they are just checking it out. In that sense, its a waste of time and resources to do so much data crunching. Look at actual conversions, which are probably a few thousand people, not millions. Reach out to those thousands and maybe a few tens will give feedback and then continue to iterate on the product based on that.
sure, most customers at an interstate gas station will only visit once or twice, but that doesn't necessarily mean they are less important to the business than the truck drivers that fill up every day. maybe the bulk of revenue actually comes from one-time customers. this could be a case where attracting new customers is more important than retaining the current ones.
There are tons of B2B saas, including regional ones, that only serve a small number of customers way under millions.
Maybe a close analogy would be: truck drivers who stop at your rest stop every time they come by... just to use the washroom. But who never go into the store itself.
And recall Mark Twain's old quip about lies & statistics. The more & bigger data that the folks who control the data & analysis have, the easier it is to make sure that those meet their own emotional & political needs.
I wonder if that's the quote they mean?
One possible reason: no one whose job it is to write Python scripts was ever promoted for making an Excel spreadsheet when that is the simpler and more practical approach. And no manager of people who write Python scripts is going to be able to use that Excel spreadsheet to sell "I need more responsibility and head count." People tend to follow incentives, rather than focusing on making wise decisions.
This is the key issue. Solving it isn't easy -- it requires people who are wise, and wisdom is a scarce commodity.
Python 2 to 3 upgrade aside, can’t really say the same about the language.
There are a number of good arguments out there that might violate an engineers perception, which one might call a cognitive data model built through training and experience.
There is no theory that makes any given engineering path “wiser” than others. Just engineers chasing incentives to be engineers.
A few slides showing the data, a boring 10 minutes about methodology, and finally the conclusion brings an air of reliability that you can't replicate for knowledge instead of data.
Of course you need to interpret it but its incredibly important and I do not think you really know what you are talking about.
https://www.ycombinator.com/library/6g-how-to-talk-to-users
Almost all the major fails I have seen in my career have been some derivative of not understanding your users.
The creation of hypothesis is often glossed over as a trivial first step in scientific or data-driven decision making, but in fact, that's where the magic lives.
Nice strawman. I have never said to never talk to your users, but to pretend that using data is meaningless and you should follow some bullshit and vague "good argument" instead is just sheer foolishness.
When you've got an early product, there are probably things you can do that 2x as many people will like as dislike. Even a small set of customers will be good for discovering this. When you've got a mature product, you should be optimizing around the edges and need a large sample size to find those 1% wins.
Likewise if you don't have scale, there are a lot of well-known best practices that probably improve your site by 5-10%. You probably don't have sufficient volume to discover test those ideas, so following general best practices is a good idea. But if you have scale, you can and should A/B test the heck out of everything. And then do it again in a couple of years in case the answer changes.
Yeah, but this just means qualitative data is challenging, not that it's useless. You have to be careful when asking questions that you're asking useful questions and not leading people into telling you what they think you want to hear (or going off on useless rabbit trails like what they think the product should be instead of what the problem they want the product to solve is).
I am guessing it's like you see of a psychologist with a patient on TV..... the customer must feel comfortable enough to open up, then flood gates can open.
I mean I worked on an app where in one part, the end user could upload CSV files to be used. What they SAID they wanted was basically a full data management system and RESTful API to enforce constraints, data validation, record retrieval and updating, etc. What they probably wanted was an excel sheet. I dislike how my employer was like "yeah sure if you pay for it" to them.
A key cause of this in many cases is that the stake-holders you talk to do not work closely with the end users of the system. Talking to the right people can help a lot, though unfortunately as a 3rd party this is not usually anywhere near your realm of control.
The other issue is them knowing what they have and wish to store, but not knowing what outputs are going to be needed down the line. That is harder to fix, but having some good industry knowledge within your company can be a great help on such matters – you can then sometimes preempt client needs if the people holding that knowledge are keeping an active eye on changes (for instance new/planned regulations that might be coming into force in X weeks/months/years).
I mean, having an Excel doc at all usually implies hour(s) of work formatting the data in structured manner. Sometimes collective decades of work depending on how much heavy lifting your 15GB .xlsx is doing.
Even a one off analysis is actually FASTER in Pandas because I've done the work of farting around with the formatting. Now I can just write the necessary analysis code, rather than deal with the formatting.
That said, my data analytics work is seriously small potatoes compared to many. But I can write a quick pivot table using Dplyr faster than I can do it in Excel.
I've seen this myself: the person who "naively" downloads that table and plays around in excel finds interesting things that the person who was using Pandas hadn't, because the code to manipulate columns and do certain types of calcs is actually more time consuming to write and modify than making a bunch of new columns in Excel with a bunch of formulas!
A good data scientist will have a more rigorous approach to their notebooks and practice reuse and so on... but that's not necesssarily easy.
I think they call that serendipity. Never underestimate its power.
That would explain why people think a <1TB is big data.
It moves with Moore's law. Big data is anything that cannot reasonably fit into memory for a single server, so yes that number is well over 1TB now.
You know, in data science, you see people spending hours writing pandas scripts that replicate a few clicks in excel for a one of analysis. You see datasets of a few gigabytes being processed with spark when SQL would be fine. You see ML techniques being thrown at questions that could be answered simply and reliably with basic statistical tests.
Especially in the B2C space a lot of companies, departments, products don't actually have a lot of customers and certainly not many decision makers. The N number is always going to be low. You can just talk to people. Let's say you are doing pretty well and running a SaS with 1000 corporate customers paying a million each - that's a billion dollar revenue - you can just talk to them. Certainly you can just talk to every single person who signs the cheque and those are the only people that matter.
And which is easier - putting together a thorough suite of A/B tests or getting some real customers to use your app on video and talking to them about what they are finding annoying, useful, missing? I see less people do that than you'd think.