News for hackers. Stuff that matters.
Re:Command-line Tools can be 235x Faster than a Hadoop Clust... by Retric on Thursday May 24, 2018 @09:55AM (#17143972)
What's non relational data?

You can have minimally relational data such as URL : website, but that's still improved by going URL : ID, ID : website because you can insert those ID's into the website data.

Now plenty of DB's have terrible designs, but there I have yet to year of actually non relational data.

Re:Command-line Tools can be 235x Faster than a Hadoop Clust... by bshacklett on Friday June 01, 2018 @12:28PM (#17207003)
That's fair and I'll concede that my terminology is incorrect. I suppose I'm really considering data for which the benefits of normalization are outweighed by the benefits that are offered by database management systems that do not fall under the standard relational model (some of those benefits being lack of schema definition/enforcement* and the availability of fully managed SaaS database management systems).

I'm also approaching this from the perspective of someone who spends more time in the ops world than development. I won't argue that NoSQL would ever "outperform" in the realms of data science and theory, but I question whether a business is going to see more positive impact from finding the perfect normal form from their data or having more flexibility in the ability to deliver new features in their application.

* I'm fully aware that this can be as much of a curse as a blessing depending on the data and the architecture of the application, which reenforces understanding the data and the workload as a significant requirement.

Re:Command-line Tools can be 235x Faster than a Hadoop Clust... by cortesoft on Thursday May 24, 2018 @12:34PM (#17145690)
Wait, why is that an improvement? If there is a 1 to 1 mapping of url to website, splitting it into two tables is just bad database design.
Re:Command-line Tools can be 235x Faster than a Hadoop Clust... by Retric on Thursday May 24, 2018 @01:10PM (#17146099)
Because there are URL's in the website data and or you want do do something with it. Looking up integers is also much faster than looking up strings. And as I said you can replace URL's in the data with strings saving space.

But, there are plenty of other ways to slice and dice that data, for example a URL is really protocol, domain name, port, path, and parameters, etc. So, it's a question of how you want to use it.

PS: Using a flat table structure (ID, URL, Data) with indexes on URL and ID is really going to be 2 or 3 tables behind the scenes depending on type of indexes used.