Re:Amazon DocumentDB, with MongoDB compatibility
by orf on Wednesday January 09, 2019 @06:58PM
(#18870290)
50GB is a really small data set.
[ Parent
| on HN ]
Re:Amazon DocumentDB, with MongoDB compatibility
by a13n on Wednesday January 09, 2019 @07:09PM
(#18870360)
Eh, by what measure? Realistically it's probably bigger than 90% of all Mongo datasets.
It's tiny if you're a massive company and it's massive if you're a tiny startup.
Re:Amazon DocumentDB, with MongoDB compatibility
by q3k on Wednesday January 09, 2019 @07:25PM
(#18870446)
50GB easily fits in RAM. It's a small dataset.
Re:Amazon DocumentDB, with MongoDB compatibility
by orf on Wednesday January 09, 2019 @07:27PM
(#18870458)
If you can run the dataset comfortably on a Macbook then it's very, very small.
Heck, you can even just use grep over 50GB reading straight from disk. It's tiny.
Re:Amazon DocumentDB, with MongoDB compatibility
by wpietri on Wednesday January 09, 2019 @08:04PM
(#18870643)
Is an argument based on the premise that relative terms have absolute meanings a good use of people's time here?
Re:Amazon DocumentDB, with MongoDB compatibility
by kstrauser on Wednesday January 09, 2019 @07:33PM
(#18870494)
A recent work Slack chat had a dev asking what a particular table contained. They were going through our data inventory and found a randomly-named table 18TB in size. When I ran "select count()" against it, I got back 5,325,451,020,708 rows (that's a copy-and-paste).
50GB isn't trivial, but it's utterly manageable.
Re:Amazon DocumentDB, with MongoDB compatibility
by Aeolun on Wednesday January 09, 2019 @07:37PM
(#18870511)
It seems a bit wrong if you have a 18TB table but no idea what it contains...
Re:Amazon DocumentDB, with MongoDB compatibility
by kstrauser on Wednesday January 09, 2019 @07:47PM
(#18870555)
It was a temp table that we hadn't garbage collected yet. We don't make a habit of leaving that much junk data around, but it bumped our monthly storage bill several percent, not like tripled it.
Re:Amazon DocumentDB, with MongoDB compatibility
by apta on Thursday January 10, 2019 @07:44PM
(#18880022)
Was this a relational or NoSQL DB?
Re:Amazon DocumentDB, with MongoDB compatibility
by kstrauser on Monday January 14, 2019 @01:31PM
(#18905134)
It's primarily in things like Spark and Snowflake that act like relational DBs as long as you squint the right way.
Re:Amazon DocumentDB, with MongoDB compatibility
by djohnston on Wednesday January 09, 2019 @07:10PM
(#18870372)
in my experience it qualifies as "medium"
Re:Amazon DocumentDB, with MongoDB compatibility
by orf on Wednesday January 09, 2019 @07:35PM
(#18870501)
If it can be stuck in a sqlite database and run on a developer laptop, then no, it is not medium by any standard.
Please elaborate why you think 50Gb is anything other than a small dataset that can fit in memory on any half-decent server though.
Re:Amazon DocumentDB, with MongoDB compatibility
by djohnston on Thursday January 10, 2019 @01:17AM
(#18871915)
[edit] in the spirit of not being a condescending tool to you, i'll replace my original reply with this: https://en.wikipedia.org/wiki/Long_tail
Re:Amazon DocumentDB, with MongoDB compatibility
by privateSFacct on Thursday January 10, 2019 @01:57AM
(#18872049)
I'm assuming this is a joke. You can run databases that size without any of the fancy scalability stuff - no sharding no anything. I'd actually recommend that, it's makes admin super easy!