Re:Large-Scale Transactional Data Lake at Uber Using Apache ...
by throwaway_pdp09 on Thursday June 11, 2020 @07:59AM
(#23487866)
I'd pretty much define big data as quantity, so yes, if it fits in ram it ain't. But that's just my definition (though reasonable, surely).
You have immediately confounded 'big data' with a ton of cloud tech. My point is that you arguably don't need big data frameworks, and therefore arguably sticking it in the cloud is pointless too. If you can buy a reasonable server and stick it in the corner of the office, do so (noise & security, yeah).
> Sure, you might only end up filtering through tens or hundreds of gigabytes ...S3 ...Parquet ...Spark ...Elastic MapReduce ...Snowflake
Dude, do you need all this stuff!? Really? For a poxy few hundred GB?
You have immediately confounded 'big data' with a ton of cloud tech. My point is that you arguably don't need big data frameworks, and therefore arguably sticking it in the cloud is pointless too. If you can buy a reasonable server and stick it in the corner of the office, do so (noise & security, yeah).
> Sure, you might only end up filtering through tens or hundreds of gigabytes ...S3 ...Parquet ...Spark ...Elastic MapReduce ...Snowflake
Dude, do you need all this stuff!? Really? For a poxy few hundred GB?