News for hackers. Stuff that matters.
Re:DynamoDB cannot store empty strings by larrik on Tuesday December 13, 2016 @04:14PM (#13171098)
I used DynamoDB for an inherited project around the time I had gotten really good with MongoDB. I don't have any idea why anyone would ever use DynamoDB. It's like Mongo, without any of the good parts, that you can't run locally or any non-Amazon server. I don't see the benefit.

Edit: Specifically, I was using some pretty advanced aggregation stuff. At the time I used it, DynamoDB had zero aggregation whatsoever.

Re:DynamoDB cannot store empty strings by junker37 on Tuesday December 13, 2016 @04:20PM (#13171143)
The reasons we use it * doesn't require any setup (no provision servers, maintining backups, etc) * automatic backups * practically free during development * scales when you need it too

That said, we've run into issues with the partitioning. If we were given access to the partition information and the ability to reduce partitions when needed, that would solve those problems.

Re:DynamoDB cannot store empty strings by thinkloop on Tuesday December 13, 2016 @04:59PM (#13171506)
Mind sharing what issues? A few keys are the target of a disproportionate amount of transactions?
Re:DynamoDB cannot store empty strings by junker37 on Wednesday December 14, 2016 @09:58AM (#13176366)
Correct, our read/write access isn't spread out. So, before knowing how the partitions worked, we increased our read/write throughput, this caused the partitions to increase, so now our throughput that was spread out across the entire table, was split between 2 partitions. This caused even more throttling.
Re:DynamoDB cannot store empty strings by rocqua on Tuesday December 13, 2016 @04:22PM (#13171173)
Your list formatting didn't quite work.
Re:DynamoDB cannot store empty strings by jonthepirate on Tuesday December 13, 2016 @04:18PM (#13171131)
I also have done 3 years of MongoDB before I (unwillingly) wrote an app with it... worst development experience I've ever had. I wouldn't have used it were it not for my corporate overlords at the time.

My new rule of thumb: Use plain databases for most things and only use DynamoDB if there's a precise feature in your system that it is well suited for.

DynamoDB is a horrible general use database.

Re:DynamoDB cannot store empty strings by cle on Tuesday December 13, 2016 @11:52PM (#13173937)
It's specifically not a general use database, so it's not surprising that it doesn't have general use database features. If you want that on AWS, use RDS or Aurora. If you need to scale in certain ways, then Dynamo can certainly be worth the restrictions it imposes, because it is very scalable.
Re:DynamoDB cannot store empty strings by wbkang on Tuesday December 13, 2016 @04:40PM (#13171348)
You had a horrible experience with MongoDB but you are drawing a conclusion about DynamoDB? Am I missing something?
Re:DynamoDB cannot store empty strings by OJFord on Tuesday December 13, 2016 @04:51PM (#13171446)
I think your parent commenter meant "3y experience using Mongo before having to use it [- DDB, subject of thread -] ..."
Re:DynamoDB cannot store empty strings by kornish on Tuesday December 13, 2016 @04:23PM (#13171184)
By "plain database" do you mean a relational database?
Re:DynamoDB cannot store empty strings by calsy on Tuesday December 13, 2016 @05:35PM (#13171849)
Care to elaborate? MongoDB for app development on paper seems like a great solution.

The NOSQL model fits great with a Restful API layer. The process of getting and posting data over HTTP should be a lot simpler with Restful API/MongoDB when compared to relational database solution.

I know Restful APIs are not perfect either, so care to fill in the blanks of why its was the worst dev experience for you.

Re:DynamoDB cannot store empty strings by andrewguenther on Tuesday December 13, 2016 @05:01PM (#13171528)
If you don't understand why anyone would use DynamoDB then MongoDB is probably perfect for you.
Re:DynamoDB cannot store empty strings by Fiahil on Tuesday December 13, 2016 @05:03PM (#13171539)
I used MongoDB in the past too. Now, I'm using Elasticsearch as a document store. Aggregations are good, and you can do some very advanced stuff with it (geographic clustering, scan left, range and exact match, range buckets, histograms). Of course it comes with a set of limitations, but it might be worth your time to investigate (especially if you want to use it also for full text search).
Re:DynamoDB cannot store empty strings by larrik on Wednesday December 14, 2016 @09:11AM (#13176033)
I've actually become quite familiar with Elasticsearch since then, whereas I stopped using Mongo (that project died). ES is indeed very powerful.
Re:DynamoDB cannot store empty strings by idbehold on Tuesday December 13, 2016 @04:17PM (#13171120)
To be fair, you can run DynamoDB locally: https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
Re:DynamoDB cannot store empty strings by niftich on Tuesday December 13, 2016 @04:26PM (#13171209)
DynamoDB-Local is a service that implements DynamoDB's API, including its query language, datatypes, and grammar, but stores everything in SQLite. It's great for playing with the API or substituting a mock implementation that doesn't drain your wallet, but it's very unlikely have the backend implementation and the performance characteristics -- or even all of the exact bugs -- of "real" DynamoDB.
Re:DynamoDB cannot store empty strings by bgentry on Tuesday December 13, 2016 @04:54PM (#13171465)
last time I used this, it had many significant differences from the real DynamoDB. Different enough that I don't really understand what purpose DynamoDB Local is actually supposed to serve.
Re:DynamoDB cannot store empty strings by larrik on Tuesday December 13, 2016 @04:26PM (#13171206)
Ah, I wonder if this was true a few years ago (early 2014) when I was working with it.
Re:DynamoDB cannot store empty strings by seanwilson on Tuesday December 13, 2016 @04:22PM (#13171170)
DynamoDB being part of the AWS ecosystem is a massive benefit.
Re:DynamoDB cannot store empty strings by bonzini on Tuesday December 13, 2016 @04:31PM (#13171266)
I like the way you spelled "lock-in". ;-)
Re:DynamoDB cannot store empty strings by dijit on Tuesday December 13, 2016 @04:55PM (#13171481)
I dislike your sarcastic winky face, but I agree with the sentiment.

Unless you plan on handing amazon your money forever, you should always steer for agnostic solutions you have the ability to host yourself.

As much as it may not 'scale' if you manage to get a passive income with decent understanding of load, then purchasing hardware is almost certainly cheaper for you.

Cheaper is good, it means more return so you can invest that income in your new project (and pay amazon for easy scaling again, if you want). But removing that ability not only locks you into their platform which may change prices at any time... it also means you can't move to self-hosting when the time is right and save quite a lot of money.

Re:DynamoDB cannot store empty strings by calsy on Tuesday December 13, 2016 @05:45PM (#13171943)
I dont believe you are locked in with AWS forever if you choose DynamoDB.

It doesnt use a unique database model that no other DB supports. Its based on NOSQL and it should be simple enough (with effort of course) to export a DynamoDB document store to another NOSQL solution.

Re:DynamoDB cannot store empty strings by dijit on Wednesday December 14, 2016 @04:44AM (#13174861)
your reply makes me believe you've never migrated database solutions for larger projects, especially ones where you're trying to actually make money and spend little time.

Migrating database solutions is really quite expensive in terms of time. I've done it twice and it's not something to take lightly. The usual recommendation is to never migrate database solutions unless there is a catastrophic problem with continuing with the current.

Re:DynamoDB cannot store empty strings by calsy on Wednesday December 14, 2016 @05:31AM (#13175031)
If you are talking relational database (I dont know, you just say databases), yes there can be issues exporting foreign keys, views you name it. But is it possible, yes it is.

NOSQL database on the other hand, completely different scenario. Exporting is a much more steamlined process. Again, its not magic, there is effort, but it is possible.

Just because you or your company are risk averse or strapped for cash, doesn't mean it's impossible to export a database. All the databases mention have export/import operations available, and as I said you are not locked into AWS if you do not wish to be.

Oh and don't assume my history thanks, you know nothing of it.

Re:DynamoDB cannot store empty strings by dijit on Wednesday December 14, 2016 @10:15AM (#13176501)
>Oh and don't assume my history thanks, you know nothing of it.

I'm going to attempt to be constructive here, but of course I'm biased so please take it with a grain of salt.

I cannot look past the words you speak, they are how I form an opinion of you on the internet. I cannot see your history, I cannot see your face/body language. Your words are your personality, they show your experience.

What you've said reminds me of similar people I know who have a distinct lack of experience, so, I lump you into that category.

In an attempt to understand you better I have peeked into your comment history. I can honestly say that I've not seen as much arrogance and bitterness for quite some time on hackernews.

Instead of trying to be snide and assuming you're better than people, or trying to passive-aggressively "win" a conversation.. Why don't we try to extract as much understanding or knowledge from every interaction on HN as possible.

most people on hackernews have some strong technical background, we have people here who are absolutely the forefront of their industry posting as regular users (Bryan Cantrill and Branden Gregg come to mind) which is absolutely humbling.

Now, with that out of the way, I invite you to read the topic.

"DynamoDB cannot store empty strings"

contrast that to your comment

>"NOSQL database on the other hand, completely different scenario. Exporting is a much more steamlined process."

NOSQL solutions aren't comparable if they handle types differently, they suffer most, if not all, of the conversion problems of relational databases.

The problem I faced when converting database solution was always types... Triggers, views and other relational-isms are a one-time investment fix.. converting types can easily lead to corruption of a few columns for a few rows.. how do you check that?

The answer is writing a lot of tests, and doing things especially carefully and incrementally.

and also, optimising for more revenue is not being cash-strapped, it's being wise enough to be able to reinvest capital in a new venture.. it's incredibly unwise to throw money up the wall for no reason other than it would cost a lot more to switch away due to vendor lock-in. (which is a reason not to move in of itself) which is what I'm warning about.

Re:DynamoDB cannot store empty strings by calsy on Wednesday December 14, 2016 @04:38PM (#13179892)
Please don't lecture on experience, it reeks of hubris. The fact the you suggest someone buys their own hardware to run a database to support their business is somehow cheaper shows a lack of understanding in regards to the difficulties companies face and the risks of managing a database inhouse.

You speak in absolute terms that are simply not true. Choosing DynamoDB as your datastore does not mean you have to pay Amazon forever. Databases can be exported successfully. Costs can vary dramatically so don't assume it will blow the budget.

Amazon has a large repository of documentation in regards to exporting data and there is huge online community for AWS support and third-party tools. If you are as experienced as you claim to be, you would know these are big factors when choosing a platform to serve your data.

These benefits along with an incredible array of services, reliability, security, ease of configuration, reduced licensing costs, scalability and support coupled with the wealth of talented people available who are well versed in AWS would be enough to persuade most companies towards this choice. Thus why it is the most popular choice to host data at the moment and why most companies are moving away from in house databases. Thats not an opinion, thats fact.

Re:DynamoDB cannot store empty strings by manigandham on Monday December 19, 2016 @12:13AM (#13209600)
There is no such thing as a "nosql database" and import/export are all specific and subjective to each database implementation.
Re:DynamoDB cannot store empty strings by calsy on Wednesday December 21, 2016 @08:49PM (#13234163)
Of course there isnt a NOSQL database groan its simply a term used to describe a category of databases that store data in a non relational manner.
Re:DynamoDB cannot store empty strings by seanwilson on Tuesday December 13, 2016 @05:03PM (#13171538)
It's a fair point, but when you stick to the same ecosystem you save a lot of time in terms of setup for credentials, monitoring, backups, devops etc.
Re:DynamoDB cannot store empty strings by kagia on Tuesday December 13, 2016 @04:17PM (#13171119)
You can run it locally for development. Just look for dynamodb-local images on the docker hub.
Re:DynamoDB cannot store empty strings by elcct on Tuesday December 13, 2016 @04:32PM (#13171275)
Dynamodb has performance guarantees and you don't have to get into nightmare of maintaining MongoDB cluster.