News for hackers. Stuff that matters.
Re:We saved $50k/year with a Go microservice coded in a hack... by cookiecaper on Tuesday April 10, 2018 @08:51PM (#16807135)
Yeah, sadly, this is not too much of an exaggeration. I've worked on teams that insisted they needed DynamoDB, because, well, Dynamo is for "Big Data", and they certainly wouldn't work somewhere that had "Small Data"! Replace the buzzwords/products as applicable; you could actually probably just scramble them and it'd work just as well, since someone out there thinks "RabbitMQ means Web Scale", etc.

SQL databases are amazing, robust examples of engineering. They are your friends and they're the appropriate choice for the vast majority of software. They are not outmoded or passe. Though I acknowledge there is a separate use case for K-V stores, I almost want to make policy preventing their use just because I know so many developers will abuse them badly and then stare back at you blankly during the semi-annual massive downtime event, muttering something like "Well, it's based on research at Google, so I'm sure there's a way to recover the data..."

Re:We saved $50k/year with a Go microservice coded in a hack... by MrMorden on Tuesday April 10, 2018 @10:33PM (#16807591)
Every fool knows that MongoDB is web scale. (Not as web scale as /dev/null, but MongoDB has a better logo.)
Re:We saved $50k/year with a Go microservice coded in a hack... by reificator on Wednesday April 11, 2018 @03:53AM (#16808935)
I disagree. /dev/null's logo is way cooler.

https://www.nasa.gov/sites/default/files/cygx1_ill.jpg

Re:We saved $50k/year with a Go microservice coded in a hack... by toddBarkus on Tuesday April 10, 2018 @09:01PM (#16807176)
I think this is a case for the return of the traditional "sysadmin" as "devops"/"SRE" is now the role of unblocking deploying a solution instead of questioning it's complexity/fitness.
Re:We saved $50k/year with a Go microservice coded in a hack... by cookiecaper on Tuesday April 10, 2018 @09:33PM (#16807311)
While I agree, I also think that overt gating and approval processes create a high tension dynamic that frequently breaks down, whether it's ops v. dev, security v. dev, or others. It's easy for people to get their pride wounded, and they end up encouraged to find workarounds to the process. The simple answers to this are pretty much imaginary, unfortunately.
Re:We saved $50k/year with a Go microservice coded in a hack... by toddBarkus on Wednesday April 11, 2018 @08:08AM (#16810272)
Sure, of course. I was hoping to point out that there's an increasingly overlooked value in having someone question complexity. The "no, you don't need React" of the frontend dev or the "our data is actually relational" of the back end dev.
Re:We saved $50k/year with a Go microservice coded in a hack... by rbranson on Tuesday April 10, 2018 @11:27PM (#16807801)
If your “SRE” team is only “unblocking deploying a solution” then I’m sad to say they are an operations team who has rebranded themselves to appear more relevant.
Re:We saved $50k/year with a Go microservice coded in a hack... by toddBarkus on Wednesday April 11, 2018 @08:05AM (#16810260)
That's most "SRE" -- it's a title arms race in that field between the underqualified and those that wish to convey they know how to do more than write system scripts in DSLs
Re:We saved $50k/year with a Go microservice coded in a hack... by EGreg on Tuesday April 10, 2018 @10:10PM (#16807508)
Question, though...

If your columns have types, how can you encrypt them using custom keys for each one? Is it possible?

I want the keys to reside on the client and search for encrypted data. Basically single row lookups at a time.

Re:We saved $50k/year with a Go microservice coded in a hack... by pandapunchpower on Tuesday April 10, 2018 @11:35PM (#16807834)
The column type would have to be the type of the encrypted value. The type of the unencrypted data could not be enforced by the DB and you would have to rely on code doing the correct thing.

I am however extremely wary of doing it that way. I don't know your requirements of course.

Re:We saved $50k/year with a Go microservice coded in a hack... by EGreg on Wednesday April 11, 2018 @10:03AM (#16811147)
Here is the thing - encrypted stuff is just a weird encoded string. So I can’t really use columns normally.

What I really need is just a huge table with two fields: “token”, “content”

And the token is basically the primary key but encrypted with whatever encryption.

You could even do foreign keys this way.

Hmm I suddenly have an idea. What about a layer above the database that basically enforces foreign keys and joins in this way to support end to end encryption? The content would reference ENCRYPTED foreign keys. Only clients would decrypt stuff.

Re:We saved $50k/year with a Go microservice coded in a hack... by pandapunchpower on Wednesday April 11, 2018 @10:40PM (#16817573)
>What I really need is just a huge table with two fields: “token”, “content”

Sounds more like a key value store and less like a relational database. Although you can store key value data in a relational db of course, there may be a better tool for the job.

Re:We saved $50k/year with a Go microservice coded in a hack... by JoachimSchipper on Thursday April 12, 2018 @02:00PM (#16823331)
You'll be interested in CryptDB.
Re:We saved $50k/year with a Go microservice coded in a hack... by LgWoodenBadger on Wednesday April 11, 2018 @11:25AM (#16812076)
Are you sure you aren't better off searching for hashed data? The only thing it doesn't really support is non-exact matches.
Re:We saved $50k/year with a Go microservice coded in a hack... by nurettin on Tuesday April 10, 2018 @11:51PM (#16807881)
Also, how do you index that?