News for hackers. Stuff that matters.
Re:How I use the good parts of AWS by jamestimmins on Sunday July 28, 2019 @12:37AM (#20545857)
Perhaps I'm alone here, but I'd say 50% of my time with AWS goes to (attempting to) properly configure the VPC as well as the IAM roles. For the majority of small-ish projects, the hardware isn't nearly as important or difficult as properly configuring access rules between services and outside parties.
Re:How I use the good parts of AWS by arwhatever on Sunday July 28, 2019 @03:45AM (#20546332)
The lack of this hassle is a big part of what has attracted me to Azure.

I had a list at one point, not much VPC and IAM Roles, but routing tables, CIDR blocks, NAT Gateways, I can't remember what all else.

I just remember thinking that, while such configuration options might be powerful in the right hands, if you expect your application developers to manage all that, your cloud services are designed at the wrong level of abstraction.

Re:How I use the good parts of AWS by freehunter on Sunday July 28, 2019 @11:40AM (#20548073)
Completely agree. I'm an information security consultant and my organization has been getting into cloud security more and more over the past few years. Fundamentally there's not a big difference in cloud vs on-premise security, the risks are the same and the technology is more or less the same so the steps you take are more or less the same... with that one huge caveat.

Far too often I see clients who move their development and operations teams to the cloud but not their network or IAM/GRC teams. And they expect that because AWS (et al) is the cloud, it's just the DevOps people who need access and they can handle it all. So now you have developers and system administrators doing networking, firewalls, IAM, etc.

There's a big difference between enterprise cloud environments and developer-friendly cloud environments.

Re:How I use the good parts of AWS by manigandham on Sunday July 28, 2019 @08:03AM (#20547077)
This is where GCP shines. The IAM and user system along with the organization/folder/project hierarchy is magnitudes better than anything in Azure or AWS. Makes managing everything from hobby projects to big corps simple.

The only annoying thing is that GCP uses json certification files for service accounts rather than just API key/secret pairs which are more portable, but you can usually avoid dealing with this by setting the account for the underlying VM or managed service.

Re:How I use the good parts of AWS by bharam on Sunday July 28, 2019 @03:58AM (#20546374)
> I had a list at one point, not much VPC and IAM Roles, but routing tables, CIDR blocks, NAT Gateways Just curious - how is the developer experience different on Azure? There are different types of developers. Some like to have full control and end up building their own cloud infrastructure while others like to have no control and go with something like Heroku (or maybe Azure, but I'm not too familiar with that hence my question).
Re:How I use the good parts of AWS by arwhatever on Monday July 29, 2019 @04:09AM (#20552472)
To set up basic App Services, Azure Functions, Web Jobs and the few other bits that I have been involved with, I have needed to do zero networking or permissions configuration.

Perhaps this is severely less secure by default, but it sure does make for a lower barrier to entry, and for reduced distraction from app development.

I also liked Heroku quite well. Felt at one point like I needed to move to Azure for some technical reason but right now, I can't recall the reason.

Originally in AWS I wanted to send emails from within a Lambda Function, and doing so required me to rework my networking configuration that I felt like I had barely gotten running AND pay ~$30/mo for some additional required networking service. IP Gateway or NAT Gateway or some other such. That was what finally ran me off from AWS.

Re:How I use the good parts of AWS by takeda on Sunday July 28, 2019 @10:10AM (#20547614)
It is awesome if you use direct connect with on prem, then use on prem as baseline and AWS for spikes.
Re:How I use the good parts of AWS by DVassallo on Sunday July 28, 2019 @12:45AM (#20545885)
Don’t you have to do it just once though? I have a CloudFormation template that I wrote in 2014 to set up all VPCs, Subnets, Gateways, Security Groups, etc, and it’s still all valid today.
Re:How I use the good parts of AWS by jamestimmins on Sunday July 28, 2019 @01:33AM (#20546017)
I think I'd agree if I had a better grasp of networking in general. VPC still requires a solid understanding of networking, which is my biggest challenge.
Re:How I use the good parts of AWS by chousuke on Sunday July 28, 2019 @04:26AM (#20546442)
What part of VPC networking is challenging to you? I'm genuinely interested, because I find the basics of internet (IP) routing and addressing to be very straightforward. Not trivial, certainly, but it's not black magic, and I don't think VPCs require anything esoteric.

It all essentially boils down to some basic operations with binary numbers; the decimal repsresentations of IP addresses are useful shortcuts, but to really get a sense of what's happening, you need to look at the binary forms.

Re:How I use the good parts of AWS by sah2ed on Sunday July 28, 2019 @07:24AM (#20546959)
> What part of VPC networking is challenging to you?

It can be a source of friction for someone that is just trying to accomplish a specific goal, in this case it might just be to use AWS to go live.

If building a database-backed prototype is your goal, designing a schema is only tangentially related to that goal. Schema design ability can slow you down if you have limited familiarity with SQL and/or the entity relationship model used by RDBMS.

NoSQL allows you to reach that specific goal faster than with an RDBMS by lowering the friction of creating database records and it does it by flipping the schema model from schema-on-write to schema-on-read which is a huge time saver. This made NoSQL great for building prototypes, especially for devs with little or no familiarity with SQL, because it eliminates the upfront effort of designing a schema (that is certain to change anyway) before you can begin to store records in the database.

In other words, the use of a far more familiar language (JavaScript vs. SQL) and the simpler programming model (document-model vs. entity relationship model) is a big part of why MongoDB was able to earn mind share among devs evaluating NoSQLs databases.

Re:How I use the good parts of AWS by chousuke on Sunday July 28, 2019 @01:34PM (#20548756)
If you're just setting up a prototype environment then yes, sure, set up a VPC and go with the defaults and don't worry overmuch about what you're doing. However, when you start setting up a production environment, the story changes. If you're a developer creating network-facing applications, then I am convinced that at least a basic understanding of computer networking is not optional.

I think the attitude that not everyone needs to know these things leads to people mistakenly thinking they can always have them be someone else's problem which furthermore results (needlessly!) in brittle systems.

I'm not asking for every developer to become a networking expert, but learning at least the very basics about your dependencies (you depend on networking, after all) is what I think a professional should do.

To address your NoSQL example, certainly it can be convenient to prototype with a dynamic schema, but you as a developer should still understand that even if it is dynamic, your data still has a schema and a concrete structure, which will affect your application in various ways. For rapid prototyping it's fine to ignore all this, but again once you start thinking about production, you no longer have the luxury.

Re:How I use the good parts of AWS by takeda on Sunday July 28, 2019 @11:08AM (#20547917)
In both, NoSQL or cloud providers that provide predefined networking it is easy to stay, but it will vote you in the butt once your project becomes bigger.

Using RDBMS has a stepper learning curve, but organizing your data right will pay back in bundles.

VPC is similar.

It's not that hard to configure it, it is actually very trivial compared to setting on prem data center, you can learn it or how someone who knows this.

Same with RDBMS, learn it (it is not hard) or hire a DBA.

I worked at one of popular car shopping sites, and while they used postgresql they stored car information as a thrift encoded blob. Every car trim was a separate row. A data had to be ETL to a NoSQL database to be extracted, that means to look up all car makes took 7 hours (the ETL happened through AMQP, don't ask) of course it was them stored in dedicated collection, but it was ridiculous. Out of curiosity I wrote code to convert the data from that form to relational database (postgresql) this actually helped to find that the were duplicates. The queries on postgresql database also had lower latency it didn't look like it would have a problem handling the traffic. This data it's also read only for users so it is trivial to scale out.

They also had a database to map up and latitude/longitude to a zip code. The data on MongoDB was taking 30GB and they had to use instances with enough RAM to hold all the data otherwise MongoDB would choke. The same exact data when stored in postgresql using right types (PostGIS and ip4r) took a bit over 600MB.

They also used Solr for storing inventory data.

I'm sure you are shaking your head that you would not do such things, bit of you have NoSQL blinkers on you often will run into problems that aren't problems at all.

Re:How I use the good parts of AWS by world32 on Sunday July 28, 2019 @09:10AM (#20547342)
I like how you managed to turn a question about VPC networking on AWS into an advertisement for MongoDB!
Re:How I use the good parts of AWS by sah2ed on Sunday July 28, 2019 @10:34AM (#20547740)
LOL :-)!

That was never my intention and I have absolutely no connection to them in any way whatsoever.

MongoDB just turned out to be an example I felt a lot of HNers could readily relate to considering some of their guffaws as they grew made front-page like this one: https://stackoverflow.com/questions/16833100/why-does-the-mo...

Re:How I use the good parts of AWS by philsnow on Sunday July 28, 2019 @01:30PM (#20548736)
Access control in vpc is strictly more powerful than in ec2 classic, but it also forces you to deal with a lot of stuff you didn't have to in ec2 classic (availability zones, subnets, routing, gateways).

If vpc is "ec2 v2", I would like to see a v3 that allows you to start off with a naive/simple setup and gracefully transition to the complicated setup of vpc when/if you need to.

Re:How I use the good parts of AWS by celim307 on Sunday July 28, 2019 @01:13AM (#20545969)
Depends on how fast your project grows and evolves. Hub and spoke simplifies a lot of things but not everything.
Re:How I use the good parts of AWS by rjurney on Sunday July 28, 2019 @12:39AM (#20545862)
IAM is a killer. You’re absolutely right. Are GCP permissions easier? I haven’t used them as much.
Re:How I use the good parts of AWS by jsmeaton on Sunday July 28, 2019 @01:07AM (#20545952)
I’ve stopped worrying about minimising IAM permissions and tend to just use the built in AWS roles for most things now.
Re:How I use the good parts of AWS by DVassallo on Sunday July 28, 2019 @01:18AM (#20545984)
Yes, I do the same. For service roles I just use the PowerUser managed role. I don’t see the need to put access control on Amazon’s ability to call it’s own services. I only restrict my EC2 instance profile, since that’s a bit more vulnerable, and I tend to know very precisely what it should have access to.
Re:How I use the good parts of AWS by poxrud on Sunday July 28, 2019 @09:15AM (#20547366)
What if you have a lambda with a full admin role that is not sanitizing its inputs? Or maybe it's using an outdated file parsing library (csv/yaml) with a vulnerability. Now your entire AWS account could potentially be compromised.
Re:How I use the good parts of AWS by DVassallo on Sunday July 28, 2019 @01:31PM (#20548737)
Yes, I would use a restricted role for Lambda too. Anything that gets creds in user space gets restricted permissions: EC2, Lambda, ECS, etc.
Re:How I use the good parts of AWS by darkr on Sunday July 28, 2019 @03:13AM (#20546245)
Some of the AWS built-in roles are an absolute car crash, no idea how they got through review (EMR is a good example). I use the built-in roles by default, but only after thoroughly reviewing the policies, I create my own based on that if I find anything I don’t like.

It’s not restricting amazon’s access that I’m worried about, more privilege escalation (e.g non-constrained iam:PassRole in combination with anything is a good one)

Re:How I use the good parts of AWS by mey on Sunday July 28, 2019 @12:54AM (#20545911)
https://github.com/tilfin/aws-extend-switch-roles/blob/maste... may be interesting to you.

The fact that this has to exist is also interesting...

Re:How I use the good parts of AWS by usmannk on Sunday July 28, 2019 @01:04AM (#20545940)
I'll hate on GCP all day long but the one thing it has going for it is the permissions.
Re:How I use the good parts of AWS by wodenokoto on Sunday July 28, 2019 @02:23AM (#20546140)
Wow, I was gonna say “haven’t tried IAM on other platforms than GCP but I can’t imagine it being more difficult than this”, but damn ...
Re:How I use the good parts of AWS by chishaku on Sunday July 28, 2019 @12:47AM (#20545894)
night and day imho