News for hackers. Stuff that matters.
Re:Announcing Google Cloud Bigtable by mataug on Wednesday May 06, 2015 @05:29AM (#9497442)
I've been quite frustrated with the google cloud platform, Just take a look at their APIs for AppEngine, CloudSQL, GCE and the sort, Its pathetic compared to their direct competitor AWS.

Lets compare trying to do create an RDS instance on AWS vs Creating one on CloudSQL

AWS:

1. Get AccessToken and AccessSecret from IAM

2. pip install boto

3. conn = boto.rds.connect_to_region("us-west-2")

4. db = conn.create_dbinstance("db-master-1", 10, 'db.m1.small', 'root', 'hunter2')

Done !

Google

1. Get a Client Id and Client Secret

2. pip install google-api-python-client

3. Go through the OAuth Flow and run a server locally to capture the access token

4. Use the discovery api to generate a service object. Good Luck finding this in the documentation

5. use the uninspectable service object to create a cloudsql instance.

The reason I don't have code for steps 3,4 and 5 is because I gave up after wasting time trying to figure this out.

My point is that they've gotten into the habit of doing half assed work so I have no hopes that the've improved this time. Practically no way to automate this The only way to use this would be from the horribly slow GUIs that google provides.

EDIT:

I ended up using google cloud sdk cli and running the automations with subprocess.check_output(['gcloud', 'sql', 'instance' ... ])

Re:Announcing Google Cloud Bigtable by forgottenacc56 on Wednesday May 06, 2015 @06:08AM (#9497576)
You missed the snakes and ladders game of following googles out of date documentation.

"You're doing it wrong, we deprecated that last year, I know we haven't yet updated the 4 separate places in which the same thing is documented all in a different context and different way. We really should fix that but we're busy coding. Did you know we're all PhDs at Google?"

What?!?!? You're using the Google console version 3? Why? That feature isn't implemented there. Stupid you. You're MEANT to be using our new Google console version X. Why are you using our old one?

Also you missed the 16 hours you'll spend trying work out why something isn't working only to find it's actually been changed or taken out of the Google developer console, without any trace or notice left in the code to say'we moved/removed the feature that you expected to be here.'

Really you should ask Google for help on StackOverflow which is now the official Google channel for ignoring support questions, and where your question will within seconds be down voted, derided and deleted by the StackOverflow Community,saving Google the effort of not reading and ignoring your question about how to resolve the catastrophic failure of your software.

Seriously though, why not entrust your critical systems to such capable hands?

Re:Announcing Google Cloud Bigtable by ianchildress on Wednesday May 06, 2015 @12:35PM (#9499869)
I signed up just to reply to this post and say THIS is exactly how our development team (which write in Go) feels. This post could not be more accurate in terms of trying to deal with the outdated documentation.

#1 The documentation is often months and many versions old.

#2 is the fact that app engine instances have to connect to back end compute engine instances using a public IP is unbelievable. The solution for an app engine instance to communicate with your compute engine app is to use their PubSub api system. To get the PubSub api system to work, see #1. The examples are months outdated and you no longer use the appengine.Context to create your OAuth token, but instead now use context.Context which you learn by going through github issue comments.

Google needs to recognize their great deficiency in documentation to be considered in the same tier as AWS.

Re:Announcing Google Cloud Bigtable by mimming on Wednesday May 06, 2015 @02:51PM (#9500965)
(Google DevRel here)

We recognize that there's out of date stuff in there :\ This is obviously on us, but if you see something wrong, please use that little 'send feedback' link the upper right corner, as demonstrated in this gifly: http://gifly.com/3SA5/

I just set up an alert set up for feedback with a table flip in the description, so those go right to my inbox.

Re:Announcing Google Cloud Bigtable by hoodoof on Wednesday May 06, 2015 @04:05PM (#9501455)
You should remove the feedback link and instead have hard requirements in your release process for all documentation to be perfect before any code is released.

"Oh the documentation is out of date, why didn't you tell us?" illustrates the problem with Google exactly.

Re:Announcing Google Cloud Bigtable by mimming on Wednesday May 06, 2015 @07:25PM (#9502413)
It looks like my previous reply vanished into the ether. Leme try again.

I don't intend to shift any blame here. The feedback link isn't a solution. I know you can't un-see docs and reclaim your lost time.

But, there are a lot of words in there for us to fix, and that feedback does help us prioritize :)

Re:Announcing Google Cloud Bigtable by hoodoof on Wednesday May 06, 2015 @07:36PM (#9502452)
It's the development process that Google needs to fix. Google's development process does not require accurate documentation prior to release.
Re:Announcing Google Cloud Bigtable by mimming on Wednesday May 06, 2015 @07:49PM (#9502500)
Understood.

The proof of the pudding is in the eating, right? The only way I can respond to that is by delivering good stuff. It will take time for that evidence to accumulate.

For today I just wanted to communicate that we are paying attention :)

Re:Announcing Google Cloud Bigtable by jivos on Thursday May 07, 2015 @03:30PM (#9507651)
I've recently applied for a few documentation/tech writer positions at google. I'll gladly help with improving your documentation and I live about 1 mile from the googleplex.
Re:Announcing Google Cloud Bigtable by jfoster on Thursday May 07, 2015 @12:26AM (#9503281)
FYI, outside of Google Cloud, it's also the things that Google doesn't document that are frustrating.

An example: Chrome on Android goes beyond the spec by adding limitations on when an Audio element will play, requiring an explicit user action. (https://code.google.com/p/chromium/issues/detail?id=178297)

In that example, the limitation (along with other Chrome limitations, such as all the features that behave differently on HTTPS sites) isn't documented anywhere. Developers have to try it on Android, notice that something is amiss, search for what is going wrong, eventually figure out that Google have made an intentional limitation, and then work around it.

I'm not a fan of Google going beyond limitations that are necessary for security when it comes to Chrome. That said, if it's going to happen, it would be nice to at least save folk some time by documenting it properly.

Re:Announcing Google Cloud Bigtable by photonios on Wednesday May 06, 2015 @02:59PM (#9501018)
Great to hear they go straight to your inbox. Usually I have the feeling the stuff that is submitted through the 'Feedback' buttons is never read.
Re:Announcing Google Cloud Bigtable by TheIronYuppie on Thursday May 07, 2015 @02:50PM (#9507363)
(Googler here)

In fact, every piece of feedback goes directly to our bug tracking, and a human being looks at every single one. That's not an excuse, and doesn't mean that we can't do better, but we track it all. The biggest problem is the conversion from item in the bug list to actual changes, and we're working on that!

Re:Announcing Google Cloud Bigtable by mimming on Wednesday May 06, 2015 @03:03PM (#9501043)
Just to clarify, they all go to someone's inbox. Only the table flip ones go to mine.
Re:Announcing Google Cloud Bigtable by Poiesis on Wednesday May 06, 2015 @06:33PM (#9502217)
That's clever with the table flip and all; it gives you a unique way to get just the responses from this. It's great, seriously, that you're using the tools you have to try to make this situation better.

I bet it's frustrating, though, that you can't do more to address the real problems, like lack of (apparent) incentive to keep this stuff up to date. You have to resort to clever sort-of hacks like "put this magic string in your request so someone will read it". A solution which, I might add, sounds a lot like the problems people are talking about when they talk about inscrutability.

Re:Announcing Google Cloud Bigtable by mimming on Wednesday May 06, 2015 @07:31PM (#9502436)
The table flip is more about whimsy than effectiveness. :) Given many options, I usually pick the one that's most fun.

Honestly, I plan to read all of the feedback. I worked on the Firebase docs, and I've see how effective a tight feedback loop can be for improving polish.

Re:Announcing Google Cloud Bigtable by biot on Wednesday May 06, 2015 @11:24PM (#9503127)
In case you're reading on iOS and see only a static image (webm doesn't play, even in Chrome): http://i.gifly.com//media_gifly/3/S/A/5/b/3SA5.mp4
Re:Announcing Google Cloud Bigtable by jfoster on Wednesday May 06, 2015 @09:32AM (#9498421)
It's a shame about the tone of this post. I think it might not suit HN. As a Google Cloud customer, though, I think you do manage to be spot on about a lot of the things Google are doing wrong when it comes to the Cloud Platform.

Today I came across an App Engine SDK bug that was reported almost 5 years ago, has a several years old patch in the comments, and still hasn't been remedied by Google. App Engine users who encounter it have to find the bug, apply the patch to the SDK, and repeat that upon each SDK update.

Re:Announcing Google Cloud Bigtable by dlor on Wednesday May 06, 2015 @10:53AM (#9499057)
Could you send me a link to the issue? I can hopefully get that fixed.
Re:Announcing Google Cloud Bigtable by d0vs on Wednesday May 06, 2015 @04:47PM (#9501686)
Not OP but here's another example: https://code.google.com/p/googleappengine/issues/detail?id=6...

The patch is trivial

Re:Announcing Google Cloud Bigtable by nulltype on Wednesday May 06, 2015 @05:28PM (#9501921)
urlfetch is the worst, at least MVM will be getting rid of that. I would just use the socket api instead of urlfetch, like I do here: https://code.google.com/p/googleappengine/issues/detail?id=6...
Re:Announcing Google Cloud Bigtable by jfoster on Thursday May 07, 2015 @02:47AM (#9503612)
Wow. That's a classic example of the lack of investment into App Engine. Filed 7 years ago, acknowledged, and still open with no one assigned to it.
Re:Announcing Google Cloud Bigtable by nulltype on Thursday May 07, 2015 @03:59PM (#9507848)
It's true, if Google wants GCP to be good, they will have to use it themselves. I definitely don't trust public bug reports to fix issues. At least if you have a support plan they are good at responding and sometimes fixing things. There's no way anyone in Google uses URLFetch, which explains why it's so bad.
Re:Announcing Google Cloud Bigtable by jfoster on Wednesday May 06, 2015 @05:28PM (#9501923)
Re:Announcing Google Cloud Bigtable by philip1209 on Wednesday May 06, 2015 @01:07PM (#9500115)
We started StaffJoy infrastructure on GCE. The per-minute billing had the potential to be a huge cost-saver with the optimization software that we use for scheduling because doubling the number of CPU cores (at twice the hourly cost) halved the calculation time, resulting in no overall cost increase but faster data.

I switched from GCE to AWS when we had a build break because Google released a new command line utility for deploys, immediately yanked the previous CLI causing it to 404, and provided no way of pulling the "latest" CLI. (We needed to reinstall the CLI for each build because we used CircleCI). It was part of a long string of frustrations of getting basic workflows to work.

Re:Announcing Google Cloud Bigtable by skj on Wednesday May 06, 2015 @03:24PM (#9501185)
It looks like https://circleci.com/docs/deploy-google-app-engine explicitly hard-coded an old version of the appengine SDK. And by old, I don't mean not-quite-the-latest, I mean quite old. When it was yanked, it was not the previous CLI, it was the CLI from a deprecation-policy-timelimit ago.
Re:Announcing Google Cloud Bigtable by dwwoelfel on Wednesday May 06, 2015 @03:31PM (#9501232)
That's just sample code to demonstrate how one would install appengine on Circle. The URL in the example is a placeholder for the one you would get from https://cloud.google.com/appengine/downloads.
Re:Announcing Google Cloud Bigtable by wslh on Wednesday May 06, 2015 @07:22AM (#9497794)
Also, it seems Google's Page Rank is sometimes optimized to show old documentation instead of the most related and current one. this is natural because there are more links pointing to the old one but needs to be fixed.
Re:Announcing Google Cloud Bigtable by jjjjoe on Wednesday May 06, 2015 @08:36PM (#9502684)
(Google Cloud Support here)

The across-the-board guidance to go to StackOverflow is not working, and we get that. It's not just that system administration questions get downvoted on StackOverflow (which I think is the parent's point) but that StackExchange isn't good for general discussion.

We (support) are trying to be much more clear that free support is not "StackOverflow or nothing." We recently updated our "community" support page at https://support.google.com/cloud/answer/3466163 to lay out all of the community support options in one place.

Bottom line: everything listed on that page has Googlers actively participating. This includes groups, StackExchange, and issue trackers. There's room for plenty of improvement, the intent is not to ignore you.

Re:Announcing Google Cloud Bigtable by hoodoof on Wednesday May 06, 2015 @10:25PM (#9502988)
Google should have a central support forum and try to focus all the questions into one place.

StackOverflow should not be on your support list at all, for the reasons raised. Having said that, Google should still be on SO answering questions that end up there.

What Google needs to understand is that its long standing public reputation that it has built is an organisation that actively tries to avoid providing support - ever since Google started it has tried to avoid support. That is now the reputation that Google carries into its efforts to woo the developer community.

Google has to be extraordinarily good at developer support to dispel the baseline assumption that developers have that Google really (genuinely) wants to avoid dealing with support questions.

There's a level of cluelessness to Google's support strategy that is concerning. Why, for goodness sake, would it EVER look like a good idea to push support to StackOverflow? Who is doing the thinking behind that sort of decision? It is self evident that Google's support interests and StackOverflow are not the same thing. It's the sort of decision made without really considering the detail, and that is the point about Google's support - it's an afterthought. Kind of like washing the dishes after dinner is eaten - has to be done but we're not enthused about it.

And in the end, lack of support is a showstopper for using a cloud computing platform. If the support looks sketchy then it just isn't worth risking your business by using that platform.

Re:Announcing Google Cloud Bigtable by jjjjoe on Thursday May 07, 2015 @10:36AM (#9505312)
You make great points. I'm new and can't really speak to your rhetorical question of "what were they thinking?!?" but I do hope for my own job security that support isn't an afterthought :-)

First, I should point out that what we are talking about here is Google's "Bronze Support." This is similar to Amazon's "Basic Support." In both cases it's not what you should be thinking about if you need a case response time SLA, or the ability to wake up engineers at midnight. If your business depends on any platform provider I really hope you buy a support plan which gets you the ability to talk to support and engineering whenever you need. Google definitely offers these. They start at $150 per month. (https://cloud.google.com/support/) End plug.

On StackOverflow: let's stop calling it "support." It's a Q&A site with good SEO. If you have a question which "belongs" there it's a fine place to ask. We're moderating our "go to StackOverflow no matter what" messaging, but I can't see tossing it completely.

Anyway, when it comes to free support, I partially agree with your point about a single forum, in that can be confusing. Again, keep an eye on https://support.google.com/cloud/answer/3466163?hl=en&ref_to...

To your point about a single forum for everything, I respectfully disagree. Free-form discussion is different from bug tracking. Highly structured Q&A has a place too. It seems to me that every product should have - a discussion forum where users can discuss the product and raise issues - an issue tracker to collect bug reports and feature requests - a designated place for Q&A

On each of those three, support staff and (preferably) engineers should participate daily. I think the situation with Compute Engine is closest to our ideal right now: it has a lively Google Group, actively triaged issue tracker and a sponsored tag on ServerFault (which I hope we can agree is a better destination than StackOverflow)

Re:Announcing Google Cloud Bigtable by nulltype on Thursday May 07, 2015 @04:19PM (#9507972)
While StackOverflow has a number of flaws, it is pretty good for finding things that I'm looking for. I've definitely found posts from the BigQuery team on there answering my BigQuery questions.

As for Google's support plan thing, it might be a little odd that it costs $150 per month, but if you're spending any significant money GCP, it's well worth it. The support response times even at the lowest level are pretty good and they sometimes fix bugs.

Of course, I think if Google employees use GCP internally, it will improve at a much faster rate.

Re:Announcing Google Cloud Bigtable by EugeneOZ on Thursday May 07, 2015 @10:51AM (#9505445)
> If your business depends on any platform provider I really hope you buy a support plan which gets you the ability to talk to support and engineering whenever you need

You can get support in 10 minutes in Linode, without any payments for support. Linode can afford it but Google can't, or don't care.

Re:Announcing Google Cloud Bigtable by SiVal on Thursday May 07, 2015 @06:18AM (#9504060)
Stack Overflow is the Soup Nazi of technical Q&A. It's where mods compete for who can be the first to find a way to prevent you from getting an answer. ("Repeat: someone asked a question about a similar topic a couple of years ago with two contradictory answers that were both wrong, so we will not only NOT answer you, but we will prevent anyone ELSE from answering you, either! You're welcome.")

If Google doesn't want support questions getting through, forcing us to submit them through Stack Overflow should do the trick nicely.

Re:Announcing Google Cloud Bigtable by EugeneOZ on Thursday May 07, 2015 @10:47AM (#9505411)
Link to StackOverflow is equal to "RTFM". With same success you could just put some search engine input box.
Re:Announcing Google Cloud Bigtable by mimming on Wednesday May 06, 2015 @02:24PM (#9500735)
Google Cloud Developer Relations here :)

Thanks for the feedback and stuff. I empathize with your frustration. We're listening to threads like these, and workin' hard to make stuff better.

Also, next time you're frustrated with Google Cloudy stuff, feel free to vent to me. It's like unfiltered usability studies.

Re:Announcing Google Cloud Bigtable by forgottenacc56 on Wednesday May 06, 2015 @04:35PM (#9501611)
OK so if you're from Google and you say Goigle is listening, here's a question I've never heard and answer to: how come Google has such terrible support for Python 3?
Re:Announcing Google Cloud Bigtable by mimming on Wednesday May 06, 2015 @04:53PM (#9501724)
This is probably outside of my area of expertise, but I'll see if I can dig up an answer.

I need more context though. Where are you trying to use Python 3 and what problem are you trying to solve with it?

Re:Announcing Google Cloud Bigtable by mimming on Wednesday May 06, 2015 @06:13PM (#9502126)
I've never done much Python stuff at all (It's a great language, I just never used it on a major project). I just spent the last hour catching up on the chronicles of Python 2 -> Python 3 (not just at Google, but everywhere).

Sorry, but here's my answer: http://i.imgur.com/tZOS8.gif

Re:Announcing Google Cloud Bigtable by forgottenacc56 on Wednesday May 06, 2015 @04:18PM (#9501515)
I just want to say this comment has 73 upvotes/points.
Re:Announcing Google Cloud Bigtable by ryanseys on Wednesday May 06, 2015 @01:15PM (#9500188)
Disclaimer: I work on gcloud-node [1]. :)

You're describing the OAuth 2.0 client dance which has always been cumbersome but necessary for browser-side auth. With Google Cloud Platform related services, you should never have to do this if you're using these services from the context of your server. As someone previously mentioned, Google has Service Accounts which greatly simplifies the auth process.

For example, to auth with gcloud-node it's as easy as:

  1. Download the key.json associated with your service account.
  2. var gcloud = require('gcloud')({ projectId: 'abc-123', keyFilename: './key.json' });
  3. There is no step 3.
We've also made some pretty nice docs [2] with clear examples to help you every step of the way. We have yet to integrate Cloud Bigtable but it's on our radar.

For a pythonista like yourself, you may like to consider trying gcloud-python [3], and it should go without saying please feel free to file issues or reach out directly with concerns if something just doesn't feel quite right!

[1]. https://github.com/GoogleCloudPlatform/gcloud-node

[2]. http://googlecloudplatform.github.io/gcloud-node/#/docs/

[3]. https://github.com/GoogleCloudPlatform/gcloud-python

Edit: Formatting.

Re:Announcing Google Cloud Bigtable by mataug on Wednesday May 06, 2015 @01:17PM (#9500205)
I'll try this out. Thanks.
Re:Announcing Google Cloud Bigtable by nulltype on Wednesday May 06, 2015 @05:14PM (#9501851)
I like the gcloud-node docs! I remember the oauth stuff being a pain just because oauth is kind of complicated and the terms are weird and there was no guide that just says "go here and click this" you had to just figure that out.
Re:Announcing Google Cloud Bigtable by TeeWEE on Wednesday May 06, 2015 @05:49AM (#9497521)
Well, it might be a bit more difficult to get started. The cool thing is that you only have to install the google-api-python once, and you can communicate with lots of services, by creating the service object. CloudPubSub, BigQuery etc etc.

Also, google comes with very good monitoring tools (they bought stackdriver and integrated it).

We are running a big appengine app, with quite some compute engine instances. Together with cloud-storage and biguery its a full blown solution, without having to setup every instance yourselve. I think AWS cant compete with that, you almost need a system-engineer when you work with AWS.

All in all: you are used to AWS. Making it cumbersome to get started with Google Cloud.

I do agree tough that the documention is lacking often.

Re:Announcing Google Cloud Bigtable by mataug on Thursday May 07, 2015 @02:16AM (#9503548)
I don't think anyone cares about having to install one or any number of client libraries for services being used.

Whats more important the ease, simplicity and clarity when using a service. The reason I like boto is because its easy to find out what is possible and what is not.

Re:Announcing Google Cloud Bigtable by piva00 on Wednesday May 06, 2015 @08:45AM (#9498175)
There's also the point about vendor lock-in, in AWS you do need a systems engineer to setup and automate infrastructure, but pretty much everything (aside from a few products like Redshift, Dynamo and EMR is what I recall from my mind) you can port out of AWS pretty easily if and when the time comes to have your own data center and cut some costs.

Google Cloud is a much bigger vendor lock-in, you could not setup the same environment outside of it.

Re:Announcing Google Cloud Bigtable by vgt on Wednesday May 06, 2015 @10:28AM (#9498847)
I'm genuinely curious why you hold that opinion, especially in context of a thread that discusses a service that's available through the standard hbase api
Re:Announcing Google Cloud Bigtable by rodionos on Thursday May 07, 2015 @01:11PM (#9506627)
Without support for co-processors in BigTable I would hardly call it 'standard' API. Also the implementation of buffered mutation, a key attraction in HBase 1+ for fast non-blocking writes, is different enough in bigtable that you might have to change application code to work it consistently. It's nice though that bigtable API have references to HBase bugs :)

> HBase 1.0.0 has a bug in it. It returns maxSize instead of the buffer size for > getWriteBufferSize. https://issues.apache.org/jira/browse/HBASE-13113

Re:Announcing Google Cloud Bigtable by 23david on Thursday May 07, 2015 @03:26AM (#9503707)
Ever try porting a customer using cloudformation templates and autoscaling groups over to another cloud service? Or reimplementing s3 or EBS semantics? It can be done, but definitely non-trivial... lockin++
Re:Announcing Google Cloud Bigtable by megaman821 on Wednesday May 06, 2015 @11:08AM (#9499186)
I think of the three AWS services you named, only Dynamo can't easily be ported out.

Redshift use Postgres drivers. EMR uses the HBase API.

Re:Announcing Google Cloud Bigtable by eropple on Wednesday May 06, 2015 @11:33AM (#9499420)
Redshift uses Postgres drivers, but does not use Postgres semantics. Important to consider if you're using any nontrivial amount of data, because porting it even to other cstore databases will require a lot of thinking and probably some munging-at-scale.
Re:Announcing Google Cloud Bigtable by erlichmen on Wednesday May 06, 2015 @05:37AM (#9497473)
You are doing it wrong:

  1. curl https://sdk.cloud.google.com | bash  
  2. gcloud auth login (one time thing)  
  3. gcloud sql instances create db-master-1
Re:Announcing Google Cloud Bigtable by estefan on Wednesday May 06, 2015 @06:39AM (#9497663)
> 2. gcloud auth login (one time thing)

That's a royal pain in the arse to automate with something like ansible... AWS nailed it with a token & secret (not some horrible expiring oauth2 object).

And that's not to mention gsutils sleeping for a second or two every time it tells you there's an update.

It's like these guys have never heard of devops.

Updates are my problem; I don't expect software to sleep whenever they're available... because you never know, I might be running it in a loop under cron and it might just piss me off when I start to lose performance...

Re:Announcing Google Cloud Bigtable by skj on Wednesday May 06, 2015 @07:57AM (#9497960)
'gcloud auth activate-service-account' will auth using a JWT, acquired from the Cloud Console.

Also, if you are running stuff in Google Compute Engine, no auth flow is needed at all: there is a metadata service that you can connect to (and gcloud connects to by default) that provides credentials associated with the machine you're on.

> It's like these guys have never heard of devops.

:\ yeah...I can't imagine trying to automate a web login flow with ansible...but no one would ever suggest you do that.

(disclosure: I wrote most of the client-side auth code for gcloud)

Re:Announcing Google Cloud Bigtable by philip1209 on Wednesday May 06, 2015 @01:11PM (#9500153)
This was also frustrating on something like CircleCI
Re:Announcing Google Cloud Bigtable by dantheman on Wednesday May 06, 2015 @05:42AM (#9497490)
This is a bad practice:

  curl * | bash
and should be discouraged. It's easy but it's like saying, hey I'm logged in as root all the time - it's easier.
Re:Announcing Google Cloud Bigtable by olalonde on Wednesday May 06, 2015 @07:10AM (#9497748)
In this case, * is "https://sdk.cloud.google.com" and I don't see how it's worse than trusting a package from PyPi. If anything, the curl command offers some guarantee that you are running code endorsed by Google.
Re:Announcing Google Cloud Bigtable by lukeschlather on Wednesday May 06, 2015 @11:26AM (#9499364)
If the HTTP connection is interrupted during download (highly likely if you're doing this routinely) you'll end up with something in a broken state. Locally running a remote stream as it arrives as code is just a bad idea, unless you're talking about something like a webpage where a partial is potentially preferable to nothing at all.
Re:Announcing Google Cloud Bigtable by cristicbz on Wednesday May 06, 2015 @12:54PM (#9500029)
Code which is meant to be piped into bash is generally written as:

    #!env bash
    f () {
      ...code
      ...code
      ...code
    }
    f
Hence a partial stream will do nothing (syntax error, missing brace to be precise).
Re:Announcing Google Cloud Bigtable by plorkyeran on Wednesday May 06, 2015 @01:15PM (#9500196)
This is a completely trivial problem to solve (wrap the logic in a function), which the script in question does solve.
Re:Announcing Google Cloud Bigtable by brlewis on Wednesday May 06, 2015 @09:00AM (#9498245)
It's no different from steps 2 and 3. At each step you're trusting that code you just downloaded from the web is doing what you want it to do.
Re:Announcing Google Cloud Bigtable by lukeschlather on Wednesday May 06, 2015 @11:32AM (#9499405)
You're also trusting that the Internet is going to stay up for the duration of the install script, which is an unreliable assumption. Imagine at some point the script does:

rm -rf ~/.config/google

and the connection gives out at

rm -rf ~/

Suddenly your script didn't install, and you've blown away your home directory. HTTP(S) is designed for reading documents, where it's OK if you can't read the document in its entirety.

Re:Announcing Google Cloud Bigtable by delroth on Wednesday May 06, 2015 @11:40AM (#9499458)
This is easily solved by putting everything in a bash function and calling the function as the last thing in the script. If you look at the Cloud SDK setup script, that is exactly what it does.
Re:Announcing Google Cloud Bigtable by lukeschlather on Wednesday May 06, 2015 @03:33PM (#9501255)
I agree it's a trivial problem to solve, I just think the right way is to download the code and store it separately, so that it's easy to add checksum/signature validation later.
Re:Announcing Google Cloud Bigtable by icebraining on Wednesday May 06, 2015 @05:47AM (#9497510)
And only it really does anyway is

  wget https://dl.google.com/dl/cloudsdk/release/install_google_cloud_sdk.bash
  chmod 775 install_google_cloud_sdk.bash
  ./install_google_cloud_sdk.bash
Re:Announcing Google Cloud Bigtable by crdoconnor on Wednesday May 06, 2015 @08:21AM (#9498061)
Only if you don't trust the source (or it's http).

Otherwise you might as well give up on downloading any software from the internet. Ever.

Re:Announcing Google Cloud Bigtable by Blackthorn on Wednesday May 06, 2015 @11:44AM (#9499485)
Have you ever run "./configure" on some open source code you downloaded? That's no more safe. In this case, curl | bash is probably more safe because at least it came over HTTPS.
Re:Announcing Google Cloud Bigtable by discardorama on Wednesday May 06, 2015 @11:56AM (#9499577)
> Have you ever run "./configure" on some open source code you downloaded? That's no more safe.

That is if you don't do a

    view ./configure
first and go through it. You do take a look at the configure scripts, don't you?
Re:Announcing Google Cloud Bigtable by Blackthorn on Wednesday May 06, 2015 @12:37PM (#9499884)
What's less readable, a ToS that you have to accept or a GNU autotools configure script?

Example: expat's autoconf script is over twenty two thousand lines long.

Re:Announcing Google Cloud Bigtable by cos on Wednesday May 06, 2015 @06:15PM (#9502141)
You could just as easily do 'curl someurl | less' before you run it, too.
Re:Announcing Google Cloud Bigtable by erlichmen on Wednesday May 06, 2015 @05:46AM (#9497509)
You don't need root to install gcloud
Re:Announcing Google Cloud Bigtable by icebraining on Wednesday May 06, 2015 @05:48AM (#9497511)
I think that was just an analogy.
Re:Announcing Google Cloud Bigtable by anthmgoogle on Wednesday May 06, 2015 @06:14PM (#9502134)
(I'm a Google developer on API Platform) A few of the follow ups mentioned that Service Accounts are more appropriate for this API and closer to the AWS experience. There have been some challenges using service accounts that have been addressed with a relatively new feature set called "Application Default Credentials": https://developers.google.com/identity/protocols/application... For most cloud APIs, you simply run "gcloud auth login" to enable auth on the development machine, and when you deploy to GCE or GAE it will use the more secure built-in service accounts there. And there is no token expiration in either case.
Re:Announcing Google Cloud Bigtable by samkone on Wednesday May 06, 2015 @06:47AM (#9497689)
Or you can use the service account to avoid dancing around gcloud and others. But I agree Google has a problem building stuff for us common humans who do not work at the big G.
Re:Announcing Google Cloud Bigtable by mataug on Wednesday May 06, 2015 @01:12PM (#9500170)
I this stems from a general lack of empathy for customers in the organisation. Take a look at the issue list on appengine for example https://code.google.com/p/googleappengine/issues/detail?id=1...

It seems like they get a bunch of engineers to come and build stuff then once they move on, the project is abandoned or poorly maintained until another team of engineers come and try to rebuild it from scratch.

Here's more proof, Google Wallet for example ... https://news.ycombinator.com/item?id=9498475

Re:Announcing Google Cloud Bigtable by estefan on Wednesday May 06, 2015 @06:40AM (#9497667)
Let's hope the oauth token you get back doesn't expire after you think you've finished! And then lets hope it doesn't also take you a day or so to have to learn oauth2client so you know which secret method you need to call request a refresh token. Such fun!
Re:Announcing Google Cloud Bigtable by mataug on Wednesday May 06, 2015 @01:06PM (#9500107)
It took me more than half a day to figure out that I was doing things wrong even though I've used oauth2client before on other projects.
Re:Announcing Google Cloud Bigtable by TheIronYuppie on Thursday May 07, 2015 @02:53PM (#9507380)
Interesting point in that you used a third party in step 2 of your AWS workflow. If you're going to use a 3rd party, have you tried Terraform?
Re:Announcing Google Cloud Bigtable by thrownaway2424 on Wednesday May 06, 2015 @10:59AM (#9499106)
Is there something in your comment regarding bigtable?