Re:The Unofficial Guide to Migrating Off of Google App Engine
by lyddonb on Friday February 25, 2011 @11:24PM
(#2264869)
1) Yep. My company moved to an HRD instance this week and our CS reps and customers have both noticed a significant difference in stability. That doesn't imply that it's perfect but to say that Google is ignoring the issue is completely false.
Re:The Unofficial Guide to Migrating Off of Google App Engine
by joshu on Friday February 25, 2011 @11:59PM
(#2264916)
How bad was the cost differential?
Re:The Unofficial Guide to Migrating Off of Google App Engine
by lyddonb on Saturday February 26, 2011 @04:33PM
(#2266576)
Not sure yet. Even though we are one of the bigger clients on App Engine we weren't getting charged much at all for our usage anyways. But I believe Google said it would be around 3x for the HRD which isn't bad at all. Our executives really didn't care though as stability and reliability to our customers is worth too much to worry 3X (GAE's non-hrd stability has not been good recently). Btw we all use EC2 for some of our other services and that is costing us quite a bit more than GAE is at the moment. We'll see if that continues to hold true as we grow.
Re:The Unofficial Guide to Migrating Off of Google App Engine
by nikcub on Saturday February 26, 2011 @06:31AM
(#2265333)
I am building everything on AppEngine atm. Initially it was just to prototype but I am thinking of launching a preview release of one app live
Can you talk about how big some of the larger customer apps are and how much traffic/uers they serve?
It would be easier for ppl in a position like mine to make decisions about AppEngine if there were big examples/case studies to point to in the same way Amazon, Heroku, Rackspace etc. do
Re:The Unofficial Guide to Migrating Off of Google App Engine
by powera on Saturday February 26, 2011 @01:53PM
(#2266204)
I can't give any current numbers, but last year we did a blog post about Gigya who had an app that peaked at 1600QPS: http://googleappengine.blogspot.com/2010/02/scalability-mean...
Re:The Unofficial Guide to Migrating Off of Google App Engine
by StavrosK on Saturday February 26, 2011 @03:09PM
(#2266381)
Not to disparage your post, but it contains only an inkling of substance. Some people managed to get 1600 QPS on GAE, that tells me absolutely nothing. What was their usage like? How much did they pay? What was their latency like? What did they think of GAE? Etc etc.
Re:The Unofficial Guide to Migrating Off of Google App Engine
by bane on Saturday February 26, 2011 @06:46PM
(#2266865)
This is what I'm most concerned about. The OP claims that 50/sec was breaking things...which doesn't sound quite right to me. Any ideas?
Re:The Unofficial Guide to Migrating Off of Google App Engine
by kamens on Saturday February 26, 2011 @09:41AM
(#2265610)
Can you elaborate on queries not having "the right indexes"? Since the development server automatically adds indexes for all queries run on local dev machines and the production machines cannot run a query for which the necessary indexes don't exist, it's easy to assume that your indexes must be right.
I'd love to hear more about tweaks to existing indexes that can alleviate these sorts of problems.
Re:The Unofficial Guide to Migrating Off of Google App Engine
by powera on Saturday February 26, 2011 @01:51PM
(#2266196)
The geomodel stuff is what I'm referring to here; it's efficient enough that you can actually execute the queries, but it's not necessarily fast (or cheap) to do so.
Re:The Unofficial Guide to Migrating Off of Google App Engine
by babyshake on Saturday February 26, 2011 @02:09AM
(#2265074)
One annoying issue with the HR datastore is the appids have a 's~' prefix which can lead to strange behaviors when setting up domains via google apps and other jankiness.
Six month roadmap?
Re:The Unofficial Guide to Migrating Off of Google App Engine
by powera on Saturday February 26, 2011 @02:31AM
(#2265103)
http://code.google.com/appengine/docs/roadmap.html
The 's~' prefix as an internal detail isn't going to go away, but we're trying to smooth out any of the situations where you have to see/deal with it.
1) It's not fair at all to say that Google "continue to ignore the two critical issues of uptime and data store latency". The HR datastore is specifically designed to address the concerns about variable latency of datastore operations, and to prevent both planned and unplanned downtime.
2) In my experience, high CPU cost for datastore operations is generally tied to things like having large numbers of indexes, or doing queries that don't have the right indexes. Each index written involves a bigtable write, if you are doing hundreds of these per entity it can become very expensive. That said, in general it isn't easy enough to know when you are doing something that will be expensive.
3) The bulkloader is definitely painful to use if you aren't familiar with it. In particular, casting schemaless data to a format such as CSV is hard because of the problem that rows can have unexpected keys. An improved "Bulk Datastore Import and Export tool" is on the 6-month roadmap for the product.