Your site will scale well - as in it will give exactly the same performance for the millionth user as for the first user.

That doesn't mean you will have a fast site, though.

On GAE the datastore is pretty slow (it's a lot better now, but it used to be terrible), so a common pattern it to use a read-though cache to improve performance.

I wrote a thing that touched on this a couple of months back: http://nicklothian.com/blog/2010/11/23/a-pragmatic-approach-...

I saw your blog. One question for you if you're using GAE/J: how heavy it is the GAE/J warm-up compare to GAE/Python?

I heard stories where some people that use Spring MVC would have their request fails because the call stack is too deep.

Java is heavy, but there are ways to work around it.

The worst offender is the data access libraries. JPA is bad, JDP slightly better, but something like Objectify works really well.

Typically, Spring isn't the problem (I guess it could be if it is doing a lot of classpath scanning stuff or something though)

I think in this reference they are not talking about storing live data in memcache, just copies of objects for faster access. So when you are getting the information about a user you check memcache first, if its not there look it up in datastore and add it to memcache. Then when the user loads a second page you can get the information directly from memcache instead of having to look it up again.

I think that's what he's talking about.

The point in question is why are you doing this on a new application? The sort of caching you describe is something you put in place after your site starts seeing a lot of traffic, and running queries for every request start to slow things down to the point where optimizing them doesn't help.

If you're designing your app well and running it on a solid stack, that should be something you need to worry about in year 3, after you're being TechCrunched on a regular basis. In the Rails/Django/AppEngine world, it seems to be the case that you need to resort to that level of caching just to see regular, day to day, 100 request/second traffic.

It raises red flags to see that.

I guess I can see that argument, but at the same time its so easy to add memcache that it really seems silly not to add memcache for you models at are accessed a lot. Here is a simple user model, adding memcahce is a total of 7 lines. I just cant see the case where you wouldn't add this to start with:

  from google.appengine.api import memcache
  from google.appengine.ext import db

  class User(db.Model):
    email = db.StringProperty()
    password = db.StringProperty()
    sessionKey = db.StringProperty()

    def userKey(self):
      return "User.session="+self.sessionKey

    def put(self):
      super(User, self).put()
      memcache.delete(self.userKey())
      memcache.add(self.userKey(), self, 600)

    def getCurrent(self, request):
      user = memcache.get(self.userKey())
      if user is None:
        user = User.all().filter('sessionKey = ', request.header.cookie.get('user'))
        memcache.add(self.userKey(), user, 600)
      return user

Easy is good. It's a similar level of effort in most frameworks, which is handy.

The thing is, if you turn it on right away you never get a chance to see and fix a bunch of low hanging fruit optimizations that can really help you out. You don't know about them until you start to run up against scaling issues despite all your caching. Once you're there, it's not any fun to try to go back, find and fix those original bottlenecks, so you don't have much option but to start throwing hardware at it.

If, on the other hand you hold off and handle your first few scaling episodes by making your queries and code faster, only adding caching after you're not seeing returns from that other stuff anymore, you'll be able to go a lot farther before you have to start adding hardware.

Or if you are using Objectify:

  @Cached
  public class MyEntity {
      @Id Long id;
      ...
  }

:)

On what setup do you serve 100 pages a second that is talking directly to the database?

C#/ASP.NET/SQL Server, all out of the box, running database and web server on a single machine that would have been a middle of the road dev box 4 years ago.

That box is running about 20 sites, some big, some small, usually no more than one of them getting TechCrunched/Redditted at a time.

Here's a writeup of one particularly heavy day in the life of that box:

http://expatsoftware.com/articles/2008/03/6-million-hits-day...