Fluxx

On the quadrupling of the estimate from $250 million to $1 billion between 1995 and 1996, the article states:

"The cost increase was the result of detailed engineering studies conducted during the year or so after the initial estimate was released. Among other things, soil testing in the Bay had revealed that bridge pilings would need to be anchored “deeper into bedrock than expected,” she writes."

Now hindsight is 20/20 and I am not an engineer in this field, but it seems that if you're floating an estimate that isn't informed by the engineering studies necessary to give an accurate estimate then you probably shouldn't have given that initial estimate in the first place? Or at least should have given the initial estimate as a range and/or with a huge disclaimer that you might get into researching the bridge and the estimate could cost multiples more?

Yeah that's totally fair. I live at Geary and Stanyan, and made liberal use of the 38L and 38[A,B]X busses, which cut that time down a lot. I used to go to the gym every morning before work and could be door to door on the 38BX in about 25 minutes most days. If you can get a seat on the busses, I kinda enjoyed the time on my phone or with a book, but that's not always possible too.

I commute by bike now, which is certainly the fastest way to get around the city - perhaps just not the safest or least-sweaty way.

There does seem to be a "center of gravity" in SF that is incredibly bizarre to me. For a lot of people, if you live west of Divis or south of 24th st, you might as well be, for all intents and purposes, in Oakland. Seems like a lot of the complacent crowd of SF lives in the Mission/SOMA/Pacific Height/Marina/Russian Hill bubble and never gets out of it.

Yet there is a whole 3/4 of the city that isn't encompassed by that part that is really, really awesome. It may not be "trendy" or anything, but they're full of great neighborhood places and "real" people. I think it's easy for everyone analyzing this situation to forget those places exist.

It's not revolutionary, but it does a lot of things right and has more features that what you mentioned above:

* maintains a semaphore of currently processing requests, and will immediately fail a request if the semaphore is full.

* tracks service latency and other statistics for you

* maintains a circuit breaker to immediately fail requests if the breaker is open (based on statistics)

* watches for health to be restored of the 3rd party service and allows requests to it again.

* built in request isolation via threadpools

* built in request collapsing and caching

Overall I like this cause it's a great demonstration of how to think about, and engineer for, failure in a distributed system.

> you're just given some basic data structures that handle pretty much everything under the sun, and you go from there

This only works because the size of your n is small, possibly a few hundred, so it doesn't matter. When you start dealing with millions or billions of records this stuff matters. Quite a lot.

So really, it's not the language, it's the size of your data - or the size of n that matters.

I'm not sure if you can count this as a startup, but I bought my King-sized Sleep Innovations (memory foam) mattress with Amazon Prime for $530. And if I wanted the less-think 10" one, it would be $400.

http://www.amazon.com/Sleep-Innovations-12-inch-Memory-Mattr...

The damn thing was ~100lbs in a giant box, and I got it shipped to me free. It's super comfortable and well worth the money - remember you sleep for like 25% of your life.

Mattresses seem like less of a specialty-item than eye glasses, so I wonder if big online retailers like Amazon can just cut out the middle man and service 80-90% of customers?

Django still powers a lot of the DISQUS infrastructure. That said, we're in the process of breaking certain parts of it off into independent services and out of Django. But Django will be the main component for the foreseeable future.

While I don't disagree that Apple's technical skill isn't best of breed, where Apple has shined - both now and in the past - is their ruthless determination and focus on UX and HCI. Apple goes further than any company on the planet to make technology devices (computers, laptops, music players, tablets, etc) that delight their users and just work and make them happier and more productive.

Apple doesn't have "cheesy blah inside" stickers because that doesn't delight users and make the product better to use. They're stupid. Apple also doesn't chase fads because fads are just that, "a fad" and rarely do fads have long lasting staying power like a good product should.

I'm a huge fan of office "suites" - smallish offices which comfortably hold 3-6 people. It's a good sweet spot between open spaces full of dozens of people and individual offices. Usually suite-mates are people on the same team/projet as you, so you still get that open collaboration, but aren't distracted (or distract) the other people in the office whose daily work is unrelated to yours.

> NodeJS is suitable for apps that do plenty of short lived requests.

I'm confused. Isn't Node's event-loop style programming ideal for long-lived requests? I.e. ones that, under a synchronous i/o, block other requests?

Hulu, Living Social, All of 37 Signals, Groupon, AirBnb, Scribd, Zendesk, Soundcloud, etc.

Twitter's scale is unlike nearly every single web app online, so I think the real story with Rails and Twitter isn't that "they had to move away from it for scalability reasons," but rather it's amazing that they were table to leverage Rails for as long as they did."

Also, Twitter is more dropping Ruby all together rather than just Rails specifically. Again, this isn't to say that Ruby isn't a great language that works for most people (it let Twitter grow quickly to where they are today), but at their scale with their demands it doesn't work well.

Oh weird. Less than 24 hours after I first saw it from Twitter, the account was no longer active. Nice to see it's back!

There was a Soundcloud account with a like every essential mix ever on it the other week, but shortly after it disappeared - I think due to licensing issues with BBC content outside of the UK. Did you look into that for your player?

I wrote a similar blog post back at my old job about our usage of Riak.

http://devblog.seomoz.org/2011/10/using-riak-for-ranking-col...

Keep in mind we used Riak pre-1.0, so YMMV.

> Isn't this just a method of implementation for your option 3? I don't really see a substantial difference between mocking the server and mocking the API.

The short answer is that there is no difference. Just as you could mock out a call to S3API.get(object_id) and have it returns my_object, you could write a server that responds to the S3 API call for getting object_id.

The long answer is that using mocks is a lot quicker to develop, easier, more straight forward and has faster run time than maintaining a real runnable copy of S3 that behaves the exact same as the real S3. With the fake S3 you're still spending CPU cycles inside your S3 client while it talks HTTP with your fake S3, which slows unit tests down a lot. Plus fake S3 may have slightly different behavior when your S3API library interacts with it, which could lead to really hard to track down bugs later on. Trusting your APIs is what unit testing is all about.

Awesome, thanks for the extra info. I think your setup sounds really good :)

In my opinion, having to replicate S3 in development and test isn't the best idea. There are a few problems I see: You have tied yourself to S3's API, you must maintain this "other" S3 by making sure it behaves like the real S3 and your test and development code never actually hits the real API you're using...until staging or production.

There are a few better strategies I can see here:

1. For test, use something like VCR[1] to record real HTTP interactions with the real S3 API during first test runs, serialize them to disk, and then replay them later.

2. Go the more OO route and create an internal business object with a defined interface that handles persistance of your objects. You could have a S3Persister for production and staging, but then you can create a LocalDiskPersister or even MemoryPersister for tests. Hell, you can even keep your own S3 and create OurS3Persister as well. The main point here is that your application code is coded to one API/interface - the "persister" - and you can easily swap in different persisters for different reasons. All the individual persisters can then have their own tests that guarantee they adhere to to Persister interface and do their own individual things correctly.

3. Mock out the calls to your S3 library. It's the job of the library to provide an API interface for you as the application developer to S3, so you can mock out those API calls and trust the library works and is doing the right thing. Since you're mocking things out, you should still have integration tests with the real S3 to verify everything is working, but for quick unit tests mocking works great.

The blog post mentioned they had GB of data, so YMMV on these ideas, but these are strategies I and others have used in the past when dealing with APIs like S3 and they work great.

[1] https://github.com/myronmarston/vcr

I think this article title is misleading. It's not 10 million hits a day on Wordpress, it's 10 million hits a day using Varnish to a single URL. Might as well say you can serve 10 million hits a day to a static file served out of memory.

If you know what these commands are doing under the covers, these commands are a true time saver. If you don't know what these commands are doing, they're basically "magic" and in the hands of inexperienced developers could be harmful to their education on Git.

> Too bad every generation has to rediscover all this stuff.

Even though this technique existed 30 years ago, doesn't mean that it's been recently "rediscovered" by the Ruby folks and excitedly implemented to the musings of, "how did we miss this technique?!?" There are likely lots of reasons why this wasn't used before. The abstract of their paper seems to shed some light on this:

[...]But Ruby interpreter runs on various platforms, and we do not have portable memory allocation API to obtain aligned memory region without wasting region. In this paper, we propose portable scheme to map from object pointers to corresponding bitmap table in constant time.[...]

I may be totally wrong, and the paper is entirely in Japanese, but I think there is more to it.

I've was bitten by this many times until I changed my global push setting to upstream, and I consider myself an advanced git user. One time I did a vanilla git push -f, wanting to add my amended commit to the remote branch. I accidentally force pushed our master, staging and production branches to whatever my local versions of them were, sometimes weeks out of date.

> But where's the full-stack test harness for a Rails app with heavy JS usage, that runs at a reasonable speed, gives us useful traces on the client and server, and doesn't break when I slightly rework my HTML?

If you're talking Python, Lettuce does this is Python, there also is Splinter. You can run selenium with a headless firefox quite easily with Xvfb.

If you're talking rails, Capybara has drivers for Webkit (QI), selenium and even PhantomJS. Their speed is variable, but it's not too bad.

On the "HTML changes break my tests," I'd argue that A) you should write your front end tests not tied to the HTML if at all possible. Refer to elements by semantic names, not XPath or CSS selectors. And be smart about your steps. Make your submit_form() method actually look for a submit button in more than one way. B) you shouldn't have more than a handful of full stack frond end tests. They're hard to write, slow, and don't give much help telling you WHAT went wrong. Most of the time the ROI is only there for verifying a few user paths, but not much more than that.

> I honestly don't see the meaningful difference between contacting Github and leaving a silly commit, except that the former would probably get the bug fixed quietly; in contrast, now everybody is aware that the bug existed in Github and is aware of the potential for it to exist everywhere. He successfully proved his point, which apparently was a pretty good point. Isn't that a better outcome?

I'm not denying that by doing what he did it certainly got the word out and made everyone understand how serious of a problem this is. It was a very good point and I think the outcome is the right one. I'm just saying that Github's actions - to suspend the user who somehow got SSH rights to the rails org - is the right thing to do. They want to minimize his damage that he will do, and until they can do a full audit and understand how his commit got there, it's the right thing o do.

> Making a silly commit did not make anyone's data more or less vulnerable, so I don't believe that "taking this shit seriously" implies flipping out over it.

I fail to see how suspending a user is "flipping out" over it? I don't think you can color unauthorized commits to github repos with different levels of responses from Github. That's a dangerous line to walk IMO.

I would disagree with this, quite a lot. He brought up an issue with the Rails team, they pointed him at the canonical, "here is where we talked about this before, sorry." Still not satisfied, he found the same exploit in Github to prove a point. Rather than do the sensible thing by creating a dummy account and contacting Github showing how he messed things up, he barged into the Rails organization and left a silly commit. Github is first and foremost a business organization where lots of companies pay them lots of money to "take this shit seriously" and protect their data, so they did the right thing by shutting him down.

> Instead of being able to use javascript libraries with one another natively, everything goes through Ruby.

You can still do that, you can turn the asset pipeline off and just throw everything in /public like normal if you want.

Re: "through Ruby." If you meant that the body of the CSS file is generated for a request by running through the ruby interpreter, that's not true.

They're not dictating how you use JS, best practices, or anything like that, it's simply a pipeline for compressing and delivering assets. You also don't have to use it if you don't want to. There is a "config.assets.enabled" option in application.rb that you can set to "false" to turn it off.

Bottom line is Rails is, and always has been, and opinionated framework, which you choose to buy in to. And even in recent years they've been better about letting you opt out of certain parts of it.