News for hackers. Stuff that matters.
Re:Join the U.S. Digital Service by wpietri on Monday February 02, 2015 @11:43PM (#8988890)
I have met some of the 18F folks here in San Francisco, and I have gradually become convinced that there is a lot of substance to this effort. People, apparently including people at the highest levels, recognize that information technology has made enormous leaps in the private sector, and they are serious about helping government catch up.

One of the most interesting bits to me was something said by Todd Park, a former CTO of the US and still a White House advisor. I was, frankly, suspicious that Washington's culture could really accept a lot of the Agile and Lean notions that are commonplace in the high-tech world. But he pointed out, correctly, that waterfall projects are enormously risky. A big reason Lean Startup advocates place such a strong emphasis on frequent small releases is that it helps us reduce risk by finding and fixing problems early. He pointed out that if bureaucrats really want to play it safe, using agile, iterative approaches is exactly what they'll have to do.

I don't know enough about Washington to say if this really will work, but improving government efficiency through smart use of tech strikes me as exactly the kind of thing people from all parties can get behind. I'm excited to see how it turns out.

Re:Join the U.S. Digital Service by trhway on Tuesday February 03, 2015 @04:24AM (#8989497)
>suspicious that Washington's culture could really accept a lot of the Agile and Lean notions that are commonplace in the high-tech world.

given my experience with thorough, to the letter, under supervision of no-expenses-spared-top-notch experts implementation of Agile/Lean/Scrum - 2 years non-stop across pretty much the whole company, only very-super-important-critical-projects got exception - the government would love it - a lot of planing, reporting, meetings and other crap boggling down everything with actual product development slowing down to the crawl as a result.

Btw, the original healthcare.gov - the one that failed and required that emergency super-team fixing - was developed using Scrum, and as result it had all the characteristics of a Scrum project, chief among them - bad, to the point of straight failure, integration between components.

Re:Join the U.S. Digital Service by bkeroack on Tuesday February 03, 2015 @09:04AM (#8990322)
I agree and share your cynical view of Agile/Scrum, at least how it's often used in top-down organizations.

In my experience there are two factors required for a successful project:

a) Developer autonomy sufficient to modify core design/implementation if necessary--and developers experienced and talented enough to recognize when this is necessary.

b) Developer ownership of the project. By that I mean minimal power difference between "stakeholders"/"business owners" and the developers actually building the product. You don't want developers to feel like consultants working on someone else's project because they won't care when bad decisions are made and they will naturally do the least amount of work possible to satisfy each Scrum ticket.

Re:Join the U.S. Digital Service by wpietri on Tuesday February 03, 2015 @12:04PM (#8991510)
I share your suspicion of large-company Agile: by and large I think it's at best Mini-Waterfall, and at worst it's the same old bullshit with some new jargon on top.

However, the large amount of bathwater doesn't mean there isn't a baby there. And the places I see doing best at making the necessary culture shift are the ones with a sense of mission. So I have hope.

And honestly, even if this merely means a transition away from pure Waterfall to Mini-Waterfall, I think that will be a step forward. In the commercial world, the rise of the Internet has made 3-year release cycles untenable for most software; people now just scoff. So if we can merely get governments to emulate business in the shift from titanic failures to medium-sized failures, as a citizen I think that's a win. But still, from what I've seen, I'm hoping for more here.

Re:Join the U.S. Digital Service by trhway on Tuesday February 03, 2015 @03:17PM (#8993024)
>However, the large amount of bathwater doesn't mean there isn't a baby there.

Applying generic systems analysis (where it doesn't matter whether it is protein building, network protocol flow or software engineering process) one can easily see that Scrum is a highly synchronous low-latency process (that is the "baby" that management loves) which comes at the cost of dramatically decreased bandwidth. In addition, changing a team's process inside, Scrum indirectly completely changes how the team interacts with up- and downstream teams/components - in particular on practice eliminating any chance for early integration and for any iterations of the integrations.

That became obvious to me during the first day at the first Scrum course, especially after i tried to ask those questions to the teaching Scrum expert from the consulting company and received blank face with some blabbing in response. Some time later, different company with different consultants - that first hand 2 year experience i described in the previous post - the Scrum drawbacks on practice happened to be even worse than i thought they may in theory. We actually have great time right now at our BigCo. as after the Scrum fiasco, and quietly dropping the Scrum company-wide as the result, the management still seems to be kind of "in shock", and they haven't brought up any new fad in the process improvement so far, so we're just doing typical old-fashioned per-feature waterfall, and it just works as usual - which is huge achievement compare to Scrum :)

Re:Join the U.S. Digital Service by wpietri on Tuesday February 03, 2015 @08:30PM (#8994792)
I can't really answer for Scrum, especially as taught by consulting companies. I think of Scrum as a set of too-basic but reasonably good ideas that quickly ended up mostly as a certification scam. I wrote more about that a number of places, including here: http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how...

I think that's especially true for dramatic company-wide, top-down Agile adoptions, which I have never heard of working. I think people doing that cannot possibly understand Agile philosophy, which promotes local control and continuous improvement of top-down control and giant leaps.

My positive experience is mostly with smaller shops, but I've definitely seen many places where companies have grown up releasing early and often and have no problem working in what I'd consider good Agile style. Wealthfront, Etsy, and Spotify are all explicitly and openly like that. But I think there are plenty of other places that aren't explicitly Agile but ended up that way as a natural evolution.

For example, when I visited YouTube to study their process, there were a great number of developers, designers, and product managers all working in relatively independent small teams. They'd release at least weekly but sometimes more often. There was no overarching plan; each team had particular goals to pursue, and coordinated with other teams as needed. They never used Agile jargon and didn't think of themselves as Agile, but I think they were acting in ways that exactly matched the spirit.

Re:Join the U.S. Digital Service by javaun on Tuesday February 03, 2015 @10:53AM (#8991012)
I spent 7 years in Washington and can attest the change has been incredible. Civic-minded hackers, non-profits, and startups went from being outsiders to permeating agencies. Much was organic, and some was top-down mandated. Obama saw what happened in 2 elections and these new agencies are peppered with the people that used agile tech to great advantage on the campaign. Agencies like HHS and FCC were pioneers in open data -- not just for government, but for anyone. Look at cfpb.gov - an example of what an agency can look like when it starts from scratch. 18F and USDS are now tasked with bringing that innovation to the rest of government. If you can get to DC, consider attending the Sunlight Foundation's transparencycamp.org this May. It's the biggest event of it's kind, bringing 600 folks from across the globe.

EDIT: changed this sentence: "Much was organic and some top-down mandated" to reflect what I think was mostly due to popular upswell, though there were key executive decisions. The O'Reilly network has a lot of influential folks in DC too.

Re:Join the U.S. Digital Service by wallawe on Tuesday February 03, 2015 @12:52AM (#8989071)
That's just the thing - I don't think 'playing it safe' is the goal in Washington, or at least not the only one. As is, the goal for contractors is to stay on a project as long as possible, earning more money over a longer period of time. Waterfall is a perfect excuse to do just that.
Re:Join the U.S. Digital Service by wpietri on Tuesday February 03, 2015 @12:10PM (#8991560)
I agree that's an issue, but I was thinking more of the decisions of the people who hire the contractors. The right Agile approach to a project is to structure it as a series of small wins and losses, rather than rolling the bones on one big win/loss. It's my hope that calculus will appeal to career-minded officials, especially since the Agile approach lets you demonstrate value earlier (thus reducing drama), lets you bet more on your wins, and dramatically reduces the chances of large failures.
Re:Join the U.S. Digital Service by panzagl on Tuesday February 03, 2015 @10:40AM (#8990922)
'Playing it safe' is also the goal of the gov't civilian managers of those projects- they have a promotion schedule to hit and having a 'failure' on their resume jeopardizes that. And since they're only going to be there 2-3 years, the goal is just to keep things propped up until they rotate out.
Re:Join the U.S. Digital Service by vpeters25 on Tuesday February 03, 2015 @08:48AM (#8990244)
I had managers yell at me when at the sprint review meeting we realize we were going the wrong way and decide to scrap some of the work of the last few weeks...

Imagine politicians with a stake in the project's failure hearing that: they will go full Benghazi on it.

Re:Join the U.S. Digital Service by jkestner on Tuesday February 03, 2015 @10:12AM (#8990732)
Just sprinkle some remote coders among their states, and they'll double down. Works for Lockheed Martin.
Re:Join the U.S. Digital Service by 14113 on Tuesday February 03, 2015 @09:19AM (#8990392)
Better a few weeks than a couple of years and millions of funding.
Re:Join the U.S. Digital Service by RHSeeger on Tuesday February 03, 2015 @09:54AM (#8990590)
Not necessarily in the eyes of certain people, and it happens in the corporate world, too. If you're only in charge of the project for a period of time (no longer in that political role, promoted, whatever), then "it doesn't need to succeed, it just can't fail" is a term used widely.

You don't want to be seen having failures in your project (even small, agile failures that are for the good)... but you don't care if the project succeeds.

If you can plan out a waterfall project that won't complete until you're "out of office", then there is no negative consequences for you... even if you KNOW it will fail once you're gone, that's the next guy's problem.

Sad, but true.

Re:Join the U.S. Digital Service by wpietri on Tuesday February 03, 2015 @12:40PM (#8991790)
This certainly true, but it's my hope the culture will shift to make the waterfall option impossible.

Back in 2004 or so, I heard of a major company that only updated their website once a year. Even then that was considered ridiculous. Consumer expectations have shifted massively; plenty of places now release every week or two.

I have hope that the government culture could shift similarly, so that software projects that don't deliver anything for long periods are seen as troubled no matter how many Powerpoint decks and GANTT charts they produce in the meantime.

And I think the way to change that culture is by having enough projects that deliver early and often, so that people say, "Well, those guys are shipping monthly. What's wrong with your team?" So instead of keeping people from being blame-y, we get them to be blame-y about long project cycles.

I think that's definitely happened a fair bit in industry, so there's some chance it will happen in government, too.

Re:Join the U.S. Digital Service by Loughla on Tuesday February 03, 2015 @10:10AM (#8990710)
Not in an election year.

If it's either a failed public project, or spending millions of dollars (that can be spun as creating jobs), what do you think they're going to take?

Re:Join the U.S. Digital Service by DeadeyeDuck on Wednesday February 04, 2015 @03:09PM (#8999375)
Ala "improving gov efficiency through smart use of tech"...I've worked in mil/Fed IT/telecom/technology-development for nearly 30 years. The problem isn't a lack of desire by the metal-benders trying to employ better/faster/cheaper tech, the problem is getting the Jabba-the-Hut career-protected SESs and their minions out of the way, along with instituting Fed procurement and contracting processes that actually facilitate a delivery timeline that at least keeps up with Moore's Law.

The challenge that USDS will need to overcome is one of culture and regulation bureaucracy, not one of throwing a unicorn cavalry at various IT dragons. You go pure "change from within", you'll fail. You go pure newbie/private sector, you'll fail. You need to find fresh and innovative eyes and pair them up with existing Fed ITers that haven't yet been jaded into submission.

Like you, I'm excited to see how it turns out. They need to succeed. If they fail, it will be a long-term failure, and that will do no one any good.

Re:Join the U.S. Digital Service by benmathes on Tuesday February 03, 2015 @12:43AM (#8989045)
It should come as no surprise that Mr. Park is on board; He built two public healthcare IT companies (ATHN, CSLT). He can definitely recruit good tech talent.

The question is: Can he convince others in government that, as another post on the HN front page says right now, we should replace middle management with APIs?

Re:Join the U.S. Digital Service by adelevie on Tuesday February 03, 2015 @09:51AM (#8990565)
That's a great question. At 18F, we <3 APIs[0][1][2][3][4][5]. Not only are we big believers ourselves, but we strive to help other agencies improve their API practices.

[0] https://github.com/18F/api-standards

[1] https://github.com/18F/jekyll_pages_api

[2] https://github.com/18F/API-All-the-X and http://18f.github.io/API-All-the-X/

[3] https://github.com/18F/api.data.gov

[4] https://github.com/18F/fbopen

[5] https://github.com/18f/?query=api etc, etc

Re:Join the U.S. Digital Service by nl on Tuesday February 03, 2015 @01:03AM (#8989100)
Given that article was completely unpersuasive to me (as a fairly long-term HN reader) I seriously doubt it[1].

But then I don't think that should be a goal. (Good) middle management is precisely what Washington needs more of.

[1] The author would be well advised reading Peopleware, and understanding that middle management is often (usually?) an exception handling layer that handles the boundary between systems.