News for hackers. Stuff that matters.

gsavit

Re:Mobile App Rollbacks on Thursday November 16, 2023 @11:46AM (#38291833)
My team and I built this! We're mostly former mobile engineers and we’ve felt the pain of app disaster scenarios in prod firsthand. No matter how extensive our test coverage, no matter how thorough our QA, no matter how much we put behind feature flags… our teams have shipped irreversibly-bad updates that break things and upset users.

And mobile hotfixes are slow. Your team has to triage the issue and assign it to the right team members, implement fixes (or carefully revert the offending changes), compile one or more RCs, upload to the stores, submit for review, then cross your fingers and pray to the store review gods for a speedy review. All of this takes time — hours in the best of cases, but potentially as long as days. And the negative impacts of even just a few hours with a critically broken app can be eye-watering: tens or hundreds of thousands of dollars in lost revenue, a flood of negative user reviews, and overwhelmed CX or ops folks.

So we decided to build a safety net.

Critical issue in prod? Roll back instantly. In the background of every normal release cycle, Runway automatically re-signs your last live production build and preps it for a possible rollback release — including preemptive submission for review (iOS only) — to remove all possible sources of delay. Should something go wrong in prod, there’s always a corresponding rollback build ready and waiting for single-click release in Runway.

Here’s how the rollback flow works in a bit more detail:

1). Your team is busy getting your next regularly-scheduled release out the door. Let’s call it version 1.2.0. 2). In the background, Runway takes your latest live build in production — say, 1.1.0 — and re-signs it, bumping the version to 1.2.1 and incrementing the build number as necessary. 3). Your team releases 1.2.0 as scheduled. Nice! 4). The 1.2.1 rollback release is surfaced in Runway, just in case. For iOS, we also preemptively create a new version for the possible rollback in App Store Connect and submit the rollback build for review. 5). Your team is monitoring the rollout of 1.2.0 and identifies a critical issue You halt the rollout. 6). Right next to the button you used to halt the rollout, there’s another button to release the rollback build that’s ready and waiting. Click that, and you’re all set.

With rollback builds primed and ready to go, your team can address critical bugs in prod with a single click — cutting out all the lead time needed to investigate and implement a fix, get builds compiled and uploaded, and make it through the store review process.

Would love to hear people's thoughts!

Mobile App Rollbacks

https://www.runway.team/blog/introducing-rollbacks-by-runway

My team and I are mostly former mobile engineers and we’ve felt the pain of app disaster scenarios in prod firsthand. No matter how extensive our test coverage, no matter how thorough our QA, no matter how much we put behind feature flags… our teams have shipped irreversibly-bad updates that break things and upset users.And mobile hotfixes are slow. Your team has to triage the issue and assign...

( Read More... | 1 comment | 3 points | on HN )
Or Runway will take care of the piping! Integrations with PagerDuty et al are certainly on the roadmap :)
Post-release monitoring and metrics is definitely an area of big interest for us as well! We’re excited to flesh out that part of the platform. Also the idea of release-to-release intelligence, so your team can see how things track over time and make sure your release practice and performance trend in a good direction.

Do reach out so we can get you set up with access :)

We're happy to hear this sounds appealing even in a non-mobile context! While some of the friction and pain points that Runway aims to solve are especially acute for mobile, it's true that releases are often a problem area for web as well, especially in team settings. We definitely have the idea of applying Runway beyond mobile in the back of our mind!
I think it's a good point. Down the road it could be smart to explore an expanded idea of release defaults and the ability to assume away new questions about edge cases that show up in the flow. For now, we're committed to having Runway perform reliably and safely - which to start with means being explicit about options/changes that occur on a third party provider's side.

In general, my opinion (or Runway's!) isn't as important as factors on the customer side - types of apps, industries/verticals, team makeup - which are likely to make a difference in which checkboxes and options are considered important or not. It probably makes sense to include those factors when thinking about how we can further streamline everyone's release process in the future.

Runway doesn't actually rely on fastlane, we're interacting directly with the App Store Connect API. So, to the extent Apple avoids breaking things on their own API, Runway should remain operational!

In terms of config and checkboxes through the submission process, Runway allows you to set defaults so you should be able to set and forget. Of course, Apple does sometimes introduce new requirements and new checkboxes - we plan to help surface those changes to teams.

Some aspects of mobile (e.g. deploying binary, dealing with app stores) make it a more gated and complicated process. That, coupled with our team's domain experience in mobile, led us to focus there to start with. But you're totally right - there's a lot of overlap with any kind of deployment and we have an eye towards a future where Runway is the single place where teams ship everything they need to ship. Plus, there's extra value to unlock in coordinating releases across platforms - e.g. coordinating backend changes with app releases, or sync'ing product launches across web and mobile.
It's definitely a problem area that's currently tricky to tackle properly for all but the largest of teams/orgs. Building in-house is often a luxury, and one which requires a big investment not just up front but into the future with upkeep and support. Of course it's nice to have a custom solution, but we've been working hard to make Runway flexible and able to adapt to every team's specific workflows and processes.

PS - thanks for the kind words - we're big fans of the book! Maybe you can squeeze us into a second edition? :)

Interesting question :) Our "full autopilot" mode isn't yet launched but once it is, if you have a recurring cadence set up with scheduled kickoff and submit days, and if each release's interim steps/gates are green, you'd theoretically be able to submit 100 out of 100 with no manual intervention. The real-world limiting factor is that in App Store Connect only one submission can be in flight at a time, and they'd usually be in flight for longer than a day.
We were super excited by the Xcode Cloud announcement! It’s a neat evolution of the Apple dev ecosystem and could be especially helpful for smaller teams wanting to more easily and quickly level-up their Apple DevOps.

We more or less see it as a substitute for any of the other cloud CI/CD providers teams are already using and integrating with Runway - on the Apple side only, of course. So, a complementary product that Runway will certainly integrate with! We’re especially excited by the prospect of it helping less mature teams get an out-of-the-box build pipeline that better positions them to take advantage of Runway.

Based on what we've heard from agencies, it seems it could be driven from either side: sometimes the agency handles the tooling and that's actually some of their value-add (having everything in place already), sometimes it's all BYO tools on the part of the company/client. Anecdotally, the former seems more common, like you say.

We hear you on the math! Our pricing is a work in progress and there's more we'll do on the lower end of the scale for sure. We're hesitant to cut/bundle the product too much but it's a likely option.

If you and your team are interested in trying us out, let's definitely chat though!

You're definitely right, in its current form Runway is aimed most squarely at mid to large sized teams. But, we do think there's still value for smaller teams - and in fact, a couple of our early customers are on the smaller side. Although some of the coordination and process overhead isn't there, there's still value in creating a source of truth and reducing context-switching between tools during releases. And, it's really valuable to get a framework like Runway in place earlier on, to better position a team for growth! We envision helping small teams out early on in a lighter-touch way and then helping them mature.

At the moment our main tier is $400/mo, but we're exploring other options for those teams that are smaller and/or at earlier stages.

Re: external clients - This is a really interesting use case that we've chatted to some agencies about. The way we've been thinking about users and roles, it would make a lot of sense to give scoped access to clients and allow them to participate in specific areas as needed!

We started with some napkin math, graduated to a complicated spreadsheet, and then tried to distill that into this rough calculator: https://runway.team/release-cost-calculator. General idea was to capture the cost of wasted time spent coordinating and herding cats instead of doing real work! We think there are even more hidden costs not captured here though - e.g. upkeep for in-house tooling or effort spent onboarding new team members.

Launch HN: Runway (YC W21) – Easier iOS and Android app releases for teams

Hi HN! I’m Gabriel, one of the co-founders of Runway (https://runway.team). We’re building a SaaS platform that makes it easier to coordinate your team’s mobile app releases. Runway is an integration layer that sits on top of mobile app teams’ existing project management and development tools, making it possible to understand at-a-glance the status and progress of release cycles and to...

( Read More... | 30 comments | 93 points | on HN )
Right now the focus is on public/prod deployment. Beta Testing is a step within Runway, but it's not so fleshed out and no integrations are pulled in there yet. It's definitely on our roadmap though - we want to help teams codify and streamline their internal distribution!
Part of what we’re trying to do with Runway is to consolidate all the various sources of information about your release.

We do currently notify your team in Slack of any changes to your review status in App Store Connect. Providing further detail about your rejection reason is 100% something that’s on our roadmap for the “App Review” step, along with making it clear who owns the task of responding to rejections if needed.

Yep, we realized early on that this is a challenge. Does Runway try to be more of a plug-and-play solution for teams that need to get a basic release pipeline spun up quickly, or does it focus more on flexibility for more established teams with stronger opinions and preferences on the individual parts of their toolchain configuration?

So far we've tried to hit a balance between the two, informed as much as possible by our pilot teams and our gauge on what will most effectively serve the most teams to start. Will definitely be an ongoing conversation.

The short answer is that we're working hard to make Runway as adaptable as possible to the varied needs that we've heard about from our partner teams.

But in general - it is tricky, and I think the perfect balance changes depending on what type of a team we're talking about. We've talked to small teams, larger ones with more complex structures (like multiple product teams all contributing code to a single binary), but also teams who are fairly agnostic about their branching and tooling setup, and those who are more opinionated.

There is certainly a version of Runway that could be much more prescriptive (and limited) for teams who prefer more guidance in setting up a release toolchain. For now, we're trying to hit a sweet spot of enough flexibility to meet the needs of most teams, while encouraging best practice in a few areas that we think will streamline everyone's process.

It's an ongoing conversation and it's really helpful to hear from individual teams on how Runway could help in their specific situation.

We're aiming to offer release-based metrics that are hard to find elsewhere. Things like the time it took for a given release cycle, frequency, finding and displaying common release bottlenecks, and volume of blockers during testing and over time are a couple examples of the types of data Runway could be uniquely positioned to provide.
We've heard from teams working on non-mobile platforms that they share a lot of these pain points.

There's also the reality that coordinating app releases often includes wrangling dependencies on backend services.

All of the above is definitely on our radar… stay tuned!

Runway definitely works without CI integrated! But, lack of build data does limit some of the cool stuff Runway can otherwise do - e.g. automatically tagging releases, and soon easier/auto build selection in ASC/Play. The CI piece definitely makes a difference, so we're exploring the possibility of offering simple out-of-the-box to teams without!
Thanks for the kind words!

We do support Bitrise! And CircleCI, GitHub Actions, GitLab CI, and Jenkins. We're adding more integrations regularly, and we prioritize as we hear from teams with specific needs.

Hey HN, I’m Gabriel from the Runway team. We’re building a SaaS platform that makes it easier to coordinate your team’s iOS and Android releases. Hook up your existing toolchain (e.g. Jira, GitHub, Bitrise, App Store Connect, Slack), and we’ll create a single source of truth for your mobile app release: providing a live timeline of the progress of the release, handling communication of status and blockers, and automating manual steps along the way.

Our team met back in 2013 building Rent the Runway’s iOS app together, and we’ve spent the years since then working on mobile app teams of all sizes. Everywhere we went, getting iOS or Android releases out the door was always an "event". Someone would be stuck spending hours with multiple Chrome tabs open checking on the status of different tools, killing time while waiting for builds to upload, Slacking the owners of various tasks, and referring back to a 40-line Google spreadsheet with the team’s specific release process. With all the context-switching and mental overhead, it made it hard to get anything else done.

While some build-centric tasks can be automated (e.g. using fastlane or scripts), we see that a lot of the overhead of releases is actually very people-centric: keeping your PM up to date on progress, looping in marketing for release notes, or syncing with QA on the status of regression testing. We also noticed that, even with a solid CI pipeline in place, there are still lots of manual tasks along the way - build selection, branching and tagging, compiling changelogs, etc.

Runway connects all those dots — it’s the tool we wish we had on our old app teams. It pulls in all Jira tickets and code relevant to the release, side-by-side, to surface and resolve any out-of-sync tickets or code. You can set up custom, interactive checklists with item-specific owners to replace the monster Google spreadsheet, and our Slack integration will ping the appropriate people or notify everyone when important milestones happen. Design/marketing can enter ‘What’s New’ release notes directly in Runway for all localizations (with a handy list of new features in the release to reference) without you having to hunt them down. Plus, Runway helps teams maintain good workflow hygiene by automatically tagging releases in GitHub and applying missing labels to Jira tickets. We’re working on further automation to make releases as hands-off as possible: from kickoff, to submit, to release.

We’re excited to talk to more teams and the HN community at large about their unique release process challenges (and general thoughts as well!)

Re:How to Ship a Mobile App on Tuesday August 18, 2020 @12:44PM (#24200914)
My team and I have been doing some thinking on the mobile app release process. It's such a necessary part of the overall mobile development lifecycle, but it often seems overlooked: having been through hundreds of mobile releases ourselves, we realized we had never paused to think about what the end-to-end process actually entails.

So, we tried to capture it all in writing. It's a high-level piece aimed at the entire cross-disciplinary mobile team (not just engineers!) and other stakeholders.

We'd love to hear your thoughts! Does anything stand out to you? Have we forgotten or overemphasized any steps in the process? How does your team ship apps?