* realistic test input: either you carefully scrub actual production traffic and replay it (these require a ton of setup that doesn't age well), or you guess what you think users are doing and create dummy requests (and miss anything wacky you didn't know users do)
* load & perf: it's rare for a team to spin up a prod-sized cluster just for a test, and before AWS it was almost unheard of to go out and buy twice the hardware you need, even when you were lucky enough not to be deploying on your customer's boxes. A change that tanks your cache hit rate can ruin your whole day yet look fine at trivial sizes.
* visual design: Microsoft famously got bitten by moving away from manual testing that would catch dialogs made unusable by layout changes
We usually had best-effort integration tests, but they never caught everything, and we always had to make tactical decisions about when to release what based on risk.
* UX review and load & performance testing are usually things that we engage in regularly on trunk or the developer/code reviewer flags up ("this change has load implications/this change requires a UX review").
* "Microsoft famously got bitten by moving away from manual testing" - I'd never advocate eliminating manual testing entirely, just that it should be A) never repetitive, B) not a gatekeeper, C) always exploratory.
>We usually had best-effort integration tests, but they never caught everything,
Nothing ever catches everything but IMHO continuous delivery, short-lived branches, good monitoring and regular exploratory testing, load/perf testing on trunk is the best way to catch as much as possible.
I've started experimenting with integration tests that generate reports with screenshots and story steps, as well, so that there's a tighter UX feedback.
In our case it was a valid concern (our team had a pretty awful track record).
The upgrade paranoia only kicked in when they were subjected to broken releases and had to roll back.
I'd quite like to load the LTS version of GMail and Facebook. I'd also like to have some say in how these apps I use every day work. Unfortunately, I have no say. I'm not even the customer.
My background in Smalltalk made me accustomed to a style where everyone would continuously merge everyone else's changes as they worked. This style makes you aware of what your team mates are doing. In fact, it facilitates communication pretty much when communication is most called for.
As always: context. The above approach is only going to work well, when there is convenient high bandwidth communications between teammates. (Not only for your version control, but also for communication between people.)
I've seen trunk-only developers not knowing how to merge a branch back to trunk.
Well of course. If they never had constant practice at merging, they wouldn't be very good at it. Merging isn't trivial.
I've seen trunks brought into a state of disarray and never fixed again.
Can't they roll back? This strikes me as a sign of badly designed process, or developer incompetence. This should be seen as something like breaking the build. (In a CI environment, it would be breaking the build, yes?)
Monticello for example,
Subversion having that TBT thing made it easy for people to respond 'yes we're doing trunk based development' when they're not
I've seen feature branches get stuck in review for weeks. I've seen trunk-only developers not knowing how to merge a branch back to trunk. I've seen trunks brought into a state of disarray and never fixed again. I've seen svn-only devs get completely bogged down and confused by a git wofkflow.
Trunk-only is a reality. Not a reality some of us would want to live it - but one that nevertheless exists.