Twitter was previously doing A/B tests. Spaces, communities, downvotes, circles, and even moderation were applied only to a subset of accounts while they actively experimented. The new policy is an objectively less scientific approach to testing functionality than what was occurring before.

It's not like A/B tests approach anything like the rigor that we expect from actual science, though.

> actual science

I'm not sure what you mean by this. Many quality studies are A/B tests. A/B just refers to the two IV states you're testing, which you're then observing a DV - sales, engagement, errors, etc.

A/B tests can be double blinded (don't tell the error monitoring people which results are from a trial), and have high number of samples, far beyond even most pharmaceutical trials.

They can also be really crappy, changing too many variables at once, etc. But they are certainly "real science".

Advertisers and marketers use the equations but measure contemporary trends.

Science is more “what’s true if humans didn’t exist.”

Marketing is more “what widget generates more revenue?”

This isn’t true. Science is ultimately the scientific method — make a hypothesis, test a change, observe the results, repeat. It’s an algorithm for learning and broadly gaining information about reality. It can equally be applied to things having to do with humans and things not having to do with humans.

Not gp, but there's a significant kink when this applies to humans; namely, that humans have the ability to reflect on publicly known outcomes, and change their behavior en-masse in light of information so gained.

I put this earler in the phrase "reflection completeness": https://sdrinf.com/reflection-completeness ie there are things which stops working when people know about it.

In particular with A/B testing, this means that the initial A/B test is intermingled from at least 3 effects: specifically it measures how the naive population's behavior changes as a function of new functionality being made available. This is heavily, heavily time-dependent; specifically there's a "novelty effect" (early data collection will not be representative to long-term usage patterns); and there's "reflection effect" (once the outcome of the test is widely known, people can change their behavior based on that). Controlling for the first is difficult, but possible; controlling for the second, beyond just "keeping everything secret", is significantly more so, as the timelines for that might be years in length.

I strongly suspect GP was pointing at this timeline factor, and specifically that market engineering, as currently, generally, widely practiced, is grounded on the immediately available signal of "does it increases sales in 2 weeks of A/B test running". Which, given novelty effects, is heavily biased towards "yes"; and these people aren't incentivized (nor have the time/energy) to measure _very_ long-term effects beyond novelty, and reflection period.

I agree that it can be a difficult thing to analyze. There's also the Hawthorne Effect at play here too. But those are just confounding variables, they do not negate the fact that A/B tests are still "real science".

An A/B test just refers to observing how a dependent variable changes when an independent variable is in two different states, State A and State B.

Drug vs placebo - is an A/B test.

Most companies (or at least the ones doing things properly) will also have a long running retro test to see if impact persists (new test group = don't use the new changes).

I feel like it's especially bad for any UI changes that have relation to long-term productivity; measuring how given change affect existing users and whether the performance will go back to previous level or get below it after few weeks or month.

Agreed. My point is I am not going to see a marketer as aiming for the same goal “as an experimental physicist.”

To borrow the Lindy effect; whether someone likes the jacket in color A or B is of such short lived value it’s a huge waste of the resources that went into the pipeline needed to come to the conclusion.

Here’s an A/B test; rethink logistics to increase customization of outputs or continue to create design jobs who define what’s trendy and acceptable?

I think we're are getting caught up on what's being tested.

In the context of what we're talking about, you can A/B test more than marketing, you're can test variables like UI/UX.

Yes clothes fall in and out of fashion, but changing the placement, color, size of the "add to cart" button isn't something that's going to be changing frequently.

Another example might be adding a "trending" tab the top navigation of a page or whether the "what's trending" vs "what you like" provide more engagement as the default page.

Youtube recently tested randomly lowering people's video resolution to see who changed it back to gauge the importance of the resolution to their customers.

I agree with your analysis of how I see things.

I disagree that I am “caught up” on anything.

I have a preference that’s been refined over time. Not a psychological error in perception.

> Youtube recently tested randomly lowering people's video resolution to see who changed it back to gauge the importance of the resolution to their customers.

I wish they start gauging how frustrating such tests are, particularly for the test group. I've been cursing at YouTube many times over the past weeks because of this very issue - and now I learn it's not even a bug, but an A/B test.

Technically it's not science because it doesn't follow the scientific method. Instead it's the close cousin, empiricism.

https://www.merriam-webster.com/dictionary/empirical

A/B testing is literally just the scientific method, A/B are the two states of the IV. A drug vs placebo experiment, is an A/B test.

For example with changing the font size of a button:

Your null hypothesis is there is no difference in the number of clicks. Your alternative hypothesis is that there is an increase in number of clicks.

Your IV is the button font size. Your DV is the number of button clicks over a set period of time.

You randomly sample 50% of the population to State A (same button size) You put the other group into State B (increased button size)

You observe the number of clicks of the button.

You analyze this data, and can determine the statistical significance between your null and alternative hypothesis.

It wasn't "scientific" to begin with but what's happening right now is pretty clearly panicked throwing stuff at the wall based on Elon's intuition and day to day demands... while that may have worked for him in the past, I don't think it's going to work as well in this domain where you're working with a complex system dependent on millions of people's behaviors and incentives.

> while that may have worked for him in the past

Did it? The nature of Tesla and his other current businesses buffer that a bit even if it has been his approach, and it seems to have gotten him thrown out as CEO at X.com twice; among the things going on Twitter seems to be Musk trying to relitigate his failure at X.com without other investors being in a position to kick him out, but he seems to be piling up existential threats without resolving them.

It does kind of seem like he's addicted to dancing on the edge of cliffs

Based on what reasoning? Data quality with respect to computer systems approaches perfection, it is far more reliable than any data you can hope to approximate in the real world. There is also plenty of "actual science" with terrible data quality, hence the replication crises spanning large sections of the scientific landscape.

That’s not at all an argument for dropping AB testing, it’s an argument for being more rigorous, especially since in principle the circumstances under which services with a large active user base can test are downright luxurious.

Sociologists will frequently not have as good access to such a large participant pool under near ideal experimental conditions with such good ways to observe behavior. And the stuff you have to keep in mind when running experiments is not terribly complex. A bit of statistics, a few things you absolutely have to get right, that’s it.

Obviously there are reasons why AB tests are often not run rigorously (statistical illiteracy and pressure to get things done quick as well as to produce tangible results as often as possible – all three of which might lead you to run underpowered experiments with too few participants and to stop testing early which will lead to too many false positives). However, stopping to do experiments (and instead just releasing new stuff and observing the reaction) isn’t really an improvement that leads to better outcomes compared to that.

If "actual science" is a real thing, then absolutely nothing humanity has ever done in the name of scientific endeavour is "actual science," given every experiment exists on a continuum of trade-offs.

A case could be made that A/B testing is insufficiently rigorous given specific goals, resources, limitations, context, etc. But that case isn't being made here.

Some pure mathematics is actually rigorous.

Hah! You're fast. I was about to comment on that: "actual math" probably exists. "actual science" probably doesn't. I'm not sure any experiment can ever be free from error, uncertainty, trade-offs. (And now I'm super curious about this...)

Thanks though. I've narrowed my original comment to more accurately represent the scope I am referring to.

to be fair whether mathematics is a science or not is debated[0], so your initial claim might very well stand true depending on which side one takes in the argument.

[0]:https://philosophy.stackexchange.com/questions/10178/is-math...

That's fair. Though that gets into semantics which I don't find interesting or fun. In this context, I think it's fair to say we're talking about "things that can be experimented on to learn more things."

Most scientists likely wish they could get the rigor of a company like Twitter's a/b testing suite.

Why wouldn't Twitter (or similarly large companies) have open-sourced their a/b testing suite? It's not like the math there is proprietary.

I mean, I'm sure the parameters to the math are proprietary. But the basic math seems simple enough.

If you're not using an off the shelf one, chances are it's tightly integrated to your application framework.

Trying to tease out the pieces that aren't coupled to Twitter's User class is probably more effort than it's worth

That makes sense. Thanks.

I guess the comment I read implied Twitter had an amazing A/B test suite, as opposed to a tightly specialized A/B test suite.

It's called DuckDuckGoose - see https://blog.twitter.com/engineering/en_us/a/2015/twitter-ex...

And this is sadly true about almost every good A/B testing system out there. They always involve tooling that has to exactly fit your stack, and it's really tough to make a general purpose product that would offer anywhere near the same value.

To some extent, you grow your company and codebase around your A/B testing system, not the other way around, because it has to slot into so many places: deployment, testing, front-end, back-end, analytics, monitoring, etc. Almost no two companies share the same stacks across all of those dimensions, and an A/B testing system that doesn't hook in tightly to every one of those systems is not complete. This is why I always get a bit scared when people think they can just use one of those "drop-in" services and be done with it: yeah, great, you can now fiddle your JavaScript from some third-party website, but you're going to have a big project ahead of you passing group assignments out to all of the different systems that will need to know about them, and almost guaranteed some part of your stack will not have a library available from the service you picked so you're going to be writing your own REST wrapper, and dammit they don't document that API very well and it seems to be responding differently since the last update, and man it'd be a lot easier if I could just pipe results straight from my own service into Big query rather than running a daily user export, and damn, their dashboard doesn't let me set exclusion criteria on my own metrics, I have to send activation events now, and etc, etc. By the end you've basically built your own A/B test system from scratch, you've just paid someone else to do the "int myGroup = Random.next()" call, which is the easiest part to build.

You are likely putting "actual science" on a pedestal here

It's about marketing, the A/B testing they do, not science.

what is not scientific about it? maybe its not good science, but it is science. really almost anything is science...

Science is just observation and experimentation.

Science doesn't dictate how you do the above. Now, someone would find it impossible to reproduce your findings, but - that would just suggest bad science

Incorrect conclusions are frequently drawn from A/B tests. To some, this makes it unscientific, to others, it just means it's bad science. I think the argument is more semantic than objective.

For example, if your metric is "time spent interacting on the platform", then a testing of a rollout of a feature ends up with longer page load times, so users spend more time there because they're waiting for pages to load would increase that metric, and management decides it's a good idea.

> Science is just observation and experimentation.

That's not enough. If you don't include some sense of both 'systematic' and 'rigorous' (and yes, these terms are slippery), you aren't doing science.

But both of those qualifiers exist on continuums and where an experiment lies on them is subject to opinion, so there's no singular threshold. It means it would depend on the person reviewing the experiment whether it's "systematic" and "rigorous" enough to (personally) be considered science. What you're describing is the quality of the science, not whether or not it is science.

Until you can objectively measure how "systematic" and "rigorous" an experiment is, your definition of science only applies to you.

"Experiment and observation" is something we know humans have been doing for literally thousands of years (and without written evidence, probably 10s-100s). At least for loose versions of "experiment" which ties back to rigor.

We haven't been doing science for very long. The primary difference is the desire and effort to add rigor and systematic thinking. The difference in efficacy is hard to understate.

> We haven't been doing science for very long.

Again, this comes back to your definition. Many would disagree in that science has existed for, at least, as long as recorded human history because an actual tenable definition of science is something along the lines of "the endeavor to build knowledge by experimenting and observing the results". The rigor of the experiment is part of the quality of the science, not whether it's science itself.

No one is arguing that rigor isn't important to good science. It is important because rigor lends to reliable and valid results. What we ultimately want is results that are reproducible and can be used to predict. If you observe bad results, it's because you did a bad experiment, thus bad science, not that you didn't do science at all.

As an analogy: if I took notes during a meeting that no one can understand or use, that doesn't mean I didn't take notes. It just means that I took bad notes.

> Many would disagree in that science has existed for, at least, as long as recorded human history

Not sensibly. Or at least, whatever the semantics, we started doing something quantifiable different recently, which has had a massive impact on our world. It is quite sensible to ask "what changed?" and try to understand it. If you want to give it a different name from "science", ok, but that's mostly likely to confuse people. If you want to claim such a shift didn't happen, you've got a hard row to hoe.

If they're done right, they are exactly as rigorous as we'd expect from science, since they are literally the same as randomized control trials. You can do even better with A/B testing because you have much tighter control over inclusion criteria, treatment compliance, and outcome analysis.

Others have addressed the ways in which A/B testing does hew quite closely to the standard of empirical observation under controlled circumstances, with which I largely agree.

Where A/B studies may go wrong in my view is a few other elements:

- A/B studies have difficulty in determining differences based on multiple interacting characteristics. In fairness, so does empirical science, and the principle of "holding all else constant" is a frequent assumption of scientific processes.

- A/B studies face an inherent self-selection / exclusion bias: the participants in this round of A/B testing are those who've not been driven off the project/product from past experiments and design changes. Given that many Web 2.0 companies eventually dance with pushing people right up to the border of tolerance, it's quite possible that A/B testing has a long-term effect of pushing those participants whose tolerance has been exceeded out of the study population entirely. I don't know how large a factor this is, though loud / rage quitters are certainly a prominent (if not necessarily large) cohort. Whether or not they're also influential, or perhaps more importantly when they become influential is another question. Again, this is a fairly common problem with any social experiment, including natural social experiments, see various forms of brain-drain and social flight.

- A/B testing tends to focus on short term changes and behaviours, which may mask longer-term outcomes. This has some overlap with the above, but also with subjects' general response to change. See the classic case of this in the Hawthorne Effect (<https://en.wikipedia.org/wiki/Hawthorne_effect>).

The upshot is that A/B testing can be valid and useful, but that experimental design, particularly in the case of social and psychological experiments, where subject feed back into the study and its methodology itself, is exceptionally thorny.

> it's quite possible that A/B testing has a long-term effect of pushing those participants whose tolerance has been exceeded out of the study population entirely.

I'd compare this to how evaporation rate increases with temperature, as more particles find themselves with enough energy to escape the liquid.

From my personal experience, even if I can tolerate a lot of UX abuse, each such "optimization" lowers my threshold of switching to a competitor. Software in general, and SaaS specifically, resists commoditization, but every now and then an actual alternative to a product/service I'm using shows up - and whether or not I switch (and when) is correlated with how much I resent the incumbent for their UX "improvements".

I'd add one bullet point to your list:

- Unlike regular scientific experimentation, A/B testing is a methodology primarily spread in business circles using regular hype channels. That is, the average practitioner is not qualified to execute it correctly, which is one of the reasons I see A/B testing more as tools to launder arbitrary decisions. Because consequences of doing it wrong are typically not immediately apparent or obvious, both companies and customers suffer (and a vast space for fraudsters is created).

I'm in a charitable mood, so I'm not passing judgement on people for not having PhD-level understanding of statistics - just pointing out that, to the degree much larger than in sciences (even soft ones, which suffer some of the same structural problems), there's little pressure to do such tests correctly (and there's lot of ways to make money or status on doing them without regards for correctness).

From what I hear, a common way of executing A/B test badly and getting bullshit results, is by terminating the test early when it shows the relevant metrics improving for the test group - vs. running it longer if no big improvements are observed (or the metrics start getting worse for the test group). This biases the experiment towards giving false positives. This problem was big enough that there was a debacle around Optimizely few years ago, whose UI was accused to promote this early termination of tests. The cynical take I'm still somewhat partial to is that it wasn't an accident (if not done on purpose, then possibly... a result of an A/B test!) - false positives make the (statistically naive) users feel they're getting more value from Optimizely than they actually are.

Three excellent points, yes.

There's a reason that the technical term for "A/B testing in SaaS products" is gaslighting.

O hai werd uv yeer!

<https://news.ycombinator.com/item?id=33784262>

A/B tests are pretty problematic for social networks, since anything effective can also influence the control group. They are excellent for single-player activities like shopping carts and signup flows.

"Network Experimentation at Scale" from Facebook describes how difficult this problem is. Most A/B test frameworks don't reach this level of sophistication. It does make some sense to just ship things if you don't have time to build out something like that. (disclosure: I worked at Twitter long ago)

They're also problematic because you'll likely optimize for whatever metric you pick with no regard to the ethics of the optimization. Sometimes a double blind test is not the right thing to do.

I've never really bought this argument. There might be unethical growth hackers in the company, but A/B tests are not the problem in that situation.

In my experience, they mostly just catch bugs. Stuff like "hmm, our much better looking signup flow underperforms... oh, the form is broken on Safari." That kind of thing.

Think of the AI optimizing for a stated goal while ignoring implied constraints (e.g. eradicating humanity to stop wars).

This can be the case for A/B testing. Sure, you can increase ad clicks by 30% ... if you trick the user into clicking it through a carefully timed layout jump.

I think GP's argumentation may go in this direction. I'd probably not say A/B testing is the problem itself, it is a tool after all, but I could imagine it's sometimes not used very well.

Another point: Spotify's core flow changes so much (feels like almost daily) that I've lost all confidence in using it.

To make it easier for everyone to understand the other posters point...

a double blind test can help you determine the most effective way to cause pain to a monkey, but it will never answer the question of whether you should be doing so.

I feel that we can hardly call that random changes "policy"...

You are suggesting that A/B testing is going away rather than this being simply an addition to widespread testing.

They've probably fired so many people that they wouldn't even know _how_ to run A/B tests even if they wanted to.

How is that different from Tesla essentially using Tesla car owners as unpaid betatesters for their autonomous driving (and other) software while disclaiming all responsibility when anything happens?

This "learn by doing" is going to end up well ... not. Especially when trying to learn by repeating mistakes with completely foreseable consequences, like that blue "verified" badge being available for anyone who pays without any verification.

> using Tesla car owners as unpaid betatesters

Almost right.

Charging Tesla owners to beta test their autonomous driving software.

Almost fully right. Charging and risking Tesla subscribers to beta test their autonomous driving software.

So close. Charging and risking Tesla subscribers along with everyone else on the road to beta test their autonomous driving software.

I'm one of those people. Running FSD Beta in my Model S Plaid in BC. And your negative connotation doesn't apply to me or many/most of the other beta testers with whom I'm regularly communicating.

It's curious how many people are unhappy that someone choses to pay for an early-access product that's in beta.

Most people are probably just amused. I'm more on the unhappy side because a vehicle can actually kill people when it crashes and Musk seems like more of a "move fast and break things" personality, which is not a great combination for safety.

Maybe because it's unsafe and misleading? The US Department of Justice has already started a criminal investigation [1]:

> Oct 25 - Tesla Inc (TSLA.O) is under criminal investigation in the United States over claims that the company's electric vehicles can drive themselves, three people familiar with the matter said.

> The U.S. Department of Justice launched the previously undisclosed probe last year following more than a dozen crashes, some of them fatal, involving Tesla’s driver assistance system Autopilot, which was activated during the accidents, the people said.

[1] https://www.reuters.com/legal/exclusive-tesla-faces-us-crimi...

I'm unhappy about it because I have to share the road with Tesla's cars, and I'm also tired of people buying wholesale into hype. I personally stay away from Tesla's on the road for fear that they have the "self-driving" software engaged.

He promised I would be able to earn income on my car driving around town all on its own while I was asleep at home by the end of 2016. Instead I have to pay him to have a computer drive full throttle into stationary objects - something that I am more than capable of doing on my own.

I think this new idea of "public testing" is really just a post hoc recasting of Elon's failed Twitter Blue rollout. Calling it by those words is...creative? But the failure was less an experiment as it was a lesson in humility.

> I think this new idea of "public testing" is really just a post hoc recasting of Elon's failed Twitter Blue rollout. Calling it by those words is...creative?

Its just a diplomatic rephrasing of Elon’s “do lots of dumb things” tweet:

https://twitter.com/elonmusk/status/1590384919829962752?t=cc...

But advertisers who have pulled out because of distrust and lack of stability aren’t likely to be reassured by rationalizations for the policy instability, they’ll just be confirmed in their decisions to wait to see how things shake out.

And regulators concerned about noncompliance with binding rules aren't going to care about a PR rationalization at all, except insofar as it provides evidence that the failures were intentional rather than inadvertent.

Exactly.

It's just a CYA statement, basically saying "oops, I meant to do that...", and leaving the door open to make more oopsies as intentional "experiments".

A typical HN comment on a SpaceX post might have once looked like:

> Unfortunately, the "move fast and break things" attitude doesn't work with rockets.

You will likely find this comment verbatim from the last 10 years. Given that is so, I can't see how someone could claim that Elon Musk is new to visible experimentation.

Well here it sounds more like "break things and be an asshole", to be honest.

I wonder what would have happened, had he fired 75% of the engineers when he took control of Tesla /s.

I largely agree with your point, but I have to say as a user I generally don’t enjoy A/B testing at all. It’s invisible to me, and some part of the app or website I’m using is now either broken or annoying in some way, but still works normally for the people I mention it to, since they’re not in my group. Additionally, the changes are sometimes totally transient, so things are disrupted for me briefly, only to be distrusted again.

A/B testing when it's like the color of something (did you know you can change the hacker news banner color?) doesn't bother me much, but when it's more invasive and I can't turn it off or switch teams, it starts to get annoying.

Especially when you're trying to help someone and they are seeing something different from what you see.

Hey remember all those terrible things we banned? Lets experiment by letting them free again! Weeeeeeeeeeee.

for our next experiment, we should re-introduce lead into paint

looking into this

The science on it is unsetttled and ideologically biased because I don't like the policies people justify on its basis.

> What a weird thing to say...

They need to explain musks antics somehow. More polite wording than idve used.

> this post is entirely to calm advertisers

Regulators, too [1].

[1] https://www.ft.com/content/a07ca1ae-9f9a-46ee-9457-27bb30e18...

Yes, but "[w]e believe that a service of this importance will benefit from feedback at scale" will just make advertisers unhappy.

To state the obvious, I don't think they are reading the room very well...

Maybe they are referring to Elon's recent unhinged polls as testing.

Hey: vox populi, vox dei!

It's Latin so it must be smart.

Apparently Twitter had some huge problem with bots when Musk was trying to get out of the purchase. Thankfully he solved the bot problem after the purchase so he could run polls and really get the will of the people and not, you know, all those previously problematic bots.

Except when a poll gets a lot of votes that he doesn’t like, in which case he says “looks like the bots are out in force today!”

Good ol' Schrödinger's bots.

> It's Latin so it must be smart.

It's perfect, in a way, that the full quote is "Nec audiendi qui solent dicere, Vox populi, vox Dei, quum tumultuositas vulgi semper insaniae proxima sit." which translates to "And those people should not be listened to who keep saying the voice of the people is the voice of God, since the riotousness of the crowd is always very close to madness."

Refs: https://twitter.com/cagrimmett/status/1595967787339743232

that blurb you quoted is just twitter rationalizing for Elon's red-bull fueled tweets to "improve twitter"

Lol, red bull...

Elon's late night tweets always make me think of that scene in The Office where the new CEO (James Spader) decides to close one of the branches without telling anyone and when asked about it he goes "I got into a case of Australian Reds... and... How should I say this, Colombian whites"

It sort of makes sense if the experiment is about judging not just individual user reactions but the wider reaction of the media, advertisers, "the conversation" to a change.

I don't think that makes it a good idea though, seems like each failed "experiment" poisons the pool and certainly makes it possible to do experiments in isolation.

the marketing team writes a Grown Up , Adult blog post to try to look normal, then commander Elon will come out tomorrow high fiving more nazis [1] and banning more non-nazi accounts for supposedly belonging to "antifa" and saying things he doesn't personally like [2].

This is the same "trapped enabler" pattern we saw with Trump in the early days, with his staff constantly coming out to try to paper over whatever horrible thing he did. They had no shame, and neither does the Twitter staff that wrote this blog post.

[1] https://archive.ph/xYeYY - TPM article without paywall

[2] https://theintercept.com/2022/11/29/elon-musk-twitter-andy-n...

It might be just an escape plan when Musk uses platform for his own needs. Banning users he does not like or allowing certain speech when filtering others and so on….

It's only weird, if you attempt to parse it with a straight face.

This is some B-grade best-effort spin on what has been uncontrolled chaos with predictably awful effects.

The only thing keeping Twitter rolling is the majority percentage of the casual niche user in non-political and non-technical niches, who haven't been paying attention to the chaos; and who by virtue of being casual users are not notably surprised at everything that has broken both culturally, wrt safety and content, and technically.

Unfortunately for Leon the money comes from corners who HAVE been paying attention and not only see what's happened, and ongoing—they see through this kind of comedic college-try at handwaving around it.

Twitter has never been more fun, useful, and engaging for me.

I signed up for an account after years of refusing to do so after elon took over

Then, if I may ask, how do you have an idea of what it was before?

I was a lurker before, I read twitter all the time with out an account. It as a pain because they kept trying to force you to create an account so I kept having to find ways around their account signup walls which was fun...

What's funny is they no longer try to force you to create an account. I used to use nitter.net to browse someone's history, but now I can do it on Twitter apparently with no restrictions.

What specific communities have gotten more fun for you since Elon took over and what is making them more fun? Genuinely curious.

That's because you're a right wing troll, no? You're exactly the target demographic Musk is rolling out the red carpet for, so it stands to reason you're having a great time with it. This clearly doesn't refute the OP though.

Will certainly be interesting to see if there are enough of y'all willing to shell out $8/mo to have your speech boosted above the free variety to make up for the lost ad spend of the Pinochet-helicopter-meme-adverse.

>That's because you're a right wing troll, no?

I am an independent libertarian that strongly believes in free expression, beyond the idea that only governments can censor people.

>>speech boosted above the free variety to make up for the lost ad spend

I think the reduced ad spend is temporary and in some ways unrelated to Musk whom they have used as an excuse to virtue signal for something they were already going to do in the first place

Also I think the plans are to expand twitters revenue model beyond Blue and ads so it will be more interesting what Elon plans to expand twitter into

̶u̶n̶s̶t̶a̶g̶e̶d̶ ̶r̶o̶l̶l̶o̶u̶t̶s̶ ̶o̶f̶ ̶f̶e̶a̶t̶u̶r̶e̶s̶ ̶t̶h̶e̶ ̶C̶E̶O̶ ̶t̶h̶i̶n̶k̶s̶ ̶a̶r̶e̶ ̶c̶o̶o̶l̶ ̶ public testing/experiments

It's just referencing the fact that the new Twitter is more open and transparent. I can't believe how many people are spinning this as a bad thing.

We heard almost nothing from Parag and little from Jack when he was running it and experiments were opaque from the outside.

Now we're hearing from the CEO and employees like George Hotz about what they are doing and planning, and they're involving the community, asking for feedback directly.

> previous Twitter wasn't as transparent as it is now

Reinstating accounts on the basis of a poll, on a platform you have spent months railing for having too many bots, is a good example of CYA transparency.

Elon specifically mentioned that his polls were actually a great way to obviate bots, as reflected by the bot activity seen on them. He was going to reinstate the accounts anyway.

That sounds like excuse after the fact someone called him on his bullshit

And are all the bots expected to vote one way?

That's your response? That polling is transparent because both sides will just turn it into a game of CoreWars, and may the best argument win (at CoreWars)?

Where does it say or even imply "rolling out new features to all users at once"?

> TLDR: this post is entirely to calm advertisers,

How though, Twitter has had week(s) to prepare this post and its so bare.

> First, none of our policies have changed. ... The team remains strong and well-resourced.

If you're Eli Lily, why would you re-advertise on twitter. Nothings changed from when you stopped!

> If you're Eli Lily, why would you re-advertise on twitter. Nothings changed from when you stopped!

Not only would I not buy ads, i'd rethinking even having a corporate presence considering the lack of protections against fraud.

Looking forward to a verified @RealEliLilleyCEO account appearing 5 minutes after Twitter Blue 2.0 3.0 comes out, which only allows you to trivially impersonate people, after which Musk will completely independently invent Verification 2.0 3.0 which will either just be the old system again or a separate Twitter Blue badge. Visionary genius at work.

Oh I agree, this post won't change anything at this point. But I guess that's why it was released (and as others noted, also for regulators / to save face after weeks of chaos).

Musk literally said "we're going to dumb things" and keep the things that aren't dumb.

This "going to do dumb things" is completely on brand for him with his 5 step manufacturing improvement process, step 1 of which is "make your requirements less dumb".

This is actually how Nike works, having worked there. They try all kinds of weird things aren't aren't afraid to, and then drop the ones that don't work with no regrets, keeping what works. Try new things, fail fast isn't a bad strategy for innovation.

Whether or not it'll work out for twitter remains to be seen. Especially with the rest of biased tech still upset that they lost their monopoly on the narrative arrayed against him.

Right, but cost to that is your company losing some lunch money on prototypes.

The costs to "fail fast" (if we're being generous) strategy Twitter employed so far far exceed that

I'm guessing that some product people finally managed to explain what an AB test is to Elon and why they use them to validate product ideas before rushing implementations out the door based on gut instinct. Aside from trying to assuage advertisers about his capricious product decisions he's also trying to act like he is now an expert on digital product development.

I read the experimentation bit as Twitter (Elon) trying to push the boundaries of what Twitter can get away with from a PR perspective.

More like Twitter now lacks staffing of its ML and experimentation infra to actually do experiments carefully.

The thing is Twitter already A/B tested a LOT and Twitter’s new owner is naive to those prior findings as well as the long-term effects of them. Any advertiser is going to want to know about attrition rates, and those were previously best reported through now-fired employees as well as earnings releases. This is Twitter’s new owner learning the hard way that there are dumb advertisers out there who do want quick lift but longer term he’ll retain only spam. No way Twitter ever gets better than Doubleclick now.

> What a weird thing to say... A/B tests are a thing, does anyone buy that experimenting with new things by rolling out new features to all users at once is a good strategy?

What it means: 'Elon looked bad when the blue checkmark fiasco happened. And more boo boos are on the way. Now Elon doesn't like to look bad. Solution? Everything we do is now covered by "it was just a test" disclaimer. Problem solved.'

It could make sense in the context of twitter. They're a "public square" so to speak, so if everyone experiences the same change, they can discuss it at the same time instead of small subsets experiencing it and being confused (or getting back data from the wrong people)

I do not think it means A/B testing and such is going away, but that public testing is going to be more widespread about things most people will probably be interested in.

He should unban Trump as an A/B test, like ones who answered Yes in that poll should be in B group, ones answered No in A group got him banned. It would be great even to separate these people completely into the A/B Twitter (2.0)

It's kind of hard to calm your advertisers when the boss has basically cut most of the marketing staff responsible for keeping contact with them. That's been the biggest issue so far, aside from the right-wing favoritism that the boss (Musk) has online (he's banned Crimethinc on a whim of Andy Ngo's hearsay). Basically, I wouldn't bother advertising on Twitter since the new boss/owner has no one to filter his nonsense out of the daily decision making. It's one thing to spew platitudes for free speech and it's another to actually lay out the rules of free speech on the platform (i.e. trying not to trigger regulations or reprisals of big govt agencies).