It did not ask him to clarify what he is doing, it did not say anything about bounds of acceptable use cases or what they might be, it did not encourage him to contact Twitter directly, or provide a link to do so. It did _not_ "inquire about why the developer needs elevated access", there is no such inquiry in the email. It _did_ say elevated access was going away.
It said he could use the free API with rate limits, or the commercial Gnip API.
It kind of half-heartedly suggested he could "reply to this email" if he had "questions". From the email, there was no reason to think elevated access might still be available, it pretty clearly said it would _not_ be, so I'm not surprised he didn't have any questions -- the email was quite clear (at saying something pretty different than you are saying).
So when you describe the email that goes out -- it does not seem to describe the one he received, according to him in the post. Are you talking about a different email? That he did not receive, or that you think he should have received in addition? Or do you actually think that email somehow communicates what you describe above? (It really really does not, which is why I think you must be thinking of a different email he did not receive or post, or you were internally misinformed about what the email was going to say).
With the email he actually pasted into the OP it is not surprising that he simply publicly notified his userbase and other interested parties that the service would be going away -- what else do you do when your upstream provider tells you the service is going away? This is all very standard and professional.
But, if you are saying that you do continue providing free elevated access to certain projects that seem worthwhile and meet some 'boudns of acceptable use', then that is nice, cool, I'm glad Twitter is doing this. (Maybe you should have told him that in the email though! And it would be great to actually advertise that fact, and what the bounds of acceptable use are, and how someone can get in touch with you to request access.)
That's not entirely correct. Under option 2 after mentioning it's commercial side, it says:
There are also other solutions that offer varying levels of access as well as historical search.
It's not prominent (likely because they only want it pursued in special cases), and it's easy to miss, but it is there.
> If they want all these special access cases handled through Gnip, whether they are discounted or free, that's their call.
There's nothing in the mail suggesting they "want special access cases handled through Gnip".
In any case, it's all speculation, there was never a dialogue, and the HN title as it current stands is probably inflaming this more than it should be. Their access was not shut down (and the blog post's title and wording does does not say it is shut down, but that they believe they will be shut down). An email was sent, it was poorly worded, and the recipient overreacted (or at least prematurely reacted). The responsible thing to do would have been to contact Twitter or the sender and discuss the needs and goals of the project, and see what they could offer.
Immediately writing a public blog post is not negotiating in good faith.
Yet this seems to have been quite effective, given that the head of Developer Relations at Twitter is replying to this story on HN..
Public shaming on twitter+blog is 50/50. It often gets the attention of an engineer and sometimes even a founder, and when it does the response is often reasonable. Why would you ever try anything else?
I've had a crazy technical issue with AWS that our account representative proved useless for about but a tweet got it fixed. Also see news articles about people suffering the most ridiculous treatment from comcast/timewarner/verizon/at&t until they get in the news and everything is finally sorted out. This is a totally standard thing you should already be familiar with. The problem is that these big companies get too many queries from crazies and people who have no idea what they're doing, they're inundated with stupid support requests. So a legitimate support request will never be noticed by anyone who knows anything. So you need a sort of public vetting process. You need the "shaming" part to express the priority in large companies where business people have all the power.
I'm not sure there's a better way, that's just how it is.
Going public means getting in front of people that care more about customers or has more time to do so. It feels a little like skipping the line though.
I think it boils down to this, if the email said "Hey, we noticed you have special access, and we are trying to clean those up. You should either use the regular API with rate limits or our commercial offerings. Let me know if you have any questions." then I think the appropriate first step would be to contact Twitter and explain your situation. Since the actual letter contains that and just more information about possible options, I don't see why it should be viewed any differently.
> Hey, we noticed you have special access, and we are trying to clean those up. You should either use the regular API with rate limits or our commercial offerings. Your special access will be terminated on Thursday, April 21st. Let me know if you have any questions.
On receiving an e-mail of that nature, I would very likely let my audience know that I was shutting down the service on Thursday, April 21st just as the blog post author did.
I would not assume "Let me know if you have questions" meant "Let me know if you'd like to clarify your situation and request an exemption from this policy". I would assume it meant I could inquire about the details and process of the shutdown, or ask questions about the Gnip service or other options that they discussed in the e-mail.
Immediately going public can backfire, depending on the specifics of your case. Now there are possibly competing interests at play, the public pressure to let them continue, and possibly some anger over at Twitter for not being given the smallest benefit of a doubt (when their project works because of the good grace your company exhibited in the first place) or even asked before playing hardball. If I was the person responsible for making the call at Twitter, I would probably acquiesce, but I would want to tell this person to go get bent.
If they don't wish to communicate that message, the message should end with at the very least something like "We don't want to terminate a popular service in error, so please contact us today to tell us more about it."
Did everyone miss this part of the blog post?
> I’ve replied to Twitter asking for an exception, but I honestly don’t expect to receive one—and even if I were to, it wouldn’t help other developers who I believe deserve to get the same level of opportunity as me.
> In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so. The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate. We have contacted the owner of this app and hope to resolve this situation, as we do with hundreds of other developers on our platform. We do occasionally provide exceptions for apps that are non-commercial (not-for-profit, no ads, etc.)
If this was the intent, the template email was kind of poorly drafted, because that's not what it says at all.
It says multiple times, without qualification, that they're going to be cut off — "this type of elevation will no longer be available," "your current elevated rate limits will no longer be available," "we will remove this elevation from your account," and "we encourage you to evaluate alternative options." At no point does it say that keeping the elevated access is an option.
So I mean, can you really blame the owner of this app for reading these absolute statements as being absolute and believing their app was cut off, rather than seeing it as some kind of sideways inquiry as to whether they think their use is acceptable?
As many others have pointed out, the e-mail from Twitter in the blog post absolutely does not state this. I add my voice to the chorus of others asking if there is an alternate e-mail, or if you and Twitter truly believe the e-mail in the blog post ever made this statement.
> The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate.
Again, joining in the chorus of voices, I want to ask why it was unfortunate that the developer made a public statement that his service would be ending? In the correspondence with him it was clearly stated "On Thursday, April 21st we will remove this elevation from your account" (contradicting your claim that you "asked them to clarify what they're doing"). I would argue that the developer has the responsibility to notify their audience that the service is going to be shut down at that time.
Why is a public response anything but appropriate given the content of the e-mail that Twitter sent to him? If the situation you describe here is truthful, then don't push blame onto the external developer when the e-mail didn't come close to communicating what the Twitter team thought it did.
Every 6 months you guys make a new change that kills a bunch of websites who were silly enough to rely on your APIs. Then you give an explanation like the one above: something about needing to grow the platform, and pushing the responsibility for the problem back on the businesses.
When you killed the tweet count a few months ago, the story was exactly the same - something about depreciating it in order to build a more dependable platform, and people shouldn't have relied on it in the first place. It's clear you guys just wanted to take control of your data.
As a social network, I love Twitter. But I would never waste any time building on your platform. At every step you leave a trail of dead businesses in your wake.
If a contract isn't feasible, at least make sure it's in the other side's best interest to keep the situation going. Getting free access when it costs the other side (in managing access, serving requests, handling support, etc, no matter how small) is not something I would feel comfortable basing a business off of.
The difference is that the person is not getting anything that's not already offered to the general public. It's unfair to use the court system to enforce a company's exclusionary political decision that "everyone can access our website EXCEPT YOU", at least barring any actual contractual terms (not generic Terms of Use). The change I propose would not force anyone to accommodate a certain API change or provide any special functionality. It would simply prevent non-neutral access rules from being enforced via legal vectors.
Twitter would still be able to charge for access to the APIs they charge for now. They'd simply be unable to use the court system to compel someone to stop accessing the data that they have no copyright interest in and which they serve up to the world for free. I'm not sure if Twitter has tried to do this yet, but it's the normal step companies take once a consumer develops the ability to evade their IP blocks.
To be honest, the law already can be interpreted this way. The problem is that it often isn't. Companies have been able to convince non-technical judges that concepts like trespass to chattels are applicable any time someone is talking to their server. We need language in the law that will clarify the matter to prevent big companies from squashing small innovators that they find inconvenient or threatening instead of leaving it up to judge roulette.
Of course it's ideal to have a contract with a company that guarantees access to the data stream for a reasonable chunk of time, but the reality is that unless you're already a big shot, platform vendors like Twitter aren't going to give you the time of day for something like that.
If we take away the ability of companies to selectively allow access to data that's available for free to the general public, it gives these guys a fighting chance. The web is a publication platform. It's equally absurd to say "I'm publishing this novel and everyone except kbenson can read it" as it is to say "I'm publishing this data to the general public and everyone except kbenson can access it". You can try to stop someone from accessing your data, but certainly no court is going to consider it reasonable to assist a company in preventing kbenson from reading a widely published manuscript. Data access is substantially the same thing, and should be treated the same way. Keeping non-disruptive entrepreneurs on the same footing as everyone else makes innovation more secure.
> It's understood that a company could stop offering some data that a company is dependent on altogether, but I think it's substantially less likely than a company threatening to terminate access to data that already exists (which is what happened here).
I'm not sure sure. There's regular griping on HN about Google cancelling services, even though they are generally beta services, because people expect them to offered. Some of these people griping are people that built products on Google APIs that were discontinued.
> Of course it's ideal to have a contract with a company that guarantees access to the data stream for a reasonable chunk of time, but the reality is that unless you're already a big shot, platform vendors like Twitter aren't going to give you the time of day for something like that.
A contract? Maybe not. But if they have a clearly defined deprecation policy, that's a start. And if you are paying them, well that's a lot better, since they are incentivized to keep it going because of your (and hopefully others') money.
As for the rest (access to public API), I'm not really interested in arguing it, as we likely agree more than we disagree. :) The only caveat is that direct consumers of the API often don't behave like regular public users, and that may put a strain on the system and degrade performance for everyone, causing real problems and the need to spend real money to fix if you still want to allow unfettered public use.
I believe that sites can implement technical measures to "enforce" a fairly normal request profile. Third-party consumers will have to comply with that because it will be technically impossible not to do so. Beyond that, there is no need for a legal method to address access. If the data source is concerned about said consumers going through a code path that is more intense on the backend, they will accommodate with a lighter API. :)
Doesn't that make some projects and uses, where it needs more than what is regularly allowed, impossible? I think the natural response to that would be "well, we'll just charge them (something/more) and give them special access beyond our normal restrictions", and we're back at square one. Enforcing everyone gets the exact same access is essentially communist theory, not mean in an inflammatory way, but in that it really does enforce some level of mediocrity on everyone (to be fair, as insurance some bad actors makeing things untenable for the rest) in comparison to an approach that allows some companies to pay for for extra access to account for the extra burden which may result in new products that people want.
I'm for making publicly available data available to use as wanted, but I'm not really for enforcing arbitrary limits that prevent whole classes of useful products that people generally might want in the name of being fair, especially when "being fair" might mean contradictory things depending on the side you approach from. The fact that the data is generated by the users, and ownership may or may not reside with users or the aggregator, and current law may not be the best with regard to that just makes it all the more complex. :/
I don't think we should force companies not to have a paid API, that's all well and good. What I want to remove is their ability to stop scrapers from gathering the data from the generic public interface just because they don't like the way the scraper is using the information.
As it pertains to this specific story, I think Twitter was well within their rights and norms to try to move a special cased user to some more traditional offering of their, or to at least make them justify their usage. I also think it's well within Emojitracker's rights and is acceptable to try to make a public stink if they think they are getting a raw deal. The only thing that I think leaves a bad taste in my mouth is that he did so prior to attempting a good faith negotiation with Twitter.
I don't see any encouragement or really any path to recourse in the email. It reads as though the decision is final. Email linked from blog post: https://cdn-images-1.medium.com/max/2000/1*MoLgPNLiQwQ8s-Y1D...
My interpretation of the email is essentially: we used to offer this for free, and we're now offering a commercial alternative. You have three options: use the paid option, use the free API that does not work for your use case, or shut down.
If I'm missing something obvious please let me know.
1. Build a successful product
2. Twitter discontinues dev-friendly policies, product gets shuttered
3. Later, tweet doubts about Twitter's dev "commitment"
4. Tweet gets removed
5. Read yet another horror story about working with Twitter
6. Twitter guys claims that Twitter is "committed"
7. Turns out, previously mentioned horror story tells a different story
8. Rinse, repeat?
As Twitter's platform business has matured to include many businesses building billions of dollars worth of social media monitoring and other tools, we've formalized the process to becoming a Twitter Official Partner (partners.twitter.com).
Simply because you're an even bigger company now that means implicitly that you need to change your processes, perhaps you could explain the reasoning of what was going wrong, previously.
I think a lot of developers would question whether or not you guys even have such a commitment at all. Twitter does NOT have a good history in this regard. If anything, you guys need to bend over backwards, sideways, upside down, and any other way you can think of, to treat developers well and woo them. I mean, somebody needs to go full Ballmer and start screaming "Developers, developers, developers!!!". [1]
Seriously, you guys are so behind the curve in terms of developer perception that "good enough" isn't even close to good enough. You should probably be reaching out to developers, offering to fly them to Twitter HQ, and treating them to steak and lobster, etc. And that's just to get back in the good graces of the development community.
Developers do not compete with Twitter, developers make the Twitter ecosystem larger and stronger and better. Twitter needs to embrace that mindset and go all in on becoming a desirable platform to develop for.
As for myself, our app offers limited Twitter integration, but we're damn sure not going to spend a lot of time and energy on Twitter given the history and what-not.
My advice? Kill Gnip, make it trivially easy to access the firehose, commit to adhering to open standards[2][3][4][5] across the board, and put developers front and center. If you want to charge for firehose access, fine, but kill any notion of some complicated application / approval process. Make it as simple as using the API now plus checking a checkbox and plugging in a credit card number, whatever. It's 2016 for crying out loud, there's no reason I should need to fill out a form and have a sales representative call me for something like that.
[1]: https://www.youtube.com/watch?v=Vhh_GeBPOhs
[3]: http://www.foaf-project.org/
[4]: https://www.w3.org/2008/09/msnws/papers/W3C_FOSN_Position_Pa...
Sure there is. The reason is that they want to guess how much money you'll be willing to pay and have your sales rep charge you that.
Not really. This follows the coloring book pretty well:
Step 1. Inflexable corporate rhetoric and/or positions expressed privately.
Step 2. Other party blogs about a scenario that puts egg on their face.
Step 3. "Nah man, we're flexable! Why didn't you just talk to us first?!"
The rest of your explanation is just specious, as pointed out by other people, and also reflects poorly on your company.
Really? Where in the email does it ask anything of that sort? The only "unfortunate" part that I see is that this person exposed the false rhetoric and lack of flexibility that Twitter has shown time and time again. Apologize and fix it, stop trying to whitewash this.
If what you just said is what it the email meant to communicate then it fails at every level. Please don't try to shift the blame for Twitters poor communication to the recipient because they "elected to blog publicly about the situation".
The "situation" is entirely created by Twitter.
Twitter has a (recent) history of terrible communication with developers and it would not surprise anyone if what seemed to be the case WAS the case. If it is NOT the case, as you say, then you really need to improve your company communication.
It is directly related.
Twitter desperately needs to decide whether you are a platform or a closed business. If you're a platform your job should be encouraging any and all innovation on there, and have clear pricing tiers which do not result in sudden jumps from free to large charges or access cut off and directly reflect the cost to you of serving the API. You should never be in a situation where you're emailing to cut off access (and yes this notice is shutting it down).
Successful products built on twitter should be celebrated, not forced off the platform.
Try the app. Talk to the developer.
Template mail should not exist.
1) There's no indication he's not US born and raised, but you're free to make assumptions based on his name, I guess.
2) Regardless of that he's been working in the US since 1994, which would put him way beyond the maximum limit for an H1B (6 years + possible extension during change of status). So no, it's not some H1B teaching you.
Hi Prashant,
I want to pay $ 100 per month to extend the rate limit of the free API. Is this possible?
Thanks,
wslh
In the past, Twitter had little formality associated with granting elevated access. It was very much, "Hey, I want to do this cool thing!" "Okay, sure, here you go." As Twitter's platform business has matured to include many businesses building billions of dollars worth of social media monitoring and other tools, we've formalized the process to becoming a Twitter Official Partner (partners.twitter.com). Over the last six months, we've started contacting all of those API key holders with elevated access and asked them to clarify what they're doing. In some cases, we know the business and business owner and reach out personally. In other cases, like this one, the business is listed in our systems as "N/A", so we send the template mail.
In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so. The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate. We have contacted the owner of this app and hope to resolve this situation, as we do with hundreds of other developers on our platform. We do occasionally provide exceptions for apps that are non-commercial (not-for-profit, no ads, etc.)
Note also that in this instance, the notice is NOT about shutting down this app. It merely inquires about why the developer needs elevated access, something that is typically reserved for our business partners.
None of this is related to continuing to use the Twitter API or our commitment to enabling developers to build on our platform (Fabric, Gnip, our Ads platform, and the Twitter API).