News for hackers. Stuff that matters.
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @04:49AM (#34365768)
Does anything stops devs from using the Web interface instead of the API? Sure it makes it a bit more challenging because parsing is involved but AFAIK it is completely legal to parse and display HTML output however you want. It's essentially like a browser add-on.

I wonder if Twitter can be killed by the "embrace, extend, extinguish" strategy?

So:

1. Make a client that combines Mastadon/Others and Twitter feeds, making it switching out of Twitter frictionless.

2. Provide extra functionality for non-Twitter posts, encouraging the use of the non-Twitter services.

3. Once have a good traction, penalize Twitter post to further push non-Twitter adoption. Eventually remove Twitter support completely.

4. Profit??

Re:Tweetbot Is Broken by samwillis on Friday January 13, 2023 @05:36AM (#34366044)
While it's quite possible to build an app that way, using scraping. The danger is that users of that app will get suspended from Twitter (users have explicitly agreed not to scrape), once thats happened to a couple of users, thats the end of your app.

Obviously you can try to obscure that you are scraping, but that is difficult to get right and there is no way back once you get it wrong.

Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @05:46AM (#34366116)
On Apple devices that wouldn’t be an issue because you you can load the Twitter Web interface into a real Safari and get the markup from there for processing. Twitter can try things like CAPTCHA but this will degrade the experience for all Twitter users and you can simply delegate this CAPTCHA to your user to solve anyway.
Re:Tweetbot Is Broken by samwillis on Friday January 13, 2023 @05:55AM (#34366184)
While for reading that works, there is an easy why via posting to spot people use that app. They can change at any moment the input names so that when posting a tweet you are exposing that you are using the scrapping client.
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @06:01AM (#34366219)
Remember that you are loading full Twitter website in the background. You will use their original interface to post Tweets.
Re:Tweetbot Is Broken by majewsky on Friday January 13, 2023 @07:13AM (#34366728)
Yes, but the app is not a human and will likely interact with the website in a more predictable (and thus, detectable) way than the human itself would.
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @07:18AM (#34366750)
But you also have a human using the app, you can transfer the interactions with your app to the Twitter's Web UI.
Re:Tweetbot Is Broken by Mindwipe on Friday January 13, 2023 @06:50AM (#34366565)
Apple wouldn't permit such an app in the store so it's a moot point.
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @06:51AM (#34366572)
Why do you think that Apple wouldn't permit it on AppStore?
Re:Tweetbot Is Broken by davidcbc on Friday January 13, 2023 @09:30AM (#34368196)
There are already apps like this on the app store
Re:Tweetbot Is Broken by alkonaut on Friday January 13, 2023 @11:31AM (#34370086)
If something is built as a browser extension and the requests it sends look like the browser would normally then it must be hard to detect.

The requirement for the extension/plug-in has to be that: it looks like someone uses Twitter normally.

Re:Tweetbot Is Broken by amelius on Friday January 13, 2023 @05:38AM (#34366070)
Where can I vote for this project?
Re:Tweetbot Is Broken by black_puppydog on Friday January 13, 2023 @05:15AM (#34365919)
you get into an armsrace where twitter will obfuscate the markup to just wear out the scraper devs. which they can, because $$$
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @05:16AM (#34365927)
Then just OCR the rendered output. It's cheap and easy these days.
Re:Tweetbot Is Broken by mlatu on Friday January 13, 2023 @05:34AM (#34366037)
i heard it works best on iphones, best of luck
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @05:40AM (#34366083)
Apple’s vision framework is actually very good, so effective on-device extraction is really just a few lines of code on iPhone(all Apple device actually).
Re:Tweetbot Is Broken by tsimionescu on Friday January 13, 2023 @06:57AM (#34366611)
Good lick OCR-ing images and videos, though. Twitter isn't just text.
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @07:12AM (#34366722)
Why do I need to OCR images and videos? Why not display those as-is?
Re:Tweetbot Is Broken by tsimionescu on Friday January 13, 2023 @08:17AM (#34367248)
Which one do you show for which tweet, if all you have is the OCR (that is, you can't read the markup)?
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @08:30AM (#34367368)
Apple's vision framework is not limited to text, can detect shapes and other things.

Twitter WebUI loads images and videos when you scroll. Which means you have a good signal what image and video is associated with what tweet.

Re:Tweetbot Is Broken by actualwitch on Friday January 13, 2023 @06:21AM (#34366353)
Doing this requires a fair amount of development time, especially when you can break something else with this and you previously removed all the guardrails.
Re:Tweetbot Is Broken by Mindwipe on Friday January 13, 2023 @06:51AM (#34366567)
That didn't stop Facebook doing it (and breaking all kinds of stuff in the process).
Re:Tweetbot Is Broken by Hamuko on Friday January 13, 2023 @07:11AM (#34366711)
Did Facebook lay off half of the company before they did it?
Re:Tweetbot Is Broken by raydev on Friday January 13, 2023 @01:03PM (#34371444)
Facebook had far more engineers available to do this sort of work.
Re:Tweetbot Is Broken by nkozyra on Friday January 13, 2023 @07:12AM (#34366718)
They don't do this when they can just disable the app without being logged in.
Re:Tweetbot Is Broken by Nextgrid on Friday January 13, 2023 @05:56AM (#34366186)
You will have 2 problems with this:

1) the law, at least in the US. Laws such as copyright and maybe even the CFAA have been abused to outlaw adversarial interoperability. You'd need to either practice good opsec and operate anonymously or be based out of reach of US-allied law enforcement.

2) Apple (and maybe Google too?) - they will remove any client for a third-party service unless said client has explicit approval from the third-party, all the way up to ridiculous levels such as rejecting an app using the LAN API of smart home devices explicitly designed for this purpose: https://community.lifx.com/t/app-store-rejection-permission-...

Re:Tweetbot Is Broken by counttheforks on Friday January 13, 2023 @06:41AM (#34366491)
Another point to show what Apple's true goal is: To kill general computing. If you buy an iPhone you should be able to run whatever app you want on it. But no, Apple knows best and will not permit you to do anything it doesn't approve of.

As for the APIs being copyrightable see Google vs Oracle.

Re:Tweetbot Is Broken by paulcole on Friday January 13, 2023 @08:05AM (#34367147)
> If you buy an iPhone you should be able to run whatever app you want on it.

I hear this sentiment quite often on HN. Maybe I’m missing something but was this tacked onto the end of those stone tablets that Moses brought down from the mountain?

If not, then it seems like people want this to be a statement of fact when the reality is much closer to:

> If I buy an iPhone I wish I could run whatever app I want on it.

Re:Tweetbot Is Broken by rchaud on Friday January 13, 2023 @10:06AM (#34368711)
Don't know about Moses, but here's what iJesus had to say:

> The full Safari engine is inside of iPhone. And so, you can write amazing Web 2.0 and Ajax apps that look exactly and behave exactly like apps on the iPhone. And these apps can integrate perfectly with iPhone services. And guess what? There’s no SDK that you need! You’ve got everything you need if you know how to write apps using the most modern web standards to write amazing apps for the iPhone today..

https://9to5mac.com/2011/10/21/jobs-original-vision-for-the-...

Re:Tweetbot Is Broken by paulcole on Friday January 13, 2023 @10:09AM (#34368754)
Yeah it's almost like he changed his mind? I don't know, I'm not too bright.

I'm also not in the habit of crying about what was said 15 years ago and what has clearly changed since then.

Re:Tweetbot Is Broken by rchaud on Friday January 13, 2023 @06:38PM (#34375438)
I never made a comment about your intelligence, but you may wish to re-evaluate your comprehension if a simple refutation strikes you as 'crying'.
Re:Tweetbot Is Broken by kmeisthax on Friday January 13, 2023 @08:39AM (#34367456)
Apple's goal was to kill malware, not general-purpose computing[0] as a whole. Unfortunately it turns out that killing general-purpose computing is an instrumental goal[1] to killing malware: if you want a phone that cannot get a virus then you need someone pre-approving every line of code that hits the processor.

Once you have any entity having control over your device, even if it is for your own benefit and you completely trust that entity, a whole bunch of other competing business considerations come into play. In the case of adversarial interoperability, that means business risks that Apple doesn't want to take on. Let's say that tomorrow Elon Musk goes absolutely insane[2] and decides that third-party Twitter clients are hacking as defined by the CFAA. If Apple knowingly distributes those clients, then they could be held criminally liable for signing and distributing the app.

Less dramatically, adversarial operability implies a lot of support headaches. Apple wants to sell working software, not stuff that needs updates every few hours to keep ahead of updates done on Twitter's side to kill it. Even if they wanted to fight for adversarial interop in court, the underlying constraint of "pre-approve software updates to keep malware away" makes updating the software in real time to work around Twitter's workaround to the software untenable.

For a non-Apple, pre-everything-being-locked-down example of how much of a pain adversarial interoperability is in practice, there's the time Microsoft tried to make MSN speak to AIM. They stopped once AOL started using buffer overflow exploits to validate that you were running AIM[3]. AOL did not need to actually sue Microsoft with a spurious application of the CFAA. They just needed to wrap their API and client in enough garbage to make interop monetarily painful to support.

[0] In the Cory Doctorow sense

[1] AI safety term for "thing you need to do to get to other things you want".

[2] More than he already has

[3] https://www.youtube.com/watch?v=w-7PjunSxLU

Re:Tweetbot Is Broken by heavyset_go on Friday January 13, 2023 @10:17AM (#34368884)
Pretty sure Section 230 of the CDA shields Apple from liability when it comes to what apps users create and share via Apple's App Store platform.

The truth is that the App Store lock-in is about securing Apple's moneyhose, and user "security" is an afterthought.

Re:Tweetbot Is Broken by tumult on Friday January 13, 2023 @10:57AM (#34369533)
The time of CFAA being wielded to suppress interoperability is over. The Supreme Court's 2021 ruling of Van Buren vs United States put an end to it.
Re:Tweetbot Is Broken by fennecfoxy on Monday January 16, 2023 @08:46AM (#34400564)
Malware/app security is always the excuse when it comes to app stores, though.

But there's no excuse that their OS can't sandbox apps as best it can to prevent issues like this. Sure there will still be scam apps, but it should be a lot easier to install apps from outside of an app store.

It's just a blatant money grab otherwise; Spotify for example is extremely trusted and unlikely to be malware - why can't they have an "install" button on their site to install their app? Because companies that run the app stores want a cut.

Why aren't mobile OS' more like desktop OS'? Everyone acts like it's about security but then we get things like Pegasus and Google's Project Zero which is constantly finding stuff anyway.

Even now the granularity on permissions and the main problem of user apathy towards this stuff are solvable. I'm on Android and I still don't understand why an app can only ask for gallery permission/access to all files and why I'm not able to only grant it read access to a particular folder in a streamlined way.

All of this stuff should be designed around a user's intentions.

Re:Tweetbot Is Broken by nhtsamera on Friday January 13, 2023 @08:59AM (#34367782)
You shouldn't even need to run an "app".

A phone is no different from a laptop, except that it is smaller and the mouse/keyboard are part of the screen. On my phone, I should be able to:

* Open an IDE and write some code.

* Compile my code (depending on the language).

* Run my code.

The whole concept of "app stores" is absurd. In 10 years, will we be denied sudo rights on our laptops? Will we need to pay a 30% cut to Apple for every program that we install, plus $99/yr if we want the privilege of building code for our own device?

Re:Tweetbot Is Broken by rchaud on Friday January 13, 2023 @10:02AM (#34368657)
> In 10 years, will we be denied sudo rights on our laptops?

More and more likely. In 10 years' time I'd love to be off the Intel/Apple Mx train and using devices that can run desktop Linux with with RISC-V and Raspberry PI-like SoCs.

Re:Tweetbot Is Broken by counttheforks on Friday January 13, 2023 @10:50AM (#34369404)
Yes, you're 100% right, of course. This just goes to show how much we've lost.
Re:Tweetbot Is Broken by fsflover on Friday January 13, 2023 @10:53AM (#34369482)
This is exactly why I bought Librem 5 smartphone, which can do all these things and can even serve as a laptop, if you connect a keyboard and a screen.
Re:Tweetbot Is Broken by dzamo_norton on Saturday January 14, 2023 @12:35AM (#34377625)
> which can do all these things and can even serve as a laptop

If it could only also serve as a smartphone I'd have one already.

Re:Tweetbot Is Broken by fsflover on Saturday January 14, 2023 @03:50AM (#34378543)
Using it as my daily driver. Depends ob your use case though. Sent from my Librem 5.
Re:Tweetbot Is Broken by l_s_ on Friday January 13, 2023 @01:00PM (#34371409)
You can open an IDE, compile and run code, have you even tried? Also if you want to develop an app and run it in your own device you can do that for free
Re:Tweetbot Is Broken by ztjio on Friday January 13, 2023 @07:53PM (#34376057)
Apple ships an app THEMSELVES that enables this (for iPad anyway) https://apps.apple.com/us/app/swift-playgrounds/id908519492 and there are third party apps for general programming as well. I have one for Python (Pythonista) and one for C# and F# (Continuous) on my iPhone right now. There is a HUGE selection of similar apps for general purpose computing on iOS.

These are unique platforms with unique properties in their design. You value them or you don't. Nobody's forcing you to use them for general purpose computing and there are thousands upon thousands of BRAND NEW general purpose computing products with zero restrictions produced every year. People in this thread are being drama queens.

Re:Tweetbot Is Broken by nier on Saturday January 14, 2023 @07:32AM (#34379779)
In ten years everything will be the same because Andreas Wendker told us that Macs will stay Macs the way we know and love them. He’s neither in marketing nor in sales but an engineer so obviously what he says must be the truth, to put it sarcastically.

«Macs will stay Macs the way you know and love them. They will run the same powerful Pro apps. They will offer the same developer APIs Macs have today. They will let users create multiple volumes on disks with different operating system versions and they will let users boot from external drives. They will support drivers for peripherals and they will be amazing UNIX machines for developers and the scientific community that can run any software they like.»

Source: https://scriptingosx.com/2020/06/macos-11/

Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @06:07AM (#34366263)
Can you elaborate on the first point?

Apple might not be such a big issue because they allow reader apps. I guess they can ban you to improve their relationship with Twitter but you can also throw a subscription service as an IAP to improve your odds :)

Re:Tweetbot Is Broken by ViViDboarder on Friday January 13, 2023 @10:42AM (#34369276)
Regarding point 1, recent judicial decisions on scraping seemed to have entrenched it as legal.

On two, it seems like they are concerned with trademark to me. A workaround could be making a website that parses and displays Twitter content and call it something else. Then, in an app, make no mention of Twitter and only the website.

Re:Tweetbot Is Broken by tumult on Friday January 13, 2023 @11:00AM (#34369584)
1) The Supreme Court's 2021 ruling of Van Buren vs United States CFAA ended the era of the CFAA being used as a legal tool to stop interoperability.

2) You can use another company's marks, names, etc. to indicate your product works with their product. You have to follow some guidelines.

Re:Tweetbot Is Broken by prirun on Friday January 13, 2023 @06:29AM (#34366408)
I think you'll have the same problem as Twitter with #4!
Re:Tweetbot Is Broken by mrtksn on Friday January 13, 2023 @06:48AM (#34366551)
Fair point. Maybe have a paid subscription with features that benefit social media professionals? There are some popular and profitable services economy built around social media.
Re:Tweetbot Is Broken by addandsubtract on Friday January 13, 2023 @02:42PM (#34372852)
I think the tools running on top of twitter are more profitable than twitter itself.
Re:Tweetbot Is Broken by dom96 on Friday January 13, 2023 @08:24AM (#34367302)
I’ve done this recently. Essentially created a browser extension that combines Mastodon and Twitter on twitter.com. It works surprisingly well, but I’m guessing catching up with Twitter.com changes is bound to be painful.

You can find it here: https://chrome.google.com/webstore/detail/mastodon-chirper/l...

Re:Tweetbot Is Broken by amelius on Friday January 13, 2023 @08:39AM (#34367461)
> catching up with Twitter.com changes is bound to be painful.

Perhaps it should be handled in the same way the YouTubeDL (YT DLP) team handles changes in the html.

Re:Tweetbot Is Broken by heavyset_go on Friday January 13, 2023 @10:20AM (#34368931)
I'm not familiar with how they do that, can you expound on it?