News for hackers. Stuff that matters.
Re:Facebook was used as a proxy by web scraping bots by seniorgarcia on Thursday November 05, 2020 @05:34PM (#25002831)
I don't get what value link previews add. Someone shares a link with me (on skype, slack, teams... whatever) and I care about the content because the person sharing it with me thinks I could/should care about it, or someone shares a link on an aggregator and then I don't think it is too much to ask for that someone to write a summary. If the link is worth sharing writing 1 sentence to explain why isn't too much to ask.

What is the value a link preview adds? And why should I, as a content provider care about the value you add? Cloudflare does something for me, what is your service doing for me and why should I whitelist you (or care about you)?

Re:Facebook was used as a proxy by web scraping bots by dasil003 on Thursday November 05, 2020 @05:39PM (#25002874)
They're sending you traffic.

Imagine Twitter or Facebook without link preview, it's much harder to use and overall reduces the change I'll click on a link. Do you think only Twitter and Facebook should be allowed publish previews?

Re:Facebook was used as a proxy by web scraping bots by def_true_false on Thursday November 05, 2020 @05:58PM (#25003027)
Half the time the link preview picks the wrong picture and sometimes even the quote. Twitter and Facebook would both be improved by disabling it. Hell, it might even stop people from thinking they need a hero image for their 2 paragraph medium shitpost.
Re:Facebook was used as a proxy by web scraping bots by input_sh on Thursday November 05, 2020 @06:25PM (#25003234)
I'd place that blame towards website owners. Both Facebook and Twitter are pretty open where they read that info from, and an owner can pretty easily pass those fields (it's just some <meta> tags in the <head> element).

They also have their own validators: https://cards-dev.twitter.com/validator and https://developers.facebook.com/tools/debug/

The only issue I'm aware of is that Facebook's crawler breaks about every two months or so.

Re:Facebook was used as a proxy by web scraping bots by seniorgarcia on Thursday November 05, 2020 @06:49PM (#25003394)
What meta tags do I have to fill and why is Twitters/FBs preview suddenly my problem?

>https://developer.twitter.com/en/docs/twitter-for-websites/c...

So, I should have to include twitter specific meta tags even though I personally don't care about twitter? Maybe twitter should make it clear which tags they read? Maybe it's SEO bullshit I don't care about? Maybe even even the OG: tags don't work all the time and result in dumb previews?

Re:Facebook was used as a proxy by web scraping bots by thatfunkymunki on Thursday November 05, 2020 @08:52PM (#25004095)
If you don't want to fill them out, don't... Filling them out lets you customize your link preview on twitter. If you don't care about Twitter, why would this affect you at all?
Re:Facebook was used as a proxy by web scraping bots by 67868018 on Friday November 06, 2020 @09:48PM (#25012895)
They're used by instant messengers too: slack, iMessage, WhatsApp, telegram, signal, ...
Re:Facebook was used as a proxy by web scraping bots by einpoklum on Thursday November 05, 2020 @05:49PM (#25002951)
> it's much harder to use

Actually it's easier to use, in that the preview doesn't take up screen real estate. Perhaps you mean the experience is less pleasant?

Re:Facebook was used as a proxy by web scraping bots by seniorgarcia on Thursday November 05, 2020 @06:05PM (#25003078)
>They're sending you traffic.

Irrelevant traffic for every metric I care about.

>Imagine Twitter or Facebook without link preview

That's exactly what I'm saying. Either I care about what that person thinks might interest me or I don't. The link preview abstract is shit anyway. Does the site title and the 2 sentence abstract really sway you? If someone wants to send traffic my way, writing an interesting abstract is not too much to ask.

>it's much harder to use and overall reduces the change I'll click on a link

Maybe you should re-evaluate who you follow on twitter? I frankly could care less about facebook.

>Do you think only Twitter and Facebook should be allowed publish previews?

I think previews are worthless regardless, I thought I made that clear. Either you care about me linking it to you or you do not.

*EDIT: And just for fun, here is the link preview stuff from my latest skype call with my brother: https://imgur.com/a/yO5OP36

Look at all the value those previews added.

Re:Facebook was used as a proxy by web scraping bots by cblconfederate on Thursday November 05, 2020 @06:08PM (#25003101)
when you paste a link on reddit and it autocompletes the title

update a bookmark title, or check if it exists.

is it not self-evident that a link being crawlable is useful?

Re:Facebook was used as a proxy by web scraping bots by seniorgarcia on Thursday November 05, 2020 @06:20PM (#25003199)
>when you paste a link on reddit and it autocompletes the title

Oh no, you have to copy/paste the title?

>update a bookmark title, or check if it exists.

I can access the site without a captcha, my browser can fetch the title.

>is it not self-evident that a link being crawlable is useful?

No, it is not. Maybe a site owner does not want crawlers to index the site?

Me being able to access the title and any html meta tags is not the same as some crawler being able to access it. It seems like your beef is with cloudflare and that is fine but please state that that is your issue and don't try to frame it as something else. What I don't get is how everybody places the blame at cloudflares feet. It is my choice as a host to use cloudflare and to use their protection features.

Re:Facebook was used as a proxy by web scraping bots by cblconfederate on Thursday November 05, 2020 @07:11PM (#25003528)
i m not sure if you re being serious

CF is so widespread that it breaks a significant part of the web for simple things like getting the page title. That's all. The End.

Re:Facebook was used as a proxy by web scraping bots by seniorgarcia on Thursday November 05, 2020 @07:20PM (#25003589)
'I' can get the page title though. That's all. The End. I don't care about your crawler. Or your ability to post the link to my site to twitter/fb and if I did maybe I'd revise my cloudflare settings.