ChrisHardie

The usual case I run in to is that a site will block requests with User-Agent header strings that don't at least try to look like a regular browser, or that appear on some list of known bots/automation tools. (If they are using Cloudflare, this is a very easy state for a site to get in to.) I'm not sure if GH actions lets you customize the user agent in the spot you're hitting the issue, but that's where I'd start.

Once I created the structure to support lots of different kinds of scraping-to-feed conversions, it’s usually fast to add a new target site in to the mix. There are definitely exceptions, and definitely the occasional maintenance when someone updates their CSS.

Articles should not have images that are misleading or confusing, but I do understand why most news articles have something when it comes to imagery. Most news website designs are optimized for at least one image per article, and social media sharing almost requires it if you expect any kind of engagement at all. But it’s not a problem the media world should have to solve at the expense of the reader experience. (Disclosure: I’m a journalist and digital editor who spends way more time than I’d like trying to pick the least harmful stock or file image in the cases where we just don’t have a good image for a story.)

I wrote about trying to do this for Facebook page events as an example. Code sample included:

https://chrishardie.com/2019/10/unlocking-community-events-f...

I believe there is a middle ground between hate speech being "allowed-but-not-thriving" and censorship, and I think we can do a better job of finding it than we do now.

Hi, original article author here. Thanks for the great comments and feedback. I just appended my post to say:

I probably should have used a question mark in my post title instead of an asterisk. I don't claim to know the future and I get that I'm talking about an industry and an area of tech that's unpredictable. I've no need to be "right" about this and am just glad to have started some conversations.

I probably also should have referred to something like "generalist retail website development for the masses" instead of just "website development." One part of my current company that is still going strong is a business unit that's performing hundreds of hours per month of custom software development, database management and consulting for a client pioneering a unique problem space where no off-the-shelf tool is going to do the trick. We just had challenges replicating that scale of project within our company, but that's been more about roadblocks to growing the way we wanted to than it has to do with the nature of the industry overall. I fully agree with the commenters who note that at the higher end of client and project requirements when it comes to creativity and complexity, there's still a great need for professional service providers with a broad range of experience and deep knowledge. As I tried to say in my post, I think the folks working with bigger budget clients and projects will have plenty of work for the foreseeable future, even if under a different name from "website developers."

Thanks for reading.