News for hackers. Stuff that matters.
Re:A web scraping CLI made for AI that is idempotent by mrdw on Saturday August 17, 2024 @01:11PM (#41276020)
why so dependent on azure?

"Decoupled architecture with Azure Queue Storage"

"Scraped content is stored in Azure Blob Storage"

"Indexed content is semantically searchable with Azure AI Search"

Re:A web scraping CLI made for AI that is idempotent by clemlesne on Saturday August 17, 2024 @02:43PM (#41276862)
Well it solve basics problems like queuing and blob storage. For example, to achieve the same features as Queue Storage, you should use RabbitMQ or similar: in enterprise environment, it means multiple instances in high availability, maintenance, people to deploy it reproductibly…