Websites are Blocking the Wrong AI Scrapers (Because AI Companies Keep Making New Ones).

  • 2024-08-07 08:00:00
  • 404 Media

Nowadays, it's not enough protecting ourselves from cybercriminals intent on stealing private information, we must also pay close attention to so-called 'AI scrapers': computer programmes created to collect data from websites and then use them in the training of generative AI models.

Following the impressive rise of this technology, several websites have fallen victim to scraping processes carried out by the AI company Anthropic, via the two bots 'ANTHROPIC-AI' and 'CLAUDE-WEB'. To deal with this threat, the companies involved have taken careful measures - however, Anthropic has also changed its game.

The latest update of the company's bot is called 'ClaudeBot' and, considering that the companies involved have moved to protect themselves against its earlier versions, the new centralised agent is able to breach their defences. Nevertheless, Anthropic recently stated that "ClaudeBot has been configured to respect all previously set guidelines; we are trying to respect the preferences of website owners."