Tech is political: The people under attack in Palestine 🇵🇸, Iran 🇮🇷, and Lebanon 🇱🇧 are people like us. They’re our brothers and sisters, too. Read up on their history, scrutinize what you’re told, and demand that they be respected. Hide

Frontend Dogma

“bots” News Archive

Definition, related topics, and tag feed

Definition · Supertopics: crawling · Subtopics: bot-detection (non-exhaustive) · “bots” RSS feed (per email)

Entry (Sources) and Additional TopicsDate#
The Free and Open Web Is Under Attack at the IETF (eff)14
open-web, ai, standards
Obituary: Farewell to robots.txt (1994–2025) (hei)13
robotstxt, search, internet-archive, ai
Wasted Open Source Efforts (jan)12
foss, community, collaboration
Perplexity Is Using Stealth, Undeclared Crawlers to Evade Website No-Crawl Directives11
ai, crawling
The Open-Source Software Saving the Internet From AI Bot Scrapers (ema/404)10
ai, scraping, tooling
A Short History of Web Bots and Bot Detection Techniques (ole)9
bot-detection, techniques, fingerprinting, traffic, history
Generic LLM Chatbot Attestation (aar)8
ai, processes
Migrating From Netlify to Cloudflare for AI Bot Protection (sia)7
migrating, netlify, cloudflare, ai, security
How to Stop Form Spam Bots With Honeypot Fields6
spam, forms
Discovering Web Automation and Scraping (gli)5
scraping, automation, tooling
Block the Bots That Feed “AI” Models by Scraping Your Website (cla)4
robotstxt, scraping, ai
Making a Twitter Bot in the Year 2022 With Node.js3
nodejs, twitter+x
How to Develop a Chat Bot With Node.js (sma)2
how-tos, nodejs, javascript, functionality
The Rise of Web Bots and Fall in Human Traffic (cra)1
web, scraping, spam, traffic, metrics