With the rise of training, retrieval, and agentic AI bots, websites are starting to block those bots to prevent them from running amok inside their systems.
I think this is a good idea to some degree - governance for these bots will be critical - but I've found myself caught in the crosshairs in a weird way.
Linux Must Be Evil
I've been running Linux as my main OS for several years, and recently I've been blocked when visiting high profile sites with Firefox.
Amazon shows a challenge page:
Marriott just stops me entirely:
Hilariously enough, while trying to visit the /r/linuxmint subreddit, I'm being blocked:
This is annoying and quite obviously a lazy measure by website admins.
What's Probably Happening
Web admins lately: "All bots come from Linux! We must block Linux traffic!"
Easy? Yes. Lazy way to do it? Also yes.
Digging into this a little more, two things are true:
- Linux powers the overwhelming majority of crawler platforms
- The user agent string for most crawlers is strikingly similar to Firefox
Yep, Mozilla/5.0 and Gecko both appear and look just like Firefox (most user agent strings do, actually), but when combined with Linux as the OS, it starts to raise suspicion with many WAFs.
The end result? Because I use Linux and Firefox in my daily life, the web is starting to become less useful for me on the desktop. And with more user-facing agent platforms (such as Muse) trying to act more human on the internet, we Linux lovers are going to be caught in the crosshairs of a very ugly cat-and-mouse game.