On January 1, I received a bill from my web hosting provider for a bandwidth overage for $155. I’ve never had this happen before. For comparison, I pay about $400/year for the hosting service, and usually the limitation is disk space.

Turns out, on December 17, my bandwidth usage jumped dramatically - see the attached graph.

I run a few different sites, but tech support was able to help me narrow it down to one site. This is a hobbyist site, with a small phpBB forum, for a very specific model of motorhome that hasn’t been built in 25 years. This is NOT a high traffic site; we might get a new post once a week…when it’s busy. I run it on my own dime; there are no ads, no donation links, etc.

Tech support found that AI bots were crawling the site repeatedly. In particular, OpenAI’s bot was hitting it extremely hard.

Here’s an example: There are about 1,500 attachments to posts (mostly images), totaling about 1.5 GB on the disc. None of these are huge; a few are into the 3-4 megabyte range, probably larger than necessary, but not outrageously large either. The bot pulled 1.5 terabytes on just those pictures. It kept pulling the same pictures repeatedly and only stopped because I locked the site down. This is insane behavior.

I locked down the pictures so you had to be logged in to see them, but the attack continued. This morning I took the site offline to stop the deluge.

My provider recommended implementing Cloudflare, which initially irritated me, until I realized there was a free tier. Cloudflare can block bots, apparently. I’ll re-enable the site in a few days after the dust settles.

I contacted OpenAI, arguing with their bot on the site, demanding the bug that caused this be fixed. The bot suggested things like “robots.txt”, which I did, but…come on, the bot shouldn’t be doing that, and I shouldn’t be on the hook to fix their mistake. It’s clearly a bug. Eventually the bot gave up talking to me, and an apparent human emailed me with the same info. I replied, trying to tell them that their bot has a bug to cause this. I doubt they care, though.

I also asked for their billing address, so I can send them a bill for the $155 and my consulting fee time. I know it’s unlikely I’ll ever see a dime. Fortunately my provider said they’d waive the fee as a courtesy, as long as I addressed the issue, but if OpenAI does end up coming through, I’ll tell my provider not to waive it. OpenAI is responsible for this and should pay for it.

This incident reinforces all of my beliefs about AI: Use everyone else’s resources and take no responsibility for it.

  • limelight79@lemmy.worldOP
    11·
    3 days ago

    Yeah that’s part of why I was so frustrated with the answer from OpenAI about it. I don’t think I mentioned it in the writeup, but I actually did modify robots.txt on Jan 1 to block OpenAI’s bot, and it didn’t stop. In fairness, there’s probably some delay before it re-reads the file, but who knows how long it would have taken for the bot to re-read it and stop flooding the site (assuming it obeys at all) - and it still would have been sucking data until that point.

    I also didn’t mention that the support bot gave me the wrong URL for the robots.txt info on their site. I pointed it out and it gave me the correct link. So, it HAD the correct link and still gave me the wrong one! Supporters say, “Oh, yeah, you have to point out its errors!” Why the fuck would I want to argue with it? Also, I’m asking questions because I don’t know the answer! If I knew the correct answer, why would I be asking?

    In the abstract, I see the possibilities of AI. I get what they’re trying to do, and I think there may be some value to AI in the future for some applications. But right now they’re shoveling shit at all of us and ripping content creators off.