The stats code introduced yesterday had a bug which only triggered about 24 hours after it had launched. It kicked in for different blogs at different times, but the result was that starting sometime last night you probably started to see really obvious spam getting past your Akismet filters.
All of our systems were up and running at the time, and spam was being correctly identified, which is why our monitoring systems didn’t notify us of the problem. However, there was extra junk being included with the response which broke the API call, which expects only “true” or “false”.
I’m really sorry about this, when things are working smoothly it’s easy to forget how much vile junk is actually being blocked day to day. I’m going to go through my blogs now and double-check that no spam actually got published, and I’d suggest other folks do the same. The “mass edit” mode under “Comments” should be useful for WordPress users.
I’ve also taken steps to make sure on the Akismet side this kind of error can’t happen again.
Well, thanks for letting us know what happened. I had a weekend of spam purging unlike anything I’d ever seen since… well, since before I’d invoked Akismet.
Seriously, I think it’s terrific that you’ve let us know what happened. I was at first wondering, “hey, how’d that get past — that one is so obvious!” And then when it happened for the 30th time I was like, “What the hell is going on over at Akismet HQ?!”
It just goes to show how much I have come to depend on your perfect, transparent performance. Sorry I freaked out for a couple of minutes there. But I never freaked out enough to complain on the Akismet forums or whatever you have.
Actually, it’s a tribute to Akismet that I have no idea where to complain.
Thanks for the open communication, as well as for fixing the spam-krieg.