Spamtastic

[altivo]

As moderator of a PHP-based writing forum, I am finding that more and more of my time is being spent trying to head off spammers. Many of these people create nonsensical accounts with no intention of participating or even reading the forum content. Instead they fill the account profile with spam links and abandon it. So I get to delete these spurious accounts, in increasing numbers all the time.

A few do go ahead and post junk messages containing phishing attempts or spurious links to phony sites selling fake drugs or forged designer items or whatever. These I get to delete, and ban the poster who doesn't care because next time he'll come back and create a new account anyway.

Yes, we have a captcha and require e-mail validation. They manage to get past both.

They don't quit, even after being banned. One IP address that I blocked last October has since tried to reconnect or re-register an account no less than 1,022 times as of today.

I realize that this is the net equivalent of random graffiti in urban areas, but I find that equally reprehensible and pointless. What is the matter with human society that it produces such stupid, moronic, and anti-social behavior in such quantities?

Complaining to the offending sites' providers is useless. Most of them ignore you, or deny that there is anything going on, or even tell you flat out "We don't care as long as they pay their bill." I've been tracking offending sites and addresses for years, and note that certain nations seem to account for the bulk of it. At the present time, Russia and Poland seem to originate more than half of the spam I see, though in the past most of it came from France, Italy, and Brazil. Before that, it was Korea, China, and other southeast Asian countries. E-mail addresses used for validation are often of the free type created on gmail, hotmail, or similar networks, so blocking an e-mail domain isn't a good idea because there are likely legitimate users on those as well. The latest increase in spam attacks has increased the number of spurious accounts generated by probably a factor of ten daily. This amounts to a DOS attack and makes me wonder if it is intentionally being directed against us. Other forums running the same software don't seem to have this problem.

Comments

[farthing]

I had the same trouble when I was running my little ten user forum thingie in the past. Webcrawlers search websites for that particular brand of forum software and automatically add new users if there's no other preventive measures in place. I considered adding captcha, but instead just renamed the file that was used for registration. Don't remember if I hid it from robots.txt too, in case they were using Google or other proper search engines.

I guess the trouble is that spamming is "outsourced" these days. Bots grab captchas, give them to a group of people making it a game who gets the most solved, with some sort of reward. My sneaky solution wouldn't probably help with that anymore...

[altivo]

This one is a bit too large and active for that solution. About 200 valid accounts, a couple dozen of them are pretty active. The owner of the site doesn't want to make registration too difficult, unfortunately.

A new version of the php software has just been released, but I'm reluctant to jump at a major version change until someone else has tested it out in production. We could require approval of all new accounts, but that would mean e-mails sent to four moderators every time someone registers, and 99% of all registrations are junk spammers.

I'm puzzled by the fact that this same software package (SMF) is used by some really large forums, customer suppert sites and such. I'm becoming paranoid that our site is being singled out for a sort of DOS attack by spam just because it's furry oriented.

[farthing]

Change would be practically invisible to regular users, but would be a pain to keep up every time the forum software updates, since one would have to fiddle with the insides to alter the internal links accordingly.

I doubt it's targeted to just this particular forum, there's more effective ways to do a denial of service attack. And bots can get practically everywhere.
I once stumbled on a video about a forum spam tool. It used Google to search some particular string in the forum software, tried automatic registration, if there was captcha/error it put the site on hold waiting for user interaction, and once it got in, it started adding spam to random subforums.
Basically the user just had to go fill captchas, and the software did the rest. It even got through the usual admin sneakiness, like renamed form labels, ignoring CSS-hidden input boxes, and it might have even figured out the registration page rename. I don't remember the numbers, but it managed to spam hundreds of sites in an hour or so.

[altivo]

I think we've got it plugged up for now. We installed a mod that checks new registrations against Stop Forum Spam's database of reported spammers. If their e-mail address or IP address matches up, they get blocked.

It seems to work amazingly well. All has been quiet since early this morning, with not one new spam account registered though I've seen quite a number of them trying. The database gives real evidence against each one that's blocked, including lists of usernames and e-mail addresses used and reported, and even samples of the actual spam they posted.

[farthing]

Ah, handy! Hopefully that'll keep the spammers away...

[baphnedia]

Oh! Depending on the forums you use, I only do account validation and a custom question - and then have a moderator queue such that new folks need 5 approved posts before they can post stuff that is visible outside the moderator team. The ones that get into trouble get deactivated (which acts as a passive email and username ban). Though, I'm sure I accidentally approve a post here and there. Now to figure out how to get actual activity... :)

http://forum.paradicegames.net/

[altivo]

We're using SMF. I wish it had an option for requiring a first post to be approved before it becomes visible, but it doesn't have that feature. The main problem though is the registrations themselves. We only get one or two genuine new users each month, but I'm getting as many as 50 spambot registrations a day. It's more like a backhanded denial of service attack than any kind of productive spamming, as 90% of these accounts never post anything after they register.

[hrrunka]

Yeah, a forum I help run has had this problem. We've just disabled registrations and asked interested folks to request a manual registration by an alternative route....

[altivo]

We may have a solution. The site owner installed a patch (we're running SMF 1.1.14) this morning. The patch code checks registration attempts against "Stop Forum Spam" and blocks any spammers recognized by e-mail address or IP address.

By this time of the morning I'd expect to have a dozen junk accounts to delete, but there is not even one. There are eight "guests" on the system at the moment, two of them trying to register. Stop Forum Spam has flagged every one of them as a spammer, and on request will show me their e-mail address and country of origin, as well as a history of their spamming activities, the user names they've tried, and even samples of the spam they've posted. It's very impressive and it seems to work.

[calydor]

A forum I frequent got rid of a lot of the bot signups by taking a look at what they all had in common - in the case of this forum, they all used the default time zone. So, the default time zone was set to an area with just a small bunch of islands in it somewhere in the Pacific, and anyone registering with that set was simply denied. According to the owner it killed off the vast majority of fake signups.

[altivo]

Automating something like that requires writing custom code, which then makes future upgrades difficult or impossible.

There are common elements to spambot registrations, but they shift over time. For a while they were generating user names with random characters inserted, like "wJohnSmithq" or "aFrancineJohnsenz" for instance. Currently I'm seeing bizarre gmail addresses with lots of extra periods in them, like "a..qui.c.k.sol..u.tion@gmail.com". This sort of thing is easy for a human to spot, and I delete those accounts immediately. But you'd need significant artificial intelligence to get computer code to find them by itself.

[calydor]

Well, the obvious answer here is to write a small script that flags any registering email address with more than four periods. That would allow addresses like firstname.middleinitial.lastname@example.co.uk - any more than that is cause for suspicion.

I will admit to not having any idea how hard future updates would be, but wouldn't it be possible to just keep a couple of changes written down in a basic text document which you then re-insert after upgrades?

[altivo]

Oh, I imagine some would consider it "no problem" to maintain special fixes across releases. My perspective after 30+ years of doing this stuff, though, is rather different. I don't think I should have to do that.

If this were my full time job, sure, it would be justified. But as something that is supposed to take up just a few minutes a day and shouldn't require me to understand the entire mass of php spaghetti on which forums run, it doesn't rate that level of effort.

[schnee]

I realize that this is the net equivalent of random graffiti in urban areas, but I find that equally reprehensible and pointless. What is the matter with human society that it produces such stupid, moronic, and anti-social behavior in such quantities?

Greed. Spam pays, and it's not as if these accounts etc. are created by actual people sitting in front of a computer right there and then — it's just bots.

I see these on a site that I help out with moderating, too; our solution is to require manual activation for all new accounts. But then, it's a semi-public site that not everyone can just sign up for, anyway.

[altivo]

It's really hard for me to believe that burying an advertisement for fake designer sunglasses in the user profiles of an obscure forum with only 200 users could possibly be worth the trouble. I know there are a lot of gullible and stupid users on the net (the continued existence of Facebook proves that) but the odds an getting any significant return from these activities are diminishingly small. I think if anyone is making money, it's some guy who sells spambot software to suckers who believe it will work.

[schnee]

You'd think it wouldn't make sense, yes, but consider this:

First, it's basically free. All the work is done automatically; all the spammer needs to do is click a button, if even that.

Second, SOME people somewhere are probably going to be stupid/naive/inexperienced enough to fall for it.

Third, it may also be about bolstering search engine ranks etc. or otherwise being picked up by other automated systems.

Fourth, I've heard the theory that spam itself is a scam these days: that quite a bit of spam is sent by people who bought spamming kits that were sold to them as a quick way to make money doing nothing. In this scenario, spam doesn't even have to be profitable in any way: it's enough if people believe it is and buy the spamming kits. (And it's not as if they're gonna go to the police when they realize it doesn't work.)

Fifth, quite a few links to fake designer sunglasses sites etc. are actually just intended to get visitors infected with malware, not actually sell anything. And with many people running insecure, unpatched browsers, just visiting a site may be enough to become part of a botnet (which then in turn gets rented out for real money — ironically, among other things, to send more spam).

All the above makes me think that spam is here to stay until and unless the people behind it (organized crime, really) are brought to justice.

[altivo]

I think your fifth expanation is probably what accounts for a lot of it.

I also think that the truth is, spam doesn't have to be so pervasive. Internet service corporations have been very resistant to implementing the measures they should take to reduce it. E-mail spam would be greatly reduced, for instance, if every smtp relay would just implement the black hole list. But the truth is, those providers are making money by selling access to the spammers. And more money by selling extra "anti-spam" services to their customers. So they dodge any attempt to hold them accountable for aiding and abetting the spammers.

Spam that is created for the purpose of raising the significance of search engine listings can be blamed on the search engine providers, and in particular Google. Their method of deciding who gets listed first on a page of results is highly suspect, and especially so if it is influenced by this sort of junk violation of private systems.

[schnee]

Perhaps; I'm not at all convinced that blacklists are useful, though. acme.com makes a good case that they're outright harmful, and judging from personal experience, I can say that quite a few seem(ed) like little more than shakedowns. Even respectable (or at least respected) lists like Spamhaus have their problems and engage in dubious behavior.

But then, they only really apply to email spam, anyway, so forum spam isn't gonna be affected one way or another.

[altivo]

Some blacklists seem reliable to me, and we've just implemented one for this particular problem that appears to be working really well. See my reply to hrrunka up above for details.

There's more that ISPs could do, of course. Accepting SMTP mail from any address that fails a reverse lookup or isn't on a white list would also be appropriate. Any ISP who accepts SMTP from its own users can, of course, validate those users and their connection, and forward their mail from its own registered server. This is neither costly nor unreliable. But they don't do it. Why?

[schnee]

I'm not an expert, but I think it's because SMTP doesn't work that way. The protocol used between servers and between end users and servers is the same.

As such, in order for their users to receive mail from anyone outside the ISP's own network at all, the ISP necessarily has to allow third parties to connect to their mail servers and send mail. In theory, one could imagine a whitelist of authorized mail servers, but questions that come to mind immediately.

How would this work in practice? How would you keep this list up to date, especially considering that even a turnaround time on the order of a few hours would not be acceptable? How would you deal with individual people running their own mail servers? How would you prevent false positives (servers listed that shouldn't be), and how would you handle them if they appeared? How would you prevent false negatives, and how would you handle them if they appeared? Who would maintain the list, anyway? How would they get paid — who'd cover the costs, the time etc.? Would you have to pay to access the list? Who would decide whether a server is "genuine"? How would server operators be able to appeal unfair decisions? What kind of oversight would there be?

All the above questions, mind you, also apply to blacklists — and they are why blacklists are problematic in practice even if they sound like a good idea in theory.

[altivo]

Every SMTP server has the ability (or can be modified) to check the identity of the node that is attempting to pass mail into its system. If the remote node is recognizable as your own user, fine, you accept the mail. If the remote node is not in your own IP address range, then you do some other checking. Does their reverse lookup match the name they tried to give you on the HELO command? (This one will kill a lot of spammers dead, because they try to impersonate well-known sites like gmail or yahoo.) If it doesn't match, then you check a bit more. Are they on your own blacklist or whitelist? Are they on the black hole list? Just doing that much would stop more than half the spam e-mail that is currently being passed through the web.

The SMTP server has the IP address of the remote node, and the name it claims for itself on the HELO command. Just checking those for validation would go a long way toward curbing spam. While it is possible to set up PTR records in the DNS system that lie about your identity, it isn't easy and would likely involve some illegal hacking somewhere other than the SMTP system itself. Does the IP address fall in the expected range for the domain the sender claims? If not, you may accept the mail but hold it in quarantine until someone can inspect it or make sure the sending node is legit. Such circumstances should be rare. Most mail transport that isn't from an ISP's own customers should come from well-identified nodes that are forwarding mail for their own customers.

[schnee]

All of that is fine and dandy, and ISPs are doing it already. But if you implement measures that weed out the stupid spammers, don't be surprised if only the smart ones remain. It's much like antibiotics.

If not, you may accept the mail but hold it in quarantine until someone can inspect it or make sure the sending node is legit.

This is unrealistic for any organization with a significant number of mailboxes, and what's more, it's legally problematic at best, unless it's the intended recipient doing the inspection.

If it IS the intended recipient who's doing it, this is precisely what is happening already when an ISP puts suspected spam mails in your "Spam" folder.

[altivo]

I disagree strongly. It isn't necessary to inspect content of the email, only the header trace. This violates nothing. And no, ISPs are not doing this job, or at least, many of them are not doing it well enough. In particular, the big relays don't always do it as well as they should. And I really do believe that is because they don't want to alienate some of their big customers, the commercial spam generators.

But this technique should in fact block spam mail generated by botnets, which has become a much greater problem in recent times. Gmail does an excellent job of blocking spam, and that alone is my main reason for using them. None of my other e-mail accounts are as well protected.

My personal experiences from when I was administrator of an academic e-mail system lead me to the conclusion that there are a lot of shady ISPs and even backbone relays who just aren't interested in taking any responsibility for what goes through their systems. All they care about is making money, and they don't care where it comes from.

[schnee]

Of course there's lots of shady ISPs. Whoever disputed that? What I'm saying is that not all ISPs are shady, which appears to be what you're implying.

[altivo]

No, I'm not saying all of them are irresponsible. In fact, I singled out Google as one that does a good job in this respect.

But there are too many irresponsible operators, some in high level positions in the net hierarchy. That's what needs to change in order to fix issues like spam and malware more effectively. No fix will be perfect, but at the present time there just isn't enough being done.

[duskwuff]

That's actually not far from the truth. I've come to the conclusion that the vast majority of forum spam is likely generated by no more than three to five different pieces of software, running through a ton of proxies.

[altivo]

Yes, that does seem to be the case. I thought someone had figured out an automated way to bypass captcha, but looking at the pattern of behavior here, I don't think that's what was happening. The bot software gets to the point where it needs to read the captcha, and holds there waiting for a human to supply the needed transcription. Sometimes this hold lasts an hour or more. That tactic should be beatable by timing out any attempt to fill out a registration form that takes too long, say, more than 10 or 15 minutes.

It's been 48 hours now since we applied the fix that checks with "Stop Forum Spam" and so far the score is 100%: no new spam accounts have been created, but I've tested twice and found that I could create a new account without any trouble. We're only blocking "known" spammers, but so far none that couldn't be recognized have tried to register.