Automated newsletter subscribers...

[uplinked]

Hi all

I have a subscription button on my blog... (http://uplinked.x10hosting.com). However, I have noticed that I get fake subscriptions because they look automatically generated and all have the same pattern:

cnAsGiIOYUvCEiP <- usernames (generated obviously)
namexx@hotmail.com <- email (name and two figures at hotmail dot com)

Any suggestions of how to sort such advertising out? I do not want to lose "ordinary" subscribers... even if automated, just the fake ones.

Thankful for help...

[essellar]

You should start by assuming that the intent here is not to get newsletters sent to Hotmail -- it's probably a really badly-written bot that sees a form with name and email address fields and assumes that it can somehow post comment spam to your site. Sending another form back to the user asking for confirmation (always good anyway, as proof of opt-in to the newsletter in accordance with CAN-SPAM and similar legislation elsewhere) should get around the problem -- most spambots aren't going to wait for a response, click on a checkbox and submit confirmation.

[uplinked]

Gee... thanks. That was simpler than I thought. I was expecting dictionary-validation etc...

Can one see if the entered email address is valid? I know that is possible... But how reliable is this method?

I will definitively implement a checkbox. =)

[essellar]

Without making it a two-step affair, where the user has to click on a link from an email you send them, there's no good way to validate an email address. A lot of large email providers will not return a 550 error for an invalid user if you try sending a test mail (it just helps the spam generators to refine their lists and get more efficient).

The problem with trying to do this through email confirmation is that if you get hammered by a bot using fake addresses at real domains, you end up sending a lot of mail to fake addresses at real domains, and that may just mark you as a spamhaus yourself. (Having left a SMTP relay open for a few hours once back in the caveman days, I have to tell you that it's really, really hard to get yourself off of a spam list once you're on one.)

It's easy for a human or a sophisticated bot to get around this simple system. Human "mechanical Turks" are responsible for a rather large amount of the comment spam in this world, and being humans they can defeat any CAPTCHA you might think of using. A sophisticated bot wouldn't really be trying your newsletter subscription form -- it would see that there are no fields it can use to deliver its payload, and try elsewhere on your site. There are ways of strengthening this simple obstacle (using time-limited value tokens unique to the user/session), but the simple approach should do it for the class of "attack" you're facing. (If not, then we can always make your back-end ridiculously complicated later.)

[uplinked]

Yes, I agree. I will try the simple solution first.

This bot dummy is hammering my domain, more precisely my "top" domain (uplinked.x10hosting.com). My subdomains, where I developed this version of the website, did not get bothered. But now, it attacked the new version of my website and I really want to get rid of it. =)

I have mailing solutions, but I agree that sending mails out in the thin air is a waste of time that I do not want to embark on...

Thank you very much for the advice!

-------

HOW MANY NEWSLETTERS CAN I SEND TO "FAKE" EMAIL-ADDRESSES BTW? Assuming that I believe they are real so to say before I GET BANNED/BLACKLISTED?

[essellar]

It's not a particularly small number, and it varies. There's no way for the system (usually) to distinguish between an account that has never existed and one that's been deactivated and then deleted, so if you're sending something weekly/monthly to the same few dead addresses and getting no (or nearly no) spam reports from your legitimate recipients, you should be okay. The system (if it's not returning a 550) will just assume that and old user forgot to cancel/change their subscription. It's just the situation where you're getting hammered by a dumb bot that's using fake addresses, and in turn hammering a mail server with fake addresses in response, that you'd trigger the weirdness detector.

If the email address looks legit, you're probably in good shape, even if the address doesn't exist. If the address belongs to somebody who didn't subscribe, and has an unsubscribe link, you're still okay, as long as the link is clear and the unsubscribe policy statement is worded to make everything sound like "it's not you, it's me".

Unfortunately, it's almost impossible to make something like this bulletproof without also making it onerous on your users. If it comes down to making it hard to subscribe (and RSS isn't a realistic alternative -- and it isn't always), then you're sort of stuck being highly apologetic as you make the users jump through hoops. If your content is valuable enough to your readers, they'll follow, but it does kind of make the relationship a little less casual.

[uplinked]

Okay, understood.

Well, I have unsubscribe links and I use them. So I shouldn't have a problem there.

I am still torn between the choice of a "less casual" method and sending mails to "fake" addresses with unsubscribe links attached... Which IS to prefer actually?

I did not really follow you on the RSS, I am planning on implementing that solution some time in the future... Is this to prefer over mailing lists? I still need to mail out internal info for example...

[essellar]

That's really what I meant by RSS not always being an answer. Before RSS, a newsletter or mailing list was the only way we had to keep users up to date without having them actively visit the site. RSS solves one part of the problem -- people can jsut sit back and let the site come to them, so to speak. But it is limited in what it can provide: you can't do personalised content or subscriber-only specials easily.

I really can't offer any more advice on the mechanism (except implementation, if you decide to go the "less casual" route). Any barrier you put between you and your users is going to cause some people to think twice, so it's a real balancing act between getting traction and creating higher security. Grandfathering in your current subscribers isn't hard -- you can just give them a link in your newsletter, along with an explanation of why you need them to click the link. If they're your main source of new users, then new subscribers will mostly know what to expect. If, on the other hand, you are counting on people "discovering" your site, a multi-step process involving a form, another form to say "I'm probably human", an email confirmation link and, no doubt, the feeling that any minute now you'll be asked to submit notarized copies of your driver's license and passport by registered mail might seem like a bit much for an email newsletter. Users, even people who have the patience of Job in the real world, are an impatient lot.

Hmmm... a thought occurs to me (and if you knew how infrequent they are...). Can you include a confirmation link for not-verified subscribers in the first few emails? You can phrase things so that it looks like you're doing them a favour by stopping automatically if they're not satisfied after, say, three months. It's all a matter of perception -- if they think it's a hoop they have to jump through, they'll resent it, but if they think you've got their best interests at heart, they'll love you for it.

[theone48]

This is one interesting thread. Anyone who thinks essellar is stupid as some bloak said earlier today ought to read this. I can't give him more rep right now, but if I could, I would. This guy knows more than meets the eye.

[uplinked]

LOL =) Great suggestions in deed!

I mean, I have some severe issues of distinguishing between the automatically generated user names which look like English first- and last names, with the same pattern of fake emails and the ones that don't even bother to use a dictionary for the subscriber name and fake emails...

You were right about the attempt of bots posting their advertising automatically, because that is the first main reason why I developed my site and took away the cms I was using (which had a FORUM with guest access!!!)!

So, I do agree, Essellar, your idea is on the roll! (",) Thank you again!