# A solution to the spam problem



## Kzach (Apr 13, 2012)

So, with the new board you've promised to import all the old threads and users.

But do you really have to?

Sometimes to move forward, you have to let go of the past. Would a lot of people complain? Sure, but let's face it, they'd get over it and their complaints would be tears in the rain anyway.

The proposal I'm making is that you change one very simple aspect of the boards. You require that accounts be made using ONLY ISP email addresses. In other words, no gmail, hotmail, etc. This alone DRASTICALLY reduces your spambot rate. It doesn't eliminate it, mind you, but if the current rate is 100%, you'd be cutting it by 99%.

At least, that's what people tell me. I got the idea from another forum I go to where I am yet to see a spambot post at all. It also severely cuts down on people creating sock-puppets and mock accounts. Also, bandwidth wouldn't be wasted on all those nuisance accounts and spambots.

So sure, everyone would have to create new accounts. And sure, they'd all lose their precious XP and post counts and join dates. But everyone, at some point, has to make sacrifices for the betterment of a society. People WILL get over it. And you'll have a far more accurate picture of your actual users.


----------



## Morrus (Apr 13, 2012)

I'd be screwed. I don't have an ISP email address!


----------



## Viking Bastard (Apr 13, 2012)

Does anyone use ISP email addresses anymore? (It's just not very smart to rely on.)


----------



## Kzach (Apr 13, 2012)

Morrus said:


> I'd be screwed. I don't have an ISP email address!



Surely you do. I mean, isn't it a bare minimum basic requirement? After all, how else would you communicate with your ISP? How do you receive bills or warning notices or service interruption notices, etc.?



Viking Bastard said:


> Does anyone use ISP email addresses anymore? (It's just not very smart to rely on.)




Hell no. I don't use it for anything other than this one forum that requires it. Which is, again, why it's such a good idea. I'd be willing to bet a good 10% (at least) of the accounts here are dummy/spam accounts. Imagine all the bandwidth those spambots use alone!

Anyone can create a gmail account (I have about seven...), but very few people have the know-how to get around having the requirement of an ISP email account in order to register a forum account.


----------



## Viking Bastard (Apr 13, 2012)

Kzach said:


> Surely you do. I mean, isn't it a bare minimum basic requirement? After all, how else would you communicate with your ISP? How do you receive bills or warning notices or service interruption notices, etc.?.




My fiancée usually gets an rare SMS about the immediate important stuff; then there's the ISP's online service panel for the rest and the bills go directly to my online bank.

I don't get a email address from my ISP, although I can get one if I request it.



> Hell no. I don't use it for anything other than this one forum that requires it. Which is, again, why it's such a good idea. I'd be willing to bet a good 10% (at least) of the accounts here are dummy/spam accounts. Imagine all the bandwidth those spambots use alone!
> 
> Anyone can create a gmail account (I have about seven...), but very few people have the know-how to get around having the requirement of an ISP email account in order to register a forum account.




But how would you restrict it to an ISP account? I mean, I've seen such restrictions for signup (though, not for years), but what they effectively were "Use any email account, except for this list of popular free-signup email services". Someone operating a spambot can probably set up his own POP3 server.

If it's actually a list of approved "real ISPs", how can you make sure my Icelandic ISP makes it to the list.

What if you just don't buy any service from any ISP?


----------



## Kzach (Apr 13, 2012)

Viking Bastard said:


> If it's actually a list of approved "real ISPs", how can you make sure my Icelandic ISP makes it to the list.




*shrug*

I just reported some spam and thought to myself, "Geez, they have a lot of spam on this site... that reminds me of this other forum I go to where they have no spam at all... maybe I should post that site's methods as a suggestion!"

I am not a web developer.


----------



## Piratecat (Apr 13, 2012)

I'd be screwed too. All gmail, all the time.

Spam is actually really easy to deal with. We get 1-3 spammers a day. 3 clicks and everything they've posted is gone, and they're IP banned. I don't mind the 3 minutes it takes. Thank you for reporting the spam, by the way - it's definitely appreciated.


----------



## chriton227 (Apr 13, 2012)

I'm all for reducing spam, but it is a very difficult problem to solve.  While filtering to only ISP email accounts is an idea worth considering, it could easily become a nightmare to maintain.  There are two possible approaches to restricting it to ISP only email:

1) Blacklist non-ISP "free" email providers.  This is probably the simplest to implement, but trivial to work around.  It cost me less than $20 to get started with my site, which included a domain name, hosting, and more email addresses than I could ever want, and I'm sure there are much cheaper options available.  You could always add more sites to the blacklist as you see spammers coming from those addresses, but that could be a time consuming job to keep up with.  You would also have to keep an eye out for new free providers or alias services as they become available. 

2) Whitelist ISP email providers.   Just building a list of the existing providers globally would be a monumental task.  You would have to accommodate obsolete ISPs (my main ISP email account is from a provider that was bought out over a decade ago, and my old account is grandfathered in).  You will have cases where the ISP service doesn't provide enough email addresses for everyone in a family to have one and the family doesn't want to go with shared email accounts.  Some ISPs don't even have the option of getting an email account (my 3g Mifi is internet access through a cell phone provider that doesn't offer email).  I also know people that have business class internet connections for one reason or another and these frequently don't come with email addresses either.  And finally you have people that don't have ISPs at all, they access the internet through other available access points like restaurant hot-spots, school, work, or libraries.

Sites like StackExchange have a much stricter process that new accounts have to go through before they can really start posting, and it is driven by feedback from other users.  Perhaps there is something similar that can be done here like adding a "report spammer" button that becomes available to registered users of a certain rep level, and if a poster accumulates a certain number of reports in a given time frame posts from the spammer account would be hidden and the account flagged for moderator review.  There would need to be some audit trail/accountability built in to deal with abuse.


----------



## Piratecat (Apr 13, 2012)

We could also blacklist certain IP ranges - for instance, much of our recent spam is from India, so they must have opened up a new shop there - but that leads to some unsavory blocking of legitimate accounts.


----------



## Umbran (Apr 13, 2012)

chriton227 said:


> Sites like StackExchange have a much stricter process that new accounts have to go through before they can really start posting, and it is driven by feedback from other users.




It isn't like most of our spammers are posting heavily.  The majority of them are "one and done".  



> Perhaps there is something similar that can be done here like adding a "report spammer" button that becomes available to registered users of a certain rep level, and if a poster accumulates a certain number of reports in a given time frame posts from the spammer account would be hidden and the account flagged for moderator review.  There would need to be some audit trail/accountability built in to deal with abuse.




I don't see how this would be superior to our current "one person reports it once, and that spammer is dealt with in a few hours" system.


----------



## Kzach (Apr 13, 2012)

I'm not sure how it would work but couldn't it be coded so that the IP of the email that is used to register is compared to the IP of the person using it to register. If I understand correctly, it should fall within a certain range that is particular to the ISP of the person registering. That would automate the process AND get rid of Piratecat and Morrus, all in one fell swoop!


----------



## Umbran (Apr 13, 2012)

Kzach said:


> I'm not sure how it would work but couldn't it be coded so that the IP of the email that is used to register is compared to the IP of the person using it to register. If I understand correctly, it should fall within a certain range that is particular to the ISP of the person registering. That would automate the process AND get rid of Piratecat and Morrus, all in one fell swoop!




It would also eliminate anyone registering from a computer that isn't at home - say, someone using a personal e-mail address, but who registers during their lunch hour at work.


----------



## ghostcat (Apr 13, 2012)

Don't forget most ISPs use "Dynamic" IP Addresses. That is a person's IP Address is dynamically allocated from an address pool and may change over time. 

This is especially noticeable if someone is using dial-up, which I believe is still used.


----------



## Kzach (Apr 13, 2012)

Umbran said:


> It would also eliminate anyone registering from a computer that isn't at home - say, someone using a personal e-mail address, but who registers during their lunch hour at work.



They shouldn't be surfing the internet at work anyway.



ghostcat said:


> Don't forget most ISPs use "Dynamic" IP Addresses. That is a person's IP Address is dynamically allocated from an address pool and may change over time.



It only matters at registration time and that 'pool' is still specific to the ISP.


----------



## Umbran (Apr 14, 2012)

Kzach said:


> They shouldn't be surfing the internet at work anyway.




Yes, well, pardon us if we set policy based upon what they do, rather than on what Kzach thinks they should or shouldn't do.  Or don't pardon us.  Whatever floats your boat.


----------



## Kzach (Apr 14, 2012)

Umbran said:


> Yes, well, pardon us if we set policy based upon what they do, rather than on what Kzach thinks they should or shouldn't do.  Or don't pardon us.  Whatever floats your boat.




I'm no expert but isn't this snark?


----------



## Umbran (Apr 14, 2012)

Kzach said:


> I'm no expert but isn't this snark?




It is perhaps a tad wry, but not intended as snide, or hurtful.


----------



## RangerWickett (Apr 14, 2012)

Kzach said:


> I'm no expert but isn't this snark?




No expert? Since when?


----------



## Kzach (Apr 14, 2012)

RangerWickett said:


> No expert? Since when?




I'm trying to be good so I'm only left now with irony and sarcasm


----------



## Lwaxy (Apr 15, 2012)

I have no ISP address and would never use one to begin with. My email regularly changes to avoid spam on my email account. ISP addresses kinda suck in that regard, even if I could get one, which I can't.


----------



## the Jester (Apr 15, 2012)

Kzach said:


> So, with the new board you've promised to import all the old threads and users.
> 
> But do you really have to?...
> 
> ...




So inconvenience a large percentage of the user base to deal with what isn't really a problem anyway?

I don't think that's a very good suggestion.


----------



## Kzach (Apr 15, 2012)

the Jester said:


> So inconvenience a large percentage of the user base to deal with what isn't really a problem anyway?
> 
> I don't think that's a very good suggestion.




Not if you wanted to inconvenience everyone for your own selfish and evil agenda.

*cackles with glee as he rides off on his broom*


----------



## drothgery (Apr 16, 2012)

FWIW, the email address I'm registered with here is my college alumni account. The email address I mostly use now is at a domain I own, but is set up to be a front end to a free email service. And while I have an email address from my ISP, the email account there is hosted by another free email service.


----------



## Rhyssa (Apr 16, 2012)

Forcing folks to use ISP emails only would alienate tons of users here who may only lurk rather than post.

For the other site that you keep touting using ISP addresses only, how do you know this for sure?  I'd think having an email confirmation would make far more sense since it's an extra step in the process plus it forces people to use real emails (even if from a free email service) rather than just making something up.  

Or if it's that tragic to see the occasional spam post, the permissions can be set that new accounts need X amount of posts before having a signature and/or putting any links into posts.  It's not that hard to do, actually.


----------



## jonesy (Apr 23, 2012)

Spacebattles.com has a system for handling spam where you can't make posts in the other sections until after you've made your first post in the introductions forum. I don't know how well that works for them though (I suppose someone could make a normal looking introductory post, and then go spam spam spam in the other sections). 



Kzach said:


> I'm trying to be good so I'm only left now with irony and sarcasm



If those would get banned I'd be in serious trouble.


----------



## frankthedm (Apr 24, 2012)

Kzach said:


> So, with the new board you've promised to import all the old threads...
> 
> But do you really have to?



 Information, thoughts and opinions don't have an expiration date.


----------



## Kzach (Apr 25, 2012)

frankthedm said:


> Information, thoughts and opinions don't have an expiration date.



Sure it does; whenever a new edition comes out!

*boom*tish*


----------



## Janx (May 24, 2012)

Viking Bastard said:


> Does anyone use ISP email addresses anymore? (It's just not very smart to rely on.)




I concur.  I tell everybody I end up helping to NEVER use an email account from your ISP.

A person should expect to fire their ISP every year and adopt a practice that makes that trivial to do.  Hotmail and Gmail are pretty much the Global Email Providers that most likely won't be going away.

Even though my ISP provides me with an email automatically, I don't check it and I told them to use the email address I give them to send me mail.  Any ISP that does not understand their role in the relationship in the 21st century is an ISP that does not understand that they are simply packet passers.


As to the general concept of blocking by IP, there are unintended consequences and complexities to that idea.

One example of good idea gone bad is where a manager told his IT staff to block all traffic from China.  Worked great until the evening, when Google shifts its load over its chinese servers.  So attempts to visit google were failing because responses were coming back from China.

Furthermore, the magic and mystery of the IP address has changed over the years.  Once upon a time, every PC got an IP address that was directly reachable from any other PC on the internet.  Nowadays with firewalls and Natural Address Translation (NAT), the IP address your PC actually has is likely to be 192.168.1.X where X is a number 1-255.  Your firewall or router may have an internet-reachable IP address, but nothing in your house or company does.  

This means that multiple users inside your firewall all show up as the same IP  address when they both browse this site.  

It's also not as simple as every address that starts with 101 to 125 is a chinese address.

It's also not hard for a Chinese spammer to lease a linux server in the US and then setup email accounts on that and to send out from there.  Or to even remote-control browse from the server to american servers (so it will have an American IP address).  There's no magic backtrace that could show that the server is being driven remotely from the perspective of Enworld.


There's better ways to block bots.  Using the goofy graphic text you have to re-enter is effective, and you're actually helping a document scanning project (those snippets come from an OCR project that has stumbled on those characters).

Requiring an initial post in an Introduction area is also probably reasonable.  As is doing an email verification to prove your really you.  Both mostly require a human to take the time to go through the process.


----------



## Umbran (May 24, 2012)

Janx said:


> A person should expect to fire their ISP every year and adopt a practice that makes that trivial to do.




For most people, ISPs (especially for residential broadband) are like utilities - they don't have many choices, such that firing them every year makes little sense. 



> This means that multiple users inside your firewall all show up as the same IP  address when they both browse this site.




EN World doesn't have so many users that this is apt to be a major issue.  There just aren't that many households that have multiple EN Worlders in residence, and fewer still that have one of them be problematic, and the other not.


----------



## Janx (May 24, 2012)

Umbran said:


> For most people, ISPs (especially for residential broadband) are like utilities - they don't have many choices, such that firing them every year makes little sense.




I don't literally expect people to actually fire their ISP every year.  But I've had 4 ISPs over 15 years.  My hotmail email address was never impacted.

People move.  Using your ISP's email address needlessly entangles you, whereas a major web-mail provider does not.

Furthermore, most ISPs set you up with a POP3 account and a crappy web-interface.  This puts your mail on your PC and not where your mobile device can get to it.

Thus far, the most superior configuration I have found is gmail, which has IMAP and Exchange opened up.  You can use Outlook with it if you are inclined.  Your iPhone, Android, and BlackBerry can all do GoogleSync and keep email, contacts and calendar in sync with the web and devices.  it's better than the limited iCloud solution Apple has.





Umbran said:


> EN World doesn't have so many users that this is apt to be a major issue.  There just aren't that many households that have multiple EN Worlders in residence, and fewer still that have one of them be problematic, and the other not.




It's not just houses.  Companies and schools do the same thing.  So one IP blocking from a bad user at work can take out an entire school of potentially valid users.  I worked at a fortune small-number company full of nerds who played D&D.  Bound to be a few enworlders there.

My point being, I support not blocking by IP because there are such complexities that aren't immediately obvious when such ideas are proposed.


----------

