CSC 379:Week 1, Group 4: Difference between revisions

From Expertiza_Wiki
Jump to navigation Jump to search
 
(12 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Internal Use Only ==
Group members:
Nick Principe / naprinci@gmail.com / AIM: mahoubaka <br />
Ken Ganong / kjganong@ncsu.edu / AIM: C4P0droid
*domain authentication
**[http://e-com.ic.gc.ca/epic/site/ecic-ceac.nsf/en/gv00298e.html what canada has to say about it]
**[http://lwn.net/Articles/188685/ seemingly useful article with links to domain authentication related concepts]
*the "goodmail" approach
**[http://www.usatoday.com/tech/news/computersecurity/2006-03-05-goodmail_x.htm goodmail article]
**[http://www.goodmailsystems.com/ goodmail web page]
*client-side filtering
[http://www.spamlaws.com/articles/usf.pdf huge paper on this subject]
== Spam Prevention Techniques ==
== Spam Prevention Techniques ==


Line 48: Line 31:
|-valign="top"
|-valign="top"
<!----------------------------------------------------------------------------->
<!----------------------------------------------------------------------------->
|[[#Require_users_to_request_permission_to_send_mail|Require users to request permission to send you e-mail]]
|[[#Require_users_to_request_permission_to_send_you_mail|Require users to request permission to send you e-mail]]
|
|
*Blocks all spam
*Blocks all spam
Line 135: Line 118:
*Possible deterrent
*Possible deterrent
|
|
*Costs government (tax-payers) money
*Very difficult for average citizens to find more information than FBI
*Bounty only paid if spammer is penalized
*Most known spammers are not brought to trial due to lack of evidence
|
|
<!----------------------------------------------------------------------------->
<!----------------------------------------------------------------------------->
Line 150: Line 135:
|[[#The_.22Goodmail.22_approach | The "Goodmail" approach]]
|[[#The_.22Goodmail.22_approach | The "Goodmail" approach]]
|
|
*Mass emails cost money so mass spammers don't work
*Accreditation of senders ensures legitimate senders
*Certification of mail ensures individual mail is legitimate
*Legitimate mail won't get caught up in spam/junk mail folders
*Accredited senders must maintain high standards
|
|
*Companies can bypass the spam filter by paying money
*ISP / email provider must support certified mail
*Depends on reliability and impartiality of one company
*Creates two classes of email
**Not in the spirit of the internet or email
|
|
<!----------------------------------------------------------------------------->
<!----------------------------------------------------------------------------->
Line 158: Line 149:
|-
|-
| style="background:#00ff00;border:1px solid white" width=2px |&nbsp;
| style="background:#00ff00;border:1px solid white" width=2px |&nbsp;
| style="background:#cccccc;border:1px solid white" width=2px | &nbsp;
| style="background:#00ff00;border:1px solid white" width=2px | &nbsp;
| style="background:#cccccc;border:1px solid white" width=2px | &nbsp;
| style="background:#00ff00;border:1px solid white" width=2px | &nbsp;
| style="background:#cccccc;border:1px solid white" width=2px | &nbsp;
| style="background:#cccccc;border:1px solid white" width=2px | &nbsp;
| style="background:#cccccc;border:1px solid white" width=2px | &nbsp;
| style="background:#cccccc;border:1px solid white" width=2px | &nbsp;
Line 186: Line 177:
<!----------------------------------------------------------------------------->
<!----------------------------------------------------------------------------->
|[[#Client-side_filtering | Client-side filtering]]
|[[#Client-side_filtering | Client-side filtering]]
|pro
|
|
*Only as good as user or algorithms/heuristics at identifying spam
* Extremely flexible.
*Spam emails are stopped, they are simply not read.
* Users typically don't read spam but can also recover all emails.
|
*Only as good as user or algorithms/heuristics at identifying spam.
*Spam emails are not stopped, they are simply not read.
* Has a potential to cause many false positives
|
|
<!----------------------------------------------------------------------------->
<!----------------------------------------------------------------------------->
Line 253: Line 247:
===== Links =====
===== Links =====
* [http://www.ftc.gov/bcp/conline/pubs/buspubs/canspam.shtm The CAN-SPAM Act]
* [http://www.ftc.gov/bcp/conline/pubs/buspubs/canspam.shtm The CAN-SPAM Act]
* [http://www.spamlaws.com/articles/usf.pdf Large paper including opt-out]


==== Domain authentication ====
==== Domain authentication ====
Line 267: Line 262:


==== Bounties ====
==== Bounties ====
Large sums of money can be offered for the capture of big-time spammers. This money can either be a flat amount or a percentage
of the civil penalty paid by the spammer. The FTC's plan was to give 20% of the collected penalty to the person who identified
the spammer.


Since the civic penalties for spammers are large amounts of money, this incentive could cause many people to strive for
incriminating evidence against spammers. With such incentive, spammers would not want to send so much email as to be at the top
of the bounty list, thereby deterring spammers from increasing their spam.
However, critics say that this ignores the main problem with capturing spammers. The agencies that would offer such bounties
already know who the spammers are, but don't have enough proof to bring said spammers to trial. Therefore, the bounty, though
offered for everyone, would only be truly available to people inside the spam rings.


===== Links =====
===== Links =====
Line 274: Line 279:


==== The "Goodmail" approach ====
==== The "Goodmail" approach ====
This technique provides for accreditation for legitimate senders, allowing their emails to bypass spam and volume mailing filters and folders and get delivered directly to the inbox of users on participating mail systems. The accreditation process costs money and sending the certified messages also costs money. The accreditation process is strict, and the policies for using the service exclude any unsolicited email.
This approach does benefit users in that certified mail is known to be from a trustworthy source and emails from companies using the Goodmail service will not be put in a junk mail folder. This benefits the companies to the same degree, though, ensuring that their messages are delivered straight to the users’ inboxes.
Much maligned when AOL first announced it would use the Goodmail service, it seems most of the criticism was reactionary – the qualifications required by and standards instituted by Goodmail, if they are actually as strict as they appear, mean that users either want the emails sent through this service or can opt out of them – and they aren’t unsolicited. This service is provided by a for-profit entity, so it is similar to a pay-for-email scheme – it also seems to divide email into two classes which is arguably not in the spirit of the email system. Also, it does require the support of mail hosts and/or ISPs.
===== Links =====
*[http://www.goodmailsystems.com/ Goodmail Systems Homepage]
*[http://www.usatoday.com/tech/news/computersecurity/2006-03-05-goodmail_x.htm USA Today Article on Goodmail]


==== Bonds with escrow agencies ====
==== Bonds with escrow agencies ====
Line 287: Line 301:


==== Client-side filtering ====
==== Client-side filtering ====
This technique for fighting spam involves software on the receiver's computer or mail server. This software will examine all incoming emails
and decide if they are spam based on a number of rules part of which can be written by the user. Once a message is decided as spam, it is
either marked as such, moved to a separate directory, or deleted. The user will typically ignore all messages deemed as spam.
The client side filter is an extremely flexible technique that can be adapted to the specific user's definition of spam. Messages can be
marked as spam by practically any criteria meaning that a well configured spam detector will prevent the user from even seeing a mass
majority of the spam that comes to his or her mail box.
However, this technique does nothing for bandwidth or mail server space. All the spam gets downloaded and there is nothing that deters
spammers from sending more future spam email. Also, client filters have to constantly be updated as spammers have consistently found ways to
circumvent the more popular filtering techniques. Since the user can configure the filter in almost any way, it could easily cause many
false positives (wanted emails marked as spam).
===== Links =====
* [http://www.spamlaws.com/articles/usf.pdf Large paper including client filters]
* [http://wiki.mozilla.org/Thunderbird:Help_Documentation:Dealing_with_Junk_E-mail Thunderbird's adaptive spam filters]

Latest revision as of 20:57, 12 July 2007

Spam Prevention Techniques

Comparison of Techniques

Technique Pros Cons Authors' Rating
Block domains of "known" spammers
  • Can block a large amount of spam
  • Low chance of blocking legitimate email
    • Notification sent to blocked senders in some implementations
  • Action to take on spam is user-definable
  • Some spam still can still get through
  • Might block individuals running their own mail server
  • Requires processing at client/receiver-side for effective blocking
         
Require users to request permission to send you e-mail
  • Blocks all spam
  • Robots cannot easily send spam
  • Hard to falsify identity
  • Can introduce large delays in user "seeing" an email
  • Impossible to implement correctly and universally at the client side
  • Requires significant action on the part of the user to make exceptions
         
Charge for email sent
  • Forces targeted selection of spam
  • Changes the operational paradigm of email
  • Lots of supporting infrastructure development necessary
  • Might impact users more than spammers
  • Where does the money go?
         
Opt-in for commercial email
  • Companies can send advertisements without sending spam
  • Users can freely restrict the influx of mail from their many online affiliations
  • Fraudulent emails have an opt-out link that sends you to an unwanted web page.
  • Only stops unwanted spam from companies that abide by this rule.
         
Domain authentication
  • Very little spam gets through
  • Makes phishing and other identity deceptions very hard
  • Easier to track down spammers
  • Current approaches can break under certain common circumstances
  • Requires infrastructure changes
  • Multiple standards competing
  • Could make it more difficult for individuals with their own mail server
         
Bounties
  • Gets rid of big spammers with incentive
  • Possible deterrent
  • Very difficult for average citizens to find more information than FBI
  • Bounty only paid if spammer is penalized
  • Most known spammers are not brought to trial due to lack of evidence
         
The "Goodmail" approach
  • Accreditation of senders ensures legitimate senders
  • Certification of mail ensures individual mail is legitimate
  • Legitimate mail won't get caught up in spam/junk mail folders
  • Accredited senders must maintain high standards
  • ISP / email provider must support certified mail
  • Depends on reliability and impartiality of one company
  • Creates two classes of email
    • Not in the spirit of the internet or email
         
Bonds with escrow agencies
  • Whitelisted email accounts don't take out a bond
  • Only spammers have to pay.
  • Lots of infrastructure and processing behind 'micro-payments'
  • Somebody has to pay for the escrow agency.
  • Users can subvert the system by collecting even when not spam.
         
Client-side filtering
  • Extremely flexible.
  • Users typically don't read spam but can also recover all emails.
  • Only as good as user or algorithms/heuristics at identifying spam.
  • Spam emails are not stopped, they are simply not read.
  • Has a potential to cause many false positives
         

Technique Details

Block domains of "known" spammers

This technique is often implemented by means of a DNS Blacklist (DNSBL) which is a frequently updated list containing IP addresses and ranges of known spammers, though similar systems exist to block domain names or URIs (uniform resource identifiers) associated with spam.

This approach has an advantage over most client-side filtering schemes, since it is better able to block spam that the particular client has never seen before since it is a shared and universal list. Depending on implementation, the DNSBL system could be implemented as a form of client-side filtering. However, it is most efficient if the DNSBL filtering system is implemented at a higher level, such as at the ISP or business. Another advantage of this system is that the action taken when a message is identified as spam is defined by the individual users of the DNSBL system – they can still deliver the message, flag it as spam, or bounce it entirely.

There is the possibility that a DNSBL system could block legitimate email – the likelihood and method of handling this possibility are dependent on the specific implementation. The Spamhaus system sends a message back to each sender of a blocked message indicating why it was bounced to prevent legitimate email from “disappearing” without a trace – however, this approach increases mail and network traffic. A DNSBL system can also make it hard for individuals to set up their own mail servers at home, since residential IPs are blocked in some systems. Also, with minimal processing of incoming messages the percentage of spam blocked is relatively low – for the Spamhaus system, only 15-25%. To get over 90% spam blockage, the headers and body of each message must be analyzed.

Links

Require users to request permission to send you mail

This draconian spam prevention technique, most commonly seen through the use of the Earthlink spamBlocker, delivers only messages from the user’s address book to their inbox – all other mail is put in a separate folder and the sender is sent a reply with information on requesting the user allow them to send the user email.

This technique doesn’t let spam through to the user’s inbox, which is a major advantage. Also, this approach makes it hard or impossible for the sender’s identity to be falsified. When allowing the sender to request the intended recipient to allow their mail through, anti-robot measures prevent the automation and abuse of this system.

There are major disadvantages to this approach. Whenever receiving email from a new person or website, the user must manually add permission to receive from the new sender. Desired or important emails could sit idle for long periods of time in the “Suspect Email” folder before the user reads them. Also, sending a reply to the sender with instructions to request permission from the user to send them mail can cause frustration: first, it is an additional step for the sender, and second, the sender might not check their email again after sending the original message, which introduces further delays.

Links

Charge for email sent

This technique strives to make email more like postal mail, shifting the burden of cost from the recipients to the senders. The goal being reduced spam due to the new costs associated with sending email – requiring more targeted marketing/spamming to be cost-effective. The costs involved could either be monetary – e-postage – or temporal – “hashcash”: requiring a complex computation for each sent email, making sending email very slow if there are many recipients.

The advantage to this technique, if it worked properly, would be reduced spam because spammers would be forced to have a more targeted selection of recipients.

Unfortunately, there are many problems in implementing a system to handle this. Vast amounts of banking infrastructure would need to be established to support an e-postage system, since many checks would have to be done to ensure against fraud. The decentralized mail infrastructure of the internet would have to become more centralized to force the usage and verification of e-postage as well. Using hashcash instead of e-postage would still require mechanisms and infrastructure to enforce the running of the hashcash algorithms. Also, most spammers have vast networks of computers, both legitimately and illegitimately, under their control, so they would have more capacity to solve hashcash problems than individual users.

Links

Opt-in for commercial email

Commercial advertisements are often considered spam even when the user has had a previous relationship with the company sending the email. In order for companies to send such advertisements without causing a lot of unwanted email, a simple opt-in or opt-out system should be implemented by the company. If an opt-out link or instructions appear in an email, the result of a user following them is that that user will no longer be sent similar advertisements. This much is required to be CAN-SPAM compliant. The preferred method, however, is an opt-in. In this way, when a company and a user first achieve contact (usually by the user making a user account with the company), there is a method for the user to configure which types of email advertisements he or she desires from the company.

Such a system allows a user to decide and configure which companies and which types of advertisements they would like to receive email about from each company for which they have an online affiliation. However, this requires the company to implement and abide by such a rule. Also, since this system is so common, many fraudulent spam emails have opt-out options which are fake. By responding to such an opt-out option, you actually submitting yourself to more spam because the sender knows that your e-mail account is active.

Links

Domain authentication

Domain authentication ensures that the domain in a sender’s email is authentic and hasn’t been forged, making phishing and other fraudulent email harder to send. There are two major techniques for accomplishing domain authentication: Sender Policy Framework (SPF) and Domain Keys (DK). Both of these approaches use DNS to authenticate sender domains.

Both approaches add support for domain authentication to the current mail framework in existence, enabling one to be sure that the email originated from the domain specified in the sender’s address. This would effectively filter out a lot of spam.

One problem with both techniques is that they fail under certain conditions: SPF fails under some store-and-forward situations, and DK fails when the message is modified after signing. Also, these techniques do not eliminate the ability of spammers to use temporary domains to send spam – though mail will have to be addressed from that domain, possibly facilitating the tracking of spammers.

Links

Bounties

Large sums of money can be offered for the capture of big-time spammers. This money can either be a flat amount or a percentage of the civil penalty paid by the spammer. The FTC's plan was to give 20% of the collected penalty to the person who identified the spammer.

Since the civic penalties for spammers are large amounts of money, this incentive could cause many people to strive for incriminating evidence against spammers. With such incentive, spammers would not want to send so much email as to be at the top of the bounty list, thereby deterring spammers from increasing their spam.

However, critics say that this ignores the main problem with capturing spammers. The agencies that would offer such bounties already know who the spammers are, but don't have enough proof to bring said spammers to trial. Therefore, the bounty, though offered for everyone, would only be truly available to people inside the spam rings.

Links

The "Goodmail" approach

This technique provides for accreditation for legitimate senders, allowing their emails to bypass spam and volume mailing filters and folders and get delivered directly to the inbox of users on participating mail systems. The accreditation process costs money and sending the certified messages also costs money. The accreditation process is strict, and the policies for using the service exclude any unsolicited email.

This approach does benefit users in that certified mail is known to be from a trustworthy source and emails from companies using the Goodmail service will not be put in a junk mail folder. This benefits the companies to the same degree, though, ensuring that their messages are delivered straight to the users’ inboxes.

Much maligned when AOL first announced it would use the Goodmail service, it seems most of the criticism was reactionary – the qualifications required by and standards instituted by Goodmail, if they are actually as strict as they appear, mean that users either want the emails sent through this service or can opt out of them – and they aren’t unsolicited. This service is provided by a for-profit entity, so it is similar to a pay-for-email scheme – it also seems to divide email into two classes which is arguably not in the spirit of the email system. Also, it does require the support of mail hosts and/or ISPs.

Links

Bonds with escrow agencies

This spam fighting technique works based on whitelists, blacklists, graylists, and a third party (escrow agency) separate from the email sender or receiver. A whitelisted sender simply sends email and it goes through without the escrow agency intercepting. A blacklisted sender cannot send email to the would-be receiver. The contents of the graylist is essentially everyone on neither of the other lists.

A graylisted sender opens a bond for a small amount of money (one cent) with the escrow agency in order to send email. If the receiver blacklists the sender as a result of the email, the bond is collected and the sender is charged. Thus, only spammers have to pay for their email unlike the charge-for-email approach.

The escrow agency, however, must be paid. One way of doing this is having the collected spammer money go to the escrow agency. There is a lot of processing for any type of internet payment, so the penny (or so) that is charged to the spammer may not be enough to cover the escrow agency's cost regarding. Also, non-profit groups would possibly often be blacklisted and therefore be forced to pay more than they can afford similar to the Goodmail approach. Since the email cost is mean to deter spammers, Users can subvert the system by blacklisting emails that aren't spam. For example, I could charge my professors for sending me email that they must send for class or users could charge ebay for requested notifications.

Links

Client-side filtering

This technique for fighting spam involves software on the receiver's computer or mail server. This software will examine all incoming emails and decide if they are spam based on a number of rules part of which can be written by the user. Once a message is decided as spam, it is either marked as such, moved to a separate directory, or deleted. The user will typically ignore all messages deemed as spam.

The client side filter is an extremely flexible technique that can be adapted to the specific user's definition of spam. Messages can be marked as spam by practically any criteria meaning that a well configured spam detector will prevent the user from even seeing a mass majority of the spam that comes to his or her mail box.

However, this technique does nothing for bandwidth or mail server space. All the spam gets downloaded and there is nothing that deters spammers from sending more future spam email. Also, client filters have to constantly be updated as spammers have consistently found ways to circumvent the more popular filtering techniques. Since the user can configure the filter in almost any way, it could easily cause many false positives (wanted emails marked as spam).

Links