Saturday, September 1, 2012

Email Marketing: Blacklist removal process of ISPs

                    

How to Remove Your IP from Gmail’s Blacklist

                       
If you cannot send emails to Gmail, your server may have been blacklisted. Here are some tips to get removed from the Gmail blacklist.
   
Before jumping through the blacklist removal hoops, you may want to double check that your emails are not simply going into the spam folder. This process will not help you with emails being dropped into the spam folder. This is for getting off of Gmail’s blacklist. I am going to outline 3 steps.
   
  1. Verify you are on the Gmail blacklist.
  2. Perform preliminary blacklist removal checks.
  3. Submit Gmail blacklist delisting request.
   

Gmail Blacklist Verification

   
If you are blacklisted, then you should be getting a delivery rejection notice from Google. If you check your server’s logs or your email bounce you may see something like this:
   
       
Remote_host_said:_550-5.7.1 Our_system_has_detected_an_unusual_rate_of unsolicited_mail_originating_from_your_IP_address._To_protect_our users_from_spam,_mail_sent_from_your_IP_address_has_been_blocked. Please_visit_http://www.google.com/mail/help/bulk_mail.html_to_review_our_Bulk_Email_Senders_Guidelines
   
   
If you are seeing this email error, then your server’s IP has likely been blocked by Google. There could be other response codes, but typically all Gmail blacklist notifications will include a 550 error plus a link to the Gmail policies pages.
If you are not seeing 550 errors, then you may not have an email blacklist problem but some other email delivery issue.
   

Preliminary Blacklist Removal Tasks

   
Before requesting removal from Gmail’s blacklist, you will want to take some steps to stop whatever caused the listing.
Make sure there is no unauthorized email going from your server.
   
  • Check the daily volume of email going to Gmail
  • Look for compromised user accounts.
  • Look for people forwarding email to Gmail.
   
Once you have reviewed these items, you should be able to determine the cause of the listing. For example, if someone is forwarding email to Gmail and then marketing it as spam, your server’s sender reputation is lowered and you can be blacklisted. Sudden spikes in email volume can also trigger the filters. The important thing is look for changes in your server’s behavior as it is likely the cause of the listing.
   

Gmail Blacklist Removal Process

   
The forms to initiate an inquiry at Gmail are buried in Google’s email help section. If you are running your own server, you will want to start with their “My domain can’t send to Gmail” form. If you answer the questions correctly, you will win a prize:
   
Report a delivery problem between your domain and Gmail.
   
Provide only what they ask, and do not complain. Just provide the requested details and drop a note of thanks into the additional information field. I suspect they receive 100’s of these a day, so be nice and wait.
   




                       




How to Remove Your IP from Yahoo’s Blacklist

                       

This is another installment of our Spam Blacklist Removal Series, so be sure to check out the series for other ISPs. For Yahoo blacklist problems, read on. So you fire up your favorite mail client only to find upset customers and bounced emails.

Appears, your server is on the email blacklist. You see tons of messages with an error:
421 4.7.0 [TS01] Messages from <1.2.3.4> temporarily deferred
due to user complaints
- <1.2.3.4> ; see http://postmaster.yahoo.com/421-ts01.html

   
So what now? Your server has been blacklisted by Yahoo!. The email queues are building up, clients are calling, and you cannot forward that latest NSFW video to your friends on Yahoo!.
   
To get removed from Yahoo!‘s blacklist, you need to follow some simple steps:
   
  1. Find the cause
  2. Fix the problem
  3. Wait …
  4. Fill out the form
  5. Wait …
   
If you are a spammer (and you know if you are), then don’t wast your time with these tips. They will not help you get removed from the email blacklist.
   
If your wearing a white hat happened to get nailed by the blacklist, read on.
   

Find the Cause

   
The first step is to find the cause. Email service providers are increasingly relying on sender reputation metrics to determine if email is spam or should be flagged for additional processing. Yahoo! works with ReturnPath for this feature. If you consistently trigger Yahoo!‘s blacklist, you may find that you get permanently blocked or your email is routed to the junk folder.
   
Yahoo! did not blacklist your server to spite you. Something triggered the listing. To prevent future listings, you must identify the trigger. If you get re-listed, your sender reputation will drop, and as a result, email will be even more difficult to delivery successfully.
   
In my experience, if you run normally clean email operations and follow these steps, your Yahoo! email delivery problems will reside within 48 hours or less.
   
Some key reasons for getting blacklisted:
   
  • Compromised Web Script
  • Compromised End-User Account
  • Legitimate Bulk Emailing
  • Forwarded Email
   
If you have a client forwarding large volumes of email to Yahoo! and the client flags these forwarded messages as spam, you could get put on the Yahoo blacklist. The reason is it is impossible for the filtering to determine if the forwarding was intentional or a trick used by the spammers. As a result, spam filters often hold the last and first links in the delivery chain responsible.
   
Historically, open relays have been prone to blacklisting, but recent versions of hosting control panels and email software often disables relaying by default. We encourage the use of SMTP AUTH on all servers to prevent unauthorized mail relay.
   

Call the Postmaster

   
All major ISPs, Yahoo! included, maintain very useful Postmaster help pages. These pages often detail the do’s and don’ts for sending to their email servers. Yahoo!‘s Postmaster help center provides a variety of topics.
   

Blacklist Removal Process

   
In my work providing help with dedicated servers, I’ve learned you have to be diligent and patient when trying to get your IP removed. There is not need to send the ISP’s multiple emails. Just follow their process and results should follow.
   
Do not try to get removed from the blacklist if you have not found the source of the problem. You don’t want cycles of de-listing and re-listing to damage your sender reputation. If you have found the problem, then simple wait. In many cases, I find that the block is removed in under 48 hours.
   
If you are still blocked after 48 hours (or impatient), you can then submit Yahoo!‘s Mail Delivery Issues Form. Before submitting, be sure you are abiding by Yahoo!‘s email policies. Also, verify that all of your DNS setting are correct. Reverse DNS (PTR), MX, A, and SPF records should all be checked If you are using DomainKeys, be sure to test that those are working as they should. The key is not to raise any red flags during the review process. Make the reviewers job easy by fixing problems first.
   
Though some fields on the delivery report are optional, it is best to provide as much information as possible. Especially, the “Enter additional information here:”. You will want to detail in 2-3 sentences your remediation efforts. For example, if the problem was a compromised web script send them a note:
   
We have identified an insecure web application on our server that permitted unauthorized email relay through our system. We have removed this script.
   
Keep it short and technical. Yahoo! postmaster staff reviews 1000’s of these requests, so being short and to the point is best to get removed from the Yahoo blacklist.
   

Bulk Senders

   
If you are sending bulk email, you will receive a special type of bounce pointing you to Yahoo!‘s error pages. Before starting the blacklist removal process, be sure to review Yahoo!‘s bulk email sending policies. Once you are compliant, send in the Yahoo! Mail Bulk Sender Form to get your account exempted from the bulk sender triggers. You must maintain a clean list. Simply submitting the form does not let your email flow freely. If you are spamming, this will not help you. If you maintain clean, double opt-in lists, then your emails will get to their recipients.
   

Quick Review

   
  1. When you get blacklisted by Yahoo!
  2. Find the trigger
  3. Fix the trigger
  4. Submit a mail delivery report
  5. Wait …
  6. Review best practices.
   
In the next couple of weeks, I will be providing quick how-to’s for getting out of the blacklists maintained by Google, MSN, Earthlink and Frontbridge. Also, if you send large volumes of email or are an email service provider, don’t forget about email feedback loops. Yahoo! now provides a Compliant Feedback Loop based on DomainKeys.
   
Lastly, if you find all of this too much to deal with, then consider our Linux support services. While we cannot guarantee removal from the yahoo blacklist, we are pretty successful (provided, of course, that you are not a spammer).
                       

How to Remove Your IP from AT&T’s Blacklist

                       
If you cannot send emails to AT&T, SBC Global or other AT&T related email servers, your server may have been blacklisted. Here are some tips to get removed from the AT&T blacklist.
   
Spam blacklist removal often requires you to follow specific steps established by the ISP. While the process is generally the same for most ISPs, finding their abuse forms and their methods of handling removal differ. I find AT&T to be one of the more responsive of the major ISPs. Typically, you will see your IP removed from AT&T’s spam blacklist within 24-72 hours, provided of course you are not spamming again.
   
Getting delisted from the AT&T email blacklist requires a few steps:
   
  1. Verify you are on the AT&T’s blacklist.
  2. Perform preliminary blacklist removal checks.
  3. Submit AT&T blacklist delisting request.
   

AT&T Blacklist Verification

   
If you are blacklisted at AT&T, you will get a bounced email from them. In this email or in the server logs, you will see something like:

Remote_host_said:_521-192.168.1.1_blocked_by_ldap:
ou=rblmx,dc=AT&amp;T,dc=net/521_Error_-_Blocked_for_abuse._
Contact_abuse_rbl@abuse-att.net./


   
The exact format of the error will vary depending on your email system. This error is from a Plesk server running Qmail.

If AT&T’s email servers return this blocked email message, then your server’s IP has likely been blocked by AT&T. There could be other response codes, but typically all AT&T blacklist notifications will include a 500 series error plus a link to the abuse_rbl@abuse-att.net email address.
   
If you are not getting a bounce or seeing 550 type error, then you may not be on a spam blacklist but have some other email delivery issue.
   

Preliminary Blacklist Removal Tasks

   
Before requesting removal from AT&T’s blacklist, you will want to stop what has caused the listing in the first place. Here are a few items to check:
   
Make sure there is no unauthorized email going from your server.    
  • Check the daily volume of email going to AT&T.
  • Look for compromised user accounts.
  • Look for people forwarding email to AT&T.net, Bellsouth.net, or SBCGlobal.net.
   
You will often need to dig through your server maillogs and access logs to try to identify the cause of the listing. The sooner you respond to bounces the greater the chance you can catch the problem before it becomes too severe. If you flood 1000’s of spam or phishing emails, your email sender reputation will drop. If you have a low sender reputation, AT&T may not remove you from their spam blacklist.
   

AT&T Blacklist Removal Process

   
While the rejected emails tell you to email abuse_rbl@abuse-att.net, the information you want is located on AT&T’s block inqury website. Here you will find a list of AT&T email error codes as well as a AT&T blacklist removal request form:
   
Tools for administrators of mail systems whose messages have been blocked.
   
After completing the form, you will usually get a reply from AT&T within a few hours. The removal of your server IP from AT&T’s spam blacklist may take a few more days. If you do not see any response within 72 hours, you will want to confirm the spamming has stopped and then email abuse_rbl@abuse-att.net with a request about your removal request.

Server Configuration: How to make Uptime 99.999%

Related Terms

Advanced ECC Memory Protection
Chipkill
Hardware RAID / DRBD
Multichannel SCSI Controllers
External Drive Array
Hot-Swappable PCI card
Hot-Swappable Power Supply
Monitoring (OS Process, Network)
Disk Mirroring

General Infrastructure

When designing a high availability solution, it should generally be remembered that even the installation of all key servers at a single location can be a potential single point of failure (SPOF) if this location is hit by disaster or power failures. The environmental conditions of the servers should also be taken into account (redundant) air conditioning systems are essential.

Hardware

Even the most sophisticated software cannot produce a high availability system without the greatest possible security from failure on a hardware level. The key hardware components that should be considered and laid out with the greatest possible redundancy are:

Power Supply : If possible, secure your servers using a UPS (uninterpretable power supply) to ensure that a brief power failure can be bridged and the systems can be shut down correctly in the event of a longer power failure. The power supply should also be configured for redundancy. Hot-Swappable Power Supply are standard on any server that claims to offer increased levels of reliability or availability.

Network Interfaces : Make sure each of your systems has several network interfaces. If      one interface fails, another must automatically take over the address and task of the failed component. Redundancy expressly relates to the two interface directions. There is no harm planning an active and backup interface for both the internal and external interfaces.    

Hard Disks : Assign several hard disks to your system and arrange the data backup (e.g., using RAID or DRBD) in such a way that if one of these disks is lost, the others always contain the intact data record. It must be possible to replace a faulty disk with a new one without stopping the system.    

Applications

All important data and applications that form the outer face of your systems must be arranged in such a way that they will not prevent a restart. If an application does not release its lock files after a crash, this prevents the relevant process from restarting. This means that the application is not suitable for a high availability environment. Ideally, the "health" of certain applications, operating system processes, and network connections should be monitored with a suitable monitoring tool.


Data

After a system fails, all key data must be available to the fail over system complete and intact. This type of high availability is achieved by distributing stored data over several systems or hard disks. For this, the contents of a disk are regularly mirrored on another disk (or several disks), which can take over with the intact data record if a failure occurs. Use a file system to ensure that a file system restarts in a consistent state after a system crash.

Network

All network infrastructure should be configured for redundancy, from the router and switch infrastructure down to the simple network cable.

Important HTTP Response Codes



CodeMeaningMessage
200OKSuccess!
304Not ModifiedThere was no new data to return.
400Bad RequestThe request was invalid. An accompanying error message will explain why. This is the status code will be returned during rate limiting.
401UnauthorizedAuthentication credentials were missing or incorrect.
403ForbiddenThe request is understood, but it has been refused. An accompanying error message will explain why. This code is used when requests are being denied due to update limits.
404Not FoundThe URI requested is invalid or the resource requested, such as a user, does not exists.
406Not AcceptableReturned by the Search API when an invalid format is specified in the request.
420Enhance Your CalmReturned by the Search and Trends API when you are being rate limited.
500Internal Server ErrorSomething is broken. Please post to the group so the Our Servers team can investigate.
502Bad GatewayOur Servers are down or being upgraded.
503Service UnavailableOur Servers are up, but overloaded with requests. Try again later.
504Gateway timeoutOur servers are up, but the request couldn't be serviced due to some failure within our stack. Try again later.