Frequently Asked Questions

General Questions

Feature Questions

Feature Suggestions

Troubleshooting

What is spamdyke and what does it do?

In a sentence, spamdyke is a drop-in qmail filter for stopping spam at connection-time.

"drop-in" means it can be installed without patching or recompiling qmail, without installing or updating libraries, without drastically reconfiguring anything and without having to become a qmail expert.

"connection-time" means spamdyke evaluates and rejects spam while the remote server is still delivering it. Other filters and anti-spam solutions focus on classifying spam after qmail has accepted it. The spam still has to go somewhere. Even if it's filed in a folder, it still occupies disk space, consumes server resources and must be deleted by someone. When spamdyke rejects the incoming spam completely, no one has to deal with it. It's never on the server at all.

For a complete description of spamdyke and all its features, see the README page.

I use email but I don't administer my own mail server. Can I use spamdyke?

Unfortunately, no. spamdyke must be installed by a mail administrator. It cannot be installed or used by end users, sorry.

How do I install spamdyke?

For installation instructions, see the INSTALL.txt file.

How do I upgrade spamdyke? What's the significance of the version numbers?

Typically, upgrading spamdyke is as simple as compiling the new version and copying the new spamdyke binary over the old one. However, sometimes the new version is not backwards-compatible with the old version and simply replacing the binary will cause problems. The UPGRADING.txt file has details on the backwards-compatibility of each version.

The version numbers are used to show the type of changes in each version:

  • When major, non-backwards-compatible changes are made, the "major" version number is incremented. For example, 3.0.0 is not completely backwards compatible with 2.6.3, so the major version number changed from 2 to 3.
  • When new, backwards-compatible features are added, only the "minor" version number is incremented. For example, 3.1.0 includes new features but is backwards compatible with 3.0.0, so the minor version number changed from 0 to 1.
  • When no features are added but bugs are fixed, only the "revision" version number is incremented. For example, 3.0.1 only contained bug fixes, so the revision version number changed from 0 to 1.

How do I get support for spamdyke? Is there a mailing list?

Yes! Visit the mailing list page to sign up: www.spamdyke.org/mailman/listinfo/spamdyke-users. The mailing list archives are searchable at: www.mail-archive.com/spamdyke-users@spamdyke.org.

All of the documentation and releases can be found at on the spamdyke website at spamdyke.org.

If you can't find answers there, send an email to me at: samc (at) silence (dot) org

I don't use qmail. Can I still use spamdyke?

Not at this time.

Some background: Qmail starts a new process (qmail-smtpd) from a listening daemon (tcpserver or xinetd) every time a new connection is established. spamdyke slips in between those two, so the daemon starts a new copy of spamdyke for each connection and spamdyke starts the qmail process.

Most other mail servers don't work this way; they use a single long-running daemon for handling incoming requests. There's no way to insert spamdyke without rewriting the mail server daemon (no thanks!).

Having said that, spamdyke could be modified to listen for incoming connections itself (replacing qmail's tcpserver) and establish a new connection to the "real" server (presumably listening on a different port, perhaps running on a different interface or machine). This requires some refactoring in the spamdyke code but it's not an insurmountable task.

Look for this feature in a future version.

Do I have to install the programs from the "utils" directory? Does spamdyke use them? Do they use spamdyke or each other?

No. domainsplit and domain2path are just small utilities for use in writing scripts. spamdyke doesn't use them or depend on them. Conversely, they don't use or depend on spamdyke.

dnsa, dnsmx, dnsns, dnsptr, dnssoa and dnstxt are just small, self-contained examples of how to make A, MX, NS, PTR, SOA and TXT DNS queries using libc. They're not really useful on their own. I made them available because I was looking for examples like them when I was writing the DNS code in spamdyke and I couldn't find any. Hopefully someone else will be able to learn from them. Aren't I just a swell guy?

When I reconfigure spamdyke, do I have to restart qmail?

Usually not but it depends on what you've done.

You do not need to restart qmail when you:

  • Replace the spamdyke binary
  • Edit spamdyke's configuration file
  • Edit a file in a configuration directory
  • Edit a file referenced from spamdyke's configuration (e.g. adding IP addresses to a whitelist file)

You must restart qmail when you:

  • Edit spamdyke's command line

What is the difference between a right-hand-side blacklist (RHSBL) and a realtime blackhole list (RBL)?

Right-hand-side blacklists (RHSBLs) are lists of domain names. The name refers to the "right-hand side" of an email address -- the portion after the "@" symbol.
Advantage: all email from a domain will be blocked, even if the spammer uses many different servers in different locations (or a botnet).
Advantage: RHSBLs can be used to validate a message in multiple ways: the rDNS name of the remote server can be checked, the envelope sender's domain name can be checked and the "From" address can be checked (spamdyke does not run this third test).
Disadvantage: RHSBLs require more network activity to check than RBLs.
Disadvantage: sender addresses are very easy to forge. If a domain name is being abused by a spammer, it could be blocked even though the real owners are sending legitimate email.

Realtime blackhole lists (RBLs) are lists of IP addresses.
Advantage: all email from a specific server will be blocked, no matter what domain it appears to come from. This makes RBLs an effective defense against botnets that use randomly generated sender addresses.
Advantage: RBLs can easily block entire ranges of IP addresses. This is useful when a spammer hosts many servers on leased/owned/stolen blocks of IP addresses.
Advantage: RBLs require less network activity to check than RHSBLs.
Disadvantage: RBLs block IP addresses without regard to the sender address; a single compromised/malicious user can cause the entire server to be blocked.

Advantages of both RHSBLs and RBLs:
By using a list maintained by someone else, you don't have to update a blacklist yourself.

Disadvantages of both RHSBLs and RBLs
Both RHSBLs and RBLs require network activity to use, which incurs a performance penalty. If too many lists are used, the time/bandwidth required to check them all may cause unacceptable delays.
By using a list maintained by someone else, your (and your users') email is at someone else's mercy. You are responsible for your server, so be sure you understand the list administrator's policies before you use their blacklist.

Which RBLs and/or RHSBLs do you recommend?

None. Blacklists are a sensitive issue. Their legal status is uncertain, depending on what country and area you (and they) live in.

Blacklist operators each have different policies about how entries are placed on the list and how they are removed. Some operators are very forgiving, others are not. For some, a simple delisting request is enough while others require more elaborate "proof" a server/domain has "been reformed". Some require money. Many operators have been known to list huge blocks of IP addresses or domains in order to pressure an ISP to cancel a spammer's service. This effectively recruits all of the list's subscribers into a large group to boycott the ISP's services.

For all of those reasons, I am not willing to endorse any specific list (although I use several myself). You must decide for yourself if you are comfortable with a list operator's policies. Be sure to look for complaints against about the list and responses from the operators. Sometimes those can be the best indicator of a list operator's true attitude.

What operating systems and versions do you test spamdyke with?

On the following systems, spamdyke's configuration script executes correctly, it compiles without errors or warnings and displays its usage message:

  • Apple OS X 10.5
  • CentOS 5.2 64-bit
  • Fedora Core 4
  • FreeBSD 2.2.2
  • FreeBSD 4.7
  • FreeBSD 6.0
  • NetBSD 3.1
  • OpenBSD 4.3
  • Solaris 10 x86

spamdyke's test scripts execute correctly on the following systems:

  • Fedora Core 4, netqmail 1.03 (patched with TLS and SMTP AUTH) and vpopmail 5.4.10
  • OpenBSD 4.3, netqmail 1.03 (no additional patches) and vpopmail 5.4.10
If you are unable to configure or compile spamdyke correctly, please send an email to samc (at) silence (dot) org so the errors can be fixed and your system can be added to the list of test environments.

I love spamdyke! How can I help? Can I send you money?

Thank you for your generosity, but I'm not soliciting donations. I've put some text advertisements on the website to try to cover the cost of hosting this server (they don't) but that's about as far as I'm willing to go. I started writing spamdyke to meet my own needs. I continue writing it as a hobby, not to make money. I get much more pleasure and motivation from reading thank-you emails than I would from the occasional donation.

If you want to help, there are several ways:

  • Spread the word about spamdyke. Let others know, create links to this site, post an article on your blog, etc. Everyone hates spam and statistics are very persuasive.
  • Join the spamdyke-users mailing list and help answer questions. The spamdyke community is growing and it needs support.
  • Join the spamdyke-dev mailing list and help test new versions before they are released. Remember Linus' Law: Given enough eyeballs, all bugs are shallow.
  • Proofread the documentation and suggest improvements. Write tutorials for installing spamdyke. Translate existing documentation into other languages.
  • If you are a C programmer, send an email to samc (at) silence (dot) org and volunteer to implement a new feature.

Does spamdyke run its filters in any particular order?

Yes. spamdyke evaluates its filters in the following order (of course a filter is skipped if it's disabled):

Check if mail is being accepted or filtered at all
Check for an rDNS name
Check for an IP address in a country code rDNS name
Check for an rDNS whitelist entry
Check for an rDNS blacklist entry
Check for an IP whitelist entry
Check for an IP blacklist entry
Check for an IP address and keyword in the rDNS name
Check if the rDNS name resolves
Check DNS whitelists
Check right-hand-side whitelists
Check DNS RBLs
Check right-hand-side blacklists
Check for earlytalkers

The intent is to order the filters from least-to-most expensive, so connections will be rejected as quickly as possible. In a typical setup, DNS queries are more expensive than file searches, pattern matching is more expensive than simply checking for a file's existence, etc.

The remaining filters are all checked during the SMTP conversation.

Limit the number of recipients
Block unqualified recipient addresses
Block relaying from unauthorized remote hosts
Check for sender's domain MX record
Graylisting
Check sender whitelists
Check sender blacklists
Check right-hand-side whitelists for the sender's domain name
Check right-hand-side blacklists for the sender's domain name
Check recipient whitelists
Check recipient blacklists
If spamdyke is passing TLS traffic to qmail without processing (i.e. spamdyke wasn't compiled with TLS support or doesn't have access to the server certificate), none of these filters are run.

When spamdyke's config-dir option is in use, no filters are run until after the remote server gives the recipient's address. This is because spamdyke must load any additional configuration files before it will know which filters to run.

My users authenticate with SMTP AUTH. Can I still use spamdyke?

Yes! As of version 2.5.0, spamdyke understands SMTP AUTH and disables all of its filtering for authenticated users.

See the README page for complete details.

My users authenticate with POP3-before-SMTP. Can I still use spamdyke?

Probably not. If your POP3 server writes authenticated IP addresses to a file, you can use that file as an IP whitelist with spamdyke. If it keeps track of authenticated IP addresses in some other way, you're out of luck.

For example, Plesk supports POP3-before-SMTP but it writes the authenticated IP addresses to its database, which spamdyke cannot read.

POP3-before-SMTP is really a kludge anyway; consider using SMTP AUTH instead.

I want to block all emails unless the sender authenticates. Can spamdyke do that?

Yes. As of version 4.0.0, spamdyke accepts the smtp-auth-level option. When it is set to require-auth, all connections are rejected unless the sender has authenticated.

Prior to version 4.0.0, the easiest way to accomplish this was to first enable SMTP AUTH. Then, create an IP blacklist file that will block all IP addresses:

0.0.0.0/0.0.0.0
spamdyke will disable its filters for authenticated users and block everyone else.

Does spamdyke support TLS?

As of version 2.6.0, spamdyke supports TLS (which is just another name for SSL). spamdyke will detect TLS and pass it through seamlessly. Obviously, none of its post-connect filters will work (e.g. graylisting) because all of the traffic will be encrypted.

However, if spamdyke has access to a server certificate, it will handle the TLS itself and all of its filters will work as before. Bonus: spamdyke will provide TLS even if your qmail has not been patched to provide TLS!

As of version 4.0.0, spamdyke supports SMTP-over-SSL (SMTPS) when the tls-level option is set to smtps.

See the README page for complete details.

I want to whitelist a large number of IP addresses; can I use wildcards?

Yes, as of spamdyke version 2.2.0. The whitelist and blacklist IP files will work with partial IP addresses to represent ranges.

As of spamdyke version 2.4.0, whitelist and blacklist IP files can also contain IP addresses as dotted quad IP address followed by a netmask as a number of bits. Also, spamdyke supports dotted quad IP addresses followed by a netmask as a dotted quad.

As of spamdyke version 3.0.0, whitelisted IP addresses can also be found by using a DNS realtime whitelist. This is like a DNS RBL that lists whitelisted IP addresses instead of blacklisted ones.

For complete details, see the README page.

On the mailing list, you often promise changes in an upcoming version. How can I find out what you're working on?

The easiest way is to simply ask. Send an email.

Other than that, check out the TODO.txt file. The file contains notes to indicate the intended major changes in the next few versions. Those notes are not a completely accurate predictor of what will happen but they'll give you an idea of what's happening.

Why doesn't spamdyke use CDB files or a database? A database would be faster and better than text files and strange directory structures!

Reasons why files and directory structures are preferable to databases:

  1. Speed. Because spamdyke doesn't run as a daemon, a database engine must be loaded and initialized for every incoming connection. That takes time, so the database engine must be very fast to keep the overall time lower than using plain text files.
  2. Memory. Because one copy of spamdyke is started for every incoming connection, memory usage is a big concern on busy servers. Most qmail installations use DJB's softlimit program to limit memory usage for exactly this reason. Database libraries use memory, often quite a bit, to load/cache/parse data. spamdyke must be able to fit within a reasonable limit.
  3. Concurrency. On a busy mail server, hundreds (possibly thousands) of spamdyke processes could be running at the same time. A database engine would have to handle that kind of simultaneous access without failing.
  4. Portability. At the moment, spamdyke runs on every Unix-like platform I've tested. The only external library it uses is OpenSSL (for TLS support) and even that is optional. I believe spamdyke's simplicity is largely responsible for its popularity -- nothing extra must be installed, no existing programs must be recompiled. Requiring a mail server administrator to install a database engine could scare away potential users.
  5. Accessibility. By using only plain text files and directories, spamdyke is easy for anyone to administer and reconfigure with standard command line tools. Any administrator can understand a file of whitelisted IP addresses without help. Very few administrators know SQL. This is very important to me -- I hate proprietary file formats that can only be accessed with special tools. I can't remember how many times I've stared at a malfunctioning Sendmail server and been unable to determine if a given option was even available, much less enabled. This point and the next point are very closely related.
  6. Safety. Plain text files and directories are easy to understand, easy to back up and easy to restore. They can be printed out, emailed, imported into other programs, etc. Most importantly, it's very difficult to corrupt a plain text file by "improperly" stopping spamdyke or losing power. In an emergency, when an administrator is trying to restore a mail server while users are screaming at him, I don't want him wondering if spamdyke's files are intact. If he has any doubts, he should be able to visually verify them with any available text editor. (I've had to restore corrupted Exchange Private Mail Stores in the middle of the night. I wouldn't wish that experience on anyone.)
  7. Availability. Mail servers should depend on as few external systems as possible. spamdyke already depends heavily on DNS but that's unfortunately unavoidable. I've done everything I can to make sure spamdyke fails gracefully if DNS is down. Fortunately, DNS servers are (usually) very stable and very reliable. Database engines are not in the same league -- while many databases are fairly reliable they still have much higher downtime than DNS servers. As stated above, I don't want to force a stressed administrator to restore his database server just to get mail flowing again. If I were forced to do that, I would choose instead to uninstall spamdyke (and I would never use it again).

Database servers like MySQL conflict with all 7 points. Embedded database engines like SQLite conflict with points 1, 2, 5 and 6. CDB files conflict with points 5 and 6.

This doesn't mean spamdyke will never use a database. It just means there's a lot to think about. A database would have to solve a really tough problem before it would be considered. Whether it saves some coding time is really not a factor -- the time required to install and administer spamdyke is more important.

Why doesn't spamdyke filter using Sender Policy Framework (SPF), Sender ID, Certified Server Validation (CSV), DomainKeys or DomainKeys Identified Mail (DKIM)?

Each of those systems is very complex, so adding support to spamdyke will not be a small task. They're also all different, so supporting all of them would be a major undertaking. At this point, there are other ways to improve spamdyke that will take less effort and provide much more benefit.

Additionally, I'm not convinced any of those systems will make any difference. They were each designed to prevent spam being sent from forged addreses so, for example, you won't get spam from president@whitehouse.gov. However, most spammers own their domains (often many thousands of them) and control their own DNS, so they can (and do) add SPF/CSV/etc records so their spam follows the SPF/CSV/etc rules. (Hint: Spammers understand and administer their DNS records better than most ISPs do.) The major ISPs (AOL, Yahoo!, GMail) use these systems and still have spam problems; this is pretty good evidence they aren't living up to the hype.

There is a (small) chance these systems could stop spam coming from botnets. If that happens however, the spammers will just start relaying their spam through the compromised machine's ISP's mail servers. (Hint: That's why having every ISP block all outbound port 25 traffic won't stop botnet spam either.)

If you really, really want to filter your email using one or more of these frameworks, try using SpamAssassin. It already supports most of them and you'll be able to judge their effectiveness for yourself.

Why can't spamdyke filter based on message headers like "From:" or "Received:"? Why can't spamdyke block large messages or strip attachments?

To understand why, a little explanation of the SMTP protocol is necessary.

After some conversational preliminaries, an SMTP client gives the server a sender address. This address is referred to as the "envelope sender" and it's used as a return address if the message bounces. Here's the part most users don't understand: The "envelope sender" doesn't have to be the same address you see on the "From" line. The two aren't related at all; the "From" line is mostly for show (this is why spam "From" lines are so often forged).

If the server accepts the "envelope sender" address, the client sends the addresses of the message recipients, one at a time. The recipient addresses are for message delivery. The server must accept or reject each recipient individually. Once again, the "To" and "Cc" lines aren't related at all; they're mostly for show.

If the server accepts one or more recipients, the client sends the message. The server accepts or rejects the message and the conversation is over.

spamdyke makes a decision whether to accept or reject senders and recipients during the first two steps because those steps must be acknowleged separately. Once the client sends the actual message, it can't be rejected.

Well, yes, technically, according to the RFCs, it can be rejected. However, most MTAs don't handle this situation correctly and bounce the message if they see anything other than a success code after sending the message. This is a Bad Thing. If the rejection was for graylisting, the client won't retry delivery. If the rejection was for one recipient out of many, the client (and therefore the sending user) won't understand why the error was given. The message will just bounce and that's it.

There are many, many tools for filtering messages based on headers and content that are far, far smarter and feature-rich than spamdyke. If you really want to filter based on message headers, please use one of them in addition to spamdyke. Personally, I use (and recommend) SpamAssassin and ClamAV.

Why can't spamdyke validate a recipient address before accepting a message?

This would be very nice to have but it's not very easy to implement. Qmail doesn't make it easy to determine if an address is valid and vpopmail (if present) only complicates matters. Since it's better to accept spam than reject legitimate email, this feature doesn't exist (yet).

If you have a suggestion or method for checking recipient validity, I'd love to know about it. Note: recompiling qmail or reusing qmail's source code is not an option.

My graylist directories are getting huge -- many, many entries. Is this is a problem? What can I do?

For most servers, it isn't a problem, since most of the files contain no data. However, it might be a problem if the server's filesystem runs out of inodes. (An inode is basically a file index within the filesystem. The number of available inodes is large but not infinite; when a file system runs out of inodes, no new files can be created.)

On most Unix-like systems, the df command will show the amount of free space on all of the mounted filesystems. To show the number of available inodes, use the command df -i.

To remove old graylist files on systems with GNU tools installed, the following command should work:

find GRAYLIST_FOLDER -type f -mmin +$[SPAMDYKE_MAX_GRAYLIST_SECS/60] -print0 | xargs -0 rm -f
Be sure to replace GRAYLIST_FOLDER with the path to your graylist directory, as given in the spamdyke command line or configuration file. Also replace SPAMDYKE_MAX_GRAYLIST_SECS with the maximum graylist age from the spamdyke command line (-M) or configuration file.

Be careful about deleting graylist entries -- it is possible to be too aggressive and delete them before the remote server(s) have finished delivering their message(s). If the entries disappear while a remote server is still trying to pass the filter, it will be graylisted again (and again and again). Eventually, the remote server wil stop retrying and the message will bounce back to the sender. To avoid this, you shouldn't delete any graylist entries that are less than two weeks old.

Why can't spamdyke automatically delete old graylist entries?

spamdyke doesn't automatically delete old graylist entries because it doesn't run as a daemon. In other words, spamdyke doesn't run constantly on the server. It only runs while a message is being received. If spamdyke suddenly decided to clean up the graylist directory, it would have to do that work while receiving an incoming email message.

If your graylist directory is large, the cleanup could cause a large enough delay to bounce the message. Also, how is spamdyke to decide when to cleanup? Because a new spamdyke process is started for each incoming connection and those processes don't communicate with each other, there's a good chance multiple spamdyke instances would attempt to cleanup the directory at the same time. That would delay multiple incoming messages and place an unnecessary load on the server.

Instead of trying to prevent relaying itself, why doesn't spamdyke just set the RELAYCLIENT environment variable based on authentication/sender address/recipient address?

Unfortunately, that isn't possible because of the way environment variables work.

Some background: Environment variables are set when a program is started. Most of the time, we care about them because the login shell (e.g. bash) keeps a list of environment variables that it sets for every process that it starts (i.e. when commands are typed at the prompt). However, each process that runs on a system has an environment, set by the process that started it (the parent process). Once the child process has been started, the parent process cannot alter its environment any more.

More background: When the RELAYCLIENT environment variable is set before qmail-smtpd is started, qmail will allow the connection to relay. Typically, RELAYCLIENT is set in the access file (e.g. /etc/tcp.smtp) based on the IP address of the remote server.

spamdyke must determine what environment variables to set before it starts the qmail-smtpd process (because after qmail-smtpd has been started, spamdyke can't change its environment). For that reason, spamdyke always sets the RELAYCLIENT environment if it has enough information to run its relaying filter. That way, qmail-smtpd will not prevent relaying if spamdyke determines it is allowed (e.g. because the connection is whitelisted).

If spamdyke's configuration does not include the access file and a list of the locally hosted domains, it cannot run its relay filter; it won't set the RELAYCLIENT environment variable.

As of version 4.0.0, the relay-level option can also be used to more precisely control spamdyke's relay filter.

Why can't spamdyke automatically blacklist (or delay) servers that are rejected too many times?

Technically, this is not possible because spamdyke of the way spamdyke works. Without a long-running daemon to keep track of the statistics and "penalties", each instance of spamdyke would have to perform the accounting work on its own. This would take too long and is very likely to be error-prone.

However, even if it were possible, this kind of feature is very problematic, for several reasons.

  1. The only way spamdyke could possibly decide to block a server would be to examine its rejection statistics (i.e. number of rejections per time period exceeds a threshold). However, if spamdyke is already rejecting messages, what is the purpose of blacklisting the server? In effect, that would enable two filters for blocking a server when one is already enough.
  2. Innocent mistakes by users of large email systems could unfairly penalize every user of that system. For example, if an AOL user sends several messages that are rejected, an automatic blacklist might block all messages from all AOL users. This is not acceptable.
  3. Malicious users could use an automatic blacklist to conduct a denial-of-service attack. By sending multiple invalid messages from a victim's server, an attacker could cause the automatic blacklist to block all messages from the victim server.

In summary, an automatic blacklist is likely to cause more problems than it solves. For that reason, spamdyke is unlikely to ever contain such a feature.

However, just because spamdyke doesn't offer an automatic blacklist doesn't mean one can't be created. A script that analyzes spamdyke's logs and modifies a blacklist file would be just as effective.

Graylisting isn't working! What am I doing wrong?

Most often, graylisting doesn't work because there are no domain directories. spamdyke is designed to allow some flexibility when configuring graylisting, so you can enable it for some domains and disable it for others.

As of version 4.0.0, spamdyke can automatically create domain directories as they are needed. To enable this, look at spamdyke's graylist-level option. If it is always, change it to always-create-dir. If it is only, change it to only-create-dir.

To enable graylisting for a domain without enabling automatic directory creation, you must create a directory within your graylist directory named for the domain you want to graylist. For example, suppose you created a graylist directory:

/home/vpopmail/graylist.d
Then you ran spamdyke with a command line like this:
/usr/local/bin/spamdyke -g /home/vpopmail/graylist.d ...
No graylisting will take place. To enable graylisting for the domain example.com, you must also create another directory:
/home/vpopmail/graylist.d/example.com
Once you've done that, graylisting will begin for all recipients within the example.com domain. Create a directory for each domain you want to graylist.

spamdyke can automatically catch configuration problems like this. See the README page for details.

I want to use spamdyke but some of my users roam and connect from strange places. How can I allow them to send email but still filter spam?

As of version 2.5.0, this is no problem. spamdyke understands SMTP AUTH, so it can authenticate your users and bypass all of its filtering just for them. Bonus: spamdyke will provide SMTP AUTH even if your qmail has not been patched to provide SMTP AUTH!

See the README page for complete details.

I installed spamdyke and now I'm seeing a lot of timeouts in my logs. Why?

Badly written software on the remote hosts. It seems a lot of spam software doesn't handle error codes at all. It just attempts delivery and expects success. When an error code is sent, the software just sits and waits for the success code it wants. Eventually, the connection times out. Sometimes, a remote server will take a long time to begin delivering a large (legitimate) message, which can cause timeouts.

qmail enforces a 20 minute idle timeout but it does so silently (no logging). It's possible you were already getting timeouts and just didn't know it until spamdyke began logging them.

If you suspect legitimate connections are timing out, there are two things you can do. First, you can increase or disable spamdyke's timeouts. Of course, qmail's 20 minute idle timeout will still apply but at least you'll be back where you were before.

Second, you can use spamdyke's full logging feature to log all incoming connections to files. The log files contain timestamps so you can see how quickly the remote server is sending data and where it's stopping. Hopefully that will yield some clues you can use to fix the problem.

I installed spamdyke and now my server is very slow! Incoming connections have to wait 20 seconds or longer before they see the greeting banner. What can I do speed it up?

Most often, delays like these are due to DNS traffic. Depending on the enabled filters, spamdyke can perform the following DNS queries for each incoming connection:

  1. Find the reverse DNS name for the remote server
  2. Find the IP address for the remote server's reverse DNS name
  3. Check the realtime whitelists for TXT records matching the remote server's IP address
  4. Check the realtime whitelists for A records matching the remote server's IP address
  5. Check the realtime blacklists for TXT records matching the remote server's IP address
  6. Check the realtime blacklists for A records matching the remote server's IP address
  7. Check the righthand-side whitelists for TXT records matching the remote server's rDNS name
  8. Check the righthand-side whitelists for A records matching the remote server's rDNS name
  9. Check the righthand-side blacklists for TXT records matching the remote server's rDNS name
  10. Check the righthand-side blacklists for A records matching the remote server's rDNS name
  11. Check the righthand-side whitelists for TXT records matching the sender's domain name
  12. Check the righthand-side whitelists for A records matching the sender's domain name
  13. Check the righthand-side blacklists for TXT records matching the sender's domain name
  14. Check the righthand-side blacklists for A records matching the sender's domain name
That's a lot of queries. If your DNS servers aren't fast enough to keep up, you're going to see delays in spamdyke. To alleviate this, try installing a caching nameserver on your mail server. If you still see delays, begin disabling DNS-based filters until spamdyke begins running faster.

If you have disabled all of the filters and spamdyke is still running slowly, you may have found a bug. Please report it!

I use spamdyke to prevent relaying (because my qmail isn't patched to provide SMTP AUTH) but SpamAssassin has stopped scanning incoming messages. What gives?

You must be using qmail-scanner from qmail-scanner.sourceforge.net. That package has an interesting flaw: it assumes any time the environment variable RELAYCLIENT is set, no scanning should be performed. When spamdyke prevents relaying, it always sets that variable to keep qmail from interfering with its relaying decision.

To reenable scanning, modify your /etc/tcp.smtp file and add QS_SPAMASSASSIN to all of the connections you want scanned. For example:

127.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue"
:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue",QS_SPAMASSASSIN=""
Rebuild the CDB file with qmailctl cdb and you should be fine.

My logs contain the message ERROR: unable to write X bytes to file descriptor 1: Broken pipe. What does it mean?

The message means spamdyke was trying to send some data to the remote server, but the connection was closed before all of the data was sent. This can indicate a problem (which is why spamdyke logs it) but most often it indicates a spammer's server disconnected as soon as it received an error message.

This error is usually safe to ignore. You can prevent it from being logged by lowering spamdyke's log-level setting.

When I use the config-test feature on my Plesk server, the SMTP AUTH tests always fail. But I know authentication works, so what's wrong?

Plesk uses a program called relaylock to provide some of the features spamdyke includes. One of relaylock's jobs is to offer (and probably process) SMTP AUTH.

However, relaylock won't offer SMTP AUTH to connections that come from the local host. It uses the TCPREMOTEIP environment variable to determine the IP address of the connecting server. Try setting it to something other than the localhost IP. Within the bash shell, you can use this command:

export TCPREMOTEIP=11.22.33.44
Within the zsh shell, this command will work:
setenv TCPREMOTEIP 11.22.33.44
If the TCPREMOTEIP environment variable isn't set at all, spamdyke should set it to 0.0.0.0 before it starts its tests.

Why isn't spamdyke blocking messages from some blacklisted servers/senders/recipients?

Most likely, the problem is TLS. If the remote server is using TLS to encrypt its transmission and spamdyke doesn't have access to the certificate (using the "tls-certificate-file" option), spamdyke can't decrypt the traffic to monitor the entire connection. In that situation, spamdyke will not block the connection if there is any chance it could be allowed.

For example, if the remote server should be blocked because its IP address is blacklisted, spamdyke normally won't reject the connection until it has exhausted every opportuntity to be allowed. If SMTP AUTH is possible, spamdyke must wait to see if the server authenticates. If sender or recipient whitelists are in use, spamdyke must wait to see if any of those whitelists are matched. However, when TLS is in use and spamdyke can't decrypt the traffic, the SMTP AUTH, the sender and the recipient will all be hidden. spamdyke must allow the connection to continue just in case.

Most of the time, when qmail provides TLS for a connection, it will use the text (DHE-RSA-AES256-SHA encrypted) in the Received header entry it adds to the top of the message. If you see that text in the headers of messages that spamdyke should be blocking, TLS is the issue. Change spamdyke's configuration so it can decrypt the TLS traffic and it will block the connections.

I can't figure out why spamdyke isn't working correctly -- some features are malfunctioning in strange ways OR I'm seeing strange/impossible error messages in my logs. What's wrong?

Of course, the most likely explanation is a configuration problem. The config-test feature can help find common mistakes. You may also have found a bug in spamdyke.

However, occassionally very strange malfunctions/errors are reported that no one is able to explain. In those cases, the culprit often turns out to be low memory. Many qmail installations use DJB's softlimit program to limit the amount of memory that can be used by a single connection. The limit must allow enough room for spamdyke, qmail and any other filters that have been added. If the limit is too low, the results are often strange behavior instead of the "out of memory" errors you might expect.

If you're seeing strange malfunctions/errors and you suspect memory might be the issue, first try setting it to a rediculously high value for a short time. If the malfunctions/errors stop, you've found the cause.

Unfortunately, there doesn't seem to be a good way to determine what the limit should be. Some tutorials recommend 10 MB, others 40 MB or even 80 MB. Some guides recommend using trial-and-error to find a limit where everything appears to work, then doubling it.

Good luck!