Configuration¶

There are multiple ways how fuglu can be integrated into Postfix:

After Queue (one filter)
After Queue (separate inbound/outbound filter)
Before Queue
Milter

Basic Configuration¶

Fuglu in After-Queue Mode¶

** Pros and Cons **

This is the basic integration mode. Postfix will first accept the mail, enqueue it locally, then pass it to Fuglu. This way connections from delivering servers are fast to close.

This setup is ideal for machines with heavy work load and the need to inspect mail thoroughly or with longer running tasks.

In this setup Fuglu should only return DUNNO or DISCARD (or DEFER for recoverable errors). If Fuglu returns REJECT then the message will create a bounce.

** Instructions **

First of all, ensure that directory /etc/fuglu is exists. If you install via pip, for example in debian, /usr/local/lib/python<version>/dist-packages/etc/ copy this directory to /etc/ and remove the .dist ending from all files in /etc/fuglu, so you have a basic default configuration.

edit /etc/fuglu/fuglu.conf and change parameters if you like

Note: If you don’t have Clamav (clamd) or Spamassassin (spamd) , you need to remove those entries from the plugins option.

eg, change

plugins=archive,clamav,spamassassin,attachment

to

plugins=archive,attachment

to disable both Antispam and Antivirus.

Make sure the logging directory exists:

mkdir -p /var/log/fuglu
chown nobody /var/log/fuglu

Then check if fuglu is happy with your configuration:

python -m fuglu --lint

example:

python -m fuglu --lint
Fuglu 0.6.1
---------- LINT MODE ----------
Checking dependencies...
sqlalchemy: installed
BeautifulSoup: installed
magic: installed
Loading extensions...
fuglu.extensions.sql: enabled (available)
Loading plugins...
Plugin loading complete
Linting  main configuration
OK

Linting Plugin  Archive Config section: ArchivePlugin
OK

Linting Plugin  Attachment Blocker Config section: FiletypePlugin
Found python-file magic library
No database configured. Using per user/domain file configuration from /etc/fuglu/rules
OK

Linting Plugin  Clam AV Config section: ClamavPlugin
Virusaction: DELETE
Got Pong: PONG

Clamav found virus {'stream': 'Eicar-Test-Signature'}
OK

Linting Plugin  SpamAssassin Config section: SAPlugin
Got: SPAMD/1.5 0 PONG

GTUBE Has been detected correctly
OK

Linting Plugin  Debugger Config section: debug
OK

Linting Plugin  Plugin Skipper Config section: PluginSkipper
OK
0 plugins reported errors.

If all went fine, run fuglu without parameters to start the daemon. In /var/log/fuglu/fuglu.log you should then see output similar to this:

2014-08-13 15:25:46,093 root        : INFO FuGLU Version 0.6.1 starting up
2014-08-13 15:25:46,094 fuglu.MainController: INFO Init Stat Engine
2014-08-13 15:25:46,094 fuglu.MainController: INFO Init Threadpool
2014-08-13 15:25:46,095 fuglu.MainController: INFO Starting interface sockets...
2014-08-13 15:25:46,095 fuglu.MainController: INFO starting connector smtp/10025
2014-08-13 15:25:46,096 fuglu.MainController: INFO starting connector smtp/10099
2014-08-13 15:25:46,096 fuglu.incoming.10025: INFO SMTP (After Queue) Server running on port 10025
2014-08-13 15:25:46,096 fuglu.MainController: INFO starting connector smtp/10888
2014-08-13 15:25:46,096 fuglu.incoming.10099: INFO SMTP (After Queue) Server running on port 10099
2014-08-13 15:25:46,097 fuglu.incoming.10888: INFO SMTP (After Queue) Server running on port 10888
2014-08-13 15:25:46,097 fuglu.MainController: INFO Startup complete
2014-08-13 15:25:46,097 fuglu.control.fuglu_control.sock: INFO Control/Info Server running on port /tmp/fuglu_control.sock

fuglu comes with init scripts/systemd service files if you want to start fuglu like other system daemons

Postfix (filter all mail equally)¶

Once fuglu is up and running, we need to tell Postfix to use it as an after queue filter.

All mail will be sent to the same filter. This is useful if this machine only processes incoming (or only outgoing) mail.

Detailed documentation is available on the postfix website, but here is a quick example.

Add to master.cf :

fuglu_default   unix  -       -       n       -       10      smtp
            -o smtp_send_xforward_command=yes
            -o disable_mime_output_conversion=yes

    localhost:10026 inet  n       -       n       -       10      smtpd
            -o content_filter=
            -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters,no_address_mappings
            -o smtpd_helo_restrictions=
            -o smtpd_client_restrictions=
            -o smtpd_sender_restrictions=
            -o smtpd_relay_restrictions=
            -o smtpd_recipient_restrictions=permit_mynetworks,reject
            -o mynetworks=127.0.0.0/8
            -o smtpd_authorized_xforward_hosts=127.0.0.0/8

In main.cf add

content_filter = fuglu_default:127.0.0.1:10025
fuglu_default_destination_concurrency_limit = 10

Run postfix reload and check your maillog for errors. from now on, messages should be filtered by fuglu!

Advanced Config¶

Postfix with separate filtering for inbound and outbound mail¶

Once fuglu is up and running, we need to tell Postfix to use it as an after queue filter.

Mail will be sent to different ports of fuglu, depending on postfix restrictions. This is useful if this machine handles incoming and outgoing mail.

Detailed documentation is available on the postfix website, but here is a quick example.

Add to master.cf :

#fuglu
fuglu_default   unix  -       -       n       -       10      smtp
        -o smtp_send_xforward_command=yes
        -o disable_mime_output_conversion=yes

fuglu_trusted   unix  -       -       n       -       10      smtp
        -o smtp_send_xforward_command=yes
        -o disable_mime_output_conversion=yes

#
localhost:10026 inet  n       -       n       -       10      smtpd
        -o content_filter=
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters,no_address_mappings
        -o smtpd_helo_restrictions=
        -o smtpd_client_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_relay_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks=127.0.0.0/8
        -o smtpd_authorized_xforward_hosts=127.0.0.0/8

This creates two filters fuglu_trusted for outgoing mail and fuglu_default for incoming mail. An additional postfix instance will listen on port 10026 where fuglu can re-inject filtered messages.

Create a new file /etc/postfix/filter_default

# filter for others
/./ FILTER fuglu_default:[127.0.0.1]:10025

This tells postfix to send untrusted messages to fuglu on port 10025

Create a new file /etc/postfix/filter_trusted

# filter for authenticated users
/./ FILTER fuglu_trusted:[127.0.0.1]:10099

This tells postfix to send trusted (outgoing) mail to fuglu on port 10099. Fuglu can be configured to treat outgoing mail differently, e.g. skip spam scanning but keep antivirus scanning

In main.cf add

smtpd_recipient_restrictions =  check_client_access pcre:/etc/postfix/filter_trusted,
                                                                permit_sasl_authenticated,
                                                                permit_mynetworks,
                                                                reject_unauth_destination,
                                                                check_client_access pcre:/etc/postfix/filter_default

fuglu_trusted_destination_recipient_limit=1
fuglu_default_destination_recipient_limit=1

Run postfix reload and check your maillog for errors. from now on, messages should be filtered by fuglu!

Running a plugin multiple times with different config sections¶

By default, plugins search your configfile for a section named like the plugin itself, for example, the ArchivePlugin looks for a [ArchivePlugin] section. You can pass the plugin definition an argument to override this with a section named to suit your needs. This can also allow running a plugin multiple times per scan with different config options:

Example:

plugins=archive(ArchiveConfig1),spamassassin,clamav,archive(ArchiveConfig2)

[ArchiveConfig1]
....

[ArchiveConfig2]

Fuglu in Before-Queue Mode¶

The fuglu ESMTP connector enables fuglu to run in before-queue mode, i.e. while the smtp session with the remote system is still active. This allows rejecting spam / infected mails.

Postfix config: edit master.cf as described in http://www.postfix.org/SMTPD_PROXY_README.html

# Before-filter SMTP server. Receive mail from the network and
# pass it to the content filter on localhost port 10025.
#
smtp      inet  n       -       n       -       20      smtpd
    -o smtpd_proxy_filter=127.0.0.1:10025
    -o smtpd_client_connection_count_limit=10
    # Postfix 2.7 and later performance feature.
    # -o smtpd_proxy_options=speed_adjust
#
# After-filter SMTP server. Receive mail from the content filter
# on localhost port 10026.
#
127.0.0.1:10026 inet n  -       n       -        -      smtpd
    -o smtpd_authorized_xforward_hosts=127.0.0.0/8
    -o smtpd_client_restrictions=
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=
    -o mynetworks=127.0.0.0/8
    -o receive_override_options=no_unknown_recipient_checks

fuglu config:

Enable the esmtp connector on the incoming port (10025) in /etc/fuglu/fuglu.conf:

incomingport=esmtp:10025,10099,10888

rejecting spam:

In the fuglu configuration set DEFAULTHIGHSPAMACTION=REJECT (do the same for DEFAULTLOWSPAMACTION if you want to reject all spam). in the Spamassassin Pluginconfiguration, you can configure a custom reject message which supports various template variables:

${from_address} : sender address

${to_address} : recipient address

${from_domain} : sender domain

${to_domain} : recipient domain

${subject} : message subject

${spamscore} : spamassassin score

${date} : current date

${time} : current time

Example:

rejectmessage=message from ${from_address} to ${to_domain} identified as spam (score=${spamscore})

rejecting viruses:

This works the same way as rejecting spam, by setting DEFAULTVIRUSACTION=REJECT and configuring the rejectmessage in the antivirus plugin section.

Example fuglu configuration

With this example pre-queue configuration fuglu will scan for spam, virus and blocked attachments. low spam will be tagged, high spam (level>=10) will be rejected.

[main]
plugins=clamav,spamassassin,attachment
incomingport=esmtp:10025,10099,10888

[spam]
DEFAULTLOWSPAMACTION=DUNNO
DEFAULTHIGHSPAMACTION=REJECT

[virus]
DEFAULTVIRUSACTION=REJECT

[FiletypePlugin]
sendbounce=0
blockaction=REJECT

[SAPlugin]
rejectmessage=Sorry, this message looks like spam to me!
highspamlevel=10

Running fuglu as a milter¶

Milter mode is a sophisticated integration of Fuglu into Postfix, allowing to communicate with Postfix at any stage of the SMTP process while handling a message.

Install libmilter Python 3¶

Don’t use pip3 to install libmilter from pypi since it will install the python 2 version
Download python3 branch from github (https://github.com/crustymonkey/python-libmilter/tree/python3) and install using “setup.py”

Configure¶

To run fuglu as a milter, add milter:<portnumber to the incomingport configuration option in fuglu.conf.

incomingport=10025,10099,10888,milter:10028

Then enable milters in your MTA. postfix example (main.cf) :

milter_protocol = 6
milter_default_action = accept
milter_content_timeout = 30s
smtpd_milters=inet:127.0.0.1:10028

The following configuration options are available and can be added to fuglu.conf in the namelist [milter]

[milter]
# available options: readonly,manual,auto,autoheaders,tags,replace_demo
milter_mode=readonly

# available options: all,body,headers,from,to
milter_mode_options=

milter_mode:

readonly -> this does not allow to change anything, but it is possible to REJECT/DEFER/DISCARD based on plugin results
replace_demo -> TESTING ONLY this will replace all messages by a dummy replacement message keeping only from/to
manual -> use options defined in “milter_mode_options”
auto -> this will check if the suspect has been changed and then replace corresponding message parts
tags -> check for actions in the suspect tag: milter_replace

milter_mode_options:

all -> replace whole message by fuglu suspect
body -> replace body by fuglu suspect body (careful to remain consistent with headers (multipart msg))
headers -> replace headers by fuglu headers (careful to remain consistent with body (multipart msg))
from -> replace envelope from by fuglu suspect from_address
to -> replace envelope to fuglu suspect to_address

NOTE:

headers added by fuglu to suspect will be added anyway
options can be combined, for example: milter_mode_options=from,to or milter_mode=manual,tags
if readonly or replace_demo is in milter_mode, no other option will apply
milter_mode options are applied in the order: auto -> manual -> tags so tags can overwrite previous values
the mode-options force replace even if milter_mode (if in auto or tags mode) would not replace the option

Control milter replacing message parts in plugins

If tags is in milter_mode it is possible to define replacement options on a per-message basis using message tags in Suspect. Same options as in milter_mode_options can be set in the tag milter_replace using the method set_tag of Suspect

suspect.set_tag["milter_replace"] = "from,to"

Running fuglu in netcat-mode¶

fuglu supports message processing by simply “piping” them into a socket. To enable this socket, add netcat:<portnumber> to the incomingport configuration option in fuglu.conf.

incomingport=10025,10099,10888,netcat:20099

When restarting fuglu, you should see something like INFO NETCAT Server running on port 20099 in the logs. you may then pipe an eml from the shell

(sleep 0.1; cat eicar.eml) | nc -c localhost 20099

In some special setups, people use fuglu as an after-delivery filter with this method.

Example procmail filter:

# netcat send msg copy to fuglu
:0c
| nc localhost 20099

Note: in netcat mode fuglu only receives message headers and body, no envelope data. Fuglu will assign the dummy value unknown@example.org for both envelope sender and recipient.

Fetching scan-time configuration values from a database¶

It is now possible to fetch certain configuration options at runtime, based on the recipient email address or domain.

For this to work you need to have sqlalchemy installed ( running python -m fuglu –lint should show enabled for the sql extension)

SQL script to create the table:

CREATE TABLE `fugluconfig` (
  `scope` varchar(255) NOT NULL,
  `section` varchar(255) NOT NULL,
  `option` varchar(255) NOT NULL,
  `value` varchar(255) NOT NULL,
  PRIMARY KEY (`scope`,`section`,`option`)
)

The process is very similar to spamassassin’s SQL configuration (http://wiki.apache.org/spamassassin/UsingSQL ), with a few minor differences:

we require two columns to identify a preference: section and option

username is called scope

fuglu does not load the full configuration per user, only the actual config options requested by plugins supporting db overrides

Enable configuration lookups by providing a standard sql alchemy connectstring in your configuration (section databaseconfig option dbconnectstring)

[databaseconfig]
dbconnectstring=mysql://root@localhost/fuglu

If you use the above table structure in MariaDB / MySQl, the built-in default sql statement should work, but you may use a custom statement (sql=SELECT …. in the same section. ) use :section, :option and the usual suspect variables like :to_domain and :to_address as placeholders.

Note: only the following plugins currently supports reading database overrides for the listed options:

Spamassassin: lowspamaction, highspamaction, highspamlevel
FiletypePlugin: sendbounce, checkarchivenames, checkarchivecontent, enabledarchivetypes

multi-processing mode¶

by default, fuglu runs multithreaded. This is usually a fine choice as many tasks are I/O-bound (waiting on sockets etc). However, for environments running CPU-bound tasks directly in fuglu plugins this can become a bottleneck. Due to the python global interpreter lock, only one operation can run at a time and fuglu does utilize only one CPU core.

For these setups, fuglu can be switched to multiprocessing mode:

[performance]
backend=process
initialprocs=0

initialprocs defines the number of worker processes that fuglu should start. If set to 0 , fuglu will default to twice the number of available virtual CPU cores.

Most things will work similarly as in threaded mode, but there are a few differences:

in threaded mode, there is only one instance per plugin. In multiprocessing there is one instance per worker. This means increased memory usage, as in-memory caches are built multiple times.
dynamic config reloading is not currently supported in multiprocessing mode. Restart fuglu for config changes to take effect.
the number of workers is not auto-adapted like in threading mode