Syslog-NG Deployment with SSL/TLS Encryption Enabled

Syslog-NG Deployment with SSL/TLS Encryption Enabled
Syslog-NG is an open source implementation of the Syslog protocol for Unix and Unix-like systems. It extends the original syslogd model with content-based filtering, rich filtering capabilities, flexible configuration options and adds important features to syslog, like using TCP for transport. As of today syslog-ng is developed by Balabit IT Security Ltd. It has two editions with a common codebase.
Syslog-NG has a much larger scope than merely transporting syslog messages and storing them to plain text log files:

• the ability to format log messages using UNIX shell-like variable expansion;
• the use of this shell-like variable expansion when naming files, thus covering thousands of destination files with a single statement;
• the ability to send log messages to local applications;
• ability to message flow-control in network transport;
• logging directly into a database (since syslog-ng OSE 2.1);
• rewrite portions of the syslog message with set and substitute primitives (since syslog-ng OSE 3.0);
• classify incoming log messages and at the same time extract structured information from the unstructured syslog message (since syslog-ng OSE 3.0);
• generic name-value support: each message is just a set of name-value pairs, which can be used to store extra information (since syslog-ng OSE 3.0);
• the ability to process structured message formats transmitted over syslog, like extract columns from CSV formatted lines (since syslog-ng OSE 3.0);

DEPLOYMENT STRATEGY
The best way to configure your server infrastructure is to have a centralized Syslog/Audit Server. Upon this server which consists of either RHEL 5.x or CentOS 5.x along with syslog-ng v3 RPM package installed, plus have the configuration setup for TLS encryption channels and a 20-Year shared key certificate. Along with the new deployment and configurations, the general OS, Apache and Windows channels will be using the SSL/TLS encrypted IP addresses and ports.The Syslog-NG v3.0.8 application uses a redesigned configuration format, new TLS encrypted IP addresses/port channels and a pair of RPM packages:

Syslog-NG Server:  syslog-ng-3.0.8-1.rhel5.x86_64.rpm
Syslog-NG Clients:  syslog-ng-client-3.0.8-1.rhel5.x86_64.rpm
All other RHEL client-servers will have the syslog-ng-client v3 RPM package installed. The global syslog-ng configuration of these RHEL clients have the added capability for monitoring the Apache web logs, but are configured in a non-enabled state. Once all RHEL client-servers have been deployed, certificates copied and configured to a running state and able to see the Syslog-NG/Audit server (as seen through the general OS log path), the Apache log reader configuration and its destination channel will be configured on the clients to start actively reading the httpd log files and sending it to the Syslog-NG/Audit server and their specialized directory path.
IMPACT STATEMENT
No server-side reboots will be required. However, local logging will be suspended until the new RPM package has been installed and the modified configuration has been deployed along with the installation of the TLS shared certificate keys per client and also on the master syslog server.

INSTALLATION OF SYSLOG-NG
Server/Syslog Repository: "yum localinstall syslog-ng-3.0.8-1.rhel5.x86_64.rpm"
Client/Workstation: "yum localinstall syslog-ng-client-3.0.8-1.rhel5.x86_64.rpm"

CREATION OF SYSLOG-NG Certificate
Step1)  On the server, generate an X.509 certificate in PEM format.  (I'm using PKI on RHEL/CentOS 5.5 for this tutorial).

Detailed steps for using PKI for this (NOTE: root privileges will be required to write the .pem file)

a) cd /etc/pki/tls/certs

b) make syslog-ng.pem

c)  Provide demographic information to be included in the certificate

NOTE:  It's important to match the COMMON NAME portion of the certificate to the short hostname of your syslog-ng server.  I have had no problems, but according to the syslog-ng Administration Guide, you may experience client authentication errors if this doesn't match.

d) You may want to <#ls> just to verify the file has been created.  If you experienced errors generating the key, try it again with absolute root privileges (i.e. su - ).  If you still have problems, try a different program to generate the key and you may want to troublshoot why it didn't work with PKI.

Step2)  Create two directories within /opt/syslog-ng/etc/ called cert.d and key.d, respectively

Step3)  Using a text editor (vi works well for this), open the PEM file you created in step 1, delete the entire RSA Private Key portion of the text (to include the ----BEGIN RSA PRIVATE KEY-- , --END RSA PRIVATE KEY---- lines) and save what's left as a file in the newly created /opt/syslog-ng/etc/cert.d directory.  You may name this file whatever you like, so long as your syslog-ng.conf file corresponds to it.  (generally, it's called syslog-ng.cert)

Step4)  Again, using a text editor, open the PEM file you created in step 1, delete the entire Certificate portion of the text (to include the ----BEGIN CERTIFICATE-- , --END CERTIFICATE---- lines) and save what's left as a file in the newly created /opt/syslog-ng/etc/key.d directory.  You may name this file whatever you like, so long as your syslog-ng.conf file corresponds to it.  (generally, it's called syslog-ng.key)

* At this point, you may discard the PEM file you created in step 1
Step5)  On the CLIENT(s), create the /opt/syslog-ng/etc/cert.d directory.

Step6)  Copy the syslog-ng.cert file from the server's /opt/syslog-ng/etc/cert.d directory to the client's /opt/syslog-ng/etc/cert.d directory.  (You could combine steps 5 and 6 by copying the entire cert.d directory in one step from the server to the client if you prefer)

Step7)  Issue the following command(s) on the certificate:

openssl x509 \-noout \-hash \-in syslog-ng.cert

The result is a hash (e.g., 6d2962a8), a series of alphanumeric characters based on the Distinguished Name of the certificate.  Issue the following command to create a symbolic link to the certificate that uses the hash returned by the previous command and the .0 suffix.

ln -s syslog-ng.cert 6d2962a8.0

SYSLOG-NG Server & Client Configurations
Step8) Server/Syslog System: Edit ('vi') /opt/syslog-ng/etc/syslog-ng.conf and replace the old data with the new configuration:

@version: 3.0
# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat. But
# it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#

options {
        chain_hostnames(no);
        time_reopen(10);
        time_reap(360);
        log_fifo_size(1000);
        group(adm);
        perm(0644);
        dir_perm(0755);
        use_dns(yes);
        use_dns(yes);
        use_fqdn(yes);
        create_dirs(yes);
        keep_hostname (yes);
        log_msg_size(32768);
        stats_freq(0);
        flush_lines(0);
};

#
# Connectivity needs, TLS encryption enabled
#
source s_localhost {
        file ("/proc/kmsg" program_override("kernel: "));
        unix-stream ("/dev/log");
        internal();
};

source s_general {
        tcp(ip(192.168.0.2) port (514) max-connections(1000)
        tls( key_file("/opt/syslog-ng/etc/key.d/syslog-ng.key")
        cert_file("/opt/syslog-ng/etc/cert.d/syslog-ng.cert")
        peer_verify(optional-untrusted)) );
};

source s_apache {
        tcp(ip(192.168.0.3) port (514) max-connections(1000)
        tls( key_file("/opt/syslog-ng/etc/key.d/syslog-ng.key")
        cert_file("/opt/syslog-ng/etc/cert.d/syslog-ng.cert")
        peer_verify(optional-untrusted)) );
};

source s_windows {
        tcp(ip(192.168.0.4) port (514) max-connections(1000)
        tls( key_file("/opt/syslog-ng/etc/key.d/syslog-ng.key")
        cert_file("/opt/syslog-ng/etc/cert.d/syslog-ng.cert")
        peer_verify(optional-untrusted)) );
};


destination d_cons { file("/dev/console"); };

destination d_mesg { file("/var/log/messages"
             template("$DATE $HOST <$FACILITY.$PRIORITY> $MSG\n")
             template_escape(no)
        );
};

destination d_auth {
        file("/var/log/secure"
             template("$DATE $HOST <$FACILITY.$PRIORITY> $MSG\n")
             template_escape(no)
        );
};

destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };

filter notdebug { level(info...emerg); };

filter f_kernel     { facility(kern); };
filter f_default    { level(info..emerg) and
                        not (facility(mail)
                        or facility(authpriv)
                        or facility(cron)); };
filter f_auth       { facility(authpriv); };
filter f_mail       { facility(mail); };
filter f_emergency  { level(emerg); };
filter f_news       { facility(uucp) or
                        (facility(news)
                        and level(crit..emerg)); };
filter f_boot   { facility(local7); };
filter f_cron   { facility(cron); };

#
# Filters needed for RHEL platforms
#
filter M_audit   { not message("Audit daemon rotating log files"); };
filter M_repeat  { not message("last message repeated"); };
filter M_stats   { not message("Log statistics"); };
filter M_cron    { not message("CMD"); };
filter M_snmp    { not message("SNMP"); };
filter M_snmp2   { not message("UDP"); };

filter windows { program(MSWinEventLog); };

#log { source(s_localhost); filter(f_kernel); destination(d_cons); };
log { source(s_localhost); filter(f_kernel); destination(d_kern); };
log { source(s_localhost); filter(f_default); filter(M_snmp); filter(M_snmp2); destination(d_mesg); };
log { source(s_localhost); filter(f_auth); destination(d_auth); };
log { source(s_localhost); filter(f_mail); destination(d_mail); };
log { source(s_localhost); filter(f_emergency); destination(d_mlal); };
log { source(s_localhost); filter(f_news); destination(d_spol); };
log { source(s_localhost); filter(f_boot); destination(d_boot); };
log { source(s_localhost); filter(f_cron); destination(d_cron); };

# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:

#
# Destinations for storing data logs
#
destination d_general {
          file ("/var/log/syslog/general/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log"
          template("$DATE $HOST <$FACILITY.$PRIORITY> $MSGHDR $MSG\n")
          template_escape(no)
        );
};

destination d_apache {
          file ("/var/log/syslog/apache/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log");
};

destination d_windows {
        file("/var/log/syslog/windows/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log"
        template("$DATE <$FACILITY.$PRIORITY> $HOST $MSGHDR $MSG\n")
        template_escape(no)
        );
};

#
# How to log and apply filters, if necessary
#
log { source(s_localhost);
       filter(M_audit);
       filter(M_repeat);
       filter(M_stats);
       filter(M_cron);
       filter(M_snmp);
       filter(M_snmp2);
       filter(notdebug);
      destination(d_general);
};

log { source(s_general);
       filter(M_audit);
       filter(M_repeat);
       filter(M_stats);
       filter(M_cron);
       filter(M_snmp);
       filter(M_snmp2);
       destination(d_general);
};

log { source(s_apache);
       destination(d_apache);
};

log { source(s_windows);
        filter(windows);
        destination(d_windows);
        flags(final);
};


Step9) Client System: Edit ('vi') /opt/syslog-ng/etc/syslog-ng.conf and replace the old data with the new configuration:

@version: 3.0
# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat. But
# it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#

options {
        chain_hostnames(no);
        time_reopen(10);
        time_reap(360);
        log_fifo_size(0);
        create_dirs(yes);
        group(adm);
        perm(0640);
        dir_perm(0755);
        use_dns(no);
        log_msg_size(32768);
        stats_freq(0);
        flush_lines(0);
};

source s_sys {
        file ("/proc/kmsg" program_override("kernel: "));
        unix-stream ("/dev/log");
        internal();
};

source s_apache {
        file("/etc/httpd/logs/access_log" follow_freq(1));
        file("/etc/httpd/logs/error_log" follow_freq(1));
};

destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };

filter f_kernel     { facility(kern); };
filter f_default    { level(info..emerg) and
                        not (facility(mail)
                        or facility(authpriv)
                        or facility(cron)); };
filter f_auth       { facility(authpriv); };
filter f_mail       { facility(mail); };
filter f_emergency  { level(emerg); };
filter f_news       { facility(uucp) or
                        (facility(news)
                        and level(crit..emerg)); };
filter f_boot   { facility(local7); };
filter f_cron   { facility(cron); };

# RHEL SNMP Filters
filter M_snmp    { not message("SNMP"); };
filter M_snmp2   { not message("UDP"); };

#log { source(s_sys); filter(f_kernel); destination(d_cons); };
log { source(s_sys); filter(f_kernel); destination(d_kern); };
log { source(s_sys); filter(f_default); filter(M_snmp); filter(M_snmp2); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_auth); };
log { source(s_sys); filter(f_mail); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_cron); };

# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:

###################################################
# destinations
###################################################

destination general_syslog {
        tcp(192.168.0.2 port(514)
        tls(ca_dir("/opt/syslog-ng/etc/cert.d")) );
};

destination apache_syslog {
        tcp(192.168.0.2 port(515)
        tls(ca_dir("/opt/syslog-ng/etc/cert.d")) );
};


log {
        source(s_sys);
        filter(M_snmp);
        filter(M_snmp2);
        destination(general_syslog);
};

log {
        source(s_apache);
        destination(apache_syslog);
};


Step10) Restart syslog-ng on both machines to initialize the new syslog-ng.conf settings.

service syslog-ng restart

or

/etc/init.d/syslog-ng restart