|
|
|
10.2 Syslog-ngAs useful and ubiquitous as syslog is, it's beginning to show its age. Modern Unix and Unix-like systems are considerably more complex than they were when syslog was invented, and they have outgrown both syslog's limited facilities and its primitive network-forwarding functionality. Syslog-ng ("syslog new generation") is an attempt to increase syslog's flexibility by adding better message filtering, better forwarding, and eventually (though not quite yet), message integrity and encryption. In addition, Syslog-ng supports remote logging over both the TCP and UDP protocols. Syslog-ng is the brainchild of and is primarily developed and maintained by Balazs ("Bazsi") Scheidler. Lest you think Syslog-ng is untested or untrusted, it's already been incorporated into Debian GNU/Linux 2.2 "Potato" as a binary package (in the "admin" section). Syslog-ng is in fact both stable and popular. Furthermore, even though its advanced security features are still works in progress, Syslog-ng can be used in conjunction with TCP "tunneling" tools such as stunnel and ssh to authenticate or encrypt log messages sent to remote hosts. 10.2.1 Compiling and Installing Syslog-ng from Source CodeThe non-Debian users among you may not wish to wait for your distribution of choice to follow suit with its own binary package of Syslog-ng. Let's start, then, with a brief description of how to compile and install Syslog-ng from source. First, you need to obtain the latest Syslog-ng source code. As of this writing, there are two concurrent branches of Syslog-ng development. Syslog-ng Version 1.4 is the stable branch, so I recommend you use the latest release of Syslog-ng 1.4. Version 1.5 is the experimental branch, and although it's officially disclaimed as unstable, some people use it on production systems due to its new "field expansion" feature, which allows you to write messages in your own custom formats. If you decide this functionality is worth the risk of running experimental code, be sure to subscribe to the Syslog-ng mailing list (see http://lists.balabit.hu/mailman/listinfo/syslog-ng to subscribe). Speaking of which, it probably behooves you to browse the archives of this mailing list periodically even if you stick to the stable branch of Syslog-ng. Bazsi Scheidler tends to prioritize bug fixes over documentation, so Syslog-ng documentation tends to be incomplete and even out of date. But Bazsi not only maintains the mailing list, he also very actively participates in it, as do other very knowledgeable and helpful Syslog-ng users and contributors. Thus the mailing list is an excellent source of Syslog-ng assistance. Before posting a question, you may wish to see if anyone else has asked it first. See the Syslog-ng mailing list archives at http://lists.balabit.hu/pipermail/syslog-ng/. Syslog-ng can be downloaded either directly from Bazsi Scheidler's web site at http://www.balabit.hu or from its Freshmeat project site at http://freshmeat.net/projects/syslog-ng/. In addition to Syslog-ng itself, you'll need the source code for libol, Syslog-ng's support library. Unzip and untar both archives. Compile and install libol first, then Syslog-ng. For both packages the procedure is the same:
This will install everything in the default locations, which for both libol and Sylog-ng are subdirectories of /usr/local (e.g., /usr/local/lib, /usr/local/sbin, etc.). If you wish to install either package somewhere else — e.g., your home directory (which is not a bad place to test new software) — then in Step 2, pass that directory to configure with the — prefix= flag as in Example 10-11. Example 10-11. Telling configure where to install the packagemylinuxbox:/usr/src/libol-0.2.23# ./configure --prefix=/your/dir/here After both libol and Syslog-ng have been compiled and installed, you need to set up a few things in Syslog-ng's operating environment. First, create the directory /etc/syslog-ng. Next, copy one or more of the example syslog-ng.conf files into this directory from the source-distribution's contrib/ and doc/ directories (unless you intend to create your syslog-ng.conf completely from scratch). Finally, you need to create a startup script for syslog-ng in /etc/init.d and symbolic links to it in the appropriate runlevel directories (for most Linux distributions, /etc/rc2.d, /etc/rc3.d, and /etc/rc5.d). Sample syslog-ng init scripts for several Linux distributions are provided in the Syslog-ng source distribution's contrib/ directory. If you don't find one there that works for you, it's a simple matter to make a copy of your old syslog or sysklogd init-script and hack it to start syslog-ng rather than syslogd. 10.2.2 Running syslog-ngIt's premature to start syslog-ng before you've created a configuration file. However, since syslog-ng has so few startup flags, I'll mention them in brief and spend the remainder of this section on syslog-ng.conf use. The only flags supported by the syslog-ng daemon are listed in Table 10-6.
In normal use, set these flags in the startup script you installed or created when you installed Syslog-ng, and use that script not only automatically at startup time, but also manually if you need to restart or stop Syslog-ng afterwards. 10.2.3 Configuring Syslog-ngThere's quite a bit more involved in configuring Syslog-ng than with syslog, but that's a symptom of its flexibility. Once you understand how syslog-ng.conf works, writing your own configurations is simple, and adapting sample configurations for your own purposes is even simpler. Its main drawback is its sketchy documentation; hopefully, what follows here will mitigate that drawback for you. By default, Syslog-ng's configuration file is named syslog-ng.conf and resides in /etc/syslog-ng/. Let's dissect a simple example of one in Example 10-12. Example 10-12. A simple syslog-ng.conf file# Simple syslog-ng.conf file.
options {
use_fqdn(no);
sync(0);
};
source s_sys { unix-stream("/dev/log"); internal( ); };
source s_net { udp( ); };
destination d_security { file("/var/log/security"); };
destination d_messages { file("/var/log/messages"); };
destination d_console { usertty("root"); };
filter f_authpriv { facility(auth, authpriv); };
filter f_messages { level(info .. emerg)
and not facility(auth, authpriv); };
filter f_emergency { level(emerg); };
log { source(s_sys); filter(f_authpriv); destination(d_security); };
log { source(s_sys); filter(f_messages); destination(d_messages); };
log { source(s_sys); filter(f_emergency); destination(d_console); };
As you can see, a syslog-ng.conf file consists of options{}, source{}, destination{}, filter{}, and log{} statements. Each of these statements may contain additional settings, usually delimited by semicolons. Syntactically, syslog-ng.conf is very similar to C and other structured programming languages. Statements are terminated by semicolons; whitespace is ignored and may therefore be used to enhance readability (e.g., by breaking up and indenting lengthy statements across several lines). After defining global options, message sources, message destinations, and message filters, combine them to create logging rules. 10.2.3.1 Global optionsGlobal options are set in syslog-ng.conf's options{} section. Some options may be used in the options{} section and in one or more other sections. Predictably, options set within source{}, destination{}, filter{}, and log{} sections overrule those set in options{}. Table 10-7 lists some of the most useful of Syslog-ng's options.
Options that deal with hostnames and their resolution (chain_hostnames( ), keep_hostname( ), use_fqdn( ), and use_dns) deal specifically with the hostnames of remote log clients and not with hostnames/IPs referenced in the body of the message. In other words, if syslog-ng.conf on a central log server contains this statement: options { use_dns(yes); };
and the remote host joe-bob, whose IP address is 10.9.8.7, sends this message: Sep 13 19:56:56 [email protected] sshd[13037]: Accepted publickey for ROOT from 10.9.8.254 port 1355 ssh2 then the log server will log: Sep 13 19:56:56 s_sys@joebob sshd[13037]: Accepted publickey for ROOT from 10.9.8.254 port 1355 ssh2 As you can see, 10.9.8.7 was resolved to joebob, but 10.9.8.254 wasn't looked up. (For now you can disregard the s_sys@ in front of the hostname; I'll explain that shortly.) The use_dns(yes) statement applies only to the hostname at the beginning of the message indicating which host sent it; it doesn't apply to other IP addresses that may occur later in the message. Note also that options related to files and directories may be specified both in the global options{} statement and as modifiers to file( ) definitions within destination{} statements. file( ) options, when different from their global counterparts, override them. This allows you to create a "rule of thumb" with specific exceptions. The chain_hostname( ) and keep_hostname( ) options are also worth mentioning. By default, keep_hostname( ) is set to no, meaning that syslog-ng will not take the hostname supplied by a remote log server at face value; syslog-ng will instead resolve the source IPs of packets from that host to determine for itself what that host's name is. This is in contrast to syslog, which takes remote hosts' names at face value. chain_hostname( ) determines whether syslog-ng should list all hosts through which each message has been relayed. By default, this option is set to yes. Example 10-13 illustrates the effects of keep_hostname(no) and chain_hostname(yes) (i.e., syslog-ng's default behavior). It shows a log message (in this case, a syslog-ng startup notification) being generated locally and then relayed twice. host1, who gives its hostname as "linux," generates the message and then sends it to host2. host2 records both "linux" and "host1," having double checked that hostname itself via DNS. Finally, the message is relayed to host3. Example 10-13. A log message relayed from one host to two othersOriginal log entry on host1: Sep 19 22:57:16 s_loc@linux syslog-ng[1656]: syslog-ng version 1.4.13 starting Entry as sent to and recorded by host2: Sep 19 22:57:16 s_loc@linux/host1 syslog-ng[1656]: syslog-ng version 1.4.13 starting Same log entry as relayed from host2 to host3: Sep 19 22:57:16 s_loc@linux/host1/host2 syslog-ng[1656]: syslog-ng version 1.4.13 starting There are several interesting things to note in this example. First, you can see that in the second entry (the one logged by host2), Syslog-ng does not clearly indicate that "linux" is actually host1 — it simply adds the "real" hostname after the "fake" one in the slash-delimited hostname chain. Second, the timestamp is identical in all three log entries. It's unlikely that three hosts would be in sync to the millisecond and be able to relay log messages amongst themselves virtually instantaneously. In fact, the timestamp given to the message by the originating host (host1 here) is preserved on each host to which the message is relayed, unless a host has its own use_time_recd( ) option set to "yes" (which causes syslog-ng to replace message-provided timestamps with the time at which the message was received locally). Finally, Example 10-13 also shows that when host1 created the message, it (actually its local syslog-ng process) appended s_loc, to the message — this is the label of the source{} on host1 from which the local syslog-ng process received the message. Example 10-14 lists host1's syslog-ng.conf file, the one responsible for the first entry shown in Example 10-13. Example 10-14. host1's syslog-ng.conf fileoptions { };
source s_loc { unix-stream("/dev/log"); internal( ); };
destination d_host2 { udp("host2" port(514)); };
destination d_local { file("/var/log/messages"); };
log { source(s_loc); source(s_net); destination(d_host2); destination(d_local); };
Which brings us to the next topic: Syslog-ng message sources. 10.2.3.2 SourcesThe syslog-ng.conf file listed in Example 10-14 contains one source{} definition, which itself contains two source "drivers" (message-inputs). syslog-ng.conf may contain many source{} definitions, each of which may, in turn, contain multiple drivers. In other words, the syntax of source definitions is as follows: source sourcelabel { driver1( [options] ); driver2( [options] ); etc. };
where sourcelabel is an arbitrary string used to identify this group of inputs, and where driver1( ), driver2( ), etc. are one or more source drivers that you wish to treat as a single group. Let's take a closer look at the source definition in Example 10-14: source s_loc { unix-stream("/dev/log"); internal( ); };
This line creates a source called s_loc that refers to messages obtained from /dev/log (i.e., the local system-log socket) and from the local syslog-ng process. Syslog-ng is quite flexible in the variety of source drivers from which it can accept messages. In addition to Unix sockets (e.g., /dev/log), syslog-ng itself, and UDP streams from remote hosts, Syslog-ng can accept messages from named pipes, TCP connections from remote hosts, and special files (e.g., /proc files). Table 10-8 lists Syslog-ng's supported source drivers.
As we just saw in Example 10-14, internal( ) is syslog-ng itself: syslog-ng sends itself startup messages, errors, and other messages via this source. Therefore, you should include internal( ) in at least one source{} definition. file( ) is used to specify special files from which syslog-ng should retrieve messages. The special file you'd most likely want syslog-ng to read messages from is /proc/kmsg. Note, however, that file( ) is not intended for use on regular text files. If you wish syslog-ng to "tail" dynamic log files written by other applications (e.g., httpd), you'll need to write a script that pipes the output from a tail -f [filename] command to logger. (For instructions on using logger, see Section 10.3 later in this chapter.) unix_stream( ) and unix_dgram( ) are important drivers: these read messages from connection-oriented and connectionless Unix sockets, respectively. As noted at the end of "Compiling and Installing Syslog-ng from Source Code," Linux kernels Versions 2.4.1 and higher use Unix datagram sockets; if you specify /dev/log as a unix_stream( ) source, kernel messages won't be captured. Therefore, use unix_dgram( ) when defining your local-system log source, e.g.: source s_loc { unix-dgram("/dev/log"); internal( ); };
If your kernel is pre-2.4.0, you should instead use unix_stream( ) for /dev/log. tcp( ) and udp( ) read messages from remote hosts via the connection-oriented TCP protocol and the connectionless UDP protocol, respectively. In both tcp( ) and udp( ), a listening address and a port number may be specified. By default, syslog-ng listens on 0.0.0.0:514 — that is, "all interfaces, port 514." (Specifically, the default for tcp( ) is 0.0.0.0:TCP514, and for udp( ), that is 0.0.0.0:UDP514.) Example 10-15 shows source statements for tcp( ) and udp( ), with IP and port options defined. Example 10-15. tcp( ) and udp( ) sourcessource s_tcpmessages { tcp( ip(192.168.190.190) port(10514) ); };
source s_udpmessages { udp( ); };
In Example 10-15, we're defining the source s_tcpmessages as all messages received on TCP port 10514, but only on the local network interface whose IP address is 192.168.190.190. The source s_udpmessages, however, accepts all UDP messages received on UDP port 514 on all local network interfaces. Besides ip( ) and port( ), there's one more source option I'd like to cover. max_connections( ), which can only be used in tcp( ) and unix_stream( ) sources, restricts the number of simultaneous connections from a given source that syslog-ng will accept. This is a tradeoff between security and performance: if this number is high, then few messages will be dropped when the server is under load, but at the expense of resources. If this number is low, the chance that logging activity will bog down the server is minimized, but whenever the number of maximum connections is reached, messages will be dropped until a connection is freed up. The correct syntax for max-connections( ) is simple: specify a positive integer between the parentheses. For example, let's adapt the tcp( ) source from Example 10-15 to accept a maximum of 100 concurrent TCP connections from remote hosts: source s_tcpmessages { tcp( ip(192.168.190.190) port(10514) max-connections(100) ); };
By default, max-connections( ) is set to 100 for unix-stream( ) sources and 10 for tcp( ) sources. By the way, TCP port 514 is the default listening port not only for syslog-ng, but also for rshd. This isn't a big deal, for the simple reason that rshd has no business running on an ostensibly secure Internet-accessible system. If, for example, you wish to use both syslog-ng and rshd on an intranet server (even then I recommend sshd instead), then you should specify a different (unused) port for syslog-ng to accept TCP connections on. 10.2.3.3 Destinationssyslog-ng can be configured to send messages to the same places syslog can: ASCII files, named pipes, remote hosts via UDP, and TTYs. In addition, syslog-ng can send messages to Unix sockets, remote hosts via TCP, and to the standard inputs of programs. Table 10-9 lists the allowed destination types (called "drivers") in Syslog-ng.
As with ordinary syslog, the most important type of destination is file( ). Unlike with syslog, Syslog-ng supports filename-expansion macros and a number of options that give one much more granular control over how log files are handled. When you specify the name of a file for syslog-ng to write messages to, you may use macros to create all or part of the filename. For example, to tell syslog-ng to write messages to a file whose name includes the current day, you could define a destination like this: destination d_dailylog { file("/var/log/messages.$WEEKDAY"); };
When Syslog-ng writes to this particular destination, it will use the filename /var/log/messages.Tues, /var/log/messages.Wed, etc., depending on what day it is. See Table 10-10 for a complete list of supported filename macros.
As with syslog, if a file specified in a file( ) destination doesn't exist, syslog-ng will create it. Unlike syslog, Syslog-ng has a number of options that can be implemented both globally and on a per-log-file basis. (Global settings are overridden by per-log-file settings, allowing you to create "general rules" with exceptions.) For example, whether and how syslog-ng creates new directories for its log files is controlled via the options create_dirs( ), dir_owner( ), dir_group( ), and dir_perm( ). Example 10-16 illustrates the use of these options within a destination{} statement. Example 10-16. Controlling a file( ) destination's directory-creating behaviordestination d_mylog { file("/var/log/ngfiles/mylog" create_dirs(yes) dir_owner(root) \
dir_group(root) dir_perm(0700)); };
Example 10-16 also happens to show the default values of the dir_owner, dir_group( ), and dir_perm( ) options. While this may seem unrealistic (why would anyone go to the trouble of setting an option to its default?), it's necessary if nondefaults are specified in a global options{} statement and you want the default values used for a specific file — remember, options set in a destination{} statement override those set in an options{} statement. Other global/file-specific options can be used to set characteristics of the log file itself: owner( ), group( ), and perm( ), which by default are set to root, root, and 0600, respectively. In case you're wondering, there is no create_file( ) option — syslog-ng has the irrevocable ability to create files (unless that file's path includes a nonexistent directory and create_dirs( ) is set to no). Example 10-17 shows a destination definition that includes these options. Example 10-17. Options that affect file propertiesdestination d_micklog { file("/var/log/micklog" owner(mick) group(wheel) perm(0640)); };
The other file( ) option we'll cover here is sync( ), which can be used to limit the frequency with which log files are synchronized. This is analogous to syslog's "-" prefix, but much more granular: whereas the "-" merely turns off synchronization, file( ) accepts a numeric value that delays synchronization to as many or as few messages as you like. The higher the value, the more messages are cached prior to filesystem synchronization and, therefore, the fewer "open for read" actions on the filesystem. The lower the number, the lower the chances of data loss and the lower the delay between a message being processed and written to disk. By default, sync( ) is set to zero, meaning "synchronize after each message." In general, the default or a low sync( ) value is preferable for low-volume scenarios, but numbers in the 100s or even 1,000s may be necessary in high-volume situations. A good rule of thumb is to set this value to the approximate number of log-message lines per second your system must handle at peak loads. 10.2.3.4 FiltersAnd now we come to some of the serious magic in Syslog-ng: message filters. Filters, while strictly optional, allow you to route messages based not only on priority/level and facility (which syslog can do), but also on the name of the program that sent the message, the name of the host that forwarded it over the network, a regular expression evaluated against the message itself, or even the name of another filter. A filter{} statement consists of a label (the filter's name) and one or more criteria connected by operators (and, or, and not are supported). Table 10-11 lists the different types of criteria that a filter{} statement may contain.
Example 10-18 shows several filter{} statements taken from the default syslog-ng.conf file included in Debian 2.2's syslog-ng package. Example 10-18. Filtersfilter f_mail { facility(mail); };
filter f_debug { not facility(auth, authpriv, news, mail); };
filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon,
mail, news); };
filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); };
The first line in Example 10-17, filter f_mail, matches all messages logged to the mail facility. The second filter, f_debug, matches all messages not logged to the auth, authpriv, news, and mail facilities. The third filter, f_messages, matches messages of priority levels info through warn, except those logged to the auth, authpriv, cron, daemon, mail, and news facilities. The last filter, called f_cother, matches all messages of priority levels debug, info, notice, and warn, and also all messages logged to the daemon and mail facilities. When you create your own filters, be sure to test them using the logger command. See Section 10.3 later in this chapter. 10.2.3.5 Log statementsNow we combine the elements we've just defined (sources, filters, and destinations) into log{} statements. Arguably, these are the simplest statements in syslog-ng.conf: each consists only of a semicolon-delimited list of source( ), destination( ), and, optionally, filter( ) references. (Filters are optional because a log{} statement containing only source( ) and destination( ) references will send all messages from the specified sources to all specified destinations.) Elements from several previous examples are combined in Example 10-19, which culminates in several log{} statements. Example 10-19. Another sample syslog-ng.conf filesource s_loc { unix-stream("/dev/log"); internal( ); };
source s_tcpmessages { tcp( ip(192.168.190.190); port(10514);); };
destination d_dailylog { file("/var/log/messages.$WEEKDAY"); };
destination d_micklog { file("/var/log/micklog" owner(mick) perm(0600)); };
filter f_mail { facility(mail); };
filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon,
mail, news); };
log { source(s_tcpmessages); destination(d_micklog); };
log { source(s_loc); filter(f_mail); destination(d_micklog); };
log { source(s_loc); filter(f_messages); destination(d_dailylog); };
As you can see in this example, all messages from the host 192.168.190.190 are written to the log file /var/log/micklog, as are all local mail messages. Messages that match the f_messages( ) filter are written to the log file /var/log/messages.$WEEKDAY, e.g., /var/log/Sun, /var/log/Mon, etc. Example 10-19 isn't very realistic, though: no nonmail messages with priority-level higher than warn are dealt with. This begs the question, "Can I get syslog-ng to filter on `none of the above?'" The answer is yes: to match all messages that haven't yet matched filters in previous log{} statements, you can use the built-in filter DEFAULT. The following line, if added to the bottom of Example 10-18, will cause all messages not processed by any of the prior three log{} statements to be written to the daily log file: log { source(s_loc); filter(DEFAULT); destination(d_dailylog); };
10.2.4 Advanced ConfigurationsAs you're hopefully convinced of by this point, Syslog-ng is extremely flexible, so much so that it isn't feasible to illustrate all possible Syslog-ng configurations. I would be remiss, however, if I didn't list at least one advanced syslog-ng.conf file. Example 10-20 shows a setup that causes syslog-ng to watch out for login failures and access denials by matching messages against a regular expression and then sending the messages to a shell script (listed in Example 10-21). Example 10-20. Using syslog-ng as its own log watcher# WARNING: while this syslog-ng.conf file is syntactically correct and complete, it is
# intended for illustrative purposes only — entire categories of message
# are ignored!
source s_local { unix_stream("dev/log"); internal( ); };
filter f_denials { match("[Dd]enied|[Ff]ail"); };
destination d_mailtomick { program("/usr/local/sbin/mailtomick.sh"); };
log { source(s_local); filter(f_denials); destination(d_mailtomick); };
Example 10-21. Script for emailing log messages#!/bin/bash # mailtomick.sh # Script which listens for standard input and emails each line to mick # while read line; do echo $line | mail -s "Weirdness on that Linux box" [email protected] done The most important lines in Example 10-20 are the filter f_denials and the destination d_mailtomick. The filter uses a match( ) directive containing a regular expression that matches the strings "denied," "Denied," "Fail," and "fail."[3] The destination d_mailtomick sends messages via a program( ) declaration to the standard input of a script I wrote called /usr/local/sbin/mailtomick.sh.
The script itself, /usr/local/sbin/mailtomick.sh, simply reads lines from the standard input and emails each line to [email protected]. Since syslog-ng needs to keep this script open, the read command is contained in an endless loop. This script will run until the syslog-ng process that invoked it is restarted or killed. In the interest of focusing on the most typical uses of Syslog-ng, I've listed some syslog-ng.conf options without giving examples of their usage and omitted a couple of other options altogether. Suffice it to say that the global/file option log_fifo_size( ) and the global options time_reap( ), time_reopen( ), gc_idle_threshold( ), and gc_busy_threshold( ) are useful for tuning syslog-ng's performance to fit your particular environment.
|
|
|
|
