23.2.1 The Queue Identifier

To ensure that new filenames are not the same as the names of files that may already be in the queue, sendmail uses the following pattern for each new ident:

AApid prior to V8.6
hourAApid beginning with V8.6

Here, pid is the process identification number of the incarnation of sendmail that is trying to create the file. Because sendmail often fork(2)'s to process the queue, that pid is likely to be unique, resulting in a unique ident. For V8 sendmail an extra letter prefixes the AA. Shown as hour, it is an uppercase letter that corresponds to the hour (in a 24-hour clock) that the identifier was created. For example, a file created in hour three of the day will have a D prefixed (the hour begins at midnight with A). [2]

[2] Programs should not depend on the lead letter actually encoding the hour. It is intended only to ensure that all identifiers be unique within any 24-hour period and as an aid to scripts that need to extract information from log files.

If sendmail cannot create an exclusive filename (because a file with that identifier already exists), it clocks the second A of the AA to a B and tries again. It continues this process, clocking the right-hand letter from A to Z and the left-hand letter from A to ~ until it succeeds:

AA             start
AB             second try
AC             third try
 ... and so on
~W
~X
~Y             last try
~Z             failure

If it never succeeds, the ident ultimately looks like the following and sendmail has failed:

hour~Zpid

This ident is unlikely to ever appear, because the clocking provides for over 1600 possibilities.

All the files associated with a given mail message share the same ident as a part of their filenames. The individual files associated with a single mail message differ only in the first letter of their names.

23.2.2 The Data (Message Body) File: df

All mail messages are composed of a header and a body. When queued, the body is stored in the df file.

Traditionally, the message body could contain only characters that had the high (most significant) bit turned off (cleared, set to 0). But under V8 sendmail, with a version 2 or higher configuration file (see Section 27.5, "The V Configuration Command"), the high bit is left as is until delivery (whereupon the F=7 delivery-agent flag, see Section 30.8.4, F=7, determines whether or not that bit will be stripped during delivery).

Because the message body can contain sensitive or personal information, the df file should be protected from reading by ordinary users. If the queue directory is world readable, then the TempFileMode (F) option (see Section 34.8.68, TempFileMode (F)) should specify minimum permissions (such as 0600) for queued files. But if the queue directory is protected by both narrow permissions and a secure machine, the TempFileMode (F) option may be relaxed for easier administration.

There is currently no plan to provide for encryption of df files. If you are concerned about the privacy of your message, you should use an end-to-end encryption package (not discussed in this book).

23.2.3 The Lock File (obsolete as of V5.62): lf

When old versions of sendmail process a queued message (attempt to redeliver it) they create an empty lock file. That lock file is necessary to signal to other running sendmail processes that the mail message is busy so that they won't try to deliver the message too. Current versions simply flock(2) or fcntl(2) the qf file.

23.2.4 The ID Creation File (obsolete as of V5.62): nf

Old versions of sendmail used an nf file when creating a message identifier to avoid race conditions. [3] But contemporary versions of sendmail create the queue identifier when first creating the qf file. The nf file is obsolete.

[3] Historical footnote: This stems from the days when the only atomic file-system call was link(2).

23.2.5 The Queue Control File: qf

A queued mail message is composed of two primary parts. The df file contains the message body. The qf file contains the message header.

In addition to the header, the qf file also contains all the information necessary to:

• Deliver the message. It contains the sender's address and a list of recipient addresses.

• Order message processing. It contains a priority that determines the current message's position in a queue run of many messages.

• Expire the message. It contains the date that the message was originally queued. That date is used to time out a message.

• Explain the message. It contains the reason that the message is in the queue and possibly the error that caused it to be queued.

The qf file is line-oriented, with one item of information per line. Each line begins with a single uppercase character (the code letter), which specifies the contents of the line. Each code letter is then followed by the information appropriate to the letter. The code letters and their meanings are shown in Table 23.2 of Section 23.9, "The qf File Internals".

Here is an example of a version 1 (for V8.8 sendmail) qf file:

V1
T826845694
K0
N0
P30016
I7/4/20
$_you@localhost
MDeferred: Host wash.dc.gov is down
[email protected]
RPFD:[email protected]
RPFD:jefferson
H?P?Return-Path: [email protected]
HReceived: (from [email protected]) by your.domain (8.8.4/8.8.4)
        id QAA06571 for [email protected]; Thu, 14 Mar 1996 16:21:34 -0700 (MST)
H?D?Date: Thu, 14 Mar 1996 16:21:34 -0700 (MST)
H?F?From: Your Name <[email protected]>
H?x?Full-Name: Your Name
H?M?Message-Id: <[email protected]>
HSubject: foo
.

This fictional qf file shows the information that will be used to send a mail message from [email protected] (the S line) to two recipients: [email protected] and jefferson (the R lines). It also shows the various headers that appear in that message (the H lines). We discuss the individual lines of the qf file in at the end of this chapter.

23.2.6 The Temporary qf Rewrite Image: tf

When processing a queued message, it is often necessary for sendmail to modify the contents of the qf file. This usually occurs if delivery has failed or if delivery for only a part of the recipient list succeeded. In either event, at least the message priority needs to be incremented.

To prevent damage to the original qf file, sendmail makes changes to a temporary copy of that file. The temporary copy has the same queue identifier as the original, but its name begins with a t.

After the tf file has been successfully written and closed, sendmail calls rename(2) to replace the original with the copy. If the renaming fails, sendmail syslog(3)'s at LOG_CRIT a message like the following:

cannot rename(tfAA00000, qfAA00000), df=dfAA00000

Failure to rename is an unusual, but serious, problem: A queued message has been processed, but its qf file contains old and incorrect information. This failure may, for example, indicate a hardware error, a corrupted queue directory, or that the system administrator accidentally removed the queue directory.

23.2.7 The Transcript File: xf

A given mail message may be destined for many recipients, requiring different delivery agents. During the process of delivery, error messages (such as "User unknown" and "Permission denied") can be printed back to sendmail by each delivery agent.

While calling the necessary delivery agents, sendmail saves all the error messages it receives in a temporary file. The name of that temporary file begins with the letters xf. After all delivery agents have been called, sendmail returns any collected error messages to the sender and deletes the temporary xf file. If there are no errors, the empty xf file is silently deleted. The -d51.104 debugging switch (see Section 37.5.171, -d51.104) can be used to prevent deletion of the xf file.