A Case for Study (sed & awk, Second Edition)

6.2. A Case for Study

Lenny, on our staff, was having difficulty converting a document coded for Scribe to our troff macro package, because of font changes. The problems he encountered are quite interesting, apart from the task he was trying to do.

The Scribe convention for putting text in a bold font is:

@f1(put this in bold)

This font change command can appear in-line and may begin on one line and end on a subsequent line. It can also appear more than once on a line. Here's a sample file that shows several different occurrences:

$ cat test
I want to see @f1(what will happen) if we put the
font change commands @f1(on a set of lines).  If I understand
things (correctly), the @f1(third) line causes problems. (No?).
Is this really the case, or is it (maybe) just something else?

Let's test having two on a line @f1(here) and @f1(there) as
well as one that begins on one line and ends @f1(somewhere 
on another line).  What if @f1(it is here) on the line?
Another @f1(one).

The sample file shows the different contexts in which the font-change commands appear. The script must match "@f1(anything)" when it occurs on a single line or multiple times on the same line or when it extends across more than one line.

The easiest way to make a single match is:

s/@f1(\(.*\))/\\fB\1\\fR/g

The regular expression matches "@f1(.*)" and saves anything inside parentheses using \( and \). In the replacement section, the saved portion of the match is recalled as "\1".

Putting this command in a sed script, we will run it on our sample file.

$ sed -f sed.len test
I want to see \fBwhat will happen\fR if we put the
font change commands \fBon a set of lines\fR.  If I understand
things (correctly), the \fBthird) line causes problems. (No?\fR.
Is this really the case, or is it (maybe) just something else?

Let's test having two on a line \fBhere) and @f1(there\fR as
well as one that begins on one line and ends @f1(somewhere 
on another line).  What if \fBit is here\fR on the line?
Another \fBone\fR.

The substitute command works properly in the first two lines. It fails on the third line. It also fails in the first line of the second paragraph where there are multiple occurrences on the same line.

Because a regular expression always makes the longest match possible, ".*" matches all the characters from "@f1(" to the last closing parenthesis on the line. In other words, the span indicated by ".*" ends with the last close parenthesis it finds, not the first.

We can fix this problem by modifying the regular expression ".*" to zero or more occurrences of any character except ")".

[^)]*

In a character class, the caret (^) reverses the sense of the operation so it matches all characters except those specified in the brackets. Here's how the revised command looks:

s/@f1(\([^)]*\))/\\fB\1\\fR/g

Now we have a command that handles one or more occurrences on a single line.

I want to see \fBwhat will happen\fR if we put the
font change commands \fBon a set of lines\fR.  If I understand
things (correctly), the \fBthird\fR line causes problems. (No?).
Is this really the case, or is it (maybe) just something else?

Let's test having two on a line \fBhere\fR and \fBthere\fR as
well as one that begins on one line and ends @f1(somewhere 
on another line).  What if \fBit is here\fR on the line?
Another \fBone\fR.

This command gets all instances except the one in the second paragraph that extends over two lines. Before solving this problem, it is interesting to look at Lenny's first solution to it and why it fails. Here's Lenny's first script:

/@f1(/,/)/{
	s/@f1(/\\fB/g
	s/)/\\fR/g
}

He tried to attack the problem of matching an occurrence over multiple lines by specifying a range of lines. Here's the result of running the script on the test file:

$ sed -f sed.len test.len
I want to see \fBwhat will happen\fR if we put the
font change commands \fBon a set of lines\fR.  If I understand
things (correctly, the \fBthird) line causes problems. (No?\fR.
Is this really the case, or is it (maybe) just something else?

Let's test having two on a line \fBhere) and (there\fR as
well as one that begins on one line and ends \fBsomewhere 
on another line\fR.  What if \fBit is here\fR on the line?
Another \fBone\fR.

Matching lines containing ")" makes unwanted matches on lines containing only parentheses. The solution to matching the pattern over more than one line is to create a multiline pattern space. If we match "@f1(" and no closing parenthesis is found, we need to read (N) another line into the buffer and try to make the same kind of match as the first case (the \n represents the newline).

s/@f1(\([^)]*\))/\\fB\1\\fR/g
/@f1(.*/{
	N
	s/@f1(\(.*\n[^)]*\))/\\fB\1\\fR/g
}

We can test it out:

$ sed -f sednew test
I want to see \fBwhat will happen\fR if we put the
font change commands \fBon a set of lines\fR.  If I understand
things (correctly), the \fBthird\fR line causes problems. (No?).
Is this really the case, or is it (maybe) just something else?

Let's test having two on a line \fBhere\fR and \fBthere\fR as
well as one that begins on one line and ends \fBsomewhere 
on another line\fR.  What if @f1(it is here) on the line?
Another \fBone\fR.

As you can see, we have caught all but the next to last font change. The N command reads a second line into the pattern space. The script matches the pattern across two lines and then outputs both lines from the pattern space. What about the second line? It needs a chance to have all the commands in the script applied to it from top to bottom. Now, perhaps you understand why we need to set up a multiline input/output loop like the one discussed in the previous section. We add the multiline Print and multiline Delete to the script.

# Scribe font change script.
s/@f1(\([^)]*\))/\\fB\1\\fR/g
/@f1(.*/{
	N
	s/@f1(\(.*\n[^)]*\))/\\fB\1\\fR/g
	P
	D
}

This can be translated as: Once making a substitution across two lines, print the first line and then delete it from the pattern space. With the second portion remaining in the pattern space, control passes to the top of the script where we see if there is an "@f1(" remaining on the line.

The revised script matches all occurrences in the sample file. However, it's not perfect, so we'll hear from Lenny again.


6. Advanced sed Commands		6.3. Hold That Line