Recipe 8.4 Reading a File Backward by Line or Paragraph

8.4.1 Problem

You want to process each line or paragraph of a text file in reverse.

8.4.2 Solution

Read all lines into an array, then process that array from the end to the start:

@lines = <FILE>;
while ($line = pop @lines) {
    # do something with $line
}

Or store an array of lines in reverse order:

@lines = reverse <FILE>;
foreach $line (@lines) {
    # do something with $line
}

Or use the Tie::File module (standard as of v5.8):

use Tie::File;
tie(@lines, "Tie::File", $FILENAME, mode => 0)
    or die "Can't tie $FILENAME: $!";
$max_lines = $#lines;
for ($i = $max_lines; $i; $i--) {
    # do something with $lines[$i], eg line number them:
    printf "%5d  %s\n", $i+1, $lines[$i],
}

8.4.3 Discussion

The limitations of file access mentioned in this chapter's Introduction prevent reading a line at a time starting from the end. You must read the lines into memory, then process them in reverse order. This requires at least as much available memory as the size of the file, unless you use tricks like Tie::File does.

The first technique moves through the array of lines in reverse order. This destructively processes the array, popping an element off the end of the array each time through the loop. We could do it non-destructively with:

for ($i = $#lines; $i != -1; $i--) {
    $line = $lines[$i];
}

The second approach generates an array of lines already in reverse order. This array can then be processed non-destructively. We get the reversed lines because the assignment to @lines confers list context on the return from reverse, and reverse confers list context on its argument of <FILE>, which returns a list of all lines in the file.

These approaches are easily extended to paragraphs just by changing $/:

# this enclosing block keeps local $/ temporary
{           
    local $/ = "";
    @paragraphs = reverse <FILE>;
}

foreach $paragraph (@paragraphs) {
    # do something
}

The Tie::File module lets you treat the file as an array of lines. The solution then becomes simply iterating through the array a line at a time from the end back to the start. It's much slower than reading everything into memory and reversing it, but works on files too big to fit into memory all at once. Be careful, though: Tie::File will rewrite the file if you change the contents of the tied @lines, so don't do that. In our example, assigning @lines = reverse(@lines) would reverse the file on disk! By opening the file with mode O_RDONLY (0), you can avoid that possibility. The default mode is O_RDWR | O_CREAT. Also, Tie::File cannot emulate the paragraph semantics of setting $/ to the empty string ("").

8.4.4 See Also

The reverse function in perlfunc(1) and in Chapter 29 of Programming Perl; the $/ entry in perlvar(1), and in Chapter 28 of Programming Perl; the documentation for the standard Tie::File module; Recipe 4.11; Recipe 1.7