4.19 Walking Directory Trees

Credit: Robin Parmar, Alex Martelli

4.19.1 Problem

You need to examine a directory, or an entire directory tree rooted in a certain directory, and obtain a list of all the files (and optionally folders) that match a certain pattern.

4.19.2 Solution

os.path.walk is sufficient for this purpose, but we can pretty it up quite at bit:

import os.path, fnmatch

def listFiles(root, patterns='*', recurse=1, return_folders=0):

    # Expand patterns from semicolon-separated string to list
    pattern_list = patterns.split(';')
    # Collect input and output arguments into one bunch
    class Bunch:
        def _ _init_ _(self, **kwds): self._ _dict_ _.update(kwds)
    arg = Bunch(recurse=recurse, pattern_list=pattern_list,
        return_folders=return_folders, results=[])

    def visit(arg, dirname, files):
        # Append to arg.results all relevant files (and perhaps folders)
        for name in files:
            fullname = os.path.normpath(os.path.join(dirname, name))
            if arg.return_folders or os.path.isfile(fullname):
                for pattern in arg.pattern_list:
                    if fnmatch.fnmatch(name, pattern):
                        arg.results.append(fullname)
                        break
        # Block recursion if recursion was disallowed
        if not arg.recurse: files[:]=[]

    os.path.walk(root, visit, arg)

    return arg.results

4.19.3 Discussion

The standard directory-tree function os.path.walk is powerful and flexible, but it can be confusing to beginners. This recipe dresses it up in a listFiles function that lets you choose the root folder, whether to recurse down through subfolders, the file patterns to match, and whether to include folder names in the result list.

The file patterns are case-insensitive but otherwise Unix-style, as supplied by the standard fnmatch module, which this recipe uses. To specify multiple patterns, join them with a semicolon. Note that this means that semicolons themselves can't be part of a pattern.

For example, you can easily get a list of all Python and HTML files in directory /tmp or any subdirectory thereof:

thefiles = listFiles('/tmp', '*.py;*.htm;*.html')

4.19.4 See Also

Documentation for the os.path module in the Library Reference.