Book: LPI Linux Certification in a Nutshell
Section: Chapter 17.  Shells, Scripting, Programming, and Compiling (Topic 1.9)



17.1 Objective 1: Customize and Use the Shell Environment

This Objective could be considered a brief "getting started with shells" overview because it details many of the basic concepts necessary to utilize the shell environment on Linux. These concepts are fundamental and very important for system administrators working on Linux and Unix systems. If you're new to shells and shell scripting, take heart. You can think of it as a combination of computer interaction (conversation) and computer programming (automation). It is nothing more than that, but the result is far more than this simplicity implies. If you're an old hand with shell programming, you may want to skip ahead to brush up on some of the particulars necessary for Exam 102.

17.1.1 An Overview of Shells

A shell is a fundamental and important part of your Linux computing environment. Shells are user programs not unlike other text-based programs and utilities. They offer a rich customizable interface to your system. Some of the main items provided by your shell are:

An interactive textual user interface to the operating system

In this the role, the shell is a command interpreter and display portal to the system. It offers you a communications channel to the kernel and is often thought of as the "shell around the kernel." That's where the name shell originates and is a good metaphor for conceptualizing how shells fit into the overall Linux picture.

An operating environment

Shells set up an environment for the execution of other programs, which affect the way some of them behave. This environment consists of any number of environment variables, each of which describes one particular environment property by defining a name=value pair. Other features such as aliases enhance your operating environment by offering shorthand notations for commonly used commands.

A facility for launching and managing commands and programs

Shells are used not only by users but also by the system to launch programs and support those programs with an operating environment.

A programming language

Shells offer their own programming languages. At its simplest, this feature allows user commands to be assembled into useful sequences. At the other end of the spectrum, complete programs can be written in shell languages, with loop control, variables, and all of the capabilities of Linux's rich set of operating system commands.

On a Linux system, you have a choice of at least five different shells. There are two basic families:

Bourne-derived shells

Many shells are related to the Bourne shell, sh, named for creator Steve Bourne. sh is the oldest of the currently available shells and lacks several features considered necessary for modern interactive use. However, it is well understood, is used regularly for programming, and can be found on nearly every Unix or Unix-like system. One of the descendants of sh is bash, which is described in the next section.

C-shells

For interactive use, many people like to use the C-shell, csh, or its descendant, tcsh. These shells have some elements in their associated programming language syntax that are similar to the C language. Despite this, many feel that programming in the C-shell is less than satisfactory due to some missing features. Additional shells include zsh and ksh.

All of the shells share some common concepts:

  • They are all distinct from the kernel and run as user programs.

  • Each shell can be customized by tuning the shell's operating environment.

  • Shells are run for both interactive use by end users and noninteractive use by the system.

  • A shell can be run from within another shell, enabling you to try a shell other than your default shell. To do this, you simply start the other shell from the command line of your current shell. In fact, this happens constantly on your system as scripts are executed and programs are launched. The new shell does not replace the shell that launched it. Instead, the new shell is a process running with the original shell as a parent process. When you terminate the child shell, you go back to the original one.

  • Shells use a series of configuration files to establish their operating environment.

  • Shells pass on environment variables to child processes.

17.1.2 The bash Shell

To enhance the Bourne shell while retaining its programming constructs, a few descendants have been written over the years. Among those descendants is bash, which stands for Bourne-again shell, from the Free Software Foundation (the FSF is famous for tongue-in-cheek command names). While there are a number of shells available to choose from on a Linux system, bash is very popular and powerful, and is the default shell for new accounts. Exam 102 concentrates on its use and configuration. The next few sections deal with general shell concepts, but the examples are specific to bash.

17.1.2.1 Shell and environment variables

Many programs running under Linux require information about you and your personal preferences to operate sensibly. While you could instruct each program you run with important details it needs to proceed, much of the information you'd convey would be redundant because you'd be telling every command you enter the same ancillary information at each invocation. For example, you'd need to tell your paging program about the size and nature of your terminal or terminal window each time you use it. You would also need to give fully qualified directory names for the programs you run.

Rather than force users to include so much detail to issue commands, the shell handles much of this information for you automatically. You've already seen that the shell creates an operating environment for you. That environment is made up of a series of variables, each of which has a value that is used by programs and other shells. There are two types of variables used by most shells:

Environment variables

These variables can be thought of as global variables because they are passed on to all processes started by the shell, including other shells. This means that child processes inherit the environment. By convention, environment variables are given uppercase names.[1] Your shell maintains many environment variables, including the following examples:

[1] bash doesn't require the case convention; it's intended for clarity to humans.

PATH

A list of directories through which the shell looks for executable programs as you enter them on the command line. All of the directories that contain programs that you'll want to execute are stored together in the PATH environment variable. Your shell looks through this list in sequence, from left to right, searching for each command you enter. Your PATH may differ from the PATH s of other users on your system because you may use programs found in different locations or you may have a local directory with your own custom programs that need to be available. The PATH variable can become quite long as more and more directories are added.

HOME

Your home directory, such as /home/bsmith.

USERNAME

Your username.

TERM

The type of terminal or terminal window you are running. This variable is likely to have a value such as xterm or xterm-color. If you are running on a physical VT100 (or compatible) terminal, TERM is set to vt100.

Shell variables

These variables can be thought of as local because they are specific only to the current shell. Child processes do not inherit them. Some shell variables are automatically set by the shell and are available for use in shell scripts. By convention, shell variables are given lowercase names.

In the csh and tcsh shells, environment variables and shell variables are differentiated by their case, and shell variables are always local while environment variables are always global. In bash, this distinction is blurred somewhat, because variables are shell variables until they are exported to the environment, making them environment variables that will be passed on to child shells and programs. In addition, nearly all the shell and environment variables you'll encounter in bash will be uppercase.

To create a new bash shell variable, simply enter a name=value pair on the command line:

# PI=3.14

To see that this value is now assigned to the local variable PI, use the echo command to display its contents:

# echo $PI
3.14

The dollar sign preceding the variable name indicates that the name will be replaced with the variable's value. Without the dollar sign, echo would just return the text that was typed, which in this case is the variable name PI. At this point, PI is a local variable and is not available to child shells or programs. To make it available to other shells or programs, the variable must be exported to the environment:

# export PI
17.1.2.2 Aliases

Among the features missing from shwas the ability to easily make new commands or modify existing commands. bash has the ability to set an alias for commonly used commands or sequences of commands. For example, if you habitually call for the older pager more but actually prefer less, an alias can be handy to get the desired behavior, regardless of the command you use:

$ alias more='less'

This has the effect of intercepting any command entries for more, substituting less instead. The revised command is passed along to the shell's command interpreter.

Another common use for an alias is to modify a command slightly so that its default behavior is more to your liking. Many people, particularly when operating with superuser privileges, will use this alias:

$ alias cp='cp -i'

With this alias in effect, the use of the cp (copy) command becomes safer, because with the -i option always enforced by the alias, cp prompts you for approval before overwriting a file of the same name. Additional options you enter on the command line are appended to the end of the new command, such that cp -p becomes cp -i -p and so on.

If the righthand side of the aliased command is bigger than a single word or if it contains multiple commands (separated by semicolons, bash's command terminator), you probably need to enclose it in single quotation marks to get your point across. This is because you need to prevent the shell in which you're working (your current bash process) from interpreting file globbing or other characters that might be part of your alias value. For example, suppose you wished to use a single alias to pair two simple commands:

$ alias lsps=ls -l;ps

Your current bash process will interpret this command not as a single alias but as two separate commands. First the alias lsps will be created for ls -l, and then a ps command will be added for immediate execution. What you really want is:

$ alias lsps='ls -l;ps'

Now, entering the command lsps will be aliased to ls -l; ps, and will correctly generate ls output immediately followed by ps output, as this example shows:

$ lsps
total 1253
drwx------  5 root  root    1024 May 27 17:15 dir1
drwxr-xr-x  3 root  root    1024 May 27 22:41 dir2
-rw-r--r--  1 root  root   23344 May 27 22:44 file1
drwxr-xr-x  2 root  root   12288 May 25 16:13 dir3
  PID TTY          TIME CMD
  892 ttyp0    00:00:00 bash
 1388 ttyp0    00:00:00 ps

Admittedly, this isn't a very useful command, but it is built upon in the next section.

After adding aliases, it may become easy to confuse which commands are aliases or native. To list the aliases defined for your current shell, simply enter the alias command by itself. This results in a listing of all the aliases currently in place:

$ alias
alias cp='cp -i'
alias lsps='ls -l;ps'
alias mv='mv -i'
alias rm='rm -i'

Note that aliases are local to your shell and are not passed down to programs or to other shells. You'll see how to ensure that your aliases are always available in the section on configuration files.

Aliases are mainly used for simple command replacement. The shell inserts your aliased text in place of your alias name before interpreting the command. Aliases don't offer logical constructs and are limited to a few simple variable replacements. Aliases can also get messy when the use of complicated quoting is necessary, usually to prevent the shell from interpreting characters in your alias.

17.1.2.3 Functions

In addition to aliases, bash also offers functions . They work in much the same way as aliases, in that some function name of your choosing is assigned to a more complex construction. However, in this case that construction is a small program rather than a simple command substitution. Functions have a simple syntax:

$ [ function ] NAME (  ) { COMMAND-LIST; }

This declaration defines a function called NAME. The word function is optional, and the parentheses after NAME are required if function is omitted. The body of the function is the COMMAND-LIST between the curly brackets ({ and }). This list is a series of commands, separated by semicolons or by new lines. The series of commands are executed whenever NAME is specified as a command. The simple lsps alias shown earlier could be implemented as a function like this:

$ lsps (  ) { ls -l; ps; }

Using this new function as a command yields exactly the same result the alias did. However, by implementing this command using a function, parameters can be added to the command. Here is a new version of the same function, this time entered on multiple lines (which eliminates the need for semicolons within the function):

$ lsps (  ) {
> ls -l $1
> ps -aux | grep `/bin/basename $1`
> }

The > characters come from bash during interactive entry, indicating that bash is awaiting additional function commands or the } character, which terminates the function definition. This new function allows us to enter a single argument to the function, which is inserted everywhere $1 is found in the function. These arguments are called positional parameters because each one's number denotes its position in the argument list. This example uses only one positional parameter; there can be many, and the number of parameters is stored for your use in a special variable $# .

The command implemented in the previous example function now returns a directory listing and process status for any program given as an argument. For example, if the Apache web server is running, the command:

$ lsps /usr/sbin/httpd

yields a directory listing for /usr/sbin/httpd and also displays all currently running processes that match httpd:

-rwxr-xr-x  1 root root 165740 Apr  7 17:17 /usr/sbin/httpd
root   3802 0.0  3.8  2384 1192 ?     S 16:34 0:00 httpd
nobody 3810 0.0  4.2  2556 1292 ?     S 16:34 0:00 httpd
nobody 3811 0.0  4.2  2556 1292 ?     S 16:34 0:00 httpd
nobody 3812 0.0  4.2  2556 1292 ?     S 16:34 0:00 httpd
nobody 3813 0.0  4.2  2556 1292 ?     S 16:34 0:00 httpd
nobody 3814 0.0  4.2  2556 1292 ?     S 16:34 0:00 httpd
root   3872 0.0  1.4  1152 432 ttyp0 S 16:45 0:00 grep httpd
17.1.2.4 Configuration files

It's a good assumption that every Linux user will want to define a few aliases, functions, and environment variables to suit his needs. However, it's undesirable to manually enter them upon each login or for each new invocation of bash. In order to set up these things automatically, bash uses a number of configuration files to set its operating environment when it starts. Some of these files are used only upon initial log in, while others are executed for each instance of bash you start, including login time. Some of these configuration files are system-wide files for all users to use, while others reside in your home directory for your use alone.

bash configuration files important to Exam 102 are listed in Table 17-1.

Table 17-1. bash Configuration Files

File

Description

/etc/profile

This is the systemwide initialization file, executed during log in. It usually contains environment variables, including an initial PATH, and startup programs.

/etc/bashrc

This is another systemwide initialization file that may be executed by a user's .bashrc for each bash shell launched. It usually contains functions and aliases.

~/.bash_ profile

If this file exists, it is executed automatically after /etc/profile during log in.

~/.bash_login

If .bash_ profile doesn't exist, this file is executed automatically during log in.

~/.profile

If neither .bash_ profile nor .bash_login exist, this file is executed automatically during log in. Note that this is the original bourne shell configuration file.

~/.bashrc

This file is executed automatically when bash starts. This includes login, as well as subsequent interactive and noninteractive invocations of bash.

~/.bash_logout

This file is executed automatically during log out.

~/.inputrc

This file contains optional key bindings and variables that affect how bash responds to keystrokes. By default, bash is configured to respond like the Emacs editor.

In practice, users will generally (and often unknowingly) use the systemwide /etc/profile configuration file to start. In addition, they'll often have three personal files in their home directory: ~/.bash_ profile, ~/.bashrc, and ~/.bash_logout. The local files are optional, and bash does not mind if one or all of them are not available in your directory.

The syntax ~/ refers to bash's "home directory." While this shortcut may not represent much of a savings in typing, some Linux configurations may place users' directories in various and sometimes nonobvious places in the filesystem. Using the tilde syntax reduces the need for you to know exactly where a user's home directory is located.

Each of these configuration files consists entirely of plain text. They are typically simple, often containing just a few commands to be executed in sequence to prepare the shell environment for the user. Since they are evaluated by bash as lines of program code, they are said to be sourced, or interpreted, when bash executes them.

Like most programming languages, shell programs allow the use of comments. Most shells including bash consider everything immediately following the hash (#) character on a single line to be a comment.[2] Comments can span an entire line or share a line by following program code. All of your shell scripts and configuration files should use comments liberally.

[2] An important exception is the $# variable, which has nothing to do with comments but contains the number of positional parameters passed to a function.

Files sourced at login time are created mainly to establish default settings. These settings include such things as where to search for programs requested by the user (the PATH) and creation of shortcut names for commonly used tasks (aliases and functions). After login, files sourced by each subsequent shell invocation won't explicitly need to do these things again because they inherit the environment established by the login shell. Regardless, it isn't unusual to see a user's .bashrc file filled with all of their personal customizations. It also doesn't hurt anything, provided the .bashrc file is small and quick to execute.

While it is not necessary to have detailed knowledge of every item in your shell configuration files, Exam 102 requires that you understand them and that you can edit them to modify their behavior and your resulting operating environment. The following examples are typical of those found on Linux systems and are annotated with comments. Example 17-1 shows a typical Linux system-wide profile. This file is executed by every user's bash process at login time. A few environment variables and other parameters are set in it.

Example 17-1. An Example System-wide bash profile
# /etc/profile
# System-wide environment and startup programs
# Functions and aliases go in system wide /etc/bashrc

# PATH was already set, this is an extension
PATH="$PATH:/usr/X11R6/bin"

# Set a default prompt string
PS1="[\u@\h \W]\\$ "

# Set an upper limit for "core" files
ulimit -c 1000000

# Set a default umask, used to set default file permissions
if [ `id -gn` = `id -un` -a `id -u` -gt 14 ]; then
   umask 002
else
   umask 022
fi

# Set up some shell variables
USER=`id -un`
LOGNAME=$USER
MAIL="/var/spool/mail/$USER"
HOSTNAME=`/bin/hostname`
HISTSIZE=1000
HISTFILESIZE=1000
INPUTRC=/etc/inputrc

# Make all these into environment variables
export PATH PS1 HOSTNAME HISTSIZE HISTFILESIZE 
   USER LOGNAME MAIL INPUTRC

# Execute a series of other files
for i in /etc/profile.d/*.sh ; do
   if [ -x $i ]; then
      . $i
   fi
done

unset I       # Clean up the variable used above

Example 17-2 shows a system-wide .bashrc file. This file is not sourced by default when bash starts. Instead, it is optionally sourced by users' local .bashrc files.

Example 17-2. An Example System-wide .bashrc File
# /etc/bashrc

alias more='less'                    # prefer the "less" pager
alias lsps='ls -l;ps'                # a dubious command

Example 17-3 shows an example user's local .bash_ profile. Note that this file sources the system-wide /etc/bashrc, then goes on to local customizations.

Example 17-3. An Example User .bash_ profile File
# .bash_profile

# Get the aliases and functions from the systems administrator
if [ -f ~/.bashrc ]; then
   . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin  # Add my binaries directory to the path
EDITOR=emacs          # Set my preferred editor to Emacs
VISUAL=emacs          # Set my preferred editor to Emacs
PAGER=less            # Set my preferred pager to less

# Make my variables part of the environment
export PATH EDITOR VISUAL PAGER      

Example 17-4 shows an individual's .bashrc file. Like the .bash_profile earlier, this file also sources the system-wide /etc/bashrc.

Example 17-4. An Example User's .bashrc File
# .bashrc

# User-specific aliases and functions

# Source global definitions
if [ -f /etc/bashrc ]; then
   . /etc/bashrc
fi

alias rm='rm -i'                     # Add a safety net to rm
alias cp='cp -i'                     # Add a safety net to cp
alias mv='mv -i'                     # Add a safety net to mv

lsps(  ) {                             # Define a personal function
   ls -l $1
   ps -aux | grep `/bin/basename $1`
}

Example 17-5 shows a short, simple, and not uncommon .bash_logout file. Probably the most likely command to find in a logout file is the clear command. Including a clear in your logout file is a nice way of being certain that whatever you were doing just before you log out won't linger on the screen for the next user to ponder. This file is intended to execute commands for a logout from a text session, such as a system console or terminal. In a GUI environment where logout and login are handled by a GUI program, .bash_logout may not be of much value.

Example 17-5. A Simple .bash_logout File
# .bash_logout
# This file is executed when a user logs out of the system
/usr/bin/clear          # clear the screen
/usr/games/fortune      # print a random adage

On the Exam

Make certain that you understand the difference between execution at login and execution at shell invocation, as well as which of the startup files serve each of those purposes.

17.1.2.5 Inputrc

Among the many enhancements added to bash is the ability to perform as if your history of commands is the buffer of an editor. That is, your command history is available to you, and you may cut, paste, and even search among command lines entered previously. This powerful capability can significantly reduce typing and increase accuracy. By default, bash is configured to emulate the Emacs editor, but a vi editing interface is also available.

The portion of bash that handles this function, and in fact handles all of the line input during interactive use, is known as readline. Readline may be customized by putting commands into an initialization file, which by default is in your home directory and called .inputrc.[3] For example, to configure bash to use vi-style editing keys, add this line to .inputrc:

[3] You may also set the INPUTRC variable to the name of another file if you prefer. On your system, this variable may be set to/etc/initrc by default, which would override any settings you put into a local .initrc. To use your own file, you must first explicitly place unset INPUTRC in your.bash_profile.

set editing-mode vi

The default editing facilities enabled in bash are extensive and are beyond the scope of this section and Exam 102. However, you need to understand the concepts of adding your own custom key bindings to the .inputrc file and how they can help automate common keystrokes unique to your daily routine for the test.

For example, suppose you often use top to watch your system's activity (top is a useful process-monitoring utility that is described inChapter 3):

$ top -Ssd1

If you do this often enough, you'll get tired of typing the command over and over and will eventually want an alias for it. To create the alias, simply alias this command to top:

$ alias top='/usr/bin/top -Ssd1'

Better yet, you can use .inputrc to create a key binding that will enter it for you. Here's how the .inputrc file would look if you were to bind your top command to the key sequence Ctrl-t:

# my .inputrc file
Control-t: "top -Ssd1 \C-m"

The lefthand side of the second line indicates the key combination you wish to use (Ctrl-t). The righthand side indicates what you wish to bind to that key sequence. In this case, bash outputs top -Ssd1 and a carriage return, denoted here by \C-m (Ctrl-m), when Ctrl-t is pressed.

Through modifications of your local configuration files, you can customize your environment and automate many of your daily tasks. You may also override system-wide settings in your personal files simply by setting variables, aliases, and functions.

On the Exam

You won't need to have detailed knowledge of this key-binding syntax, but be aware of the .inputrc file and the kinds of things it enables bash to do.