10.1 GNU and Unix Commands (Topic 1.3)

10.1.1 Objective 1: Work Effectively on the Unix Command Line

10.1.1.1 The interactive shell and shell variables

• A shell provides the command prompt and interprets commands.

• A shell variable holds a value that is accessible to shell programs.

• PATH is a shell variable that contains a listing of directories that hold executable programs.

• Commands must be bash built-ins, found in the PATH, or explicitly defined in order to succeed.

• When shell variables are exported, they become part of the environment.

10.1.1.2 Entering commands

• Commands are comprised of a valid command, with or without one or more options and arguments, followed by a carriage return.

• Interactive commands can include looping structures more often used in shell scripts.

10.1.1.3 Command history, editing, and substitution

• Shell sessions can be viewed as a conversation. History, expansion, and editing make that dialogue more productive.

• Commands can be reissued, modified, and edited. Examples are shown in Table 10-1.

• Command substitution allows the result of a command to be placed into a shell variable.

Table 10-1. Shell Expansion, Editing, and Substitution Examples

History Type	Examples
Expansion	!! !n ^string1^string2
Editing	Ctrl-P, previous line Ctrl-K, kill to end of line Ctrl-Y, paste (yank) text
Substitution	VAR=$(command)

10.1.1.4 Recursive execution

• Many commands contain either a -r or -R option for recursive execution through a directory hierarchy.

• The find command is inherently recursive, and is intended to descend through directories looking for files with certain attributes or executing commands.

10.1.2 Objective 2: Process Text Streams Using Text Processing Filters

The following programs modify or manipulate text from files and standard input:

cut [files]

Cut out selected columns or fields from one or more files.

expand files

Convert tabs to spaces in files.

fmt [files]

Format text in files to a specified width by filling lines and removing newline characters.

head [files]

Print the first few lines of files.

join file1 file2

Print a line for each pair of input lines, one each from file1 and file2, that have identical join fields.

nl [files]

Number the lines of files, which are concatenated in the output.

od [files]

Dump files in octal, hexadecimal, ASCII, and other formats.

paste files

Paste together corresponding lines of one or more files into vertical columns.

pr [file]

Convert a text file into a paginated, columnar version, with headers and page fills.

split [infile] [outfile]

Split infile into a specified number of line groups; the output will go into a succession of files of outfileaa, outfileab, and so on.

tac [file]

Print file to standard output in reverse line order.

tail [files]

Print the last few lines of one or more files.

tr [string1 [string2]]

Translate characters by mapping from string1 to the corresponding character in string2.

wc [files]

Print counts of characters, words, and lines for files.

10.1.2.1 The stream editor, sed

sed is a popular text-filtering program found on nearly every Unix system; it has the following syntax:

sed command [files]

sed -e command1 [-e command2] [files]

sed -f script [files]

Execute sed commands, or those found in script, on standard input or files.

10.1.3 Objective 3: Perform Basic File Management

• Filesystem creation is called formatting, which prepares a disk partition for use. Linux usually uses the native ext2 (second extended) filesystem.

• The Linux filesystem is arranged into a hierarchical structure anchored at the root directory, or /. Beneath this is a tree of directories and files.

• Identification information for a filesystem object is stored in its inode, which holds location, modification, and security information. Filesystems are created with a finite number of inodes.

10.1.3.1 File and directory management commands

The following commands are essential for the management of files and directories:

cp file1 file2

cp files directory

Copy file1 to file2, or copy files to directory.

mkdir directories

Create one or more directories.

mv source target

Move or rename files and directories.

rm files

Delete one or more files from the filesystem. When used recursively (with the -r option), rm also removes directories.

rmdir directories

Delete directories, which must be empty.

touch files

Change the access and/or modification times of files by default to the present time.

10.1.3.2 File-naming wildcards

Wildcards (also called file globs) allow the specification of many files at once. A list of commonly used wildcards can be found in Table 10-2.

Table 10-2. File-Naming Wildcards

Wildcard	Function
*	Match zero or more characters.
?	Match exactly one character.
[characters]	Match any single character from among characters listed between brackets
[!characters]	Match any single character other than characters listed between brackets.
[a-z]	Match any single character from among the range of characters listed between brackets.
[!a-z]	Match any single character from among the characters not in the range listed between brackets.
{frag1, frag2, frag3, ...}	Brace expansion: create strings frag1, frag2, and frag3, etc., such that {file_one,two,three} yields file_one, file_two, and file_three.

10.1.4 Objective 4: Use Unix Streams, Pipes, and Redirects

• A central concept for Linux and Unix systems is that everything is a file.

• Many system devices are represented in the filesystem using a device file, such as /dev/ttyS0 for a serial port.

10.1.4.1 Standard I/O

• The shell provides the standard I/O capability, offering three default file descriptors to running programs.

• Standard input (stdin) is a text input stream, by default attached to the keyboard.

• Standard output (stdout) is an output stream for normal program output.

• Standard error (stderr) is an additional output stream meant for error messages.

10.1.4.2 Pipes and redirection

• It is possible to tie the output of one program to the input of another. This is known as a pipe and is created by joining commands using the pipe symbol (|).

• Pipes are a special form of redirection, which allows you to manage the origin of input streams and the destination of output streams. Redirection syntax for various shells differs slightly. See Table 10-3 for examples of common redirection operators.

Table 10-3. Common Redirection Operators

Redirection Function	Syntax for bash
Send stdout to file.	$ cmd > file $ cmd 1> file
Send stderr to file.	$ cmd 2> file
Send both stdout and stderr to file.	$ cmd > file 2>&1
Receive stdin from fil.e	$ cmd < file
Append stdout to fil.e	$ cmd >> file $ cmd 1>> file
Append stderr to file.	$ cmd 2>> file
Append both stdout and stderr to file.	$ cmd >> file 2>&1
Pipe stdout from cmd1 to cmd2.	$ cmd1 | cmd2
Pipe stdout and stderr from cmd1 to cmd2.	$ cmd1 2>&1 | cmd2
Pipe stdout from cmd1 to cmd2 while simultaneously writing it to file1 using tee.	$ cmd1 tee file1 | cmd2

10.1.5 Objective 5: Create, Monitor, and Kill Processes

• Processes have:

  • A lifetime.

  • A PID.

  • A UID.

  • A GID.

  • A parent process.

  • An environment.

  • A current working directory.

10.1.5.1 Monitoring commands

ps

Generate a one-time snapshot of the current processes on standard output.

pstree

Display a hierarchical list of processes in a tree format.

top

Generate a continuous, formatted, real-time process activity display on a terminal or in a terminal window.

10.1.5.2 Signaling processes

• Processes listen for signals sent by the kernel or users using the kill command:

kill

-sigspec [pids]

Send sigspec to pids.

• Common kill signals are listed in Table 10-4.

Table 10-4. Common Signals

Signal	Number	Meaning
HUP	1	Hangup, reread configuration.
INT	2	Interrupt, stop running.
KILL	9	Stop immediately.
TERM	15	Terminate nicely.
TSTP	18	Stop executing, ready to continue.

10.1.5.3 Shell job control

Shells can run processes in the background, where they execute on their own, or in the foreground, attached to a terminal. Each process handled in this way is known as a job. Jobs are manipulated using job control commands:

bg [jobspec]

Place jobspec in the background as if it had been started with &.

fg [jobspec]

Place jobspec in the foreground, making it the current job.

jobs [jobspecs]

List jobspecs on standard output.

10.1.6 Objective 6: Modify Process Execution Priorities

• A process' execution priority is managed by the kernel.

• You can bias the execution priority by specifying a nice number in the range of -20 to +19 (default is 0).

• Positive nice numbers reduce priority; negative nice numbers increase priority and are reserved for the superuser.

nice -adjustment [command]

Apply nice number adjustment to the process created to run command.

renice [+|-]nicenumber targets

Alter the nicenumber, and thus the scheduling priority, of one or more running target processes.

10.1.7 Objective 7: Perform Searches of Text Files Making Use of Regular Expressions

• Regular expressions are used to match text. The term is used to describe the loosely defined text-matching language as well as the patterns themselves. A regular expression is often called a regex or a regexp.

• Regular expressions are made up of metacharacters (with special meaning) and literals (everything that is not a metacharacter).

10.1.7.1 Position anchors

These operators match line position:

^

Match the beginning of a line.

$

Match the end of a line.

10.1.7.2 Character sets

These operators match text:

[abc]

[a-z]

Match any single character from among listed characters or from among the characters comprising a range.

[^abc]

[^a-z]

Match any single character not among listed characters or ranges.

\<word\>

Match words bounded by whitespace.

. (A single period, or dot)

Match any single character except a newline.

\

Turn off (escape) the special meaning of a metacharacter that follows.

10.1.7.3 Modifiers

These operators modify the way other operators are interpreted:

*

Match zero or more of the character that precedes it.

?

Match zero or one instance of the preceding regex.

+

Match one or more instances of the preceding regex.

\{ n,m\}

Match a range of occurrences of the single character or regex that precedes this construct.

|

Match the character or expression to the left or right of the vertical bar.