Secure Copy with scp (SSH, The Secure Shell: The Definitive Guide)

7.5. Secure Copy with scp

The secure copy program, scp, obeys keywords in your client configuration file just as ssh does. In addition, scp provides other features and options that we'll cover in this section.

7.5.1. Full Syntax

So far, we've described the syntax of scp only in general: [Section 2.2.1, "File Transfer with scp"]

scp name-of-source name-of-destination

Each of the two names, or path specifications, on the command line represents files or directories in the following manner (it is fairly consistent with the behavior of Unix cp or rcp):

If name-of-source is a file, name-of-destination may be a file (existing or not) or a directory (which must exist). In other words, a single file may be copied to another file or into a directory.
If name-of-source is two or more files, one or more directories, or a combination, name-of-destination must be an existing directory into which the copy takes place.[106] In other words, multiple files and directories may be copied only into a directory.
[106]We say "must," but technically you may specify a file as a destination in some cases. However, this behavior is probably not what you want. As your multiple files get copied into a single destination file, each is overwritten by the next.

Both name-of-source and name-of-destination may have the following form from left to right:

The username of the account containing the file or directory, followed by @. This part is optional and if omitted, the value is the username of the user invoking scp.
The hostname of the host containing the file or directory, followed by a colon. This part is optional if the path is present, and the username isn't; if omitted, the value is localhost. SSH2 permits an optional TCP port number for the SSH connection to be inserted between the hostname and the colon, preceded by a hash sign.
The directory path to the file or directory. (Optional if the hostname is present.) Relative pathnames are assumed relative to the default directory, which is the current directory (for local paths) or the user's home directory (for remote paths). If omitted entirely, the path is assumed to be the default directory.

Although each field is optional, you can't omit them all at the same time, yielding the empty string. Either the hostname (2) or the directory path (3) must be present. Some examples:

MyFile: The file . /MyFile on localhost
MyDirectory: The directory ./MyDirectory on localhost
.: The current directory on localhost
server.example.com:: The directory ~username on server.example.com
server.example.com: A local file named "server.example.com" (oops: did you forget the trailing colon -- a common mistake)
server.example.com:MyFile: The file MyFile in the remote user's home directory on server.example.com
[email protected]:: The directory ~bob on server.example.com
[email protected]: A local file named "[email protected]" (oops; forgot the trailing colon again)
[email protected]:MyFile: The file ~bob/MyFile on server.example.com
server.example.com:dir/MyFile: The file dir/MyFile in the remote user's home directory on server.example.com
server.example.com:/dir/MyFile: The file /dir/MyFile on server.example.com (note the absolute path)
[email protected]:dir/MyFile: The file ~bob/dir/MyFile on server.example.com
[email protected]:/dir/MyFile: The file /dir/MyFile on server.example.com (although you authenticate as bob, the path is absolute)
server.example.com#2000:: The remote user's home directory on server.example.com, via TCP port 2000 (SSH2 only)

Here are a few complete examples:

$ scp myfile myfile2
A local copy just like cp
$ scp myfile bob@host1:
Copy . /myfile to ~bob on host1
$ scp bob@host1:myfile .
Copy ~bob/myfile on host1 to ./myfile
$ scp host1:file1 host2:file2
Copy file1 from host1 to file2 on
host2
$ scp bob@host1:file1 jen@host2:file2
Same as above, but copying from bob's to jen's account

Table 7-3 summarizes the syntax of an scp path.

Table 7-3. scp Path Specifications

Field	Other Syntax	Optional?	Default for Local Host	Default for Remote Host
Username	Followed by @	Yes	Invoking user's username	Invoking user's username
Hostname	Followed by :	Only if username is omitted and path is present	None, file is accessed locally	N/A
Port number [107]	Preceded by #	Yes	22	22
Directory path	N/A	Only if hostname is present	Current (invoking) directory	Username's remote home directory

[107]SSH2 only.

7.5.2. Handling of Wildcards

scp for SSH1 and OpenSSH has no special support for wildcards in filenames. It simply lets the shell expand them:

$ scp *.txt server.example.com:

Watch out for wildcards in remote file specifications, as they are evaluated on the local machine, not the remote. For example, this attempt is likely to fail:

$ scp1 server.example.com:*.txt .              Bad idea!

The Unix shell attempts to expand the wildcard before scp1 is invoked, but the current directory contains no filename matching "server.example.com:*.txt". C shell and its derivatives will report "no match" and not execute scp1. Bourne-style shells, noticing no match in the current directory, will pass the unexpanded wildcard to scp1, and the copy may succeed as planned, but this coincidental behavior shouldn't be relied on. Always escape your wildcards so they are explicitly ignored by the shell and passed to scp1:

$ scp1 server.example.com:\*.txt .

scp2 does its own regular expression matching after shell-wildcard expansion is complete. The sshregex manpage for SSH2 (see Appendix A, "SSH2 Manpage for sshregex") describes the supported operators. Even so, escape your wildcard characters if you want your local shell to leave them alone.

7.5.3. Recursive Copy of Directories

Sometimes you want to copy not just a single file but a directory hierarchy. In this case, use the -r option, which stands for recursive. If you are familiar with rcp, its -r option has the same effect.

For example, to securely copy the directory /usr/local/bin and all its files and subdirectories to another machine:

# SSH1, SSH2, OpenSSH
$ scp -r /usr/local/bin server.example.com:

If you forget the -r option when copying directories, scp complains:

$ scp /usr/local/bin server.example.com:
/usr/local/bin: not a regular file

Although scp can copy directories, it isn't necessarily the best method. If your directory contains hard links or soft links, they won't be duplicated. Links are copied as plain files (the link targets), and worse, circular directory links cause scp1 to loop indefinitely. (scp2 detects symbolic links and copies their targets instead.) Other types of special files, such as named pipes, also aren't copied correctly.[108] A better solution is to use tar, which handles special files correctly, and send it to the remote machine to be untarred, via SSH:

[108]These limitations also are true when copying single files, but at least you see the erroneous result quickly. With directories, you can copy a hierarchy incorrectly and not notice.

$ tar cf - /usr/local/bin | ssh server.example.com tar xf -

7.5.4. Preserving Permissions

When scp copies files, the destination files are created with certain file attributes. By default, the file permissions adhere to a umask on the destination host, and the modification and last access times will be the time of the copy. Alternatively, you can tell scp to duplicate the permissions and timestamps of the original files. The -p option accomplishes this:

# SSH1, SSH2, OpenSSH
$ scp -p myfile server.example.com:

For example, if you transfer your entire home directory to a remote machine, you probably want to keep the file attributes the same as the original:

$ scp -rp $HOME server.example.com:myhome/

7.5.5. Automatic Removal of Original File

After copying a file, scp2 can optionally remove the original if desired. The -u command-line option specifies this:[109]

[109]In some earlier versions of SSH2, this option has no effect.

# SSH2 only
$ scp2 myfile server.example.com:
$ ls myfile
myfile
$ scp2 -u myfile server.example.com:
$ ls myfile
myfile: No such file or directory

If you've ever wanted a "secure move" command in addition to secure copy, you can define one in terms of scp2 -u:

$ alias smv='scp2 -u'

7.5.6. Safety Features

scp has two features to protect you from running dangerous commands. Suppose you want to copy a local file myfile to a remote directory. You type:

# SSH1, SSH2, OpenSSH
$ scp2 myfile server.example.com:mydir
$ rm myfile

Then you connect to server.example.com and find, to your horror, that mydir was a file, not a directory, and you just overwrote it! The -d option prevents this tragedy. If the destination isn't a directory, scp complains and exits without copying the file.

# SSH1, SSH2, OpenSSH
$ scp2 -d myfile server.example.com:mydir
warning: Destination file is not a directory.
warning: Exiting.

This option is necessary only if you are copying a single file. If you are copying multiple files or a directory, all the scp implementations check by default that the remote destination is a directory.[110]

[110]There's one degenerate case. If your copy occurs on a single machine, e.g., scp *.c mydir, the scp client doesn't necessarily check that mydir is a directory.

Another safety feature of scp2 is the -n option, which instructs the program to describe its actions but not perform any copying. This is useful for verifying the behavior of scp2 before executing a potentially risky command.

# SSH2 only
$ scp2 -n myfile server.example.com:
Not transferring myfile -> server.example.com:./myfile  (1k)

7.5.7. Statistics Display

As scp copies files, it may print statistics about its progress.

7.5.7.1. scp1 statistics

For scp1, the statistics display is configurable by command-line options and environment variables:[111]

[111]For starters, scp1 must be compiled with the configuration flag -- with-scp-stats, or else statistics will be unavailable. [Section 4.1.5.11, "scp behavior"]

$ scp1 myfile* server.example.com:
myfile1         |         50 KB |  50.0 kB/s | ETA: 00:00:00 | 100%
myfile2         |         31 KB |  31.3 kB/s | ETA: 00:00:00 | 100%
myfile3         |          3 KB |   3.8 kB/s | ETA: 00:00:00 | 100%

For each file, scp1 displays the name, the size, the transfer rate, and a two-part progress meter about the transmission. "ETA" (Estimated Time of Arrival) is the estimated transfer time, and the final number is the percentage of the file transmitted so far. While the file is transferring, the ETA value counts down to zero and the percentage increases to 100, though you can't see this on the printed page.

Although the statistics are informative, you might want to change or disable them. For example, you might prefer to turn them off when scp1 is part of a batch job that shouldn't produce screen output.

This statistics display can be configured in several ways, using command-line options and environment variables (see Table 7-4: note that command-line options take precedence over environment variables).

Table 7-4. Controlling Statistics in scp1

Desired Outcome	Using Options	Setting Environment Variables
No output [112]	scp1 -q	SSH_NO_SCP_STATS
Output, but not file-by-file	scp1 `-`Q `-`A	SSH_NO_ALL_SCP_STATS SSH_SCP_STATS
Output file-by-file	scp1 `-`Q `-`a	SSH_ALL_SCP_STATSSSH_SCP_STATS

Desired Outcome

Using Options

Setting Environment Variables

No output [112]

scp1 -q

SSH_NO_SCP_STATS

Output, but not file-by-file

scp1 -Q -A

SSH_NO_ALL_SCP_STATS

SSH_SCP_STATS

Output file-by-file

scp1 -Q -a

SSH_ALL_SCP_STATSSSH_SCP_STATS

[112]Also works for OpenSSH's scp client.

First, you may control the presence or absence of statistics at all. This is done with the options -q and -Q, or the environment variables SSH_SCP_STATS and SSH_NO_SCP_STATS. To disable statistics, use either of the following:

# SSH1, OpenSSH
$ scp -q myfile server.example.com:

# SSH1 only
$ setenv SSH_NO_SCP_STATS 1
$ scp1 myfile server.example.com:

To enable statistics, use either of these:

# SSH1 only
$ scp1 -Q myfile server.example.com:

# SSH1 only
$ setenv SSH_SCP_STATS 1
$ scp1 myfile server.example.com:

If statistics are enabled, you may also choose to print file-by-file statistics. This is done with the options -a and -A, or the environment variables SSH_ALL_SCP_STATS and SSH_NO_ALL_SCP_STATS. To print file-by-file statistics, use either of these:

# SSH1 only
$ scp1 -Q -a myfile server.example.com:

# SSH1 only
$ setenv SSH_ALL_SCP_STATS 1
$ scp1 myfile server.example.com:

or to print a single, cumulative statistic:

# SSH1 only
$ scp1 -Q -A myfile server.example.com:

# SSH1 only
$ setenv SSH_NO_ALL_SCP_STATS 1
$ scp1 myfile server.example.com:

7.5.7.2. scp2 statistics

The statistics display for scp2 is configurable as well, but as of SSH2 2.1.0, this information is missing from the manpage. By default, the statistics display is enabled, and there's no compile-time option like SSH1's -- with-scp-stats to disable it. The display looks different from that of scp1:

$ scp2 myfile* server.example.com:
Transfering myfile1 -> server.example.com:./myfile1  (50k)
|................................................................|
51200 bytes transferred in 1.00 seconds [50.0 kB/sec].
Transfering myfile2 -> server.example.com:./myfile2  (30k)
|................................................................|
31744 bytes transferred in 1.03 seconds [31.3 kB/sec].
Transfering myfile3 -> server.example.com:./myfile3  (3k)
|................................................................|
3068 bytes transferred in 0.79 seconds [3.8 kB/sec].

The progress indicators (dotted lines) change as the files are transferred, but frankly we find them unintuitive. To suppress the statistics display, use the -Q command-line option (yes, it has the opposite meaning of SSH1's -Q option):

$ scp2 -Q myfile server.example.com:

7.5.8. Locating the ssh Executable

To copy files securely, scp invokes ssh internally. Therefore, scp needs to know where the ssh executable resides on disk. Normally, the path to ssh is made known to scp at compile time (by the compile-time flag -- prefix), but you can specify the path manually if you like. [Section 4.1.5.2, "Installation, files, and directories"] For instance, you can test a new version of ssh with an old version of scp. The command-line option -S specifies the path:

# SSH1, SSH2
$ scp -S /usr/alternative/bin/ssh myfile server.example.com:

7.5.9. For Internal Use Only

scp for SSH1 and OpenSSH has two undocumented options, -t and -f, for internal use. Most likely you will never need to use them explicitly. They inform scp of the direction of the copy: from the local to the remote machine, or from remote to local. The -t option means copying to a remote machine and -f means copying from a remote machine.

Whenever you invoke scp, it invisibly runs a second scp process on the remote host that includes either -t or -f on its command line. You can see this if you run scp in verbose mode. If copying from the local to the remote machine, you see:

$ scp -v myfile server.example.com:
Executing: host server.example.com, ..., command scp -v -t .
...

On the other hand, if you copy from the remote to the local machine, you see:

$ scp -v server.example.com:myfile .
Executing: host server.example.com, ..., command scp -v -f .
...

Again, it's likely you'll never use these options, but they're useful to know when reading scp's output in verbose mode. Also, the scp2 manpage mentions them, so it's good to understand what they are.


7.4. Client Configuration in Depth		7.6. Summary