Never Ending Security

It starts all here

Tag Archives: SCP

SCP Commands Cheatsheet


SCP Command Examples

Secure copy or SCP is a means of securely transferring computer files between a local host and a remote host or between two remote hosts. It is based on the Secure Shell (SSH) protocol

Basic syntax of SCP

scp source_file_name username@destination_host:destination_folder

SCP – detail information

Basic SCP command without parameter will copy all your files in the background. User will not see the progress or any other related information – unless there’s an error. To see what is really going on behind the curtain you would use “-v” switch. This will print all debug information into you console. If your SCP commands fails for some reason, this switch will help you ‘debug’ connection, see authentication process and closer inspect those pesky configuration problems.

cyberpunk@n0where.net ~/Documents $ scp -v somefile.txt root@192.x.x.x:.
SAMPLE OUTPUT
Executing: program /usr/bin/ssh host 192.x.x.x, user cyberpunk, command scp -v -t .
OpenSSH_x.x Debian, OpenSSL x.x.x 10 Sep 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.x.x.x [192.x.x.x] port 22.
debug1: Connection established.
debug1: Host '192.x.x.x' is known and matches the RSA host key.
debug1: Found key in /home/cyberpunk/.ssh/known_hosts:1
debug1: ssh_rsa_verify: signature correct
debug1: Next authentication method: password
root@192.x.x.x's password:
debug1: Authentication succeeded (password).
Authenticated to 192.x.x.x ([192.x.x.x]:22).
Sending file modes: C0770 3760348 somefile.txt
Sink: C0770 3760348 somefile.txt
somefile.txt 100% 3672KB 136.0KB/s 00:27
Transferred: sent 3766304, received 3000 bytes, in 65.2 seconds
Bytes per second: sent 57766.4, received 46.0
debug1: Exit status 0

SCP … Estimated Time ?

With “-p” switch you can get estimated time of execution and the connection speed

cyberpunk@n0where.net ~/Documents $ scp -p somefile.txt root@192.x.x.x:.
SAMPLE OUTPUT
root@192.x.x.x's password:
somefile.txt 100% 3672KB 126.6KB/s 00:29

Compress during transfer

There’s a chance ( depending on a file type) that you can speed up your transfer by compressing the content during network transmission. You can try this with “-C” switch but the result will vary from file to file.

Test case: 93 Mb file:

cyberpunk@n0where.net ~/Documents $ scp -pv big.log root@192.x.x.x:.
SAMPLE OUTPUT
Executing: program /usr/bin/ssh host 192.x.x.x, user cyberpunk, command scp -v -p -t .
OpenSSH_x.x Debian, OpenSSL x.x 10 Sep 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.x.x.x [192.x.x.x] port 22.
debug1: Connection established.
debug1: identity file /home/cyberpunk/.ssh/id_rsa type -1
debug1: Found key in /home/cyberpunk/.ssh/known_hosts:1
debug1: ssh_rsa_verify: signature correct
debug1: Trying private key: /home/cyberpunk/.ssh/id_rsa
debug1: Next authentication method: password
root@192.x.x.x's password:
debug1: Authentication succeeded (password).
Authenticated to 192.x.x.x ([192.x.x.x]:22).
debug1: Sending command: scp -v -p -t .
File mtime 1323853868 atime 1380425711
Sending file timestamps: T1323853868 0 1380425711 0
big.log 100% 93MB 58.6KB/s 27:05
Transferred: sent 97614832, received 25976 bytes, in 1661.3 seconds
Bytes per second: sent 58758.4, received 15.6
debug1: Exit status 0

SCP Encryption

By default SCP is using “AES-128” to encrypt files. If you want to that to another cipher use “-c” switch

cyberpunk@n0where.net ~/Documents $ scp -c 3des somefile.txt root@192.x.x.x:.

root@192.x.x.x's password:
somefile.txt 100% 3672KB 282.5KB/s 00:13

Limit bandwidth usage

The “-l” parameter will limit the bandwidth use:

cyberpunk@n0where.net ~/Documents $ scp -l 400 somefile.txt root@192.x.x.x:.

root@192.x.x.x's password:
somefile.txt 100% 3672KB 50.3KB/s 01:13

The 400 value behind “-l” parameter tells scp to limit bandwidth to 50 KB/sec.

Use specific port

Usually SCP is using port 22 as a default port. But for security reason, you may change the port to something else:

cyberpunk@n0where.net ~/Documents $ scp -P 2249 somefile.txt root@192.x.x.x:.

root@192.x.x.x's password:
somefile.txt 100% 3672KB 262.3KB/s 00:14

Copy files recursively

Sometimes we need to copy directory and all files recursively . SCP support recursive copy with the “-r” switch

cyberpunk@n0where.net ~/Documents $ scp -r documents root@192.x.x.x:.

root@192.x.x.x's password:
somefile.txt 100% 3672KB 282.5KB/s 00:13
scp.txt 100% 10KB 9.8KB/s 00:00

Disable diagnostic messages

If you choose not to see progress meter and warning / diagnostic messages from SCP, you may disable it using “-q” switch:

cyberpunk@n0where.net ~/Documents $ scp -q somefile.txt root@192.x.x.x:.

root@192.x.x.x's password:
cyberpunk@n0where.net ~/Documents $

Copy files via proxy

Proxy server is usually used in office environment. Natively, SCP is not proxy configured. When your environment has proxy server in-between you and your destination, you have to “tell” SCP to use that proxy.

The proxy address at 192.168.1.1 on port 8080. First, you need to create “~/.ssh/config” file with:

ProxyCommand /usr/bin/corkscrew 192.168.1.1 8080 %h %p ~/.ssh/proxyauth

after that, you need to create file “~/.ssh/proxyauth” with

myusername:mypassword

After that you can do SCP transparently as usual.

12 scp command examples to transfer files on Linux


Secure copy

Scp (Secure Copy) is a command line tool to copy or transfer files across hosts. It uses the same kind of security mechanism like the ssh program. Infact it uses an ssh connection in the background to perform the file transfer. scp refers both to the “protocol” that defines how secure copy should work and the “program” (command) which is installed as a part of OpenSSH suite of tools.

In this quick tutorial we shall look at a few examples the scp command and how it can be used to transfer files securely.

Installing scp

Scp is generally installed by default on most linux distros as a part of openssh packages. On ubuntu/debian for example, the openssh-client package provides the scp program.

$ dpkg -L openssh-client | grep scp
/usr/bin/scp
/usr/share/man/man1/scp.1.gz

Its the OpenSSH package that provides the ssh, scp, sftp programs along with many other tools. So we do not have to do anything extra in here, except to use and learn the program.

Using scp

The basic syntax of scp is very simple to memorize. It looks like this

$ scp source_file_path destination_file_path

Depending on the host, the file path should include the full host address, port number, username and password along with the directory path.

So if you are “sending” file from your local machine to a remote machine (uploading) the syntax would look like this

$ scp ~/my_local_file.txt user@remote_host.com:/some/remote/directory

When copying file from remote host to local host (downloading), its looks just the reverse

$ scp user@remote_host.com:/some/remote/directory ~/my_local_file.txt

# just download the file
$ scp user@192.168.1.3:/some/path/file.txt .

That is pretty much about using scp for regular tasks. Apart from it, there are a couple of extra options and functions that scp supports. Lets take a quick overview of those.

And yes, by default scp will always overwrite files on the destination. If you need to avoid that, use a more powerful tool called rsync.

1. Verbose output

With verbose output, the scp program would output lots of information about what it does in the background. This is often useful when the program fails or is unable to complete the request. The verbose output would then indicate the exact point where the program ran into issues.

$ scp -v ~/test.txt root@192.168.1.3:/root/help2356.txt
Executing: program /usr/bin/ssh host 192.168.1.3, user root, command scp -v -t /root/help2356.txt
OpenSSH_6.2p2 Ubuntu-6ubuntu0.1, OpenSSL 1.0.1e 11 Feb 2013
debug1: Reading configuration data /home/enlightened/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.168.1.3 [192.168.1.3] port 22.
debug1: Connection established.
..... OUTPUT TRUNCATED

The output would be big and contain detailed information about how the connection is made, what configuration and identity files are being used and so on.

2. Transfer multiple files

Multiple files can be specified separated by a space like this

$ scp foo.txt bar.txt username@remotehost:/path/directory/

To copy multiple files from remote host to current local directory

$ scp username@remotehost:/path/directory/\{foo.txt,bar.txt\} .

$ scp root@192.168.1.3:~/\{abc.log,cde.txt\} .

3. Copy entire directory (recursively)

To copy an entire directory from one host to another use the r switch and specify the directory

$ scp -v -r ~/Downloads root@192.168.1.3:/root/Downloads

4. Copy files across 2 remote hosts

Scp can copy files from 1 remote host to another remote host as well.

$ scp user1@remotehost1:/some/remote/dir/foobar.txt user2@remotehost2:/some/remote/dir/

5. Speed up the transfer with compression

A super cool option to speed up the transfer to save time and bandwidth. All you need to do is use the C option to enable compression. The files are compressed on the fly and decompressed on the destination.

$ scp -vrC ~/Downloads root@192.168.1.3:/root/Downloads

In the above example we moved the entire directory with compression enabled. The speed gain would depend on how much the files could be compressed.

6. Limit the bandwidth usage

If you do not want scp to take up the entire available bandwidth, then use the l option to limit the maximum speed in Kbit/s.

$ scp -vrC -l 400 ~/Downloads root@192.168.1.3:/root/Downloads

7. Connect to a different port number on remote host

If the remote server has ssh daemon running on a different port (default is 22), then you need to tell scp to use that particular port number using the ‘-P’ option.

$ scp -vC -P 2200 ~/test.txt root@192.168.1.3:/some/path/test.txt

8. Preserve file attributes

The ‘-p’ option (smallcase), would preserve modification times, access times, and modes from the original file.

$ scp -C -p ~/test.txt root@192.168.1.3:/some/path/test.txt

9. Quiet mode

In quiet mode ( ‘-q’ option ), the scp output would get suppressed, and would disable the progress meter as well as warning and diagnostic messages.

$ scp -vCq ~/test.txt root@192.168.1.3:/some/path/test.txt

10. Specify identity file

When using key based (passwordless) authentication, you would need to specify the identity file which contains the private key. This option is directly passed to the ssh command and works the same way.

$ scp -vCq -i private_key.pem ~/test.txt root@192.168.1.3:/some/path/test.txt

11. Use a different ssh_config file

Use the ‘-F’ option to specify a different ssh_config file.

$ scp -vC -F /home/user/my_ssh_config ~/test.txt root@192.168.1.3:/some/path/test.txt

12. Use different cipher

Scp by default uses the AES cipher/encryption. Sometimes you might want to use a different cipher. Using a different cipher can speed up the transfer process. For example blowfish and arcfour are known to be faster than AES (but less secure).

$ scp -c blowfish -C ~/local_file.txt username@remotehost:/remote/path/file.txt

In the above example we use the blowfish cipher along with compression. This can give significant speed boost depending on available bandwidth.

Summary

Although scp is very efficient at transferring file securely, it lacks necessary features of a file synchronisation tool. All it can do is copy paste all the mentioned files from one location to another.

A more powerful tool is Rsync which not only has all functions of scp but adds more features to intelligently synchronise files across 2 hosts. For example, it can check and upload only the modified files, ignore existing files and so on.