ssh

Working with SSH in Bash: Remote Command Execution

SSH (Secure Shell) is a powerful tool that allows secure communication between a local machine and a remote machine over a network. It’s widely used for remote login, file transfers, and executing commands on remote servers. When combined with Bash scripting, SSH can help automate remote system management, configuration tasks, and even run commands remotely without manually logging into the server.

This guide will explore how to work with SSH in Bash for remote command execution.

---

1. What is SSH?

SSH provides a secure way to connect to remote systems and execute commands as if you were physically logged in to the server. It uses encryption to protect data, ensuring that communications between systems are secure.

The basic SSH command syntax is:

ssh user@remote_host 'command'

• user: The username on the remote machine.
• remote_host: The IP address or domain name of the remote machine.
• command: The command to execute on the remote machine.

---

2. Setting Up SSH Key Authentication

While you can authenticate with SSH using a password, it's more secure and efficient to use SSH key-based authentication. This method involves generating an SSH key pair (a public key and a private key), and storing the public key on the remote server.

Steps to set up SSH key authentication:

1. Generate an SSH key pair on the local machine:

   ssh-keygen -t rsa -b 2048
   

   This generates two files:

   • ~/.ssh/id_rsa (private key)
   • ~/.ssh/id_rsa.pub (public key)

2. Copy the public key to the remote server:

   ssh-copy-id user@remote_host
   

   This will add the public key to the remote server's ~/.ssh/authorized_keys file.

3. Test the connection: Now, you can SSH into the remote server without needing to enter a password:

   ssh user@remote_host
   

---

3. Executing Commands Remotely with SSH

Once SSH is set up, you can use it in Bash scripts to remotely execute commands. The syntax for running a command on a remote server is:

ssh user@remote_host 'command_to_execute'

• Example: Check disk usage on a remote server: bash ssh user@remote_host 'df -h'

This will run the df -h command on the remote server, showing disk usage in human-readable format.

---

4. Running Multiple Commands Remotely

You can run multiple commands on the remote server by separating them with semicolons (;), or use && to run commands conditionally.

• Example: Run multiple commands on a remote server: bash ssh user@remote_host 'cd /var/www && ls -l && df -h'

This command changes the directory to /var/www, lists the contents of the directory, and then shows the disk usage.

---

5. Running Commands in the Background

If you want to run a command on a remote server without keeping the SSH session open, you can use nohup to run the command in the background.

• Example: Run a script on a remote server in the background: bash ssh user@remote_host 'nohup /path/to/long_running_script.sh &'

This will start the script in the background, and the output will be written to nohup.out on the remote server.

---

6. Passing Arguments to Remote Commands

You can pass arguments to the remote command just like you would on the local machine. If you need to pass dynamic values (like variables from a script), you can use quotes and variable substitution.

• Example: Passing arguments to a remote command: bash my_file="example.txt" ssh user@remote_host "cat $my_file"

In this case, the $my_file variable will be replaced with example.txt when the command is executed on the remote server.

---

7. Using SSH in Bash Scripts

SSH can be integrated into Bash scripts to automate tasks on remote servers. Below is an example of a Bash script that uses SSH to check disk space and memory usage on multiple remote servers.

#!/bin/bash

# List of remote hosts
hosts=("server1" "server2" "server3")

# Loop through each host and execute commands
for host in "${hosts[@]}"; do
    echo "Checking disk usage on $host..."
    ssh user@$host 'df -h'

    echo "Checking memory usage on $host..."
    ssh user@$host 'free -m'

    echo "------------------------------------"
done

This script loops through each server, checks the disk and memory usage remotely, and displays the output.

---

8. Copying Files Using SSH

In addition to executing commands, SSH allows you to securely copy files between the local and remote systems using scp (secure copy) or rsync.

• Example: Copy a file from local to remote server:

  scp local_file.txt user@remote_host:/path/to/destination/
  

• Example: Copy a directory from remote to local:

  scp -r user@remote_host:/path/to/remote_dir /local/destination/
  

• Example: Using rsync for efficient file transfer:

  rsync -avz local_file.txt user@remote_host:/path/to/destination/
  

rsync is useful for copying files while minimizing data transfer by only copying changed parts of files.

---

9. Managing Remote SSH Sessions

To manage long-running SSH sessions or prevent them from timing out, you can adjust the SSH configuration on the server or use the screen or tmux utilities to keep sessions persistent.

• Example: Start a new session with screen: bash ssh user@remote_host screen -S my_session

This opens a new terminal session that will stay active even if the SSH connection is lost.

---

10. Automating SSH Connections with SSH Config File

If you frequently connect to the same remote servers, you can simplify your SSH commands by creating an SSH config file (~/.ssh/config).

• Example of an SSH config entry: Host myserver HostName remote_host User user IdentityFile ~/.ssh/id_rsa

After configuring this file, you can connect to the server with:

ssh myserver

---

Conclusion

Using SSH in combination with Bash scripting enables automation of remote tasks, making it easier to manage multiple servers or perform system administration tasks without manually logging into each machine. By mastering SSH command execution, file transfer, and automating processes via scripts, you can significantly improve productivity and streamline server management. Whether you're running one-off commands or setting up complex automation workflows, SSH is an essential tool for efficient remote administration.

Introduction

After reading this document you should be able to identify why Linux defines its filesystem hierarchy in one big tree and explain the role of the filesystem hierarchy standard, explain what is available at boot in the root directory (/), explain each subdirectory purpose and typical contents. The aim here is to be able to create a working bash script which knows where to put its different data stores including lockfiles, database(s) or temporary files; including the script itself.

One Big Filesystem

As with all Linux installations there is a set protocol to follow which could be looked at as one big tree starting from its root, /. This often contains different access points not just typical file or folder components, but in fact mounted drives, USB or CD/DVD media volumes and so on. Even more adapt, these can span many partitions; as one filesystem. Regardless, the end result is one big filesystem, meaning applications generally do not care what volume or partition the data resides upon. The only drawback you may encounter is different naming conventions however there are now standards to follow in the Linux-ecosystem for cross-platform conformity.

Defining The Data

There has to be one method for defining all data in order to satisfy clear distinctions. Firstly, you may examine data and identify whether it is shareable or not. For instance, /home data may be shared across many hosts however .pid lock files will not. Another angle you may look at is are the files variable or static, meaning if no administrator input is given they will remain the same, ie. static - or else the data changes when the filesystem is in operation without human interaction hence it is called variable. With this in mind, you must identify which trees and sub-trees are accessible by your application or command prompt to identify if they can be manipulated at runtime and where they should reside if you are creating these filetypes.

To summarise: - Shared Data is Common To All Systems - Non-Shareable Data is Local To One System - Static Data Never Changes When Left Alone - Variable Data Will Change During Application Processing

The Filesystem Hierarchy Standard aims to help achieve unity across all platforms however different distributions can often invent new methodologies that will generally become standard over time. While the FHS (Filesystem Hierarchy Standard) publishes its standard new conventions are currently in play despite this document, see here: linuxfoundation.org...

DIRECTORY	FHS Approved	PURPOSE
/	Yes	Primary directory of the entire filesystem hierarchy.
/bin	Yes	Essential executable programs that must be available in single user mode.
/boot	Yes	Files needed to boot the system, such as the kernel, initrd or initramfs |images, and boot configuration files and bootloader programs.
/dev	Yes	Device Nodes, used to interact with hardware and software devices.
/etc	Yes	System-wide configuration files.
/home	Yes	User home directories, including personal settings, files, etc.
/lib	Yes	Libraries required by executable binaries in /bin and /sbin.
/lib64	No	64-bit libraries required by executable binaries in /bin and /sbin, for systems which can run both 32-bit and 64-bit programs.
/media	Yes	Mount points for removable media such as CDs, DVDs, USB sticks, etc.
/mnt	Yes	Temporarily mounted filesystems.
/opt	Yes	Optional application software packages.
/proc	Yes	Virtual pseudo-filesystem giving information about the system and processes running on it. Can be used to alter system parameters.
/sys	No	Virtual pseudo-filesystem giving information about the system and processes running on it. Can be used to alter system parameters. Similar to a device tree and is part of the Unified Device Model.
/root	Yes	Home directory for the root user.
/sbin	Yes	Essential system binaries.
/srv	Yes	Site-specific data served up by the system. Seldom used.
/tmp	Yes	Temporary files; on many distributions lost across a reboot and may be a ramdisk in memory.
/usr	Yes	Multi-user applications, utilities and data; theoretically read-only.
/var	Yes	Variable data that changes during system operation.

Run du --max-depth=1 -hx / to see the output of your root filesystem hierarchy.

The Root Directory (/)

Starting with our first directory, the root directory (/) this is often the access point mounted across multiple (or single) partitions with other locations such as /home, /var and /opt mounted later. This root partition must contain all root directories and files at boot in order to serve the system. Therefore it needs boot loader information and configuration files plus other essential startup data, which must be adequate to perform the following operations: - Boot the system - Restore the system on external devices such as USB, CD/DVD or NAS - Recover and/or repair the system (ie. in rescue mode)

The root directory / should never have folders created directly within it; period.

Binary Files (/bin)

• The /bin directory must be present for a system to function containing scripts which are used indirectly by other scripts. It's important because non-privileged users and system administrators all have access to this directory plus it contains scripts needed to be served before the filesystem is even mounted. It is common place to store non-essential scripts which do not merit going in /bin to be served from /usr/bin, however /bin is becoming more acceptable to be used in common-place operation, in fact in RHEL they are the same directory. Often symbolic links are used from/bin to other folder locations in order to preserve two-way folder listings.

They are as follows: cat, chgrp, chmod, chown, cp, date, dd, df, dmesg, echo, false, hostname, kill, ln, login, ls, mkdir, mknod, more, mount, mv, ps, pwd, rm, rmdir, sed, sh, stty, su, sync, true, umount and uname

Other binaries that may be present during boot up and in normal operation are: test, csh, ed, tar, cpio, gunzip, zcat, netstat, ping

The Boot Directory (/boot)

This folder contains the vmlinuz and intramfs (also known as initrd) files which are put there in order to serve the boot operation, the first is a compressed kernel and the second is the initial RAM filesystem. Other files include config and system.map.

Device Files (/dev)

Device files are often used to store device nodes; commonly serving various hardware references including nodes - network cards however are more likely to be named eth0 or wlan0 meaning they are referenced by name. The directory /dev/ will automatically create nodes using udev when system hardware is found. Quite aptly, ls /dev | grep std will show you some useful output references which can be used to process data either to the terminal or into the obis.

Configuration Files (/etc)

Used to contain config directives (or contained folders with config directives) for system-wide programs and more importantly system services.

Some examples are: csh.login, exports, fstab, ftpusers, gateways, gettydefs, group, host.conf, hosts.allow, hosts.deny, hosts,equiv, hosts.lpd, inetd.conf, inittab, issue, ld.so.conf, motd, mtab, mtools.conf, networks, passwd, printcap, profile, protocols, resolv.conf, rpc, securetty, services, shells, syslog.conf

More crucially, the following helps keep the system correctly configured: - /etc/skel This folder contains the skeleton of any new users /home directory - /etc/systemd - Points to or contains configuration scripts for system services, called by service - /etc/init.d - Contains startup and shutdown scripts used by System V initialisation

System Users (/home)

On Linux, users working directories are given in the /home/{username} format and are typically named in a naming convention such as /home/admin /home/projects/home/stagingor/home/production. Typically, this could be their name or nickname or purpose, eg./home/steve,/home/steve-work` and so on.

With Linux, this folder can be accessed via the ~ symbol which is given to system users in order to direct the user to the currently logged in users home directory, eg. ls ~/new-folder etc; which is also accessible by using $home. The only caveat to this is that the root user is placed in /root - all other users reside in /home, typically mirroring /etc/skel as previously outlined. (see “Configuration Files (/etc)”)

System Libraries (/lib and /lib64)

These folders are for libraries serving binary files found in /bin or other locations where scripts are found. These libraries are important because they maintain the upkeep of essential system programs (binaries) which help boot the system and then are used by the system once booted, fundamentally. Kernel modules (device or system drivers) are stored in /lib/modules and PAM (Pluggable Authentication Modules) are stored in /lib/security.

For systems running 32bit and 64bit the /lib64 is usually present. More common place is to use the one folder with symbolic links to the actual destination of the library, similar to how /bin has reformed back into one folder, providing the same structure with separation of differing program importance (using symbolic links) while maintaining the single source for all scripts of that type.

External Devices (/media)

The /media folder is often found to be a single source for all removable media. USB, CD, DVD even the ancient Floppy Disk Drive. Linux mounts these automatically using “udev” which in turn creates references inside /media making for simple access to external devices, autonomously. As the device is removed, the name of the file in this directory is removed also.

Temporary Mounts (/mnt)

This folder is used for mount points, usually temporary ones. During the development of the FHS this would typically contain removable devices however /media is favoured on modern systems.

Typical use scenarios are: - NFS - Samba - CIFS - AFS

Generically, this should not be used by applications, instead mounted disks should be located elsewhere on the system.

Software Packages (/opt)

This location is where you would put system-wide software packages with everything included in one place. This is for services that want to provide everything in one place, so you would have /opt/project/bin etc. all in this folder. The directories /opt/bin, /opt/doc, /opt/include, /opt/info, /opt/lib, and /opt/man are saved for administrator usage.

System Processes (/proc)

These are special files which are mounted like with /dev and are constantly changing. They only contain data at the point you make the request, so typically a file may be 0kb but if you look, may contain many lines of data; this is accessed only when you run the cat or vi operation, it does indeed remain empty. Important pseudo files are /proc/interrupts, /proc/meminfo, /proc/mounts, and /proc/partitions.

System Filesystems (/sys)

This directory is the mount point for the sysfs pseudo-filesystem where all information resides only in memory, not on disk. This is again very much like /dev/ and /proc in that it contains mounted volumes which are created on system boot. Containing information about devices and drivers, kernel modules and so on.

Root (/root)

This is generally called “slash-route” pronounced phonetically and is simply the primary system administrators home folder. Other user accounts are encouraged with specific access details for better security.

System Binaries (/sbin)

This is very similar to /bin and as mentioned may very well have symbolic link references inside /bin with the actual program residing here. This allows for a one-solution-fits-all because /bin will display all programs whereas /sbin would be designated to the programs listed there. Some of these programs include: fdisk, fsck, getty, halt, ifconfig, init, mkfs, mkswap, reboot, route, swapon, swapoff and update.

System Services (/srv)

Popular for some administrators this is designed to provide system service functionality. You can be fairly lax with naming conventions here you may want to group applications into folders such as ftp, rsync, www, and cvs etc. Popular by those that use it and may be overlooked by those what don’t.

Temporary Files (/tmp)

Used by programs that do not want to keep the data between system boots and it may be periodically refreshed by the system. Use at your own discretion. Be aware as this is truly temporary data any large files may cause issues as often the information is stored in memory.

System User (/usr)

This should be thought of as a second system hierarchy. Containing non-local data it is best practice to serve administrator applications here and is often used for files and packages or software that is not needed for booting.

DIRECTORY	PURPOSE
/usr/bin	Non-essential command binaries
/usr/etc	Non-essential configuration files (usually empty)
/usr/games	Game data
/usr/include	Header files used to compile applications
/usr/lib	Library files
/usr/lib64	Library files for 64-bit
/usr/local	Third-level hierarchy (for machine local files)
/usr/sbin	Non-essential system binaries
/usr/share	Read-only architecture-independent files
/usr/src	Source code and headers for the Linux kernel
/usr/tmp	Secondary temporary directory

Other common destinations are /usr/share/man and /usr/local, the former is for manual pages and the latter is for predominantly read-only binaries.

Variable Data (/var)

This directory is intended for variable (volatile) data and as such is often updated quite frequently. Contains log files, spool directories and files administrator files and transient files such as cache data.

SUBDIRECTORY	PURPOSE
/var/ftp	Used for ftp server base
/var/lib	Persistent data modified by programs as they run
/var/lock	Lock files used to control simultaneous access to resources
/var/log	Log files
/var/mail	User mailboxes
/var/run	Information about the running system since the last boot
/var/spool	Tasks spooled or waiting to be processed, such as print queues
/var/tmp	Temporary files to be preserved across system reboot. Sometimes linked to /tmp
/var/www	Root for website hierarchies

Transient Files (/run)

These are meant to be files that are updated quite regularly and are not often maintained during reboots, useful for containing temporary files and runtime information. The use of run is quite new and you may find /var/run and /var/lock symbolic link references. The use of this is more common place in modern systems.

Safe and Secure SSH Connections

In a modern world where cyber-warfare is common place and every-day users are targets from organised crime, it goes without saying that you are likely to run into problems rather quickly if you don't use every available means of security.

The scope of this article is to connect via SSH Keys however you should also be doing some other more mundane tasks like encrypting the connection (preferably with a VPN on your router) and using altered ports, plus limiting access to SSH users, if you have them.

So what is the safest way to connect to your remote Linux OS distribution, by command line? Well quite simply, it is done with SSH Keys which you generate so that the connection can be established. These keys are then used as a form of password and where the remote user has these pre-generated keys on their system, SSH shares them and if allowed, serves the connection.

Generating Your Keys

From command line on the machine you are connecting from, do the following:

ssh-keygen - Leave as default values

This creates files inside your home directories .ssh folder. This is a hidden folder that you usually don't need access to. To see what's inside, do ls .ssh from your home path.

Now, do the following, from your home path:

cat .ssh/id_rsa.pub

This is your public password. Share this with unlimited amounts of remote servers and while you are using this account, you will have access.

Sharing Your Keys

On a mundane level, you can provide the key you generated via any method you like, only your machine and account will be able to use it.

Now, take the output of cat .ssh/id_rsa.pub, and do echo "key-here" >> .ssh/authorized_keys and voila, the magic is done. You can now do ssh user@example.com, password-free.

So that's one way of achieving passwordless login via SSH, although there is an easier way. Do:

ssh-copy-id user@example.com

This will auto-install the keys for you, assuming you can connect to the server via SSH using other authentication methods - such as password.

Removing Keys

To remove access to a users account, do vi .ssh/authorized_keys and delete the line corresponding to the users account.

It really is that simple!

Voila

Congratulations, you're all set up! Don't forget, while it is perfectly safe to share your id_rsa.pub key, do so with caution. Using it on your website homepage may attract unwanted attention!

Peace.