command line

All posts tagged command line by Linux Bash
  • Posted on

    Introduction to Bash Scripting: A Beginner's Guide

    Bash scripting is a way to automate tasks in Linux using Bash (Bourne Again Shell), a widely used shell on Unix-like systems. This guide introduces the basics of Bash scripting, enabling you to create and execute your first scripts.


    1. What is a Bash Script?

    A Bash script is a plain text file containing a series of commands executed sequentially by the Bash shell. It’s useful for automating repetitive tasks, system administration, and process management.


    2. Basics of a Bash Script

    File Format:

    1. A Bash script is a text file.
    2. It typically has a .sh extension (e.g., myscript.sh), though this isn’t mandatory.

    The Shebang Line:

    The first line of a Bash script starts with a shebang (#!), which tells the system which interpreter to use.

    Example:

    #!/bin/bash
    

    3. Creating Your First Bash Script

    Step-by-Step Process:

    1. Create a new file: bash nano myscript.sh
    2. Write your script: bash #!/bin/bash echo "Hello, World!"
    3. Save and exit:
      In nano, press CTRL+O to save, then CTRL+X to exit.

    4. Make the script executable:

      chmod +x myscript.sh
      
    5. Run the script:

      ./myscript.sh
      

      Output:

      Hello, World!
      

    4. Key Concepts in Bash Scripting

    Variables:

    Store and manipulate data.

    #!/bin/bash
    name="Alice"
    echo "Hello, $name!"
    

    User Input:

    Prompt users for input.

    #!/bin/bash
    echo "Enter your name:"
    read name
    echo "Hello, $name!"
    

    Control Structures:

    Add logic to your scripts.

    • Conditionals:

      #!/bin/bash
      if [ $1 -gt 10 ]; then
        echo "Number is greater than 10."
      else
        echo "Number is 10 or less."
      fi
      
    • Loops:

      #!/bin/bash
      for i in {1..5}; do
        echo "Iteration $i"
      done
      

    Functions:

    Reusable blocks of code.

    #!/bin/bash
    greet() {
        echo "Hello, $1!"
    }
    greet "Alice"
    

    5. Common Bash Commands Used in Scripts

    • echo: Print messages.
    • read: Read user input.
    • ls, pwd, cd: File and directory management.
    • cat, grep, awk, sed: Text processing.
    • date, whoami, df: System information.

    6. Debugging Bash Scripts

    • Use set for debugging options:
      • set -x: Prints commands as they are executed.
      • set -e: Stops execution on errors.

    Example:

    #!/bin/bash
    set -x
    echo "Debugging mode enabled"
    
    • Debug manually by printing variables: bash echo "Variable value is: $var"

    7. Best Practices for Writing Bash Scripts

    1. Use Comments: Explain your code with #. bash # This script prints a greeting echo "Hello, World!"
    2. Check User Input: Validate input to prevent errors. bash if [ -z "$1" ]; then echo "Please provide an argument." exit 1 fi
    3. Use Meaningful Variable Names:
      Instead of x=5, use counter=5.

    4. Follow File Permissions: Make scripts executable (chmod +x).

    5. Test Thoroughly: Test scripts in a safe environment before using them on critical systems.


    8. Example: A Simple Backup Script

    This script creates a compressed backup of a specified directory.

    #!/bin/bash
    
    # Prompt for the directory to back up
    echo "Enter the directory to back up:"
    read dir
    
    # Set backup file name
    backup_file="backup_$(date +%Y%m%d_%H%M%S).tar.gz"
    
    # Create the backup
    tar -czvf $backup_file $dir
    
    echo "Backup created: $backup_file"
    

    Conclusion

    Bash scripting is a powerful tool for automating tasks and enhancing productivity. By mastering the basics, you can create scripts to handle repetitive operations, simplify system management, and execute complex workflows with ease. With practice, you’ll soon be able to write advanced scripts tailored to your specific needs.

  • Posted on

    How to Customise Your Bash Prompt for Better Productivity

    The Bash prompt is the text that appears in your terminal before you type a command. By default, it displays minimal information, such as your username and current directory. Customizing your Bash prompt can enhance productivity by providing quick access to important information and making your terminal visually appealing.


    What is the Bash Prompt?

    The Bash prompt is controlled by the PS1 variable, which defines its appearance. For example:

    PS1="\u@\h:\w\$ "
    
    • \u: Username.
    • \h: Hostname.
    • \w: Current working directory.
    • \$: Displays $ for normal users and # for the root user.

    Why Customise Your Bash Prompt?

    1. Enhanced Information: Display details like the current Git branch, exit status of the last command, or time.
    2. Improved Visuals: Use colors and formatting to make the prompt easier to read.
    3. Increased Productivity: Quickly identify useful information without typing additional commands.

    Steps to Customise Your Bash Prompt

    1. Temporary Customization

    You can modify your prompt temporarily by setting the PS1 variable:

    PS1="\[\e[32m\]\u@\h:\w\$\[\e[0m\] "
    

    This changes the prompt to green text with the username, hostname, and working directory.

    2. Permanent Customisation

    To make changes permanent:

    1. Edit the .bashrc file in your home directory: bash vi ~/.bashrc
    2. Add your custom PS1 line at the end of the file.
    3. Save and reload the file: bash source ~/.bashrc

    Common Customizations

    Add Colors

    Use ANSI escape codes for colors:

    • \[\e[31m\]: Red

    • \[\e[32m\]: Green

    • \[\e[34m\]: Blue

    • \[\e[0m\]: Reset to default color.

    Example:

    PS1="\[\e[34m\]\u@\h:\[\e[32m\]\w\$\[\e[0m\] "
    

    This makes the username and hostname blue and the working directory green.

    Include Time

    Display the current time:

    PS1="\[\e[33m\]\t \[\e[34m\]\u@\h:\w\$ \[\e[0m\]"
    
    • \t: Time in HH:MM:SS format.

    • \@: Time in 12-hour AM/PM format.

    Show Git Branch

    Display the current Git branch when inside a repository:

    PS1="\[\e[32m\]\u@\h:\[\e[34m\]\w\[\e[31m\]\$(__git_ps1)\[\e[0m\] \$ "
    
    • Ensure you have Git installed and the git-prompt.sh script sourced in your .bashrc file.

    Add Command Exit Status

    Show the exit status of the last command:

    PS1="\[\e[31m\]\$? \[\e[34m\]\u@\h:\w\$ \[\e[0m\]"
    
    • \$?: Exit status of the last command.

    Advanced Customisations with Tools

    Starship

    Starship is a modern, highly customizable prompt written in Rust. Install it and add this line to your .bashrc:

    eval "$(starship init bash)"
    

    Best Practices for Customizing Your Prompt

    1. Keep it Simple: Avoid cluttering the prompt with too much information.
    2. Use Colors Sparingly: Highlight only the most critical details.
    3. Test Changes: Test new prompts before making them permanent.
    4. Backup Your .bashrc: Keep a backup before making extensive changes.

    Example Custom Prompt

    Here’s a full-featured example:

    PS1="\[\e[32m\]\u@\h \[\e[36m\]\w \[\e[33m\]\t \[\e[31m\]$(git_branch)\[\e[0m\]\$ "
    
    • Green username and hostname.
    • Cyan working directory.
    • Yellow current time.
    • Red Git branch (requires Git integration).

    Customizing your Bash prompt is a great way to make your terminal more functional and visually appealing. Experiment with different configurations and find the one that works best for you!

  • Posted on

    Introduction

    A computer doing more than one thing at a time is using processes, these require resources, CPU time, memory and access to other devices like CD/DVD/USB drives, etc. Each process is allocated an amount of system resources to perform its function which is controlled by the operating system whose job it is to facilitate these processes. Signals have an important part to play on the interaction of the processes, usually these send exit signals and other information to each other, or to itself.

    Programs, Processes, and Threads

    A program is a set of instructions to be carried out which may local data such as information for output to the terminal via read or external data which may come from a database. Common programs include ls, cat and rm which would reside outside of the user operating system thus they have their own operating program on disk.

    A process is a program which is executing which has associated resources, furthermore. These might include environments, open files or signal handlers etc. At the same time, two or more tasks or processes may share system resources meaning they are multi-threaded processes.

    In other operating systems there may be a distinction between heavy-weight and light-weight processes; furthermore, heavy-processes may contain a number of lightweight processes. However, in Linux, each process is considered heavy or light by the amount of shared resources they consume, for faster context switching. Again, unlike other operating systems, Linux is much faster at switching between processes, creating and destroying them too. This means the model for multi-threaded processes is similar to that of simultaneous processes as they are treated as one. All the same, Linux respects POSIX and other standards for multi-threaded processes, where each thread returns the same process ID plus returning a thread ID.

    Processes

    Processes are running programs in execution, either running or sleeping. Every process has a process ID (pid), parent process ID (ppid) and a parent group ID (ppgid). In addition every process has program code, data, variables, file descriptions and an environment.

    In Linux, init is the first process that is run, thus becoming the ancestor to all other programs executed on the system; unless they are started directly from the Linux Kernel, for which they are grouped with brackets ([]) in the ps listing. Other commonalities are the processes pid numbering, if a parent process ends before the child, the process will become adopted by init, setting its process ID (pid) to 1. If the system is more recent, this will result in the pid being set to 2 inline with systemd’s kernel thread kthreadd. In the circumstance where a child dies before its parent it enters a zombie state, using no resources and retaining only the exit code. It is the job of init to allow these processes to die gracefully and is often referred to as being the zombie killer or child reaper. Finally, the processes ID can not go above 16bit definition hence 32768 is the largest pid you will find on a Linux system; to alter this value see /proc/sys/kernel/pid_max; once this limit is reached they are restarted at 300.

    Process Attributes

    • All processes have certain attributes, as shown below:
      • Program
      • Context
      • Permissions
      • Resources

    The program is the process itself, maybe it is a set of commands in a script or a loop which never ends checking against external data and performing an action when necessary.

    The context is the state which the program is in at any point in time, thus a snapshot of CPU registers, what is in RAM and other information. Furthermore, processes can be scheduled in and out when sharing CPU time or put to sleep when waiting for user input, etc. Being able to swap out and put back the process context is known as context switching.

    Permissions are inherited from the user who executed the program however programs themselves can have an s property assigned to them In order to define their effective user ID, rather than their real ID and are referred to as setuid programs. These setuid programs run with the permissions of the creator of the program, not the user who ran it. A commonly found setuid program is passwd.

    Process Resource Isolation

    This is the practice of isolating the process from other processes upon execution, promoting security and stability. Furthermore, processes do not have direct access to hardware devices, etc. Hardware is managed by the kernel meaning system calls must be made in order to interact with said devices.

    Controlling Processes (ulimit)

    This program, umlimit, reports and resets a number of resource limits associated with processes running under its shell. See below for a typical output of ulimit -a.

    ulimit -a

    Here you see a screenshot of ulimit -a which contains values important for a system administrator to be aware of because it identifies the allocation of resources. You may want to restrict or expand resource allocation depending on the requirements of the processes and / or file access limits.

    Hard and soft resource allocation terms come into play here, with hard limits imposed by the administrator and soft limited by those restrictions but allowing for user-level limits to be imposed. Run ulimit -H -n to see hard limits and ulimit -S -n for soft limits. File descriptors are probably the most common thing that may need changing, to allow more resources, typically this is set to 1024 which may make execution of an application virtually impossible so you can change this with ulimit -n 1600 which would change it to 1600. In order to make more permanent changes you would need to edit /etc/security/limits.conf then perform a reboot.

    Process States

    Processes can take on many different states, which is managed by the scheduler. See below for the main process states: - Running - Sleeping (waiting) - Stopped - Zombie

    A running process is either using the CPU or will be waiting for its next CPU time execution slice, usually cited in the run queue - resuming when the scheduler deems it satisfactory to re-enter CPU execution.

    A sleeping process is one that is waiting on its request that can not be processed until completed. Upon completion, the kernel will wake the process and reposition it into the run queue.

    A stopped process is one that has been suspended and is used normally by developers to take a look at resource usage at any given point. This can be activated with CTRL-Z or by using a debugger.

    A zombie process is one that has entered terminated state and no other process has inquired about it, namely “reaped it” - this can often be called a “defunct process”. The process releases all of its resources except its exit state.

  • Posted on

    So yeah, getting used to Bash is about finding the right way to do things. However, learning one-liners and picking up information here and there is all very useful, finding something you don't need to Google in order to recall is extremely important.

    Take the case of recursive find and replace. We've all been there, it needs to be done frequently but you're either a) scared or b) forgetful. Then you use the same snippet from a web resource again and again and eventually make a serious error and boom, simplicity is now lost on you!

    So here it is, something you can remember so that you don't use different methods depending on what Google throws up today.

    grep -Rl newertext . | xargs sed -i 's/newertext/newesttext/g'

    Notice, we use grep to search -R recursively with results fed to xargs which does a simple sed replace with g global/infinite properties.

    Say you want to keep it simple though and find and review the files before doing the "simple" replace. Well, do this.

    grep -R -l "foo" ./*

    The uppercase -R denotes to follow symlinks and the ./* indicates the current directory. The -l requests a list of filenames matched. Neat.