Bash Commands Basics

| pipe

| pipe takes the standard output of one command and passes it as the input to another. Pipes allow you to use the output of one command as the input of another

• There is no limit to the number of times you can chain pipes in a row

[command 1] | [command 2] | [command 3] ... | [command n]

• -o tells grep to only return the matching portion

• -E tells grep to be able to use extended regex symbols such as ?

• \"price\" matches the string "price"

• \s* matches any number (including 0) of whitespace (\s) characters

• : matches :

• [0-9]* matches any number of digits (from 0 to 9)

• ?\. optionally matches a .

# Chain filter commands together this will list and sort in reverse order (Desc)
ls | sort -r

#Pipe the output of manual page for ls to head to display the first 20 lines
man ls | head -20

# Extract column names from a csv and drop duplicate names
cut -d "," -f1 names.csv | sort | uniq

tr

Used to translate one set of chars into another. It is simple and quick but awk is more powerful

Only accepts standard input (text entered from keyboard, but not strings or filenames0

• -d : Delete the characters in the first set
• -c : Complement the first set of characters i.e., operate only on the characters NOT in the first set
• -s : Remove multiple adjacent occurrence of the characters in the first set
• -t : truncate the first set

tr [OPTIONS] [target characters] [replacement characters]

Replace Vowels with _

$ echo "Linux and shell scripting are awesome\!" | tr "aeiou" "_"
L_n_x _nd sh_ll scr_pt_ng _r_ _w_s_m_!

Replace Consonants Not a vowel to _

$ echo "Linux and shell scripting are awesome\!" | tr -c "aeiou" "_"
_i_u__a_____e______i__i___a_e_a_e_o_e_

Cat & tr in Files

With files, you can use cat in combination with tr to change all of the text in a file to uppercase as follows:

$ cat pets.txt | tr "[a-z]" "[A-Z]"
GOLDFISH
DOG
CAT
PARROT
DOG
GOLDFISH
GOLDFISH

Extract info from JSON

• Here is the information you extracted in JSON format
• Save it in blah.txt

{
  "coin": {
    "id": "bitcoin",
    "icon": "https://static.coinstats.app/coins/Bitcoin6l39t.png",
    "name": "Bitcoin",
    "symbol": "BTC",
    "rank": 1,
    "price": 57907.78008618953,
    "priceBtc": 1,
    "volume": 48430621052.9856,
    "marketCap": 1093175428640.1146,
    "availableSupply": 18877868,
    "totalSupply": 21000000,
    "priceChange1h": -0.19,
    "priceChange1d": -0.4,
    "priceChange1w": -9.36,
    "websiteUrl": "http://www.bitcoin.org",
    "twitterUrl": "https://twitter.com/bitcoin",
    "exp": [
      "https://blockchair.com/bitcoin/",
      "https://btc.com/",
      "https://btc.tokenview.com/"
    ]
  }
}

• The JSON field you want to grab here is "price": [numbers].[numbers]".
• To get this, you can use the following grep command to extract it from the JSON text
• Here is the breakdown of the code below
• -o tells grep to only return the matching portion
• -E tells grep to be able to use extended regex symbols such as ?
• \"price\" matches the string "price"
• \s* matches any number (including 0) of whitespace (\s) characters
• : matches :
• [0-9]* matches any number of digits (from 0 to 9)
• ?\. optionally matches a .

grep -oE "\"price\"\s*:\s*[0-9]*?\.[0-9]*"

# use cat to get the output of the JSON file and pipe it to the grep
cat blah.txt | grep -oE "\"price\"\s*:\s*[0-9]*?\.[0-9]*"

Examples

# ___ Convert lower to upper
filename | tr 'a-z' 'A-Z'    # or '[a-z]' '[A-Z]'   OR  [:lower:] [:upper:]

# ___ Replace booh with santa  if second is shorter then last character from replacement is repeated
filename | tr 'booh' 'santa'

# ___ To delete an instance of chars  (to remove is)
filename | tr -d 'is'

# ___ Replace repetitive chars like multiple spaces with a single space
filename | tr -s " "

# ___ Replace repetitive chars like multiple spaces with ;
filename | tr -s " " ";"

# ___ DELETE all digits
filename | tr -d [:digits:]

# ___ Keep numbers in a letter and numbers string: -c complement of digits with d to delete 
filename | tr -cd [:digit:]

# ___ Truncate one char string with a shorter string. Will do this in two parts
filename | tr 'isef' '12'
# ___ this will take all occurences of isef and replace them with 1222 if you remember it will use the last char
# repeatedly if second string is shorter than the first

# ___ Now let's use -t, it will truncate isef at is and leave the rest untouched >> '12ef'
filename | tr -t 'isef' '12'

which

Get the path to a command. In order to know how to set your shebang path you need to know the path

$ which bash
/bin/bash

Metacharacters

---

Metacharacters are characters having special meaning that the shell interprets as instructions.

Metacharacter	Meaning
#	Precedes a comment
;	Command separator
*	Filename expansion wildcard
?	Single character wildcard in filename expansion

#

The pound # metacharacter is used to represent comments in shell scripts or configuration files. Any text that appears after a # on a line is treated as a comment and is ignored by the shell.

#!/bin/bash
# This is a comment
$ echo "Hello, world!"  # This is another comment

,

The semicolon ; metacharacter is used to separate multiple commands on a single command line. When multiple commands are separated by a semicolon, they are executed sequentially in the order they appear on the command line.

$ echo "Hello, "; echo "world!"
Hello,
world!

*

The asterisk * metacharacter is used as a wildcard character to represent any sequence of characters, including none.

$ ls *.txt

?

The question mark ? metacharacter is used as a wildcard character to represent any single character. In this example, file?.txt is a wildcard pattern that matches any file in the current directory with a name starting with file, followed by any single character, and ending with the .txt extension.

$ ls file?.txt

Quoting

---

Quoting is a mechanism that allows you to remove the special meaning of characters, spaces, or other metacharacters in a command argument or shell script. You use quoting when you want the shell to interpret characters literally.

Symbol	Meaning
\	Escape metacharacter interpretation
" "	Interpret metacharacters within string
' '	Escape all metacharacters within string

\

The backslash character is used as an escape character. It instructs the shell to preserve the literal interpretation of special characters such as space, tab, and $

For example, if you have a file with spaces in its name, you can use backslashes followed by a space to handle those spaces literally

$ touch file\ with\ space.txt

” ”

When a string is enclosed in double quotes, most characters are interpreted literally, but metacharacters are interpreted according to their special meaning.

For example, you can access variable values using the dollar $ character:

$ echo "Hello $USER"
Hello <username>

’ ’

When a string is enclosed in single quotes, all characters and metacharacters enclosed within the quotes are interpreted literally.

• Single quotes alter the above example to produce the following output
• Notice that instead of printing the value of $USER, single quotes cause the terminal to print the string "$USER"

$ echo 'Hello $USER'
Hello $USER

I/O Redirection

---

Symbol	Meaning
>	Redirect output to file, overwrite
>>	Redirect output to file, append
2>	Redirect standard error to file, overwrite
2>>	Redirect standard error to file, append
<	Redirect file contents to standard input

Input/output (IO) redirection is the process of directing the flow of data between a program and its input/output sources.

By default, a program reads input from standard input, the keyboard, and writes output to standard output, the terminal. However, using IO redirection, you can redirect a program’s input or output to or from a file or another program.

Redirect output >

This symbol is used to redirect the standard output of a command to a specified file.

ls > files.txt will create a file called files.txt if it doesn’t exist, and write the output of the ls command to it.

Redirect and append output >>

This notation is used to redirect and append the output of a command to the end of a file. For example,

ls >> files.txt appends the output of the ls command to the end of file files.txt, and preserves any content that already existed in the file.

Redirect standard output 2>

This notation is used to redirect the standard error output of a command to a file. For example, if you run the ls command on a non-existing directory as follows,

ls non-existent-directory 2> error.txt the shell will create a file called error.txt if it doesn’t exist, and redirect the error output of the ls command to the file.

Warning: When the file already exists, the error message overwrites all of the file’s contents!

Append standard error 2>>

This symbol redirects the standard error output of a command and appends the error message to the end of a file without overwriting its contents.

ls non-existent-directory 2>> error.txt will append the error output of the ls command to the end of the error.txt file.

Redirect input <

This symbol is used to redirect the standard input of a command from a file or another command. For example,

sort < data.txt will sort the contents of the data.txt file.

Command Substitution

---

Command substitution allows you to run command and use its output as a component of another command’s argument. Command substitution is denoted by enclosing a command in either backticks (`command`) or using the $() syntax.

When the encapsulate command is executed, its output is substituted in place, and it can be used as an argument within another command. This is particularly useful for automating tasks that require the use of a command’s output as input for another command.

For example, you could store the path to your current directory in a variable by applying command substitution on the pwd command, then move to another directory, and finally return to your original directory by invoking the cd command on the variable you stored, as follows

$ here=$(pwd)
$ cd path_to_some_other_directory
$ cd $here

Command Line Arguments

---

Command line arguments are additional inputs that can be passed to a program when the program is run from a command line interface. These arguments are specified after the name of the program, and they can be used to modify the behavior of the program, provide input data, or provide output locations. Command line arguments are used to pass arguments to a shell script.

For example, the following command provides two arguments, arg1, and arg2, that can be accessed from within your Bash script

$ ./MyBashScript.sh arg1 arg2

Conditionals

---

Conditionals, or if statements, are a way of telling a script to do something only under a specific condition.

Bash script conditionals use the following if-then-else syntax

You must always put spaces around your condition inside the [ ]

Every if condition block must be paired with a fi to tell Bash where the condition block ends.

The else block is optional but recommended. If the condition evaluates to false without an else block, then nothing happens within the if condition block.

Consider options such as echoing a comment in statement_block_2 to indicate that the condition was evaluated as false.

• If the condition is true, then Bash executes the statements in statement_block_1 before exiting the conditional block of code. After exiting, it will continue to run any commands after the closing fi.
• In the following example, the condition is checking whether the number of command-line arguments read by some Bash script, $#, is equal to 2.

Notice the use of the double square brackets, which is the syntax required for making integer comparisons in the condition [[ $# == 2 ]]

if [ condition ]
then
    statement_block_1  
else
    statement_block_2  
fi

# Example
if [[ $# == 2 ]]
then
  echo "number of arguments is equal to 2"
else
  echo "number of arguments is not equal to 2"
fi

String Conditionals

• You can also make string comparisons. For example, assume you have a variable called string_var that has the value "Yes" assigned to it. Then the following statement evaluates to true

`[ $string_var == "Yes" ]`

Notice you only need single square brackets when making string comparisons.

You can also include multiple conditions to be satified by using the “and” operator && or the “or” operator ||. For example

if [ condition1 ] && [ condition2 ]
then
    echo "conditions 1 and 2 are both true"
else
    echo "one or both conditions are false"
fi

# _______  OR _______

if [ condition1 ] || [ condition2 ]
then
    echo "conditions 1 or 2 are true"
else
    echo "both conditions are false"
fi

Logicals

---

The following logical operators can be used to compare integers within a condition in an if condition block.

• ==: is equal to
• != not equal to
• <= or -le

If a variable a has a value of 2, the following condition evaluates to true; otherwise it evalutes to false

$ a == 2
$ a != 2
$ a <= 1
$ a -le 1
$ if [ $a -le $b ]

Arithmetic Calculations

---

You can perform integer addition, subtraction, multiplication, and division using the notation $(())

• Bash natively handles integer arithmetic but does not handle floating-point arithmetic. As a result, it will always truncate the decimal portion of a calculation result.

a=3
b=2
c=$(($a+$b))
echo $c

Symbol	Operation
+	addition
-	subtraction
*	multiplication
/	division

Arrays

---

The array is a Bash built-in data structure. An array is a space-delimited list contained in parentheses.ʊTo create an array, declare its name and contents

my_array=(1 2 "three" "four" 5)

# append elements to an existing array, here we add "six" and 7
my_array+=("six")
my_array+=(7)

# print the first item of the array:
echo ${my_array[0]}
# print the third item of the array:
echo ${my_array[2]}
# print all array elements:
echo ${my_array[@]}

• This statement creates and populates the array my_array with the items in the parentheses: 1, 2, "three", "four", and 5
• If you want to add items to your array after creating it, you can add to your array by appending one element at a time
• You can access individual or multiple elements of an array

Loops

---

You can use a construct called a for loop along with indexing to iterate over all elements of an array.

For example, the following for loops will continue to run over and over again until every element is printed

for item in ${my_array[@]}; do
  echo $item
done

# ---- OR ----
for i in ${!my_array[@]}; do
  echo ${my_array[$i]}
done

The for loop requires a ; do component in order to cycle through the loop. Additionally, you need to terminate the for loop block with a done statement.

Another way to implement a for loop when you know how many iterations you want is as follows. For example, the following code prints the number 0 through 6.

N=6
for (( i=0; i<=$N; i++ )) ; do
  echo $i
done

Count the number of items in an array, or sum up its elements as shown in this Bash script

#!/usr/bin/env bash
# initialize array, count, and sum
my_array=(1 2 3)
count=0
sum=0
for i in ${!my_array[@]}; do
  # print the ith array element
  echo ${my_array[$i]}
  # increment the count by one
  count=$(($count+1))
  # add the current value of the array to the sum
  sum=$(($sum+${my_array[$i]}))
done
echo $count
echo $sum