Elements in Bash Scripts
So far, we have been executing commands inside the Bash shell. To create the simplest Bash script, you may gather a list of commands, put it into a file, e.g.myscript.sh, and then executebash ./myscript.sh.
Nevertheless, we can do more than executing commands sequentially and independently, e.g. establish a control flow for automating repeated tasks. Let us learn the basic elements for writing Bash scripts.
Running a script
To run a bash script, you may pass it as an argument to the commandbash, e.g. to executemyscript.sh
$ bash ./myscript.sh
Alternatively, we may put a shebang,#!/bin/bash, at the beginning of a script. This shebang tells the OS to use/bin/bashto parse and run the script.
Then, mark the script as executable usingchmod(option+xmeans marking the file as executable, and option abefore +x means this change applies to all users"),
$ chmod a+x ./myscript.sh
And the script can be run directly,
$ ./myscript.sh
Variable
Global variables
Since there are no data type in Bash script(unlike C), the same variable may hold numbers or characters at different times. Simply define and assign a value to a variable using the syntax[variable name]=[value], e.g.
mynum=3150
mystr="hello world"
Beware that spaces are not allowed before or after the assignment operator=.
By default, variables are global and its assigned value are visible to statements that run after the assignment.
To make a variable local, specifylocalexplicitly before the variable name, e.g.
local mynum=315
Other than assigning a value, we may "grow" a variable by appending values using the operator+=, e.g.
mystr="csci"
mystr+="3150"
andmystrnow contains "csci3150".
To retrieve the value of a variable, we use the syntax${variable name}. The$means value retrieval, while{}is used to separate the variable name from other characters. The following script variable.sh shows an example on appending values to a variable, and when{}is necessary.
#!/bin/bash
mynum=3150
course_code="csci"
mystr=${course_code}
# grow mystr from csci to "csci3150"
echo "Before: ${mystr}"
mystr+=${mynum}
echo "After: ${mystr}"
# when curly braces are necessary
echo "${course_code}3150 is a course about OS"
Array
To declare an empty arraymyarray,
myarray=()
We may also assign values on declaration as follow, with each value separated by a space,
myarray=("three" 1 "five" 0)
The array index starts from 0 (as in C), so if we want to replace the third item aseight, we can execute
myarray[2]="eight"
We may add an item with any positive index, e.g.
myarray[5]="!"
To get the first item,
${myarray[0]}
To get the whole array,
${myarray[@]}
To get the number of items in the array,
${#myarray[@]}
To get the keys used in the array,
${!myarray[@]}
Look into and execute array.sh which includes examples on the usage listed above.
Strings
Strings can be though as an array of characters, and we may get its length by
${#mystr}
Also, the string can be split as if an array, see string.sh for an example.
#!/bin/bash
mylongstr="this is a long string"
echo "My string: ${mylongstr}"
echo ""
echo "Number of characters in string: ${#mylongstr}"
echo ""
echo "Splitting my string like an array:"
for word in ${mylongstr[@]}; do
echo "${word}"
done
echo ""
Arithmetic
To do simple arithmetic on whole numbers in Bash, you can put the arithmetic expression inside a double parentheses after a dollar sign$(()).
For example, to store the sum of two variablesaandbintomysum,
#!/bin/bash
a=2
b=3
mysum=$((a+b))
echo "Sum of a=${a} and b=${b} is ${mysum}"
Other than summation, we can do subtraction (-), multiplication (*), division (/) and also get the remainder of division (%).
Besides, you can increment/decrement counters using the operator++and--, e.g. to increment or decrement variablecounter
(( counter++ )), or(( counter-- ))
The$can be omitted in this case, as we are not retrieving the value. We will find the increment/decrement syntax handy in the examples on loops later on.
Conditional statements, if ... then ... else ...
The format of "If-then-else" statements in Bash script is as follows:
if expression then statement else statement fi
or
if expression then statement elif expression then statement else statement fi
Let illustrate the syntax with an example condition.sh,
#!/bin/bash
A=1
if [ $((A)) -eq 0 ]; then
echo "A equals to 0"
elif [ $((A)) -gt 0 ]; then
echo "A is greater than 0"
else
echo "A is smaller than 0"
fi
which prints "A is greater than 0" forA=1.
Note that we use arithmetic expression on variableA, i.e.$((A)), instead of simply use${A}whenAcontains a number, let us look into the difference between the two.
First, we assign an expression toAinstead of a number, e.g. replaceA=1withA=1+2+3+4. The script runs correctly without an error, since$((A))is expanded to$((1+2+3+4))and is evaluated to10when the conditions,$((A)) -eq 0and$((A)) -gt 0, are checked.
Then, we replace$((A))with${A}, and run the script. The script ends up with errors, since the expression1+2+3+4is not evaluated when the synxtax is${A}.
To summarize,$(())works whenAis either a number or an arithmetic expression, while${}works only whenAis a number.
Useful expressions
Numeric values
- Less than,
-lt - Less than or equal,
-le - Equal,
-eq - Not equal,
-ne - Greater than or equal,
-ge - Greater than,
-gt
and you may refer to the example on conditional statement.
Strings
We may check if a variable is empty using-z, e.g.
#!/bin/bash
mystr="This is an OS course"
if [ -z "${mystr}" ]; then
echo "Ops... the string is empty."
else
echo "The string is not empty."
fi
Files and directories
We may check if a file exists and is regular(see definition of regular files) using-f, e.g.
#!/bin/bash
if [ -f example.txt ]; then
echo "File example.txt exists."
else
echo "Ops... example.txt does not exist."
fi
We may also check if a directory exists using-d, e.g. to create directorytestdirif it does exists, otherwise report its existence
#!/bin/bash
if [ ! -d testdir ]; then
mkdir testdir
else
echo "Directory testdir exists"
fi
See the GNU manual here for more expressions for files and strings
Loops
for-loop
The format is
for item in list do statements done
An example forloop.sh,
#!/bin/bash
A=("a" "b" "c")
for i in {1..10}; do
echo "${i}"
done
for char in ${A[@]}; do
echo "${char}"
done
prints 1 to 10 and then 'a' to 'c', with each number or character on a new line.
while-loop
The format is
while expression do statements done
An example whileloop.sh,
#!/bin/bash
A=0
while [ $((A)) -lt 10 ]; do
echo $((A))
(( A++ ))
done
prints 0 to 9, with each number on a new line.
Input / output
Input arguments (applies to both functions and scripts)
The input arguments are obtainable from$@.
The number of arguments is obtainable from$#.
To specify the n-th argument, use$n, e.g. first argument is$1.$0is a special argument that stores the name of the script. For example, this script (argument.sh) prints the arguments to the script.
#!/bin/bash
echo "You called $0, with"
if [ $# -eq 0 ]; then
echo "no arguments..."
exit 0
fi
counter=0
for i in "$@"; do
(( counter++ ))
echo "Arg[${counter}]: ${i}"
done
Try: Please remove the double quotes from"$@"in above script and try the following command./arguments.sh First "Second argument"
Capturestdoutof commands
We may capture and store the output of a command to a variable, e.g. to execute the commandlsand store the result into the variableoutput
output=$(ls)
Note that the syntax$(command)means executingcommand(which is different from$((expression))for evaluating arithmetic expressions).
The exit status of the last executed command is always stored in$?.
See the following script, capture.sh, which includes examples on the above usage,
#!/bin/bash
# capture stdout
output=$(ls)
echo ""
echo "Output of ls: ${output}"
# where is the exit status?
echo ""
haha; echo "haha gives $?";
echo ""
echo "hello_world"; echo "echo gives $?"
Go through a file
We may loop through a file using a while-loop (file_displayer.sh),
#!/bin/bash
if [ $# -lt 1 ]; then
echo "$0 [file to display]"
exit 1
fi
if [ ! -f $1 ]; then
echo "Cannot display non-regular files"
exit 1
fi
while read line; do
echo "${line}"
done < $1
The script prints the file specified as the first input argument, e.g. to printforloop.sh,
$ ./file_displayer.sh forloop.sh
Except that trailing spaces are trimmed, e.g. indentations beforeechoandexit(you may compare the output against that from$ cat forloop.shto see the difference).
Thewhile-loop evaluatesread line, with content read (operator<) from the file named in$f. Note that if we omit both the redirection operator (<) and file name,readwill read input from standard input stream (stdin) instead.
Functions
The format is
function function_name { statements }
For example in function.sh,
#!/bin/bash
function addition {
result=$(($1 + $2))
}
function main {
local a=1
local b=2
result=
addition ${a} ${b}
echo "${a}+${b}=${result}"
}
main
we have two functions. The first one namedadditionadds up the first two input arguments. The second one namedmaincallsadditionand prints the result of addinga=1andb=2.
After executing it, you get "1+2=3" printed.
We can useaddition "$@"to pass the input arguments to the function. For example in function2.sh,
Examples on Content Processing
String splitting withIFS
IFSstands for "Internal Field Separator", which is an environment variable that determines boundaries of words, i.e., it contains word delimiters. By default, the delimiters are space, tab, and newline character.
We may changeIFSduring runtime. For example, to print only words between full-stops (.), we may assign a full-stop toIFS, see ifs.sh
#!/bin/bash
mystr="name.email.phone.remarks"
IFS='.'
for word in ${mystr[@]}; do
echo $word
done
# restore IFS
IFS=" "$'\n'$'\t'
String splitting usingawk, and print designated fields
If we want to
- split a string "first_name,middle_name,last_name" based on a comma (
,), and - print only "first_name" and "last_name" with a space in between
we may use the commandawkas follows
$ echo "first_name,middle_name,last_name" | awk -F',' '{print $1" "$3}'
This command first pipes the string output byechoas the input ofawk. Then,awktakes the character specified after the option-Fas a delimiter to break the string.awkprints the first field ($1) and the third field ($3), with a space (" ") in between.
Given a comma separated value (csv) file data.csv, we may list only the first and third column of the file using the following script process_csv.sh,
#!/bin/bash
while read line; do
echo "${line}" | awk -F',' '{print $1" "$3}'
done < data.csv
Counting lines, words, and characters,wc
wcallows us to count characters (with option-c), words (with option-w) and lines (with option-l). It goes through files when they are provided. Otherwise, it processes input fromstdin.
For example in file_summary.sh,
#!/bin/bash
for f in "$@"; do
counter=0
wc -l ${f}
while read line; do
(( counter ++ ))
chars=$(echo -n ${line} | wc -c)
words=$(echo -n ${line} | wc -w)
echo "Line $((counter)): ${chars} chars, ${words} words"
done < ${f}
done
files are accepted as input arguments. For each file, the script prints the total number of lines together with the file name, followed by the number of characters and words in each line. Note the option -n suppresses echofrom adding a newline character at the end of a string.
Exercise
Variables
- Run global_vs_local.sh and study the difference between global variables (
global1andglobal2) and local variables (local1)
Content processing
- How to print each matching phrase on a newline using
grep? - Write a script to output a summary on a set of files in csv format. The set of files are given as input arguments. Each line shows (1) the file name, (2) the total number of characters, (3) the total number of words, and (4) the total number of lines, in a file
- How to modify the script in this example to avoid trailing spaces being trimmed? (or think about why the spaces are "trimmed" in the first place? :) )