Book: LPI Linux Certification in a Nutshell Section: Chapter 17. Shells, Scripting, Programming, and Compiling (Topic 1.9)
### 17.2 Objective 2: Customize or Write Simple Scripts You've seen how the use of bash configuration files, aliases, functions, variables, and key bindings can customize and make interaction with your Linux system efficient. The next step in your relationship with the shell is to use its natural programming capability, or scripting language. The scripting language of the original Bourne shell is found throughout a Linux system, and bash is fully compatible with it. This section covers essential bash scripting language concepts as required for Exam 102. In order to have a full appreciation of shell scripting on Linux, it's important to look at your Linux system as a collection of unique and powerful tools. Each of the commands available on your Linux system, along with those you create yourself, has some special capability. Bringing these capabilities together to solve problems is among the basic philosophies of the Unix world. #### 17.2.1 Script Files Just as the configuration files discussed in the last section are plain text files, so are the scripts for your shell. In addition, unlike compiled languages such as C or Pascal, no compilation of a shell program is necessary before it is executed. You can use any editor to create script files, and you'll find that many scripts you write are portable from Linux to other Unix systems. ##### 17.2.1.1 Creating a simple bash script The simplest scripts are those that simply string together some basic commands and perhaps do something useful with the output. Of course, this can be done with a simple alias or function, but eventually you'll have a requirement that exceeds a one-line request, and a shell script is the natural solution. Aliases and functions have already been used to create a rudimentary new command, lsps. Now let's look at a shell script (Example 17-6) that accomplishes the same thing. ##### Example 17-6. The lsps Script
    # a basic lsps command script for bash
    ls -l $1
    ps -aux | grep `/bin/basename $1`
As you can see, the commands used in this simple script are identical to those used in the alias and in the function created earlier. To make use of this new file, instruct your currently running bash shell to source it, giving it an option for the `$1` positional parameter:
    $ source ./lsps /usr/sbin/httpd
If you have /usr/sbin/httpd running, you should receive output similar to that found previously for the alias. By replacing the word source with a single dot, you can create an alternate shorthand notation to tell bash to source a file, as follows:
    $ . ./lsps /usr/sbin/httpd
Another way to invoke a script is to start a new invocation of bash and tell that process to source the file. To do this, simply start bash and pass the script name and argument to it:
    $ /bin/bash ./lsps /usr/sbin/httpd
This last example gives us the same result; however, it is significantly different from the alias, the function, or the sourcing of the lsps file. In this particular case, a new invocation of bash was started to execute the commands in the script. This is important, because the environment in which the commands are running is distinct from the environment where the user is typing. This is described in more detail later.
| The `./` syntax indicates that the file you're referring to is in the current working directory. To avoid specifying `./` for users other than the superuser, put the directory `.` in the `PATH` . The `PATH` of the superuser should not include the current working directory, as a security precaution against Trojan horse-style attacks.
Thus far, a shell script has been created and invoked in a variety of ways, but it hasn't been made into a command. A script really becomes useful when it can be called by name like any other command. ##### 17.2.1.2 Executable files On a Linux system, programs are said to be executable if they have content that can be run by the processor (native execution) or by another program such as a shell (interpreted execution). However, in order to be eligible for execution when called at the command line, the files must have attributes that indicate to the shell that they are executable. Conspicuously absent is anything in the filename that indicates that the file is executable, such as the file extension of .exe found on MS-DOS and Windows applications. It would be possible to name our example file lsps.exe if desired, or for that matter lsps.sh or lsps.bin. None of these extensions has any meaning to the shell, though, and the extension would become part of the command entered when executing the program. For this reason, most executable Linux programs and scripts don't have filename extensions. To make a file executable, it must have at least one of its executable bits set. To turn our example script from a plain text file to an executable program, that bit must be set using the chmod command:
    $ chmod a+x lsps
Once this is done, the script is executable by owner, group members, and everyone else on the system. At this point, running the new command from the bash prompt yields the familiar output:
    $ ./lsps /usr/sbin/httpd
When lsps is called by name, the commands in the script are interpreted and executed by the bash shell. However, this isn't ultimately what is desired. In many cases, users will be running some other shell interactively but will still want to program in bash. Programmers also use other scripting languages such as Perl. To have our scripts interpreted correctly, the system must be told which program should interpret the commands in our scripts. ##### 17.2.1.3 She-bang! There are many kinds of script files found on a Linux system, and each interpreted language comes with a unique and specific command structure. There needs to be a way to tell Linux which interpreter to use. This is accomplished by using a special line at the top of the script naming the appropriate interpreter. Linux examines this line and launches the specified interpreter program, which then reads the rest of the file. The special line must begin with `#!`, a construct often called "she-bang." For bash, the she-bang line is:
    #!/bin/bash
This command explicitly states that the program named bash can be found in the /bin directory and designates bash to be the interpreter for the script. You'll also see other types of lines on script files, including: #!/bin/sh The bourne shell. #!/bin/csh The C-shell. #!/bin/tcsh The enhanced C-shell. #!/bin/sed The stream editor. #!/usr/bin/awk The awk programming language. #!/usr/bin/perl The Perl programming language. Each of these lines specifies a unique command interpreter use for the script lines that follow.[4] [4] bash is fully backward compatible with sh; sh is just a link to bash on Linux systems.
## On the Exam An incorrectly stated she-bang line can cause the wrong interpreter to attempt to execute commands in a script.
##### 17.2.1.4 The shell script's environment When running a script with #!/bin/bash, a new invocation of bash with its own environment is started to execute the script's commands as the parent shell waits. Exported variables in the parent shell are copied into the child's environment; the child shell executes the appropriate shell configuration files (such as .bash_profile). Because configuration files will be run, additional shell variables may be set and environment variables may be overwritten. If you are depending upon a variable in your shell script, be sure that it is either set by the shell configuration files or exported into the environment for your use, but not both. Another important concept regarding your shell's environment is one-way inheritance. Although your current shell's environment is passed into a shell script, that environment is not passed back to the original shell when your program terminates. This means that changes made to variables during the execution of your script are not preserved when the script exits. Instead, the values in the parent shell's variables are the same as they were before the script executed. This is a basic Unix construct; inheritance goes from parent process to child process, and not the other way around.
## On the Exam It is important to remember how variables are set, how they are inherited, and that they are inherited only from parent process to child process.
##### 17.2.1.5 Location, ownership, and permissions The ability to run any executable program, including a script, under Linux depends in part upon its location in the filesystem. Either the user must explicitly specify the location of the file to run or it must be located in a directory known by the shell to contain executables. Such directories are listed in the `PATH` environment variable. For example, the shells on a Linux system (including bash) are located in /bin. This directory is usually in the `PATH`, because you're likely to run programs that are stored there. When you create shell programs or other utilities of your own, you may want to keep them together and add the location to your own `PATH`. If you maintain your own bin directory, you might add the following line to your .bash_ profile:
    PATH=$PATH:$HOME/bin
This statement modifies your path to include your /home/bin directory. If you add personal scripts and programs to this directory, bash finds them automatically. Execute permissions (covered in Section 4.5) also affect your ability to run a script. Since a script is just a text file, execute permission must be granted to them before they are considered executable, as shown earlier. You may wish to limit access to the file from other users using:
    $ chmod 700 ~/bin/lsps
This prevents anyone but the owner from making changes to the script. The issue of file ownership is dovetailed with making a script executable. By default, you own all of the files you create. However, if you are the system administrator, you'll often be working as the superuser and will be creating files with username root as well. It is important to assign the correct ownership and permission to scripts to ensure that they are secured. ##### 17.2.1.6 SUID and GUID rights On rare occasions, it may become necessary to allow a user to run a program under the name of a different user. This is usually associated with programs run by nonprivileged users who need special privileges to execute correctly. Linux offers two such rights, known as set user ID (SUID) and set group ID (SGID). When an executable file is granted the SUID right, processes created to execute it are owned by the user who owns the file instead of the user who launched the program. This is a security enhancement in that the delegation of a privileged task or ability does not imply that the superuser password must be widely known. On the other hand, any process whose file is owned by root and which has the SUID set will run as root for everyone. This could represent an opportunity to break the security of a system if the file itself is easy to attack (as a script is). For this reason, Linux systems will ignore SUID and SGID attributes for script files. Setting SUID and SGID attributes is detailed in Section 4.5.
## On the Exam Be sure to think through any questions that require you to determine a user's right to execute a file. Consider location, ownership, execute permissions, and SUID/SGID rights together. Also, watch for new scripts that haven't been granted any execute privileges.
#### 17.2.2 Basic bash Scripts Now that some of the requirements for creating and using executable scripts are established, some of the features that make them so powerful can be introduced. This section contains basic information needed to customize and create new bash scripts. ##### 17.2.2.1 Return values As shell scripts execute, it is important to confirm that their constituent commands complete successfully. Most commands offer a return value to the shell when they terminate. This value is a simple integer and has meaning specific to the program you're using. Almost all programs return the value when they are successful, and return a nonzero value when a problem is encountered. The value is stored in the special bash variable `$?`, which can be tested in your scripts to check for successful command execution. This variable is reset for every command executed by the shell, so you must test it immediately after execution of the command you're verifying. As a simple example, try using the cat program on a nonexistent file:
    $ cat bogus_file
    cat: bogus_file: No such file or directory
Then immediately examine the status variable twice:
    $ echo $?
    1
    $ echo $?
    0
  The first echo yielded `1` (failure) because the cat program failed to find the file you specified. The second echo yielded `0` (success) because the first echo command succeeded. A good script makes use of these status flags to exit gracefully in case of errors. If it sounds backward to equate zero with success and nonzero with failure, consider how these results are used in practice: Error detection Scripts that check for errors include if-then code to evaluate a command's return status:
      command
      if (failure_returned) {
        ...error recovery code...
      }
In a bash script, `failure_returned` is simply the `$?` variable, which contains the result of the command's execution. Error classification Since commands can fail for multiple reasons, many return more than one failure code. For example, grep returns `0` if matches are found and `1` if no matches are found; it returns `2` if there is a problem with the search pattern or input files. Scripts may need to respond differently to various error conditions.
## On the Exam Make certain you understand the meaning of return values in general and that they are stored in the `$?` variable.
##### 17.2.2.2 File tests During the execution of a shell script, specific information about a file -- such as whether it exists, is writable, is a directory or a file, and so on -- may sometimes be required. In bash, the built-in command test performs this function.[5] test has two general forms: [5] There is also a standalone executable version of test available in /usr/bin for non-bash shells. test expression In this form, test and an expression are explicitly stated. [ expression ] In this form, test isn't mentioned; instead, the expression is enclosed inside brackets. The expression can be formed to look for such things as empty files, the existence of files, the existence of directories, equality of strings, and others. (See the more complete list with their operators in the next section.) When used in a script's `if` or `while` statement, the brackets (`[` and `]`) may appear to be grouping the test logically. In reality, `[` is simply another form of the test command, which requires the trailing `]`. A side effect of this bit of trickery is that the spaces around `[` and `]` are mandatory, a detail that is sure to get you into trouble eventually. See the later section, "Abbreviated bash command reference," for some of the available tests. ##### 17.2.2.3 Command substitution Bash offers a handy ability to do command substitution. This feature allows you to replace `$(``command` `)` with the result of `command`, usually in a script. That is, wherever `$(``command``)` is found, its output is substituted prior to interpretation by the shell. For example, to set a variable to the number of lines in your .bashrc file, you could use wc -l:
    $ RCSIZE=$(wc -l ~/.bashrc)
An older form of command substitution encloses `command` in backquotes:
    $ RCSIZE=`wc -l ~/.bashrc`
The result is the same, except that the backquote syntax allows the backslash character to escape the dollar symbol (`$`), the backquote (```), and another backslash (`\` ). The `$(``command``)` syntax avoids this nuance by treating all characters between the parentheses literally. ##### 17.2.2.4 Mailing from scripts The scripts you write will often be rummaging around your system at night when you're asleep or at least while you're not watching. Since you're too busy to check on every script's progress, a script will sometimes need to send some mail to you or another administrator. This is particularly important when something big goes wrong or when something important depends on the script's outcome. Sending mail is as simple as piping into the mail command:
    echo "Backup failure 5" | mail -s "Backup failed" root
The -s option indicates that a quoted subject for the email follows. The recipient could be yourself, root, or if your system is configured correctly, any Internet email address. If you need to send a log file, redirect the input of mail from that file:
    mail -s "subject" recipient < logfile
Sending email from scripts is easy and makes tracking status easier than reviewing log files every day. On the downside, having an inbox full of "success" messages can be a nuisance too, so many scripts are written so that mail is sent only in response to an important event, such as a fatal error. ##### 17.2.2.5 Abbreviated bash command reference This section lists some of the important bash built-in commands used when writing scripts. Please note that not all of the bash commands are listed here; for a complete overview of the bash shell, see Learning the bash Shell by Cameron Newham and Bill Rosenblatt (O'Reilly & Associates).
break |
#### Syntax
    break [n]
#### Description Exit from the innermost (most deeply nested) for, while, or until loop or from the n innermost levels of the loop.
case |
#### Syntax
    case string
    in
       regex1)
       commands1
       ;;
       regex2)
       commands2
       ;;
       ...
    esac
#### Description Choose string from among a series of possible regular expressions. If string matches regular expression regex1, perform the subsequent commands1. If string matches regex2, performcommands2. Proceed down the list of regular expressions until one is found. To catch all remaining strings, use `*)` at the end.
continue |
#### Syntax
    continue [n]
#### Description Skip remaining commands in a for, while, or until loop, resuming with the next iteration of the loop (or skipping n loops).
echo |
#### Syntax
    echo [options] [string]
#### Description Write string to standard output, terminated by a newline. If no string is supplied, echo only a newline. #### Frequently used options -e Enables interpretation of escape characters. -n Suppresses the trailing newline in the output. \a Sounds an audible alert. \b Inserts a backspace. \c Suppresses the trailing newline (same as -n). \f Form feed.
exit |
#### Syntax
    exit [n]
#### Description Exit a shell script with status n. The value for n can be (success) or nonzero (failure). If n is not given, the exit status is that of the most recent command. #### Example
    if ! test -f somefile
    then
      echo "Error: Missing file somefile"
      exit 1
    fi
    for x [in list]
    do
       commands
    done
for |
#### Syntax
    for x in list
    do
       commands
    done
#### Description Assign each word in list to x in turn and execute commands. If list is omitted, it is assumed that positional parameters from the command line, which are stored in `$@`, are to be used. #### Example
    for filename in bigfile*
    {
      echo "Compressing $filename"
      gzip $filename
    }
function |
#### Syntax
    function name
    {
       commands
    }
#### Description Define function name. Positional parameters (`$1`, `$2`, ...) can be used within commands. #### Example
    # function myfunc
    {
      echo "parameter is $1"
    }
    # myfunc 1
    parameter is 1
    # myfunc two
    parameter is two
getopts |
#### Syntax
    getopts  string name [args]
#### Description Process command-line arguments (or args, if specified) and check for legal options. The getopts command is used in shell script loops and is intended to ensure standard syntax for command-line options. The string contains the option letters to be recognized by getopts when running the script. Valid options are processed in turn and stored in the shell variable name. If an option letter is followed by a colon, the option must be followed by one or more arguments when the command is entered by the user.
if |
#### Syntax
    if expression1
    then
        commands1
    elif expression2
    then
        commands2
    else
        commands
    fi
#### Description The `if` command is used to define a conditional statement. There are three possible formats for using the `if` command:
    if-then-fi
    if-then-else-fi
    if-then-elif-then-...fi
The expressions are made up of tests (or `[]` commands).
kill |
#### Syntax
    kill [options] IDs
#### Description Send signals to each specified process or job ID, which you must own unless you are a privileged user. The default signal sent with the kill command is `TERM`, instructing processes to shut down. #### Options -l List the signal names. -s signal or -signal Specifies the signal number or name.
read |
#### Syntax
    read [options] variable1 [variable2...]
#### Description Read one line of standard input, and assign each word to the corresponding variable, with all remaining words assigned to the last variable. #### Example
    echo -n "Enter last-name, age, height, and weight > "
    read lastname everythingelse
    echo $lastname
    echo $everythingelese
The name entered is placed in variable `$lastname`; all of the other values, including the spaces between them, are placed in `$everythingelse`.
return |
#### Syntax
    return [n]
#### Description This command is used inside a function definition to exit the function with status n. If n is omitted, the exit status of the previously executed command is returned.
shift |
#### Syntax
    shift [n]
#### Description Shift positional parameters down n elements. If n is omitted, the default is 1, so `$2` becomes `$1`, `$3` becomes `$2`, and so on.
source |
#### Syntax
    source file [arguments]
    . file [arguments]
#### Description Read and execute lines in file. The file does not need to be executable but must be in a directory listed in `PATH`. The "dot" syntax is equivalent to stating source.
test |
#### Syntax
    test expression
    [ expression ]
#### Description Evaluate the conditional expression and return a status of (true) or 1 (false). The first form explicitly calls out the test command. The second form implies the test command. The spaces around expression are required in the second form. expression is constructed using options. #### Frequently used options -d file True if file exists and is a directory -e file True if file exists -f file True if file exists and is a regular file -L file True if file exists and is a symbolic link -n string True if the length of string is nonzero -r file True if file exists and is readable -s file True if file exists and has a size greater than zero -w file True if file exists and is writable -x file True if file exists and is executable -z string True if the length of string is zero file1 -ot file2 True if file1 is older than file2 string1 = string2 True if the strings are equal string1 != string2 True if the strings are not equal #### Example To determine if a file exists and is readable, use the -r option:
    if test -r file
    then
       echo "file exists"
    fi
Using the `[ ]` form instead, the same test looks like this:
    if [ -r file ]
    then
       echo "file exists"
    fi
until |
#### Syntax
    until
        test-commands
    do
        commands
    done
#### Description Execute test-commands (usually a test command) and if the exit status is nonzero (that is, the test fails), perform commands ; repeat. Opposite of while.
while |
#### Syntax
    while
        test-commands
    do
        commands
    done
#### Description Execute test-commands (usually a test command) and if the exit status is zero, perform commands; repeat. Opposite of until. #### Example Example 17-7 shows a typical script from a Linux system. This example is /etc/rc.d/init.d/sendmail, which is the script that starts and stops sendmail. This script demonstrates many of the built-in commands referenced in the last section. ##### Example 17-7. Sample sendmail Startup Script
    #!/bin/sh
    #
    # sendmail     This shell script takes care of starting
    #              and stopping sendmail.
    #
    # chkconfig: 2345 80 30
    # description: Sendmail is a Mail Transport Agent, which
    #              is the program that moves mail from one
    #              machine to another.
    # processname: sendmail
    # config: /etc/sendmail.cf
    # pidfile: /var/run/sendmail.pid

    # Source function library.
    . /etc/rc.d/init.d/functions

    # Source networking configuration.
    . /etc/sysconfig/network

    # Source sendmail configuration.
    if [ -f /etc/sysconfig/sendmail ] ; then
      . /etc/sysconfig/sendmail
    else
      DAEMON=yes
      QUEUE=1h
    fi

    # Check that networking is up.
    [ ${NETWORKING} = "no" ] && exit 0

    [ -f /usr/sbin/sendmail ] || exit 0

    # See how we were called.
    case "$1" in
      start)
      # Start daemons.
      echo -n "Starting sendmail: "
      /usr/bin/newaliases > /dev/null 2>&1
      for i in virtusertable access domaintable mailertable ; do
        if [ -f /etc/mail/$i ] ; then
          makemap hash /etc/mail/$i < /etc/mail/$i
        fi
      done
      daemon /usr/sbin/sendmail $([ "$DAEMON" = yes ] \
        && echo -bd) $([ -n "$QUEUE" ] && echo -q$QUEUE)
      echo
      touch /var/lock/subsys/sendmail
      ;;

      stop)
      # Stop daemons.
      echo -n "Shutting down sendmail: "
      killproc sendmail
      echo
      rm -f /var/lock/subsys/sendmail
      ;;

      restart)
      $0 stop
      $0 start
      ;;

      status)
      status sendmail
      ;;

      *)
      echo "Usage: sendmail {start|stop|restart|status}"
      exit 1
    esac

    exit 0
## On the Exam You should be familiar with a script's general structure, as well as the use of she-bang, test, if statements and their syntax (including the trailing fi ), return values, exit values, and so on.