## 2.4. Using awk

Like sed, awk executes a set of instructions for each line of input. You can specify instructions on the command line or create a script file.

### 2.4.1. Running awk

For command lines, the syntax is:

>     awk 'instructions' files

Input is read a line at a time from one or more files or from standard input. The instructions must be enclosed in single quotes to protect them from the shell. (Instructions almost always contain curly braces and/or dollar signs, which are interpreted as special characters by the shell.) Multiple command lines can be entered in the same way as shown for sed: separating commands with semicolons or using the multiline input capability of the Bourne shell.

Awk programs are usually placed in a file where they can be tested and modified. The syntax for invoking awk with a script file is:

>     awk -f script files

The -f option works the same way as it does with sed.

While awk instructions have the same structure as sed, consisting of pattern and procedure sections, the procedures themselves are quite different. Here is where awk looks less like an editor and more like a programming language. There are statements and functions instead of one- or two-character command sequences. For instance, you use the print statement to print the value of an expression or to print the contents of the current input line.

Awk, in the usual case, interprets each input line as a record and each word on that line, delimited by spaces or tabs, as a field. (These defaults can be changed.) One or more consecutive spaces or tabs count as a single delimiter. Awk allows you to reference these fields, in either patterns or procedures. $0 represents the entire input line. $1, $2, ... refer to the individual fields on the input line. Awk splits the input record before the script is applied. Let's look at a few examples, using the sample input file list.

The first example contains a single instruction that prints the first field of each line in the input file.

>     $ awk '{ print $1 }' list
>     John
>     Alice
>     Orville
>     Terry
>     Eric
>     Hubert
>     Amy
>     Sal

"$1" refers to the value of the first field on each input line. Because there is no pattern specified, the print statement is applied to all lines. In the next example, a pattern "/MA/" is specified but there is no procedure. The default action is to print each line that matches the pattern.

>     $ awk '/MA/' list
>     John Daggett, 341 King Road, Plymouth MA
>     Eric Adams, 20 Post Road, Sudbury MA
>     Sal Carpenter, 73 6th Street, Boston MA

Three lines are printed. As mentioned in the first chapter, an awk program can be used more like a query language, extracting useful information from a file. We might say that the pattern placed a condition on the selection of records to be included in a report, namely that they must contain the string "MA". Now we can also specify what portion of a record to include in the report. The next example uses a print statement to limit the output to the first field of each record.

>     $ awk '/MA/ { print $1 }' list
>     John
>     Eric
>     Sal

It helps to understand the above instruction if we try to read it aloud: Print the first word of each line containing the string "MA". We can say "word" because by default awk separates the input into fields using either spaces or tabs as the field separator.

In the next example, we use the -F option to change the field separator to a comma. This allows us to retrieve any of three fields: the full name, the street address, or the city and state.

>     $ awk -F, '/MA/ { print $1 }' list
>     John Daggett
>     Eric Adams
>     Sal Carpenter

Do not confuse the -F option to change the field separator with the -f option to specify the name of a script file.

In the next example, we print each field on its own line. Multiple commands are separated by semicolons.

>     $ awk -F, '{ print $1; print $2; print $3 }' list
>     John Daggett
>      341 King Road
>      Plymouth MA
>     Alice Ford
>      22 East Broadway
>      Richmond VA
>     Orville Thomas
>      11345 Oak Bridge Road
>      Tulsa OK
>     Terry Kalkas
>      402 Lans Road
>      Beaver Falls PA
>     Eric Adams
>      20 Post Road
>      Sudbury MA
>     Hubert Sims
>      328A Brook Road
>      Roanoke VA
>     Amy Wilde
>      334 Bayshore Pkwy
>      Mountain View CA
>     Sal Carpenter
>      73 6th Street
>      Boston MA

Our examples using sed changed the content of incoming data. Our examples using awk rearrange the data. In the preceding awk example, note how the leading blank is now considered part of the second and third fields.

### 2.4.2. Error Messages

Each implementation of awk gives you different error messages when it encounters problems in your program. Thus, we won't quote a particular version's messages here; it'll be obvious when there's a problem. Messages can be caused by any of the following:

- Not enclosing a procedure within braces ({})

- Not surrounding the instructions within single quotes ('')

- Not enclosing regular expressions within slashes (//)

### 2.4.3. Summary of Options

Table 2.2 summarizes the awk command-line options.

#### Table 2.2. Command-Line Options for awk

Option | Description
--- | ---
-f | Filename of script follows.
-F | Change field separator.
-v | var=value follows.

The -v option for specifying parameters on the command line is discussed in Chapter 7, "Writing Scripts for awk".

Copyright © 2003 O'Reilly & Associates. All rights reserved.