Learn How to Use Awk Special Patterns ‘BEGIN and END’ – Part 9

In Part 8 of this Awk series, we introduced some powerful Awk command features, that is variables, numeric expressions and assignment operators.

As we advance, in this segment, we shall cover more Awk features, and that is the special patterns: BEGIN and END.

Learn Awk Patterns BEGIN and END

Learn Awk Patterns BEGIN and END

These special features will prove helpful as we try to expand on and explore more methods of building complex Awk operations.

To get started, let us drive our thoughts back to the introduction of the Awk series, remember when we started this series, I pointed out that the general syntax of a running an Awk command is:

# awk 'script' filenames  

And in the syntax above, the Awk script has the form:

/pattern/ { actions } 

When you consider the pattern in the script, it is normally a regular expression, additionally, you can also think of pattern as special patterns BEGIN and END. Therefore, we can also write an Awk command in the form below:

awk '
 	BEGIN { actions } 
 	/pattern/ { actions }
 	/pattern/ { actions }
            ……….
	 END { actions } 
' filenames  

In the event that you use the special patterns: BEGIN and END in an Awk script, this is what each of them means:

  1. BEGIN pattern: means that Awk will execute the action(s) specified in BEGIN once before any input lines are read.
  2. END pattern: means that Awk will execute the action(s) specified in END before it actually exits.

And the flow of execution of the an Awk command script which contains these special patterns is as follows:

  1. When the BEGIN pattern is used in a script, all the actions for BEGIN are executed once before any input line is read.
  2. Then an input line is read and parsed into the different fields.
  3. Next, each of the non-special patterns specified is compared with the input line for a match, when a match is found, the action(s) for that pattern are then executed. This stage will be repeated for all the patterns you have specified.
  4. Next, stage 2 and 3 are repeated for all input lines.
  5. When all input lines have been read and dealt with, in case you specify the END pattern, the action(s) will be executed.

You should always remember this sequence of execution when working with the special patterns to achieve the best results in an Awk operation.

To understand it all, let us illustrate using the example from part 8, about the list of domains owned by Tecmint, as stored in a file named domains.txt.

news.tecmint.com
tecmint.com
linuxsay.com
windows.tecmint.com
tecmint.com
news.tecmint.com
tecmint.com
linuxsay.com
tecmint.com
news.tecmint.com
tecmint.com
linuxsay.com
windows.tecmint.com
tecmint.com
$ cat ~/domains.txt
View Contents of File

View Contents of File

In this example, we want to count the number of times the domain tecmint.com is listed in the file domains.txt. So we wrote a small shell script to help us do that using the idea of variables, numeric expressions and assignment operators which has the following content:

#!/bin/bash
for file in [email protected]; do
        if [ -f $file ] ; then
                #print out filename
                echo "File is: $file"
                #print a number incrementally for every line containing tecmint.com 
                awk '/^tecmint.com/ { counter+=1 ; printf "%s\n", counter ; }' $file
        else
                #print error info incase input is not a file
                echo "$file is not a file, please specify a file." >&2 && exit 1
        fi
done
#terminate script with exit code 0 in case of successful execution 
exit 0

Let us now employ the two special patterns: BEGIN and END in the Awk command in the script above as follows:

We shall alter the script:

awk '/^tecmint.com/ { counter+=1 ; printf "%s\n", counter ; }' $file

To:

awk ' BEGIN {  print "The number of times tecmint.com appears in the file is:" ; }
                      /^tecmint.com/ {  counter+=1  ;  }
                      END {  printf "%s\n",  counter  ; } 
                    '  $file

After making the changes to the Awk command, the complete shell script now looks like this:

#!/bin/bash
for file in [email protected]; do
        if [ -f $file ] ; then
                #print out filename
                echo "File is: $file"
                #print the total number of times tecmint.com appears in the file
                awk ' BEGIN {  print "The number of times tecmint.com appears in the file is:" ; }
                      /^tecmint.com/ {  counter+=1  ;  }
                      END {  printf "%s\n",  counter  ; } 
                    '  $file
        else
                #print error info incase input is not a file
                echo "$file is not a file, please specify a file." >&2 && exit 1
        fi
done
#terminate script with exit code 0 in case of successful execution 
exit 0
Awk BEGIN and END Patterns

Awk BEGIN and END Patterns

When we run the script above, it will first of all print the location of the file domains.txt, then the Awk command script is executed, where the BEGIN special pattern helps us print out the message “The number of times tecmint.com appears in the file is:” before any input lines are read from the file.

Then our pattern, /^tecmint.com/ is compared against every input line and the action, { counter+=1 ; } is executed for each input line, which counts the number of times tecmint.com appears in the file.

Finally, the END pattern will print the total the number of times the domain tecmint.com appears in the file.

$ ./script.sh ~/domains.txt 
Script to Count Number of Times String Appears

Script to Count Number of Times String Appears

To conclude, we walked through more Awk features exploring on the concepts of special pattern: BEGIN and END.

As I pointed out before, these Awk features will help us build more complex text filtering operations, there is more to cover under Awk features and in part 10, we shall approach the idea of Awk built-in variables, so stay connected.

Best Affordable Linux and WordPress Services For Your Business
Outsource Your Linux and WordPress Project and Get it Promptly Completed Remotely and Delivered Online.

If You Appreciate What We Do Here On TecMint, You Should Consider:

  1. Stay Connected to: Twitter | Facebook | Google Plus
  2. Subscribe to our email updates: Sign Up Now
  3. Get your own self-hosted blog with a Free Domain at ($3.45/month).
  4. Become a Supporter - Make a contribution via PayPal
  5. Support us by purchasing our premium books in PDF format.
  6. Support us by taking our online Linux courses

We are thankful for your never ending support.

Aaron Kili

Aaron Kili is a Linux and F.O.S.S enthusiast, an upcoming Linux SysAdmin, web developer, and currently a content creator for TecMint who loves working with computers and strongly believes in sharing knowledge.

Your name can also be listed here. Got a tip? Submit it here to become an TecMint author.

RedHat RHCE and RHCSA Certification Book
Linux Foundation LFCS and LFCE Certification Preparation Guide

You may also like...

2 Responses

  1. Cyprian ambe says:

    What are the steps to install Apache from source code

Got something to say? Join the discussion.

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.