How to Read Awk Input from STDIN in Linux – Part 7

In the previous parts of the Awk tool series, we looked at reading input mostly from a file(s), but what if you want to read input from STDIN.

In this Part 7 of Awk series, we shall look at few examples where you can filter the output of other commands instead of reading input from a file.

We shall start with the dir utility that works similar to ls command, in the first example below, we use the output of dir -l command as input for Awk to print owner’s username, groupname and the files he/she owns in the current directory:

# dir -l | awk '{print $3, $4, $9;}'
List Files Owned By User in Directory

List Files Owned By User in Directory

Take a look at another example where we employ awk expressions, here, we want to print files owned by the root user by using an expression to filter strings as in the awk command below:

# dir -l | awk '$3=="root" {print $1,$3,$4, $9;} '
List Files Owned by Root User

List Files Owned by Root User

The command above includes the (==) comparison operator to help us filter out files in the current directory which are owned by the root user. This is achieved using the expression $3==”root”.

Let us look at another example of where we use a awk comparison operator to match a certain string.

Here, we have used the cat utility to view the contents of a file named tecmint_deals.txt and we want to view the deals of type Tech only, so we shall run the following commands:

# cat tecmint_deals.txt
# cat tecmint_deals.txt | awk '$4 ~ /tech/{print}'
# cat tecmint_deals.txt | awk '$4 ~ /Tech/{print}'
Use Awk Comparison Operator to Match String

Use Awk Comparison Operator to Match String

In the example above, we have used the value ~ /pattern/ comparison operator, but there are two commands to try and bring out something very important.

When you run the command with pattern tech nothing is printed out because there is no deal of that type, but with Tech, you get deals of type Tech.

So always be careful when using this comparison operator, it is case sensitive as we have seen above.

You can always use the output of another command instead as input for awk instead of reading input from a file, this is very simple as we have looked at in the examples above.

Hope the examples were clear enough for you to understand, if you have any concerns, you can express them through the comment section below and remember to check the next part of the series where we shall look at awk features such as variables, numeric expressions and assignment operators.

If You Appreciate What We Do Here On TecMint, You Should Consider:

TecMint is the fastest growing and most trusted community site for any kind of Linux Articles, Guides and Books on the web. Millions of people visit TecMint! to search or browse the thousands of published articles available FREELY to all.

If you like what you are reading, please consider buying us a coffee ( or 2 ) as a token of appreciation.

Support Us

We are thankful for your never ending support.

Aaron Kili

Aaron Kili is a Linux and F.O.S.S enthusiast, an upcoming Linux SysAdmin, web developer, and currently a content creator for TecMint who loves working with computers and strongly believes in sharing knowledge.

Your name can also be listed here. Got a tip? Submit it here to become an TecMint author.

RedHat RHCE and RHCSA Certification Book
Linux Foundation LFCS and LFCE Certification Preparation Guide

You may also like...

17 Responses

  1. Giuseppe says:

    Hello Mr Kili, I have two other questions.
    1 – I would like to add some code. Starting from this:
    tail -f /opt/red5-server108m2/log/red5.log | awk ‘{if (/Stream start:/){

    *insertion point*

    }

    … in that marked position I woud like to add a parse of an other file to search the presence of a name. So, if the name is not present no action start.

    2 – at the same position I woul like to operate a sting operation of $9 variable.
    ex: $9 is filled with this value “namex:name”
    How to split to have “name” only? I try several times but with syntax error only.

    Thank you in advance
    (sorry for my bad english)

    • Aaron Kili K says:

      In this case, you need to approach the problem using the concept of built-in Awk variables, and that is the FS(input field separator) built-in variable.

      take for example, that the value of $9=”namex:name”

      therefore, you can use two awk commands, first, you can print out the value of $9 then pipe it to a second awk command specifying the FS value as in the command below:

      awk ‘ { print $9 } ‘ | awk -F ‘:’ ‘ { print $2 } ‘

      In the example above, the FS value is :, the value of $2 will be name, from the original value in $9

      I hope this will help, in case it does not, post the question on linuxsay.com and i will answer with some screen shots for better illustration.

    • Giuseppe says:

      Thank you for fast, kind and comprensive reply to an old noob 60 years old.
      Now I begin to write some simple code about strings operation.

      Have you some gift for the 1st question?

      Giuseppe

      • Aaron Kili K says:

        You are welcome @Giuseppe

        If i got the question right, i suppose you want to parse another file then search for a name in that file. The difficulty is therefore reading input from a second file apart from piped input.

        Let me look into the problem and figure out the solution, it will require a little time. I hope you can wait a little more, after all the solution is the most important.

        • Giuseppe says:

          Hello Mr. Kili, you are correct. In that point marked I want sto scan a second text file where a list of user is stored. So if user is present the stream can start else not.

          Have a good day, Giuseppe

          • Aaron Kili K says:

            @Guiseppe

            Sorry for the delay, but i hope this method with help solve your problem.

            Try to use the approach of an Awk script as follows:

            tail -f /opt/red5-server108m2/log/red5.log | awk -f script.awk file

            put awk command in script.awk and use the -f option to read the script as in the command above.

            script.awk

            ————————————————————————————–
            #!/bin/awk -f

            ‘{if (/Stream start:/){

            *insertion point*

            }

            Check this guide for more information: https://www.gnu.org/software/gawk/manual/html_node/Naming-Standard-Input.html

          • giuseppe says:

            Thank you. Going to found a solution out of the pipe.
            Possibly reading the second file before the tail command. So I store results in an array to scan in the second pipe.

            Giuseppe

  2. Giuseppe says:

    Ho Mr. Kili,
    The problem was the file lock on the .m3u8 file. So I create a folder and fill it with the stream files. The I remove the folder.
    Here the final script:

    tail -f /opt/red5-server108m2/log/red5.log | awk ‘{if (/Stream start:/ “){

    a=” -nostats -loglevel 0 -i rtmp://xxx.xxx.xxx.xxx:1935/live/”;
    b=” -hls_time 2 -hls_list_size 5 -hls_wrap 10 -start_number 0 -f hls /home/tv/hls/”;
    c=”/home/tv/hls/”;
    system (“mkdir ” c $9);
    system (“/opt/ffmpeg/ffmpeg” a $9 b $9 “/” $9 “.m3u8 < /dev/null &");
    }
    else if (/Stream close:/)
    {
    system ("rm -rf " c $9);
    }
    }'

    Thank you again to point me in the right direction.

  3. Giuseppe says:

    Hi, problem solved. Thanks again.

    • Aaron Kili K says:

      @Guiseppe

      Thanks for letting us know, can you please point out how you solved it. That would be useful in the future to other users in case they encounter a similar issues.

      Many thanks…

  4. Giuseppe says:

    Hi Mr. Kili

    the problem seem to be the “else” statement. The script erase the .ts files but won’t erase the .m3u8 file. Going crazy!

    • Aaron Kili K says:

      Sorry that i did not get back, in the mean time, let me look into the script and follow up. I will get back to you as soon as i figure out and we shall share more. Hope that will help.

  5. Giuseppe says:

    Here the code:

    tail -f /opt/red5-server108m2/log/red5.log | awk ‘{if (/Stream start:/){
    print “start ” $9 ” streaming”;

    a=” -nostats -loglevel 0 -i rtmp://xxx.xxx.xxx.xxx:1935/live/”;
    b=” -hls_time 2 -hls_list_size 5 -hls_wrap 10 -start_number 0 -f hls /home/tv/hls/”;

    system (“/opt/ffmpeg/ffmpeg” a $9 b $9 “.m3u8 < /dev/null ");

    }
    else if (/Stream close:/)
    {

    print "stop " $9 " streaming";

    c="/home/tv/hls/";

    system ("rm " c $9 "*" );

    }
    }'

    But if I run:
    tail -f /opt/red5-server108m2/log/red5.log | awk '{if (/Stream start:/){
    print "start " $9 " streaming";

    }
    else if (/Stream close:/)
    {

    print "stop " $9 " streaming";

    }

    }'

    work nice.

    Thank you in advance

  6. Giuseppe says:

    Here the code:

    tail -f /opt/red5-server108m2/log/red5.log | awk ‘{if (/Stream start:/){
    print “start ” $9 ” streaming”;

    a=” -nostats -loglevel 0 -i rtmp://xxx.xxx.xxx.xxx:1935/live/”;
    b=” -hls_time 2 -hls_list_size 5 -hls_wrap 10 -start_number 0 -f hls /home/tv/hls/”;

    system (“/opt/ffmpeg/ffmpeg” a $9 b $9 “.m3u8 < /dev/null ");

    }
    else if (/Stream close:/)
    {

    print "stop " $9 " streaming";

    c="/home/tv/hls/";

    system ("rm " c $9 "*" );

    }
    }'

  7. Giuseppe says:

    Hi Mr. Kili, I was out of office, back again.

    I wrote a simple bash to start or stop ffmpeg recoding on parsing the red5 server log. All seem to work fine with one session but if I run a second session with another recoding something strange happen.

    Can I add here my code?

  8. Giuseppe says:

    Hello, first, thank you a lot for your job. I have some problem to get running a little script to run/stop a program parsing a log.

Leave a Reply to Giuseppe Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.