We have already seen in Chapter 11, Summarizing Logs with Awk, how we can create complex reports from large amounts of data mined from purely text files. Similarly, we can create extensive reports using the output from standard command-line tools, such as the lastlog tool. In itself lastlog can report the last login time for all users. Often though, we may wish to filter the output from lastlog. Perhaps you need to exclude the user accounts that have never been used to login to the system. It may also be irrelevant to report on root, as the account may be predominately used for sudo only and not used to record regularly for standard logins.
In working through this chapter, we will work both with lastlog and formatting of XML data. As this is the last chapter in which we investigate awk, we will configure record separators. We have already seen the use of field separators in awk but we can change the default record separator from a newline to something more specific to our need. More specifically, within this chapter we will cover:
- Using awk ranges to exclude data
- Conditions based on the number of fields in a row
- Manipulating the awk record separator to report on XML data
So far in this book, we have predominately looked at including data with ranges either for sed or for awk. With both of these tools, we can negate the range so that we exclude the specified rows. This suits the need that we have been using the output from lastlog. This will print all the login data for all the users, including accounts that have never been logged in. These accounts that have never been logged in might be service accounts or for new users that have not logged into the system so far.
If we look at the output from lastlog, when it is used without any options, we can begin to understand the issue. From the command line, we execute the command as a standard user. There is no requirement to run it as the root account. The command is shown in the following example:
$ lastlog
The partial output is shown within the following screenshot:
We can see, even from this limited output that we have a cluttered output due to the virtual noise being created by the accounts that have not logged in. It is possible to alleviate this to some degree using the lastlog options but it may not entirely resolve the issue. To demonstrate this, we can add an option to lastlog that only users accounts usually used by standard accounts should be included. This may vary on your system but on the sample CentOS 6 host that I am using, the first user will be UID 500.
If we use the lastlog -u 500-5000 command, we will only print data for those users with a UID within this range. On the simple demonstration system, we have just three user accounts for which the output is acceptable. However, we can understand that we may still have some clutter die to these accounts that have not yet been used. This is shown in the following screenshot:
In addition to the superfluous data being printed from the Never logged in accounts, we may only be interested in the Username and Latest fields. This is another reason to support the need to use awk as our data filter. In this way, we can provide both horizontal and vertical data filtering, rows, and columns.
To provide this filtering using awk, we will pipe the data from lastlog directly to awk. We will make use of a simple control file initially providing the horizontal filtering or reducing the rows that we see. First, the command pipeline will be as simple as the following command example:
$ lastlog | awk -f lastlog.awk
Of course, the complexity is abstracted from the command line and concealed within the control file that we use. Initially, the control file is kept simple and would read as follows:
!(/Never logged in/ || /^Username/ || /^root/) {
print $0;
}The range is setup as we have seen previously and precedes the main code block. Using the exclamation mark in front of the parentheses negates or reverses the selected range. The double vertical bar acts as a logical OR. We do not include lines that contain Never logged in, nor do we include lines that start with Username. This removes the header-line that is printed by lastlog. Finally, we exclude the root account from the display. This initiates the rows that we work with and the main code block will print those lines.
We may also want to count the number of rows that are returned by the filter. For example, using the internal NR variable will show all rows and not just the matched rows; for us to be able to report the number of users that have logged in, we must use our own variable. The following code will maintain the count within the variable that we name cnt. We increment this using the C style ++ for each iteration of the main code block. We can use the END code block to display the closing value of this variable:
!(/Never logged in/ || /^Username/ || /^root/) {
cnt++
print $0;
}
END {
print "========================"
print "Total Number of Users Processed: ", cnt
}We can see from the following code and output how this appears on my system:
From the display output, we can now see that we show only users that have logged in and in this case, it is just the single user. However, we may also decide that we want to abstract the data further and display only certain fields from the matched rows. This should be a simple task but it is complicated, as the number of fields will vary depending on how the login was executed.