Filter Logs with Grep : linux Command Grep
Filter Logs with Grep : linux Command Grep
Let’s understand the use of grep is to extract useful information from system logs:
grep -Eoc "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 200" /srv/www/example.com/logs/access.log
In this command, grep filters an Apache access log for all lines that begin with an IP address, followed by a number of characters, a space and then the characters 200 (where 200 represents a successful HTTP connection).
The -c option outputs only a count of the number of matches.
To get the output of the IP address of the visitor and the path of the requested file for successful requests, omit the -c flag:
grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 200" /srv/www/example.com/logs/access.log
The curly brackets specify the number of instances of the pattern. {1,3} requires that the previous character occur at least once, but no more than three times.
The character class [0-9] will match against one or more numeric digits. You can also generate similar output but report on unsuccessful attempts to access content:
grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 404" /srv/www/example.com/logs/access.log
The following command generates a list of all IP addresses that have attempted to connect to your web server. Using the -o option, only the matching strings are sent to standard output.
This output is filtered through the utility uniq with the pipe operator (|) to filter out duplicate entries:
grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" /srv/www/example.com/logs/access.log | uniq
The next example uses an alternative pattern for matching an IP address in a different log. The following command searches the most recent /var/log/auth.log file for invalid login attempts:
grep -Eo "Invalid user.*([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log
You can split the above command into two layers to output a list of IP addresses with failed login attempts to your system:
grep "Invalid user" /var/log/auth.log | grep -Eo "([0-9]{1,3}\.){3}[0-9]{1,3}" | uniq
grep can filter the output of commands such as tail -F to provide real-time monitoring of specific log events:
tail ~/.procmail/procmail.log -F | grep "Subject"
In this case, tail follows the ~/procmail/procmail.log file. This output is passed to grep, which filters the stream and prints only lines that contain the string “Subject”.
