Filter Logs with Grep : linux Command Grep

5 (100%) 4 votes

Filter Logs with Grep : linux Command Grep

Let’s understand the use of grep is to extract useful information from system logs:

grep -Eoc "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 200"  /srv/www/example.com/logs/access.log

 

In this command, grep filters an Apache access log for all lines that begin with an IP address, followed by a number of characters, a space and then the characters 200 (where 200 represents a successful HTTP connection).

The -c option outputs only a count of the number of matches.

To get the output of the IP address of the visitor and the path of the requested file for successful requests, omit the -c flag:

grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 200"  /srv/www/example.com/logs/access.log

The curly brackets specify the number of instances of the pattern. {1,3} requires that the previous character occur at least once, but no more than three times.

The character class [0-9] will match against one or more numeric digits. You can also generate similar output but report on unsuccessful attempts to access content:

grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}.* 404"  /srv/www/example.com/logs/access.log

The following command generates a list of all IP addresses that have attempted to connect to your web server. Using the -o option, only the matching strings are sent to standard output.

This output is filtered through the utility uniq with the pipe operator (|) to filter out duplicate entries:

grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" /srv/www/example.com/logs/access.log | uniq


Find-Failed-SSH-Logins
Find-Failed-SSH-Logins

The next example uses an alternative pattern for matching an IP address in a different log. The following command searches the most recent /var/log/auth.log file for invalid login attempts:

grep -Eo "Invalid user.*([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log

You can split the above command into two layers to output a list of IP addresses with failed login attempts to your system:

grep "Invalid user" /var/log/auth.log | grep -Eo "([0-9]{1,3}\.){3}[0-9]{1,3}" | uniq

grep can filter the output of commands such as tail -F to provide real-time monitoring of specific log events:

tail ~/.procmail/procmail.log -F | grep "Subject"

In this case, tail follows the ~/procmail/procmail.log file. This output is passed to grep, which filters the stream and prints only lines that contain the string “Subject”.

For any query or issue, feel free to discuss on http://discuss.eduguru.in
%d bloggers like this: