How can I recursively grep through sub-directories ? | Unix Tutorial |Linux / UNIX Recursively Search All Files For A String

Question :How do i recursively grep (search text files) in unix?

For example, if i want to find the word "hello" in a unix directory /tmp and all its subdirectories in the .log files?  i want the file name, file path, and line number with context.

Solution:


find /tmp -name \*.log -exec grep pall {} \; -exec ls -l {} \;

The find command you probably know. It searches for files or directorys where you want him to search and to what pattern to match.

So :
"find /tmp" says only start searching in /tmp
"-name \*.log" says to look for all files or directory ending with "log" the \ before the star says to not look for a file or directory named "*log" but to make it a pattern
"-exec grep pall {} \;" says that the result of find in /tmp with the pattern ending on "log" should do the following : "grep hello /tmp/script" so {} \; is replaced with the current result. It's hard to explain the syntax, I think you should just take this as it is and copy the syntax when you need it
"-exec ls -l {} \;" has the same syntax and executes the ls -l command for the current result. If you don't do this last -exec you don't know what file he grep the text from.

Awk Basic Structure-Unix Tutorials

The essential organization of an AWK program follows the form:

pattern { action }

The pattern specifies when the action is performed. Like most UNIX utilities, AWK is line oriented. That is, the pattern specifies a test that is performed with each line read as input. If the condition is true, then the action is taken. The default pattern is something that matches every line. This is the blank or null pattern. Two other important patterns are specified by the keywords "BEGIN" and "END." As you might expect, these two words specify actions to be taken before any lines are read, and after the last line is read. The AWK program below:

BEGIN { print "START" }
      { print         }
END   { print "STOP"  }

adds one line before and one line after the input file. This isn't very useful, but with a simple change, we can make this into a typical AWK program:

BEGIN { print "File\tOwner"," } { print $8, "\t", $3} END { print " - DONE -" }

I'll improve the script in the next sections, but we'll call it "FileOwner." But let's not put it into a script or file yet. I will cover that part in a bit. Hang on and follow with me so you get the flavor of AWK.
The characters "\t" Indicates a tab character so the output lines up on even boundries. The "$8" and "$3" have a meaning similar to a shell script. Instead of the eighth and third argument, they mean the eighth and third field of the input line. You can think of a field as a column, and the action you specify operates on each line or row read in.
There are two differences between AWK and a shell processing the characters within double quotes. AWK understands special characters follow the "\" character like "t". The Bourne and C UNIX shells do not. Also, unlike the shell (and PERL) AWK does not evaluate variables within strings. To explain, the second line could not be written like this:

{print "$8\t$3" }

That example would print "$8 $3." Inside the quotes, the dollar sign is not a special character. Outside, it corresponds to a field. What do I mean by the third and eight field? Consider the Solaris "/usr/bin/ls -l" command, which has eight columns of information. The System V version (Similar to the Linux version), "/usr/5bin/ls -l," has 9 columns. The third column is the owner, and the eighth (or nineth) column in the name of the file. This AWK program can be used to process the output of the "ls -l" command, printing out the filename, then the owner, for each file. I'll show you how.
Update: On a linux system, change "$8" to "$9".
One more point about the use of a dollar sign. In scripting languages like Perl and the various shells, a dollar sign means the word following is the name of the variable. Awk is different. The dollar sign means that we are refering to a field or column in the current line. When switching between Perl and AWK you must remener that "$" has a different meaning. So the following piece of code prints two "fields" to standard out. The first field printed is the number "5", the second is the fifth field (or column) on the input line.
BEGIN { x=5 }
{ print x, $x}

How to Search & Replace Multiple Files in UNIX

Step 1 Open a blank file in any text editor.

Step 2 Place the following script into the the text editor:

#!/bin/bash
#
echo "Type your find string followed by [Enter]:"
read fstring
echo "Type your replace string followed by [Enter]:"
read rstring
for y in `ls *`;
do sed "s/$fstring/$rstring/g" $y > temp; mv temp $y;
done

Step 3Save the script with the name "findrep.sh" in the same directory as the files you want to manipulate.

Step 4Open a Terminal Window. The terminal window will be found in the operating system's main "Application" menu, under either "System Tools" or "Utilities." You will be presented with a command prompt where you will type the following commands.

Step 5Type the command "mv " to move into the directory with the script. For example, the command would be "mv Files/" if the script was held in the "Files" directory in your home directory.

Step 6Type the command "chmod +x findrep.sh" to make the file executable.

Step 7Type the command "./findrep.sh" to execute the script.

How to Remove Duplicate Lines in Unix

File Can be Sorted Alphabetically
Step 1Make a backup of the file you are working with:
cp document.txt document.txt.bkup

Step 2Issue the command:
sort -u document.txt
This command will sort the file and remove all duplicate lines.

Step 3Remove the blank lines with the command:
uniq document.txt

File Cannot be Sorted Alphabetically
Step 1Make a backup file:
cp document.txt document.txt.bkup

Step 2Issue the following awk command:
awk '!($0 in a) {a[$0];print}' document.txt > unique.txt
Your unique entries will be found in the file named unique.txt

Step 3Rename the text file with the unique lines.
cp unique.txt document.txt
This puts the unique entries back into the original file.

Combine Two Files and Find the Duplicate Lines
Step 1Make a backup file:
cp document.txt document.txt.bkup

Step 2Issue the command:
cat doc1.txt doc2.txt > combine.txt
This command combines doc1.txt and doc2.txt into the file combine.txt

Step 3Remove the duplicate lines.
Use either the sort and uniq commands or the awk command specified above.

Awk Tutorial in Unix,Examples of Using AWK

The Unix operating system is one of the most diverse and powerful operating systems ever devised. Created in the late 1960s, Unix provides a stable platform for many business operations globally to this day. Unix administrators use a robust set of commands to tune and support the Unix operating system, and AWK is a part of the language that processes files of text. In essence, AWK allows one to grab certain bits of data from output in the Unix operating system.
Examples of Using AWK
We will continue to look at this command line "# more samplefile | awk '{print $1}' and discuss how AWK can pull data from the "samplefile" file.

If you were to read "samplefile" on its own, the data output might look like this:

# more samplefile

Device File ALPA Tgt Lun Port CU:LDev Type Serial#
=======================================================
/dev/hai/567hyu 00 00 00 CL5A 07:6c OPEN-V 00030052
/dev/hai/c37t8d0 00 08 00 CL6A 07:6c OPEN-V 00030052

But let's say you only want the first field of data from this output. You could use the following AWK command to accomplish this.

# more samplefile | awk '{print $1}' | more

Device
======================================================
/dev/hai/567hyu
/dev/hai/c37t8d0

You could even take it a step further. Let us say that you only want the number that is listed in the output and not the /dev/hai/ part. In that case you would use the operator "awk --F" to act as a field separator. The great thing is that you can decide the field separator. Here is the command you would use.

# more samplefile | awk '{print $1}' | awk --F / '{print $4}' | more
567hyu
c37t8d0

In this case you are using AWK twice. The first time you are grabbing the first field of data, and the second time you are taking that first field, dividing it at the "/", and then pulling out the fourth field in this new group, which leaves you with only the numbers you wanted.

This was just a short example of how AWK comes in handy so that Unix Administrators can quickly filter and grab out only the most important bits of data.