A number of Unix text-processing utilities let you search for, and in some cases change, text patterns rather than fixed strings. These utilities include the editing programs ed, ex, vi, and sed, the awk programming language, and the commands grep and egrep. Text patterns (formally called regular expressions) contain normal characters mixed with special characters (called metacharacters).
This section presents the following topics:
Filenames versus patterns
List of metacharacters available to each program
Description of metacharacters
Examples
Metacharacters used in pattern matching are different from metacharacters used for filename expansion. When you issue a command on the command line, special characters are seen first by the shell, then by the program; therefore, unquoted metacharacters are interpreted by the shell for filename expansion. The command:
$ grep [A-Z]* chap[12]
could, for example, be transformed by the shell into:
$ grep Array.c Bug.c Comp.c chap1 chap2
and would then try to find the pattern Array.c in files Bug.c, Comp.c, chap1, and chap2. To bypass the shell and pass the special characters to grep, use quotes:
$ grep "[A-Z]*" chap[12]
Double quotes suffice in most cases, but single quotes are the safest bet.
Note also that in pattern matching, ? matches zero or one instance of a regular expression; in filename expansion, ? matches a single character.
The characters in the following table have special meaning only in search patterns:
Many Unix systems allow the use of POSIX "character classes" within the square brackets that enclose a group of characters. These are typed enclosed in [: and :]. For example, [[:alnum:]] matches a single alphanumeric character.
Class | Characters Matched |
---|---|
alnum | Alphanumeric characters |
alpha | Alphabetic characters |
blank | Space or tab |
cntrl | Control characters |
digit | Decimal digits |
graph | Non-space characters |
lower | Lowercase characters |
Printable characters | |
space | White-space characters |
upper | Uppercase characters |
xdigit | Hexadecimal digits |
The characters in the following table have special meaning only in replacement patterns.
Character | Pattern |
---|---|
Turn off the special meaning of the following character. | |
n | Restore the text matched by the nth pattern previously saved by ( and ). n is a number from 1 to 9, with 1 starting on the left. |
& | Reuse the text matched by the search pattern as part of the replacement pattern. |
~ | Reuse the previous replacement pattern in the current replacement pattern. Must be the only character in the replacement pattern. (ex and vi). |
% | Reuse the previous replacement pattern in the current replacement pattern. Must be the only character in the replacement pattern. (ed). |
u | Convert first character of replacement pattern to uppercase. |
U | Convert entire replacement pattern to uppercase. |
l | Convert first character of replacement pattern to lowercase. |
L | Convert entire replacement pattern to lowercase. |
Some metacharacters are valid for one program but not for another. Those that are available to a Unix program are marked by a bullet () in the following table. (This table is correct for SVR4 and Solaris and most commerical Unix systems, but it's always a good idea to verify your system's behavior.) Items marked with a "P" are specified by POSIX; double check your system's version. Full descriptions were provided in the previous section.
Symbol | ed | exvi | sedgrep | awkegrep | Action |
---|---|---|---|---|---|
. | Match any character. | ||||
* | Match zero or more preceding. | ||||
^ | Match beginning of line/string. | ||||
$ | Match end of line/string. | ||||
Escape following character. | |||||
[ ] | Match one from a set. | ||||
( ) | Store pattern for later replay.[1] | ||||
n | Replay sub-pattern in match. | ||||
{ } | P | Match a range of instances. | |||
{ } | Match a range of instances. | ||||
Match word's beginning or end. | |||||
+ | Match one or more preceding. | ||||
? | Match zero or one preceding. | ||||
| | Separate choices to match. | ||||
( ) | Group expressions to match. |
[1] Stored sub-patterns can be "replayed" during matching. See the examples, below.
Note that in ed, ex, vi, and sed, you specify both a search pattern (on the left) and a replacement pattern (on the right). The metacharacters above are meaningful only in a search pattern.
In ed, ex, vi, and sed, the following metacharacters are valid only in a replacement pattern:
Symbol | ex | vi | sed | ed | Action |
---|---|---|---|---|---|
Escape following character. | |||||
n | Text matching pattern stored in ( ). | ||||
& | Text matching search pattern. | ||||
~ | Reuse previous replacement pattern. | ||||
% | Reuse previous replacement pattern. | ||||
u U | Change character(s) to uppercase. | ||||
l L | Change character(s) to lowercase. | ||||
E | Turn off previous U or L. | ||||
e | Turn off previous u or l. |
When used with grep or egrep, regular expressions should be surrounded by quotes. (If the pattern contains a $, you must use single quotes; e.g., 'pattern'.) When used with ed, ex, sed, and awk, regular expressions are usually surrounded by / although (except for awk), any delimiter works. Here are some example patterns.
Pattern | What Does It Match? |
---|---|
bag | The string bag. |
^bag | bag at the beginning of the line. |
bag$ | bag at the end of the line. |
^bag$ | bag as the only word on the line. |
[Bb]ag | Bag or bag. |
b[aeiou]g | Second letter is a vowel. |
b[^aeiou]g | Second letter is a consonant (or uppercase or symbol). |
b.g | Second letter is any character. |
^…$ | Any line containing exactly three characters. |
^. | Any line that begins with a dot. |
^.[a-z][a-z] | Same, followed by two lowercase letters (e.g., troff requests). |
^.[a-z]{2} | Same as previous, ed, grep and sed only. |
^[^.] | Any line that doesn't begin with a dot. |
bugs* | bug, bugs, bugss, etc. |
"word" | A word in quotes. |
"*word"* | A word, with or without quotes. |
[A-Z][A-Z]* | One or more uppercase letters. |
[A-Z]+ | Same as previous, egrep or awk only. |
[[:upper:]]+ | Same as previous, POSIX egrep or awk. |
[A-Z].* | An uppercase letter, followed by zero or more characters. |
[A-Z]* | Zero or more uppercase letters. |
[a-zA-Z] | Any letter, either lower- or uppercase. |
[^0-9A-Za-z] | Any symbol or space (not a letter or a number). |
[^[:alnum:]] | Same, using POSIX character class. |
egrep or awk pattern | What Does It Match? |
---|---|
[567] | One of the numbers 5, 6, or 7. |
five|six|seven | One of the words five, six, or seven. |
80[2-4]?86 | 8086, 80286, 80386, or 80486. |
80[2-4]?86|(Pentium(-II)?) | 8086, 80286, 80386, 80486, Pentium, or Pentium-II. |
compan(y|ies) | company or companies. |
ex or vi pattern | What Does It Match? |
---|---|
<the | Words like theater, there or the. |
the> | Words like breathe, seethe or the. |
<the> | The word the. |
ed, sed, or grep pattern | What Does It Match? |
---|---|
0{5,} | Five or more zeros in a row. |
[0-9]{3}-[0-9]{2}-[0-9]{4} | U.S. Social Security number (nnn-nn-nnnn). |
(why).*1 | A line with two occurrences of why. |
([[:alpha:]_][[:alnum:]_.]*) = 1; | C/C++ simple assignment statements. |
The following examples show the metacharacters available to sed or ex. Note that ex commands begin with a colon. A space is marked by a ; a tab is marked by a .
Finally, some sed examples for transposing words. A simple transposition of two words might look like this:
s/die or do/do or die/Transpose words
The real trick is to use hold buffers to transpose variable patterns. For example:
s/([Dd]ie) or ([Dd]o)/2 or 1/Transpose, using hold buffers
3.141.30.162