=head1 NAME
-perlfaq6 - Regexes ($Revision: 1.27 $, $Date: 1999/05/23 16:08:30 $)
+perlfaq6 - Regular Expressions ($Revision: 1.12 $, $Date: 2002/06/01 22:31:09 $)
=head1 DESCRIPTION
littered with answers involving regular expressions. For example,
decoding a URL and checking whether something is a number are handled
with regular expressions, but those answers are found elsewhere in
-this document (in the section on Data and the Networking one on
-networking, to be precise).
+this document (in L<perlfaq9>: ``How do I decode or create those %-encodings
+on the web'' and L<perlfaq4>: ``How do I determine whether a scalar is
+a number/whole/integer/float'', to be precise).
=head2 How can I hope to use regular expressions without creating illegible and unmaintainable code?
=head2 I'm having trouble matching over more than one line. What's wrong?
-Either you don't have more than one line in the string you're looking at
-(probably), or else you aren't using the correct modifier(s) on your
-pattern (possibly).
+Either you don't have more than one line in the string you're looking
+at (probably), or else you aren't using the correct modifier(s) on
+your pattern (possibly).
There are many ways to get multiline data into a string. If you want
it to happen automatically while reading input, you'll want to set $/
undef $/; # read in whole file, not just one line or paragraph
while ( <> ) {
- while ( /START(.*?)END/sm ) { # /s makes . cross line boundaries
+ while ( /START(.*?)END/sgm ) { # /s makes . cross line boundaries
print "$1\n";
}
}
close FH;
## Get a read/write filehandle to it.
- $fh = new FileHandle "+<file";
+ $fh = new IO::File "+<file";
## Attach it to a "stream" object.
use Net::Telnet;
$file->waitfor('/second line\n/');
print $file->getline;
-=head2 How do I substitute case insensitively on the LHS, but preserving case on the RHS?
+=head2 How do I substitute case insensitively on the LHS while preserving case on the RHS?
Here's a lovely Perlish solution by Larry Rosler. It exploits
properties of bitwise xor on ASCII strings.
$old = 'test';
$new = 'success';
- s{(\Q$old\E}
+ s{(\Q$old\E)}
{ uc $new | (uc $1 ^ $1) .
(uc(substr $1, -1) ^ substr $1, -1) x
(length($new) - length $1)
print;
-And here it is as a subroutine, modelled after the above:
+And here it is as a subroutine, modeled after the above:
sub preserve_case($$) {
my ($old, $new) = @_;
this is a SUcCESS case
+As an alternative, to keep the case of the replacement word if it is
+longer than the original, you can use this code, by Jeff Pinyan:
+
+ sub preserve_case {
+ my ($from, $to) = @_;
+ my ($lf, $lt) = map length, @_;
+
+ if ($lt < $lf) { $from = substr $from, 0, $lt }
+ else { $from .= substr $to, $lf }
+
+ return uc $to | ($from ^ uc $from);
+ }
+
+This changes the sentence to "this is a SUcCess case."
+
Just to show that C programmers can write C in any programming language,
if you prefer a more C-like solution, the following script makes the
substitution have the same case, letter by letter, as the original.
=head2 What is C</o> really for?
Using a variable in a regular expression match forces a re-evaluation
-(and perhaps recompilation) each time through. The C</o> modifier
-locks in the regex the first time it's used. This always happens in a
-constant regular expression, and in fact, the pattern was compiled
-into the internal format at the same time your entire program was.
+(and perhaps recompilation) each time the regular expression is
+encountered. The C</o> modifier locks in the regex the first time
+it's used. This always happens in a constant regular expression, and
+in fact, the pattern was compiled into the internal format at the same
+time your entire program was.
Use of C</o> is irrelevant unless variable interpolation is used in
the pattern, and if so, the regex engine will neither know nor care
=head2 Can I use Perl regular expressions to match balanced text?
-Although Perl regular expressions are more powerful than "mathematical"
-regular expressions, because they feature conveniences like backreferences
-(C<\1> and its ilk), they still aren't powerful enough -- with
-the possible exception of bizarre and experimental features in the
-development-track releases of Perl. You still need to use non-regex
-techniques to parse balanced text, such as the text enclosed between
-matching parentheses or braces, for example.
+Historically, Perl regular expressions were not capable of matching
+balanced text. As of more recent versions of perl including 5.6.1
+experimental features have been added that make it possible to do this.
+Look at the documentation for the (??{ }) construct in recent perlre manual
+pages to see an example of matching balanced parentheses. Be sure to take
+special notice of the warnings present in the manual before making use
+of this feature.
+
+CPAN contains many modules that can be useful for matching text
+depending on the context. Damian Conway provides some useful
+patterns in Regexp::Common. The module Text::Balanced provides a
+general solution to this problem.
+
+One of the common applications of balanced text matching is working
+with XML and HTML. There are many modules available that support
+these needs. Two examples are HTML::Parser and XML::Parser. There
+are many others.
An elaborate subroutine (for 7-bit ASCII only) to pull out balanced
and possibly nested single chars, like C<`> and C<'>, C<{> and C<}>,
or C<(> and C<)> can be found in
-http://www.perl.com/CPAN/authors/id/TOMC/scripts/pull_quotes.gz .
+http://www.cpan.org/authors/id/TOMC/scripts/pull_quotes.gz .
-The C::Scan module from CPAN contains such subs for internal usage,
+The C::Scan module from CPAN also contains such subs for internal use,
but they are undocumented.
=head2 What does it mean that regexes are greedy? How can I get around it?
control on to whatever is next in line, like you would if you were
playing hot potato.
-=head2 How do I process each word on each line?
+=head2 How do I process each word on each line?
Use the split function:
Note that this isn't really a word in the English sense; it's just
chunks of consecutive non-whitespace characters.
-To work with only alphanumeric sequences, you might consider
+To work with only alphanumeric sequences (including underscores), you
+might consider
while (<>) {
foreach $word (m/(\w+)/g) {
print "$count $line";
}
-If you want these output in a sorted order, see the section on Hashes.
+If you want these output in a sorted order, see L<perlfaq4>: ``How do I
+sort a hash (optionally by value instead of key)?''.
=head2 How can I do approximate matching?
=head2 Why don't word-boundary searches with C<\b> work for me?
-Two common misconceptions are that C<\b> is a synonym for C<\s+>, and
+Two common misconceptions are that C<\b> is a synonym for C<\s+> and
that it's the edge between whitespace characters and non-whitespace
characters. Neither is correct. C<\b> is the place between a C<\w>
character and a C<\W> character (that is, C<\b> is the edge of a
=head2 Why does using $&, $`, or $' slow my program down?
-Because once Perl sees that you need one of these variables anywhere in
-the program, it has to provide them on each and every pattern match.
+Once Perl sees that you need one of these variables anywhere in
+the program, it provides them on each and every pattern match.
The same mechanism that handles these provides for the use of $1, $2,
etc., so you pay the same price for each regex that contains capturing
-parentheses. But if you never use $&, etc., in your script, then regexes
+parentheses. If you never use $&, etc., in your script, then regexes
I<without> capturing parentheses won't be penalized. So avoid $&, $',
and $` if you can, but if you can't, once you've used them at all, use
them at will because you've already paid the price. Remember that some
=head2 What good is C<\G> in a regular expression?
-The notation C<\G> is used in a match or substitution in conjunction the
-C</g> modifier (and ignored if there's no C</g>) to anchor the regular
-expression to the point just past where the last match occurred, i.e. the
-pos() point. A failed match resets the position of C<\G> unless the
-C</c> modifier is in effect.
+The notation C<\G> is used in a match or substitution in conjunction with
+the C</g> modifier to anchor the regular expression to the point just past
+where the last match occurred, i.e. the pos() point. A failed match resets
+the position of C<\G> unless the C</c> modifier is in effect. C<\G> can be
+used in a match without the C</g> modifier; it acts the same (i.e. still
+anchors at the pos() point) but of course only matches once and does not
+update pos(), as non-C</g> expressions never do. C<\G> in an expression
+applied to a target string that has never been matched against a C</g>
+expression before or has had its pos() reset is functionally equivalent to
+C<\A>, which matches at the beginning of the string.
For example, suppose you had a line of text quoted in standard mail
-and Usenet notation, (that is, with leading C<E<gt>> characters), and
-you want change each leading C<E<gt>> into a corresponding C<:>. You
+and Usenet notation, (that is, with leading C<< > >> characters), and
+you want change each leading C<< > >> into a corresponding C<:>. You
could do so in this way:
s/^(>+)/':' x length($1)/gem;
}
}
-But then you lose the vertical alignment of the regular expressions.
+but then you lose the vertical alignment of the regular expressions.
=head2 Are Perl regexes DFAs or NFAs? Are they POSIX compliant?
=head2 What's wrong with using grep or map in a void context?
-Both grep and map build a return list, regardless of their context.
-This means you're making Perl go to the trouble of building up a
-return list that you then just ignore. That's no way to treat a
-programming language, you insensitive scoundrel!
+The problem is that both grep and map build a return list,
+regardless of the context. This means you're making Perl go
+to the trouble of building a list that you then just throw away.
+If the list is large, you waste both time and space. If your
+intent is to iterate over the list then use a for loop for this
+purpose.
=head2 How can I match strings with multibyte characters?
-This is hard, and there's no good way. Perl does not directly support
-wide characters. It pretends that a byte and a character are
-synonymous. The following set of approaches was offered by Jeffrey
-Friedl, whose article in issue #5 of The Perl Journal talks about this
-very matter.
+Starting from Perl 5.6 Perl has had some level of multibyte character
+support. Perl 5.8 or later is recommended. Supported multibyte
+character repertoires include Unicode, and legacy encodings
+through the Encode module. See L<perluniintro>, L<perlunicode>,
+and L<Encode>.
+
+If you are stuck with older Perls, you can do Unicode with the
+C<Unicode::String> module, and character conversions using the
+C<Unicode::Map8> and C<Unicode::Map> modules. If you are using
+Japanese encodings, you might try using the jperl 5.005_03.
+
+Finally, the following set of approaches was offered by Jeffrey
+Friedl, whose article in issue #5 of The Perl Journal talks about
+this very matter.
Let's suppose you have some weird Martian encoding where pairs of
ASCII uppercase letters encode single Martian letters (i.e. the two
chomp($pattern = <STDIN>);
if ($line =~ /$pattern/) { }
-Or, since you have no guarantee that your user entered
+Alternatively, since you have no guarantee that your user entered
a valid regular expression, trap the exception this way:
if (eval { $line =~ /$pattern/ }) { }
-But if all you really want to search for a string, not a pattern,
+If all you really want to search for a string, not a pattern,
then you should either use the index() function, which is made for
string searching, or if you can't be disabused of using a pattern
match on a non-pattern, then be sure to use C<\Q>...C<\E>, documented
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-1999 Tom Christiansen and Nathan Torkington.
+Copyright (c) 1997-2002 Tom Christiansen and Nathan Torkington.
All rights reserved.
-When included as part of the Standard Version of Perl, or as part of
-its complete documentation whether printed or otherwise, this work
-may be distributed only under the terms of Perl's Artistic License.
-Any distribution of this file or derivatives thereof I<outside>
-of that package require that special arrangements be made with
-copyright holder.
+This documentation is free; you can redistribute it and/or modify it
+under the same terms as Perl itself.
Irrespective of its distribution, all code examples in this file
are hereby placed into the public domain. You are permitted and