X-Git-Url: http://git.shadowcat.co.uk/gitweb/gitweb.cgi?a=blobdiff_plain;f=pod%2Fperlfaq4.pod;h=22ea1ae5814fb300ba5b04db36d4c2761341caf6;hb=58103a2e295c15d87c7ce0bd8dd83d7e110adac4;hp=de7feeea5719f7212931f423a9b9bedc791d99ed;hpb=571e049f835793c9e2ebba49d438be94a271f3ee;p=p5sagit%2Fp5-mst-13.2.git diff --git a/pod/perlfaq4.pod b/pod/perlfaq4.pod index de7feee..22ea1ae 100644 --- a/pod/perlfaq4.pod +++ b/pod/perlfaq4.pod @@ -1,6 +1,6 @@ =head1 NAME -perlfaq4 - Data Manipulation ($Revision: 1.61 $, $Date: 2005/03/11 16:27:53 $) +perlfaq4 - Data Manipulation ($Revision: 1.73 $, $Date: 2005/12/31 00:54:37 $) =head1 DESCRIPTION @@ -349,16 +349,16 @@ Computers are good at being predictable and bad at being random (despite appearances caused by bugs in your programs :-). see the F article in the "Far More Than You Ever Wanted To Know" collection in http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz , courtesy of -Tom Phoenix, talks more about this. John von Neumann said, ``Anyone +Tom Phoenix, talks more about this. John von Neumann said, "Anyone who attempts to generate random numbers by deterministic means is, of -course, living in a state of sin.'' +course, living in a state of sin." If you want numbers that are more random than C with C provides, you should also check out the Math::TrulyRandom module from CPAN. It uses the imperfections in your system's timer to generate random numbers, but this takes quite a while. If you want a better pseudorandom generator than comes with your operating system, look at -``Numerical Recipes in C'' at http://www.nr.com/ . +"Numerical Recipes in C" at http://www.nr.com/ . =head2 How do I get a random number between X and Y? @@ -392,7 +392,7 @@ integers (inclusive), For example: C. The localtime function returns the day of the year. Without an argument localtime uses the current time. - $day_of_year = (localtime)[7]; + $day_of_year = (localtime)[7]; The POSIX module can also format a date as the day of the year or week of the year. @@ -406,10 +406,10 @@ a time in epoch seconds for the argument to localtime. use POSIX qw/strftime/; use Time::Local; - my $week_of_year = strftime "%W", + my $week_of_year = strftime "%W", localtime( timelocal( 0, 0, 0, 18, 11, 1987 ) ); -The Date::Calc module provides two functions for to calculate these. +The Date::Calc module provides two functions to calculate these. use Date::Calc; my $day_of_year = Day_of_Year( 1987, 12, 18 ); @@ -422,7 +422,7 @@ Use the following simple functions: sub get_century { return int((((localtime(shift || time))[5] + 1999))/100); } - + sub get_millennium { return 1+int((((localtime(shift || time))[5] + 1899))/1000); } @@ -436,15 +436,12 @@ reliably determine the current century or millennium. =head2 How can I compare two dates and find the difference? -If you're storing your dates as epoch seconds then simply subtract one -from the other. If you've got a structured date (distinct year, day, -month, hour, minute, seconds values), then for reasons of accessibility, -simplicity, and efficiency, merely use either timelocal or timegm (from -the Time::Local module in the standard distribution) to reduce structured -dates to epoch seconds. However, if you don't know the precise format of -your dates, then you should probably use either of the Date::Manip and -Date::Calc modules from CPAN before you go hacking up your own parsing -routine to handle arbitrary date formats. +(contributed by brian d foy) + +You could just store all your dates as a number and then subtract. Life +isn't always that simple though. If you want to work with formatted +dates, the Date::Manip, Date::Calc, or DateTime modules can help you. + =head2 How can I take a string and turn it into epoch seconds? @@ -481,60 +478,30 @@ Julian day) =head2 How do I find yesterday's date? -If you only need to find the date (and not the same time), you -can use the Date::Calc module. - - use Date::Calc qw(Today Add_Delta_Days); +(contributed by brian d foy) - my @date = Add_Delta_Days( Today(), -1 ); +Use one of the Date modules. The C module makes it simple, and +give you the same time of day, only the day before. - print "@date\n"; + use DateTime; -Most people try to use the time rather than the calendar to -figure out dates, but that assumes that your days are -twenty-four hours each. For most people, there are two days -a year when they aren't: the switch to and from summer time -throws this off. Russ Allbery offers this solution. - - sub yesterday { - my $now = defined $_[0] ? $_[0] : time; - my $then = $now - 60 * 60 * 24; - my $ndst = (localtime $now)[8] > 0; - my $tdst = (localtime $then)[8] > 0; - $then - ($tdst - $ndst) * 60 * 60; - } + my $yesterday = DateTime->now->subtract( days => 1 ); -Should give you "this time yesterday" in seconds since epoch relative to -the first argument or the current time if no argument is given and -suitable for passing to localtime or whatever else you need to do with -it. $ndst is whether we're currently in daylight savings time; $tdst is -whether the point 24 hours ago was in daylight savings time. If $tdst -and $ndst are the same, a boundary wasn't crossed, and the correction -will subtract 0. If $tdst is 1 and $ndst is 0, subtract an hour more -from yesterday's time since we gained an extra hour while going off -daylight savings time. If $tdst is 0 and $ndst is 1, subtract a -negative hour (add an hour) to yesterday's time since we lost an hour. + print "Yesterday was $yesterday\n"; -All of this is because during those days when one switches off or onto -DST, a "day" isn't 24 hours long; it's either 23 or 25. +You can also use the C module using its Today_and_Now +function. -The explicit settings of $ndst and $tdst are necessary because localtime -only says it returns the system tm struct, and the system tm struct at -least on Solaris doesn't guarantee any particular positive value (like, -say, 1) for isdst, just a positive value. And that value can -potentially be negative, if DST information isn't available (this sub -just treats those cases like no DST). + use Date::Calc qw( Today_and_Now Add_Delta_DHMS ); -Note that between 2am and 3am on the day after the time zone switches -off daylight savings time, the exact hour of "yesterday" corresponding -to the current hour is not clearly defined. Note also that if used -between 2am and 3am the day after the change to daylight savings time, -the result will be between 3am and 4am of the previous day; it's -arguable whether this is correct. - -This sub does not attempt to deal with leap seconds (most things don't). + my @date_time = Add_Delta_DHMS( Today_and_Now(), -1, 0, 0, 0 ); + print "@date\n"; +Most people try to use the time rather than the calendar to figure out +dates, but that assumes that days are twenty-four hours each. For +most people, there are two days a year when they aren't: the switch to +and from summer time throws this off. Let the modules do the work. =head2 Does Perl have a Year 2000 problem? Is Perl Y2K compliant? @@ -562,21 +529,28 @@ C<$timestamp = gmtime(1005613200)> sets $timestamp to "Tue Nov 13 01:00:00 That doesn't mean that Perl can't be used to create non-Y2K compliant programs. It can. But so can your pencil. It's the fault of the user, -not the language. At the risk of inflaming the NRA: ``Perl doesn't -break Y2K, people do.'' See http://www.perl.org/about/y2k.html for +not the language. At the risk of inflaming the NRA: "Perl doesn't +break Y2K, people do." See http://www.perl.org/about/y2k.html for a longer exposition. =head1 Data: Strings =head2 How do I validate input? -The answer to this question is usually a regular expression, perhaps -with auxiliary logic. See the more specific questions (numbers, mail -addresses, etc.) for details. +(contributed by brian d foy) + +There are many ways to ensure that values are what you expect or +want to accept. Besides the specific examples that we cover in the +perlfaq, you can also look at the modules with "Assert" and "Validate" +in their names, along with other modules such as C. + +Some modules have validation for particular types of input, such +as C, C, C, +and C. =head2 How do I unescape a string? -It depends just what you mean by ``escape''. URL escapes are dealt +It depends just what you mean by "escape". URL escapes are dealt with in L. Shell escapes with the backslash (C<\>) character are removed with @@ -586,21 +560,69 @@ This won't expand C<"\n"> or C<"\t"> or any other special escapes. =head2 How do I remove consecutive pairs of characters? -To turn C<"abbcccd"> into C<"abccd">: +(contributed by brian d foy) + +You can use the substitution operator to find pairs of characters (or +runs of characters) and replace them with a single instance. In this +substitution, we find a character in C<(.)>. The memory parentheses +store the matched character in the back-reference C<\1> and we use +that to require that the same thing immediately follow it. We replace +that part of the string with the character in C<$1>. - s/(.)\1/$1/g; # add /s to include newlines + s/(.)\1/$1/g; -Here's a solution that turns "abbcccd" to "abcd": +We can also use the transliteration operator, C. In this +example, the search list side of our C contains nothing, but +the C option complements that so it contains everything. The +replacement list also contains nothing, so the transliteration is +almost a no-op since it won't do any replacements (or more exactly, +replace the character with itself). However, the C option squashes +duplicated and consecutive characters in the string so a character +does not show up next to itself - y///cs; # y == tr, but shorter :-) + my $str = 'Haarlem'; # in the Netherlands + $str =~ tr///cs; # Now Harlem, like in New York =head2 How do I expand function calls in a string? -This is documented in L. In general, this is fraught with -quoting and readability problems, but it is possible. To interpolate -a subroutine call (in list context) into a string: +(contributed by brian d foy) + +This is documented in L, and although it's not the easiest +thing to read, it does work. In each of these examples, we call the +function inside the braces used to dereference a reference. If we +have a more than one return value, we can construct and dereference an +anonymous array. In this case, we call the function in list context. + + print "The time values are @{ [localtime] }.\n"; + +If we want to call the function in scalar context, we have to do a bit +more work. We can really have any code we like inside the braces, so +we simply have to end with the scalar reference, although how you do +that is up to you, and you can use code inside the braces. + + print "The time is ${\(scalar localtime)}.\n" - print "My sub returned @{[mysub(1,2,3)]} that time.\n"; + print "The time is ${ my $x = localtime; \$x }.\n"; + +If your function already returns a reference, you don't need to create +the reference yourself. + + sub timestamp { my $t = localtime; \$t } + + print "The time is ${ timestamp() }.\n"; + +The C module can also do a lot of magic for you. You can +specify a variable name, in this case C, to set up a tied hash that +does the interpolation for you. It has several other methods to do this +as well. + + use Interpolation E => 'eval'; + print "The time values are $E{localtime()}.\n"; + +In most cases, it is probably easier to simply use string concatenation, +which also forces scalar context. + + print "The time is " . localtime . ".\n"; =head2 How do I find matching/nesting anything? @@ -609,15 +631,16 @@ matter how complicated. To find something between two single characters, a pattern like C will get the intervening bits in $1. For multiple ones, then something more like C would be needed. But none of these deals with -nested patterns. For balanced expressions using C<(>, C<{>, C<[> -or C<< < >> as delimiters, use the CPAN module Regexp::Common, or see -L. For other cases, you'll have to write a parser. +nested patterns. For balanced expressions using C<(>, C<{>, C<[> or +C<< < >> as delimiters, use the CPAN module Regexp::Common, or see +L. For other cases, you'll have to write a +parser. If you are serious about writing a parser, there are a number of modules or oddities that will make your life a lot easier. There are the CPAN modules Parse::RecDescent, Parse::Yapp, and Text::Balanced; -and the byacc program. Starting from perl 5.8 the Text::Balanced -is part of the standard distribution. +and the byacc program. Starting from perl 5.8 the Text::Balanced is +part of the standard distribution. One simple destructive, inside-out approach that you might try is to pull out the smallest nesting parts one at a time: @@ -841,34 +864,52 @@ There's also a Text::CSV (Comma-Separated Values) module on CPAN. =head2 How do I strip blank space from the beginning/end of a string? -Although the simplest approach would seem to be +(contributed by brian d foy) - $string =~ s/^\s*(.*?)\s*$/$1/; +A substitution can do this for you. For a single line, you want to +replace all the leading or trailing whitespace with nothing. You +can do that with a pair of substitutions. -not only is this unnecessarily slow and destructive, it also fails with -embedded newlines. It is much faster to do this operation in two steps: + s/^\s+//; + s/\s+$//; - $string =~ s/^\s+//; - $string =~ s/\s+$//; +You can also write that as a single substitution, although it turns +out the combined statement is slower than the separate ones. That +might not matter to you, though. -Or more nicely written as: + s/^\s+|\s+$//g; - for ($string) { - s/^\s+//; - s/\s+$//; - } +In this regular expression, the alternation matches either at the +beginning or the end of the string since the anchors have a lower +precedence than the alternation. With the C flag, the substitution +makes all possible matches, so it gets both. Remember, the trailing +newline matches the C<\s+>, and the C<$> anchor can match to the +physical end of the string, so the newline disappears too. Just add +the newline to the output, which has the added benefit of preserving +"blank" (consisting entirely of whitespace) lines which the C<^\s+> +would remove all by itself. -This idiom takes advantage of the C loop's aliasing -behavior to factor out common code. You can do this -on several strings at once, or arrays, or even the -values of a hash if you use a slice: + while( <> ) + { + s/^\s+|\s+$//g; + print "$_\n"; + } - # trim whitespace in the scalar, the array, - # and all the values in the hash - foreach ($scalar, @array, @hash{keys %hash}) { - s/^\s+//; - s/\s+$//; - } +For a multi-line string, you can apply the regular expression +to each logical line in the string by adding the C flag (for +"multi-line"). With the C flag, the C<$> matches I an +embedded newline, so it doesn't remove it. It still removes the +newline at the end of the string. + + $string =~ s/^\s+|\s+$//gm; + +Remember that lines consisting entirely of whitespace will disappear, +since the first part of the alternation can match the entire string +and replace it with nothing. If need to keep embedded blank lines, +you have to do a little more work. Instead of matching any whitespace +(since that includes a newline), just match the other whitespace. + + $string =~ s/^[\t\f ]+|[\t\f ]+$//mg; =head2 How do I pad a string with blanks or pad a number with zeroes? @@ -1136,59 +1177,51 @@ matters. =head2 How can I remove duplicate elements from a list or array? -There are several possible ways, depending on whether the array is -ordered and whether you wish to preserve the ordering. - -=over 4 - -=item a) - -If @in is sorted, and you want @out to be sorted: -(this assumes all true values in the array) - - $prev = "not equal to $in[0]"; - @out = grep($_ ne $prev && ($prev = $_, 1), @in); - -This is nice in that it doesn't use much extra memory, simulating -uniq(1)'s behavior of removing only adjacent duplicates. The ", 1" -guarantees that the expression is true (so that grep picks it up) -even if the $_ is 0, "", or undef. - -=item b) - -If you don't know whether @in is sorted: - - undef %saw; - @out = grep(!$saw{$_}++, @in); - -=item c) - -Like (b), but @in contains only small integers: +(contributed by brian d foy) - @out = grep(!$saw[$_]++, @in); +Use a hash. When you think the words "unique" or "duplicated", think +"hash keys". -=item d) +If you don't care about the order of the elements, you could just +create the hash then extract the keys. It's not important how you +create that hash: just that you use C to get the unique +elements. -A way to do (b) without any loops or greps: + my %hash = map { $_, 1 } @array; + # or a hash slice: @hash{ @array } = (); + # or a foreach: $hash{$_} = 1 foreach ( @array ); - undef %saw; - @saw{@in} = (); - @out = sort keys %saw; # remove sort if undesired + my @unique = keys %hash; -=item e) +You can also go through each element and skip the ones you've seen +before. Use a hash to keep track. The first time the loop sees an +element, that element has no key in C<%Seen>. The C statement +creates the key and immediately uses its value, which is C, so +the loop continues to the C and increments the value for that +key. The next time the loop sees that same element, its key exists in +the hash I the value for that key is true (since it's not 0 or +undef), so the next skips that iteration and the loop goes to the next +element. -Like (d), but @in contains only small positive integers: + my @unique = (); + my %seen = (); - undef @ary; - @ary[@in] = @in; - @out = grep {defined} @ary; + foreach my $elem ( @array ) + { + next if $seen{ $elem }++; + push @unique, $elem; + } -=back +You can write this more briefly using a grep, which does the +same thing. -But perhaps you should have been using a hash all along, eh? + my %seen = (); + my @unique = grep { ! $seen{ $_ }++ } @array; =head2 How can I tell whether a certain element is contained in a list or array? +(portions of this answer contributed by Anno Siegel) + Hearing the word "in" is an Idication that you probably should have used a hash, not a list or array, to store your data. Hashes are designed to answer this question quickly and efficiently. Arrays aren't. @@ -1224,27 +1257,34 @@ quite a lot of space by using bit strings instead: Now check whether C is true for some C<$n>. -Please do not use +These methods guarantee fast individual tests but require a re-organization +of the original list or array. They only pay off if you have to test +multiple values against the same array. - ($is_there) = grep $_ eq $whatever, @array; +If you are testing only once, the standard module List::Util exports +the function C for this purpose. It works by stopping once it +finds the element. It's written in C for speed, and its Perl equivalant +looks like this subroutine: -or worse yet + sub first (&@) { + my $code = shift; + foreach (@_) { + return $_ if &{$code}(); + } + undef; + } - ($is_there) = grep /$whatever/, @array; +If speed is of little concern, the common idiom uses grep in scalar context +(which returns the number of items that passed its condition) to traverse the +entire list. This does have the benefit of telling you how many matches it +found, though. -These are slow (checks every element even if the first matches), -inefficient (same reason), and potentially buggy (what if there are -regex characters in $whatever?). If you're only testing once, then -use: + my $is_there = grep $_ eq $whatever, @array; - $is_there = 0; - foreach $elt (@array) { - if ($elt eq $elt_to_find) { - $is_there = 1; - last; - } - } - if ($is_there) { ... } +If you want to actually extract the matching elements, simply use grep in +list context. + + my @matches = grep $_ eq $whatever, @array; =head2 How do I compute the difference of two arrays? How do I compute the intersection of two arrays? @@ -1325,9 +1365,9 @@ If you cannot use List::Util, you can make your own loop to do the same thing. Once you find the element, you stop the loop with last. my $found; - foreach my $element ( @array ) + foreach ( @array ) { - if( /Perl/ ) { $found = $element; last } + if( /Perl/ ) { $found = $_; last } } If you want the array index, you can iterate through the indices @@ -1335,15 +1375,15 @@ and check the array element at each index until you find one that satisfies the condition. my( $found, $index ) = ( undef, -1 ); - for( $i = 0; $i < @array; $i++ ) - { - if( $array[$i] =~ /Perl/ ) - { - $found = $array[$i]; - $index = $i; - last; - } - } + for( $i = 0; $i < @array; $i++ ) + { + if( $array[$i] =~ /Perl/ ) + { + $found = $array[$i]; + $index = $i; + last; + } + } =head2 How do I handle linked lists? @@ -1416,7 +1456,7 @@ If not, you can use a Fisher-Yates shuffle. sub fisher_yates_shuffle { my $deck = shift; # $deck is a reference to an array my $i = @$deck; - while ($i--) { + while (--$i) { my $j = int rand ($i+1); @$deck[$i,$j] = @$deck[$j,$i]; } @@ -1452,15 +1492,15 @@ this until you have rather largish arrays. Use C/C: for (@lines) { - s/foo/bar/; # change that word - y/XZ/ZX/; # swap those letters + s/foo/bar/; # change that word + tr/XZ/ZX/; # swap those letters } Here's another; let's compute spherical volumes: for (@volumes = @radii) { # @volumes has changed parts - $_ **= 3; - $_ *= (4/3) * 3.14159; # this will be constant folded + $_ **= 3; + $_ *= (4/3) * 3.14159; # this will be constant folded } which can also be done with map() which is made to transform @@ -1474,7 +1514,7 @@ the values are not copied, so if you modify $orbit (in this case), you modify the value. for $orbit ( values %orbits ) { - ($orbit **= 3) *= (4/3) * 3.14159; + ($orbit **= 3) *= (4/3) * 3.14159; } Prior to perl 5.6 C returned copies of the values, @@ -1754,27 +1794,50 @@ such as each(). =head2 How do I sort a hash (optionally by value instead of key)? -Internally, hashes are stored in a way that prevents you from imposing -an order on key-value pairs. Instead, you have to sort a list of the -keys or values: +(contributed by brian d foy) + +To sort a hash, start with the keys. In this example, we give the list of +keys to the sort function which then compares them ASCIIbetically (which +might be affected by your locale settings). The output list has the keys +in ASCIIbetical order. Once we have the keys, we can go through them to +create a report which lists the keys in ASCIIbetical order. + + my @keys = sort { $a cmp $b } keys %hash; - @keys = sort keys %hash; # sorted by key - @keys = sort { - $hash{$a} cmp $hash{$b} - } keys %hash; # and by value + foreach my $key ( @keys ) + { + printf "%-20s %6d\n", $key, $hash{$value}; + } -Here we'll do a reverse numeric sort by value, and if two keys are -identical, sort by length of key, or if that fails, by straight ASCII -comparison of the keys (well, possibly modified by your locale--see -L). +We could get more fancy in the C block though. Instead of +comparing the keys, we can compute a value with them and use that +value as the comparison. - @keys = sort { - $hash{$b} <=> $hash{$a} - || - length($b) <=> length($a) - || - $a cmp $b - } keys %hash; +For instance, to make our report order case-insensitive, we use +the C<\L> sequence in a double-quoted string to make everything +lowercase. The C block then compares the lowercased +values to determine in which order to put the keys. + + my @keys = sort { "\L$a" cmp "\L$b" } keys %hash; + +Note: if the computation is expensive or the hash has many elements, +you may want to look at the Schwartzian Transform to cache the +computation results. + +If we want to sort by the hash value instead, we use the hash key +to look it up. We still get out a list of keys, but this time they +are ordered by their value. + + my @keys = sort { $hash{$a} <=> $hash{$b} } keys %hash; + +From there we can get more complex. If the hash values are the same, +we can provide a secondary sort on the hash key. + + my @keys = sort { + $hash{$a} <=> $hash{$b} + or + "\L$a" cmp "\L$b" + } keys %hash; =head2 How can I always keep my hash sorted? @@ -1959,8 +2022,18 @@ in L. =head2 How can I use a reference as a hash key? -You can't do this directly, but you could use the standard Tie::RefHash -module distributed with Perl. +(contributed by brian d foy) + +Hash keys are strings, so you can't really use a reference as the key. +When you try to do that, perl turns the reference into its stringified +form (for instance, C). From there you can't get back +the reference from the stringified form, at least without doing some +extra work on your own. Also remember that hash keys must be unique, but +two different variables can store the same reference (and those variables +can change later). + +The Tie::RefHash module, which is distributed with perl, might be what +you want. It handles that extra work. =head1 Data: Misc @@ -2011,7 +2084,7 @@ function. Its semantics are somewhat cumbersome, so here's a C wrapper function for more convenient access. This function takes a string and returns the number it found, or C for input that isn't a C float. The C function is a front end to C -if you just want to say, ``Is this a float?'' +if you just want to say, "Is this a float?" sub getnum { use POSIX qw(strtod); @@ -2082,7 +2155,7 @@ the PDL module from CPAN instead--it makes number-crunching easy. =head1 AUTHOR AND COPYRIGHT -Copyright (c) 1997-2005 Tom Christiansen, Nathan Torkington, and +Copyright (c) 1997-2006 Tom Christiansen, Nathan Torkington, and other authors as noted. All rights reserved. This documentation is free; you can redistribute it and/or modify it