extra code in pp_concat, Take 2
[p5sagit/p5-mst-13.2.git] / pod / perlfaq5.pod
CommitLineData
68dc0745 1=head1 NAME
2
7678cced 3perlfaq5 - Files and Formats ($Revision: 1.35 $, $Date: 2005/01/21 12:26:11 $)
68dc0745 4
5=head1 DESCRIPTION
6
7This section deals with I/O and the "f" issues: filehandles, flushing,
8formats, and footers.
9
5a964f20 10=head2 How do I flush/unbuffer an output filehandle? Why must I do this?
68dc0745 11
c90536be 12Perl does not support truly unbuffered output (except
13insofar as you can C<syswrite(OUT, $char, 1)>), although it
14does support is "command buffering", in which a physical
15write is performed after every output command.
16
17The C standard I/O library (stdio) normally buffers
18characters sent to devices so that there isn't a system call
19for each byte. In most stdio implementations, the type of
20output buffering and the size of the buffer varies according
21to the type of device. Perl's print() and write() functions
22normally buffer output, while syswrite() bypasses buffering
23all together.
24
25If you want your output to be sent immediately when you
26execute print() or write() (for instance, for some network
27protocols), you must set the handle's autoflush flag. This
28flag is the Perl variable $| and when it is set to a true
29value, Perl will flush the handle's buffer after each
30print() or write(). Setting $| affects buffering only for
31the currently selected default file handle. You choose this
32handle with the one argument select() call (see
197aec24 33L<perlvar/$E<verbar>> and L<perlfunc/select>).
c90536be 34
35Use select() to choose the desired handle, then set its
36per-filehandle variables.
5a964f20 37
38 $old_fh = select(OUTPUT_HANDLE);
39 $| = 1;
40 select($old_fh);
41
c90536be 42Some idioms can handle this in a single statement:
5a964f20 43
44 select((select(OUTPUT_HANDLE), $| = 1)[0]);
818c4caa 45
c90536be 46 $| = 1, select $_ for select OUTPUT_HANDLE;
5a964f20 47
c90536be 48Some modules offer object-oriented access to handles and their
49variables, although they may be overkill if this is the only
50thing you do with them. You can use IO::Handle:
68dc0745 51
52 use IO::Handle;
53 open(DEV, ">/dev/printer"); # but is this?
54 DEV->autoflush(1);
55
c90536be 56or IO::Socket:
68dc0745 57
58 use IO::Socket; # this one is kinda a pipe?
c90536be 59 my $sock = IO::Socket::INET->new( 'www.example.com:80' ) ;
68dc0745 60
61 $sock->autoflush();
68dc0745 62
63=head2 How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file?
64
1f089b22 65Use the Tie::File module, which is included in the standard
66distribution since Perl 5.8.0.
68dc0745 67
68=head2 How do I count the number of lines in a file?
69
70One fairly efficient way is to count newlines in the file. The
71following program uses a feature of tr///, as documented in L<perlop>.
72If your text file doesn't end with a newline, then it's not really a
73proper text file, so this may report one fewer line than you expect.
74
75 $lines = 0;
76 open(FILE, $filename) or die "Can't open `$filename': $!";
77 while (sysread FILE, $buffer, 4096) {
78 $lines += ($buffer =~ tr/\n//);
79 }
80 close FILE;
81
5a964f20 82This assumes no funny games with newline translations.
83
4750257b 84=head2 How can I use Perl's C<-i> option from within a program?
85
86C<-i> sets the value of Perl's C<$^I> variable, which in turn affects
87the behavior of C<< <> >>; see L<perlrun> for more details. By
88modifying the appropriate variables directly, you can get the same
89behavior within a larger program. For example:
90
91 # ...
92 {
93 local($^I, @ARGV) = ('.orig', glob("*.c"));
94 while (<>) {
95 if ($. == 1) {
96 print "This line should appear at the top of each file\n";
97 }
98 s/\b(p)earl\b/${1}erl/i; # Correct typos, preserving case
99 print;
100 close ARGV if eof; # Reset $.
101 }
102 }
103 # $^I and @ARGV return to their old values here
104
105This block modifies all the C<.c> files in the current directory,
106leaving a backup of the original data from each file in a new
107C<.c.orig> file.
108
7678cced 109=head2 How can I copy a file?
110
111(contributed by brian d foy)
112
113Use the File::Copy module. It comes with Perl and can do a
114true copy across file systems, and it does its magic in
115a portable fashion.
116
117 use File::Copy;
118
119 copy( $original, $new_copy ) or die "Copy failed: $!";
120
121If you can't use File::Copy, you'll have to do the work yourself:
122open the original file, open the destination file, then print
123to the destination file as you read the original.
124
68dc0745 125=head2 How do I make a temporary file name?
126
7678cced 127If you don't need to know the name of the file, you can use C<open()>
128with C<undef> in place of the file name. The C<open()> function
129creates an anonymous temporary file.
130
131 open my $tmp, '+>', undef or die $!;
132
133Otherwise, you can use the File::Temp module.
68dc0745 134
197aec24 135 use File::Temp qw/ tempfile tempdir /;
a6dd486b 136
16394a69 137 $dir = tempdir( CLEANUP => 1 );
138 ($fh, $filename) = tempfile( DIR => $dir );
5a964f20 139
16394a69 140 # or if you don't need to know the filename
5a964f20 141
16394a69 142 $fh = tempfile( DIR => $dir );
5a964f20 143
16394a69 144The File::Temp has been a standard module since Perl 5.6.1. If you
145don't have a modern enough Perl installed, use the C<new_tmpfile>
146class method from the IO::File module to get a filehandle opened for
147reading and writing. Use it if you don't need to know the file's name:
5a964f20 148
16394a69 149 use IO::File;
150 $fh = IO::File->new_tmpfile()
151 or die "Unable to make new temporary file: $!";
5a964f20 152
a6dd486b 153If you're committed to creating a temporary file by hand, use the
154process ID and/or the current time-value. If you need to have many
155temporary files in one process, use a counter:
5a964f20 156
157 BEGIN {
68dc0745 158 use Fcntl;
16394a69 159 my $temp_dir = -d '/tmp' ? '/tmp' : $ENV{TMPDIR} || $ENV{TEMP};
68dc0745 160 my $base_name = sprintf("%s/%d-%d-0000", $temp_dir, $$, time());
161 sub temp_file {
5a964f20 162 local *FH;
68dc0745 163 my $count = 0;
5a964f20 164 until (defined(fileno(FH)) || $count++ > 100) {
68dc0745 165 $base_name =~ s/-(\d+)$/"-" . (1 + $1)/e;
2359510d 166 # O_EXCL is required for security reasons.
5a964f20 167 sysopen(FH, $base_name, O_WRONLY|O_EXCL|O_CREAT);
68dc0745 168 }
5a964f20 169 if (defined(fileno(FH))
170 return (*FH, $base_name);
68dc0745 171 } else {
172 return ();
173 }
174 }
175 }
176
68dc0745 177=head2 How can I manipulate fixed-record-length files?
178
793f5136 179The most efficient way is using L<pack()|perlfunc/"pack"> and
180L<unpack()|perlfunc/"unpack">. This is faster than using
181L<substr()|perlfunc/"substr"> when taking many, many strings. It is
182slower for just a few.
5a964f20 183
184Here is a sample chunk of code to break up and put back together again
185some fixed-format input lines, in this case from the output of a normal,
186Berkeley-style ps:
68dc0745 187
188 # sample input line:
189 # 15158 p5 T 0:00 perl /home/tchrist/scripts/now-what
793f5136 190 my $PS_T = 'A6 A4 A7 A5 A*';
191 open my $ps, '-|', 'ps';
192 print scalar <$ps>;
193 my @fields = qw( pid tt stat time command );
194 while (<$ps>) {
195 my %process;
196 @process{@fields} = unpack($PS_T, $_);
197 for my $field ( @fields ) {
198 print "$field: <$process{$field}>\n";
68dc0745 199 }
793f5136 200 print 'line=', pack($PS_T, @process{@fields} ), "\n";
68dc0745 201 }
202
793f5136 203We've used a hash slice in order to easily handle the fields of each row.
204Storing the keys in an array means it's easy to operate on them as a
205group or loop over them with for. It also avoids polluting the program
206with global variables and using symbolic references.
5a964f20 207
68dc0745 208=head2 How can I make a filehandle local to a subroutine? How do I pass filehandles between subroutines? How do I make an array of filehandles?
209
c90536be 210As of perl5.6, open() autovivifies file and directory handles
211as references if you pass it an uninitialized scalar variable.
212You can then pass these references just like any other scalar,
213and use them in the place of named handles.
68dc0745 214
c90536be 215 open my $fh, $file_name;
818c4caa 216
c90536be 217 open local $fh, $file_name;
818c4caa 218
c90536be 219 print $fh "Hello World!\n";
818c4caa 220
c90536be 221 process_file( $fh );
68dc0745 222
c90536be 223Before perl5.6, you had to deal with various typeglob idioms
224which you may see in older code.
68dc0745 225
c90536be 226 open FILE, "> $filename";
227 process_typeglob( *FILE );
228 process_reference( \*FILE );
818c4caa 229
c90536be 230 sub process_typeglob { local *FH = shift; print FH "Typeglob!" }
231 sub process_reference { local $fh = shift; print $fh "Reference!" }
5a964f20 232
c90536be 233If you want to create many anonymous handles, you should
234check out the Symbol or IO::Handle modules.
5a964f20 235
236=head2 How can I use a filehandle indirectly?
237
238An indirect filehandle is using something other than a symbol
239in a place that a filehandle is expected. Here are ways
a6dd486b 240to get indirect filehandles:
5a964f20 241
242 $fh = SOME_FH; # bareword is strict-subs hostile
243 $fh = "SOME_FH"; # strict-refs hostile; same package only
244 $fh = *SOME_FH; # typeglob
245 $fh = \*SOME_FH; # ref to typeglob (bless-able)
246 $fh = *SOME_FH{IO}; # blessed IO::Handle from *SOME_FH typeglob
247
c90536be 248Or, you can use the C<new> method from one of the IO::* modules to
5a964f20 249create an anonymous filehandle, store that in a scalar variable,
250and use it as though it were a normal filehandle.
251
5a964f20 252 use IO::Handle; # 5.004 or higher
253 $fh = IO::Handle->new();
254
255Then use any of those as you would a normal filehandle. Anywhere that
256Perl is expecting a filehandle, an indirect filehandle may be used
257instead. An indirect filehandle is just a scalar variable that contains
368c9434 258a filehandle. Functions like C<print>, C<open>, C<seek>, or
c90536be 259the C<< <FH> >> diamond operator will accept either a named filehandle
5a964f20 260or a scalar variable containing one:
261
262 ($ifh, $ofh, $efh) = (*STDIN, *STDOUT, *STDERR);
263 print $ofh "Type it: ";
264 $got = <$ifh>
265 print $efh "What was that: $got";
266
368c9434 267If you're passing a filehandle to a function, you can write
5a964f20 268the function in two ways:
269
270 sub accept_fh {
271 my $fh = shift;
272 print $fh "Sending to indirect filehandle\n";
46fc3d4c 273 }
274
5a964f20 275Or it can localize a typeglob and use the filehandle directly:
46fc3d4c 276
5a964f20 277 sub accept_fh {
278 local *FH = shift;
279 print FH "Sending to localized filehandle\n";
46fc3d4c 280 }
281
5a964f20 282Both styles work with either objects or typeglobs of real filehandles.
283(They might also work with strings under some circumstances, but this
284is risky.)
285
286 accept_fh(*STDOUT);
287 accept_fh($handle);
288
289In the examples above, we assigned the filehandle to a scalar variable
a6dd486b 290before using it. That is because only simple scalar variables, not
291expressions or subscripts of hashes or arrays, can be used with
292built-ins like C<print>, C<printf>, or the diamond operator. Using
8305e449 293something other than a simple scalar variable as a filehandle is
5a964f20 294illegal and won't even compile:
295
296 @fd = (*STDIN, *STDOUT, *STDERR);
297 print $fd[1] "Type it: "; # WRONG
298 $got = <$fd[0]> # WRONG
299 print $fd[2] "What was that: $got"; # WRONG
300
301With C<print> and C<printf>, you get around this by using a block and
302an expression where you would place the filehandle:
303
304 print { $fd[1] } "funny stuff\n";
305 printf { $fd[1] } "Pity the poor %x.\n", 3_735_928_559;
306 # Pity the poor deadbeef.
307
308That block is a proper block like any other, so you can put more
309complicated code there. This sends the message out to one of two places:
310
197aec24 311 $ok = -x "/bin/cat";
5a964f20 312 print { $ok ? $fd[1] : $fd[2] } "cat stat $ok\n";
197aec24 313 print { $fd[ 1+ ($ok || 0) ] } "cat stat $ok\n";
5a964f20 314
315This approach of treating C<print> and C<printf> like object methods
316calls doesn't work for the diamond operator. That's because it's a
317real operator, not just a function with a comma-less argument. Assuming
318you've been storing typeglobs in your structure as we did above, you
c90536be 319can use the built-in function named C<readline> to read a record just
c47ff5f1 320as C<< <> >> does. Given the initialization shown above for @fd, this
c90536be 321would work, but only because readline() requires a typeglob. It doesn't
5a964f20 322work with objects or strings, which might be a bug we haven't fixed yet.
323
324 $got = readline($fd[0]);
325
326Let it be noted that the flakiness of indirect filehandles is not
327related to whether they're strings, typeglobs, objects, or anything else.
328It's the syntax of the fundamental operators. Playing the object
329game doesn't help you at all here.
46fc3d4c 330
68dc0745 331=head2 How can I set up a footer format to be used with write()?
332
54310121 333There's no builtin way to do this, but L<perlform> has a couple of
68dc0745 334techniques to make it possible for the intrepid hacker.
335
336=head2 How can I write() into a string?
337
65acb1b1 338See L<perlform/"Accessing Formatting Internals"> for an swrite() function.
68dc0745 339
340=head2 How can I output my numbers with commas added?
341
49d635f9 342This subroutine will add commas to your number:
343
344 sub commify {
345 local $_ = shift;
346 1 while s/^([-+]?\d+)(\d{3})/$1,$2/;
347 return $_;
348 }
349
350This regex from Benjamin Goldberg will add commas to numbers:
68dc0745 351
881bdbd4 352 s/(^[-+]?\d+?(?=(?>(?:\d{3})+)(?!\d))|\G\d{3}(?=\d))/$1,/g;
68dc0745 353
49d635f9 354It is easier to see with comments:
68dc0745 355
881bdbd4 356 s/(
357 ^[-+]? # beginning of number.
7678cced 358 \d+? # first digits before first comma
881bdbd4 359 (?= # followed by, (but not included in the match) :
360 (?>(?:\d{3})+) # some positive multiple of three digits.
361 (?!\d) # an *exact* multiple, not x * 3 + 1 or whatever.
362 )
363 | # or:
364 \G\d{3} # after the last group, get three digits
365 (?=\d) # but they have to have more digits after them.
366 )/$1,/xg;
46fc3d4c 367
68dc0745 368=head2 How can I translate tildes (~) in a filename?
369
575cc754 370Use the <> (glob()) operator, documented in L<perlfunc>. Older
371versions of Perl require that you have a shell installed that groks
372tildes. Recent perl versions have this feature built in. The
d6260402 373File::KGlob module (available from CPAN) gives more portable glob
575cc754 374functionality.
68dc0745 375
376Within Perl, you may use this directly:
377
378 $filename =~ s{
379 ^ ~ # find a leading tilde
380 ( # save this in $1
381 [^/] # a non-slash character
382 * # repeated 0 or more times (0 means me)
383 )
384 }{
385 $1
386 ? (getpwnam($1))[7]
387 : ( $ENV{HOME} || $ENV{LOGDIR} )
388 }ex;
389
5a964f20 390=head2 How come when I open a file read-write it wipes it out?
68dc0745 391
392Because you're using something like this, which truncates the file and
393I<then> gives you read-write access:
394
5a964f20 395 open(FH, "+> /path/name"); # WRONG (almost always)
68dc0745 396
397Whoops. You should instead use this, which will fail if the file
197aec24 398doesn't exist.
d92eb7b0 399
400 open(FH, "+< /path/name"); # open for update
401
c47ff5f1 402Using ">" always clobbers or creates. Using "<" never does
d92eb7b0 403either. The "+" doesn't change this.
68dc0745 404
5a964f20 405Here are examples of many kinds of file opens. Those using sysopen()
406all assume
68dc0745 407
5a964f20 408 use Fcntl;
68dc0745 409
5a964f20 410To open file for reading:
68dc0745 411
5a964f20 412 open(FH, "< $path") || die $!;
413 sysopen(FH, $path, O_RDONLY) || die $!;
414
415To open file for writing, create new file if needed or else truncate old file:
416
417 open(FH, "> $path") || die $!;
418 sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT) || die $!;
419 sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT, 0666) || die $!;
420
421To open file for writing, create new file, file must not exist:
422
423 sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT) || die $!;
424 sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT, 0666) || die $!;
425
426To open file for appending, create if necessary:
427
428 open(FH, ">> $path") || die $!;
429 sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT) || die $!;
430 sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT, 0666) || die $!;
431
432To open file for appending, file must exist:
433
434 sysopen(FH, $path, O_WRONLY|O_APPEND) || die $!;
435
436To open file for update, file must exist:
437
438 open(FH, "+< $path") || die $!;
439 sysopen(FH, $path, O_RDWR) || die $!;
440
441To open file for update, create file if necessary:
442
443 sysopen(FH, $path, O_RDWR|O_CREAT) || die $!;
444 sysopen(FH, $path, O_RDWR|O_CREAT, 0666) || die $!;
445
446To open file for update, file must not exist:
447
448 sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT) || die $!;
449 sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT, 0666) || die $!;
450
451To open a file without blocking, creating if necessary:
452
2359510d 453 sysopen(FH, "/foo/somefile", O_WRONLY|O_NDELAY|O_CREAT)
454 or die "can't open /foo/somefile: $!":
5a964f20 455
456Be warned that neither creation nor deletion of files is guaranteed to
457be an atomic operation over NFS. That is, two processes might both
a6dd486b 458successfully create or unlink the same file! Therefore O_EXCL
459isn't as exclusive as you might wish.
68dc0745 460
87275199 461See also the new L<perlopentut> if you have it (new for 5.6).
65acb1b1 462
04d666b1 463=head2 Why do I sometimes get an "Argument list too long" when I use E<lt>*E<gt>?
68dc0745 464
c47ff5f1 465The C<< <> >> operator performs a globbing operation (see above).
3a4b19e4 466In Perl versions earlier than v5.6.0, the internal glob() operator forks
467csh(1) to do the actual glob expansion, but
68dc0745 468csh can't handle more than 127 items and so gives the error message
469C<Argument list too long>. People who installed tcsh as csh won't
470have this problem, but their users may be surprised by it.
471
3a4b19e4 472To get around this, either upgrade to Perl v5.6.0 or later, do the glob
d6260402 473yourself with readdir() and patterns, or use a module like File::KGlob,
3a4b19e4 474one that doesn't use the shell to do globbing.
68dc0745 475
476=head2 Is there a leak/bug in glob()?
477
478Due to the current implementation on some operating systems, when you
479use the glob() function or its angle-bracket alias in a scalar
a6dd486b 480context, you may cause a memory leak and/or unpredictable behavior. It's
68dc0745 481best therefore to use glob() only in list context.
482
c47ff5f1 483=head2 How can I open a file with a leading ">" or trailing blanks?
68dc0745 484
485Normally perl ignores trailing blanks in filenames, and interprets
486certain leading characters (or a trailing "|") to mean something
197aec24 487special.
68dc0745 488
881bdbd4 489The three argument form of open() lets you specify the mode
490separately from the filename. The open() function treats
197aec24 491special mode characters and whitespace in the filename as
881bdbd4 492literals
65acb1b1 493
881bdbd4 494 open FILE, "<", " file "; # filename is " file "
495 open FILE, ">", ">file"; # filename is ">file"
65acb1b1 496
881bdbd4 497It may be a lot clearer to use sysopen(), though:
65acb1b1 498
499 use Fcntl;
500 $badpath = "<<<something really wicked ";
a6dd486b 501 sysopen (FH, $badpath, O_WRONLY | O_CREAT | O_TRUNC)
65acb1b1 502 or die "can't open $badpath: $!";
68dc0745 503
68dc0745 504=head2 How can I reliably rename a file?
505
49d635f9 506If your operating system supports a proper mv(1) utility or its
507functional equivalent, this works:
68dc0745 508
509 rename($old, $new) or system("mv", $old, $new);
510
d2321c93 511It may be more portable to use the File::Copy module instead.
512You just copy to the new file to the new name (checking return
513values), then delete the old one. This isn't really the same
514semantically as a rename(), which preserves meta-information like
68dc0745 515permissions, timestamps, inode info, etc.
516
d2321c93 517Newer versions of File::Copy export a move() function.
5a964f20 518
68dc0745 519=head2 How can I lock a file?
520
54310121 521Perl's builtin flock() function (see L<perlfunc> for details) will call
68dc0745 522flock(2) if that exists, fcntl(2) if it doesn't (on perl version 5.004 and
523later), and lockf(3) if neither of the two previous system calls exists.
524On some systems, it may even use a different form of native locking.
525Here are some gotchas with Perl's flock():
526
527=over 4
528
529=item 1
530
531Produces a fatal error if none of the three system calls (or their
532close equivalent) exists.
533
534=item 2
535
536lockf(3) does not provide shared locking, and requires that the
537filehandle be open for writing (or appending, or read/writing).
538
539=item 3
540
d92eb7b0 541Some versions of flock() can't lock files over a network (e.g. on NFS file
542systems), so you'd need to force the use of fcntl(2) when you build Perl.
a6dd486b 543But even this is dubious at best. See the flock entry of L<perlfunc>
d92eb7b0 544and the F<INSTALL> file in the source distribution for information on
545building Perl to do this.
546
547Two potentially non-obvious but traditional flock semantics are that
a6dd486b 548it waits indefinitely until the lock is granted, and that its locks are
d92eb7b0 549I<merely advisory>. Such discretionary locks are more flexible, but
550offer fewer guarantees. This means that files locked with flock() may
551be modified by programs that do not also use flock(). Cars that stop
552for red lights get on well with each other, but not with cars that don't
553stop for red lights. See the perlport manpage, your port's specific
554documentation, or your system-specific local manpages for details. It's
555best to assume traditional behavior if you're writing portable programs.
a6dd486b 556(If you're not, you should as always feel perfectly free to write
d92eb7b0 557for your own system's idiosyncrasies (sometimes called "features").
558Slavish adherence to portability concerns shouldn't get in the way of
559your getting your job done.)
68dc0745 560
197aec24 561For more information on file locking, see also
13a2d996 562L<perlopentut/"File Locking"> if you have it (new for 5.6).
65acb1b1 563
68dc0745 564=back
565
04d666b1 566=head2 Why can't I just open(FH, "E<gt>file.lock")?
68dc0745 567
568A common bit of code B<NOT TO USE> is this:
569
570 sleep(3) while -e "file.lock"; # PLEASE DO NOT USE
571 open(LCK, "> file.lock"); # THIS BROKEN CODE
572
573This is a classic race condition: you take two steps to do something
574which must be done in one. That's why computer hardware provides an
575atomic test-and-set instruction. In theory, this "ought" to work:
576
5a964f20 577 sysopen(FH, "file.lock", O_WRONLY|O_EXCL|O_CREAT)
9b55d3ab 578 or die "can't open file.lock: $!";
68dc0745 579
580except that lamentably, file creation (and deletion) is not atomic
581over NFS, so this won't work (at least, not every time) over the net.
65acb1b1 582Various schemes involving link() have been suggested, but
46fc3d4c 583these tend to involve busy-wait, which is also subdesirable.
68dc0745 584
fc36a67e 585=head2 I still don't get locking. I just want to increment the number in the file. How can I do this?
68dc0745 586
46fc3d4c 587Didn't anyone ever tell you web-page hit counters were useless?
5a964f20 588They don't count number of hits, they're a waste of time, and they serve
a6dd486b 589only to stroke the writer's vanity. It's better to pick a random number;
590they're more realistic.
68dc0745 591
5a964f20 592Anyway, this is what you can do if you can't help yourself.
68dc0745 593
e2c57c3e 594 use Fcntl qw(:DEFAULT :flock);
5a964f20 595 sysopen(FH, "numfile", O_RDWR|O_CREAT) or die "can't open numfile: $!";
65acb1b1 596 flock(FH, LOCK_EX) or die "can't flock numfile: $!";
68dc0745 597 $num = <FH> || 0;
598 seek(FH, 0, 0) or die "can't rewind numfile: $!";
599 truncate(FH, 0) or die "can't truncate numfile: $!";
600 (print FH $num+1, "\n") or die "can't write numfile: $!";
68dc0745 601 close FH or die "can't close numfile: $!";
602
46fc3d4c 603Here's a much better web-page hit counter:
68dc0745 604
605 $hits = int( (time() - 850_000_000) / rand(1_000) );
606
607If the count doesn't impress your friends, then the code might. :-)
608
f52f3be2 609=head2 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking?
05caf3a7 610
611If you are on a system that correctly implements flock() and you use the
612example appending code from "perldoc -f flock" everything will be OK
613even if the OS you are on doesn't implement append mode correctly (if
614such a system exists.) So if you are happy to restrict yourself to OSs
615that implement flock() (and that's not really much of a restriction)
616then that is what you should do.
617
618If you know you are only going to use a system that does correctly
619implement appending (i.e. not Win32) then you can omit the seek() from
620the above code.
621
622If you know you are only writing code to run on an OS and filesystem that
623does implement append mode correctly (a local filesystem on a modern
624Unix for example), and you keep the file in block-buffered mode and you
625write less than one buffer-full of output between each manual flushing
8305e449 626of the buffer then each bufferload is almost guaranteed to be written to
05caf3a7 627the end of the file in one chunk without getting intermingled with
628anyone else's output. You can also use the syswrite() function which is
629simply a wrapper around your systems write(2) system call.
630
631There is still a small theoretical chance that a signal will interrupt
632the system level write() operation before completion. There is also a
633possibility that some STDIO implementations may call multiple system
634level write()s even if the buffer was empty to start. There may be some
635systems where this probability is reduced to zero.
636
68dc0745 637=head2 How do I randomly update a binary file?
638
639If you're just trying to patch a binary, in many cases something as
640simple as this works:
641
642 perl -i -pe 's{window manager}{window mangler}g' /usr/bin/emacs
643
644However, if you have fixed sized records, then you might do something more
645like this:
646
647 $RECSIZE = 220; # size of record, in bytes
648 $recno = 37; # which record to update
649 open(FH, "+<somewhere") || die "can't update somewhere: $!";
650 seek(FH, $recno * $RECSIZE, 0);
651 read(FH, $record, $RECSIZE) == $RECSIZE || die "can't read record $recno: $!";
652 # munge the record
65acb1b1 653 seek(FH, -$RECSIZE, 1);
68dc0745 654 print FH $record;
655 close FH;
656
657Locking and error checking are left as an exercise for the reader.
a6dd486b 658Don't forget them or you'll be quite sorry.
68dc0745 659
68dc0745 660=head2 How do I get a file's timestamp in perl?
661
881bdbd4 662If you want to retrieve the time at which the file was last
663read, written, or had its meta-data (owner, etc) changed,
664you use the B<-M>, B<-A>, or B<-C> file test operations as
665documented in L<perlfunc>. These retrieve the age of the
666file (measured against the start-time of your program) in
667days as a floating point number. Some platforms may not have
668all of these times. See L<perlport> for details. To
669retrieve the "raw" time in seconds since the epoch, you
670would call the stat function, then use localtime(),
671gmtime(), or POSIX::strftime() to convert this into
672human-readable form.
68dc0745 673
674Here's an example:
675
676 $write_secs = (stat($file))[9];
c8db1d39 677 printf "file %s updated at %s\n", $file,
678 scalar localtime($write_secs);
68dc0745 679
680If you prefer something more legible, use the File::stat module
681(part of the standard distribution in version 5.004 and later):
682
65acb1b1 683 # error checking left as an exercise for reader.
68dc0745 684 use File::stat;
685 use Time::localtime;
686 $date_string = ctime(stat($file)->mtime);
687 print "file $file updated at $date_string\n";
688
65acb1b1 689The POSIX::strftime() approach has the benefit of being,
690in theory, independent of the current locale. See L<perllocale>
691for details.
68dc0745 692
693=head2 How do I set a file's timestamp in perl?
694
695You use the utime() function documented in L<perlfunc/utime>.
696By way of example, here's a little program that copies the
697read and write times from its first argument to all the rest
698of them.
699
700 if (@ARGV < 2) {
701 die "usage: cptimes timestamp_file other_files ...\n";
702 }
703 $timestamp = shift;
704 ($atime, $mtime) = (stat($timestamp))[8,9];
705 utime $atime, $mtime, @ARGV;
706
65acb1b1 707Error checking is, as usual, left as an exercise for the reader.
68dc0745 708
19a1cd16 709The perldoc for utime also has an example that has the same
710effect as touch(1) on files that I<already exist>.
711
712Certain file systems have a limited ability to store the times
713on a file at the expected level of precision. For example, the
714FAT and HPFS filesystem are unable to create dates on files with
715a finer granularity than two seconds. This is a limitation of
716the filesystems, not of utime().
68dc0745 717
718=head2 How do I print to more than one file at once?
719
49d635f9 720To connect one filehandle to several output filehandles,
721you can use the IO::Tee or Tie::FileHandle::Multiplex modules.
68dc0745 722
49d635f9 723If you only have to do this once, you can print individually
724to each filehandle.
68dc0745 725
49d635f9 726 for $fh (FH1, FH2, FH3) { print $fh "whatever\n" }
5a964f20 727
49d635f9 728=head2 How can I read in an entire file all at once?
68dc0745 729
49d635f9 730You can use the File::Slurp module to do it in one step.
68dc0745 731
49d635f9 732 use File::Slurp;
197aec24 733
49d635f9 734 $all_of_it = read_file($filename); # entire file in scalar
735 @all_lines = read_file($filename); # one line perl element
d92eb7b0 736
737The customary Perl approach for processing all the lines in a file is to
738do so one line at a time:
739
740 open (INPUT, $file) || die "can't open $file: $!";
741 while (<INPUT>) {
742 chomp;
743 # do something with $_
197aec24 744 }
d92eb7b0 745 close(INPUT) || die "can't close $file: $!";
746
747This is tremendously more efficient than reading the entire file into
748memory as an array of lines and then processing it one element at a time,
a6dd486b 749which is often--if not almost always--the wrong approach. Whenever
d92eb7b0 750you see someone do this:
751
752 @lines = <INPUT>;
753
30852c57 754you should think long and hard about why you need everything loaded at
755once. It's just not a scalable solution. You might also find it more
756fun to use the standard Tie::File module, or the DB_File module's
757$DB_RECNO bindings, which allow you to tie an array to a file so that
758accessing an element the array actually accesses the corresponding
759line in the file.
d92eb7b0 760
f05bbc40 761You can read the entire filehandle contents into a scalar.
d92eb7b0 762
763 {
764 local(*INPUT, $/);
765 open (INPUT, $file) || die "can't open $file: $!";
766 $var = <INPUT>;
767 }
768
197aec24 769That temporarily undefs your record separator, and will automatically
d92eb7b0 770close the file at block exit. If the file is already open, just use this:
771
772 $var = do { local $/; <INPUT> };
773
f05bbc40 774For ordinary files you can also use the read function.
775
776 read( INPUT, $var, -s INPUT );
777
778The third argument tests the byte size of the data on the INPUT filehandle
779and reads that many bytes into the buffer $var.
780
68dc0745 781=head2 How can I read in a file by paragraphs?
782
65acb1b1 783Use the C<$/> variable (see L<perlvar> for details). You can either
68dc0745 784set it to C<""> to eliminate empty paragraphs (C<"abc\n\n\n\ndef">,
785for instance, gets treated as two paragraphs and not three), or
786C<"\n\n"> to accept empty paragraphs.
787
197aec24 788Note that a blank line must have no blanks in it. Thus
c4db748a 789S<C<"fred\n \nstuff\n\n">> is one paragraph, but C<"fred\n\nstuff\n\n"> is two.
65acb1b1 790
68dc0745 791=head2 How can I read a single character from a file? From the keyboard?
792
793You can use the builtin C<getc()> function for most filehandles, but
794it won't (easily) work on a terminal device. For STDIN, either use
a6dd486b 795the Term::ReadKey module from CPAN or use the sample code in
68dc0745 796L<perlfunc/getc>.
797
65acb1b1 798If your system supports the portable operating system programming
799interface (POSIX), you can use the following code, which you'll note
800turns off echo processing as well.
68dc0745 801
802 #!/usr/bin/perl -w
803 use strict;
804 $| = 1;
805 for (1..4) {
806 my $got;
807 print "gimme: ";
808 $got = getone();
809 print "--> $got\n";
810 }
811 exit;
812
813 BEGIN {
814 use POSIX qw(:termios_h);
815
816 my ($term, $oterm, $echo, $noecho, $fd_stdin);
817
818 $fd_stdin = fileno(STDIN);
819
820 $term = POSIX::Termios->new();
821 $term->getattr($fd_stdin);
822 $oterm = $term->getlflag();
823
824 $echo = ECHO | ECHOK | ICANON;
825 $noecho = $oterm & ~$echo;
826
827 sub cbreak {
828 $term->setlflag($noecho);
829 $term->setcc(VTIME, 1);
830 $term->setattr($fd_stdin, TCSANOW);
831 }
832
833 sub cooked {
834 $term->setlflag($oterm);
835 $term->setcc(VTIME, 0);
836 $term->setattr($fd_stdin, TCSANOW);
837 }
838
839 sub getone {
840 my $key = '';
841 cbreak();
842 sysread(STDIN, $key, 1);
843 cooked();
844 return $key;
845 }
846
847 }
848
849 END { cooked() }
850
a6dd486b 851The Term::ReadKey module from CPAN may be easier to use. Recent versions
65acb1b1 852include also support for non-portable systems as well.
68dc0745 853
854 use Term::ReadKey;
855 open(TTY, "</dev/tty");
856 print "Gimme a char: ";
857 ReadMode "raw";
858 $key = ReadKey 0, *TTY;
859 ReadMode "normal";
860 printf "\nYou said %s, char number %03d\n",
861 $key, ord $key;
862
65acb1b1 863=head2 How can I tell whether there's a character waiting on a filehandle?
68dc0745 864
5a964f20 865The very first thing you should do is look into getting the Term::ReadKey
65acb1b1 866extension from CPAN. As we mentioned earlier, it now even has limited
867support for non-portable (read: not open systems, closed, proprietary,
868not POSIX, not Unix, etc) systems.
5a964f20 869
870You should also check out the Frequently Asked Questions list in
68dc0745 871comp.unix.* for things like this: the answer is essentially the same.
872It's very system dependent. Here's one solution that works on BSD
873systems:
874
875 sub key_ready {
876 my($rin, $nfd);
877 vec($rin, fileno(STDIN), 1) = 1;
878 return $nfd = select($rin,undef,undef,0);
879 }
880
65acb1b1 881If you want to find out how many characters are waiting, there's
882also the FIONREAD ioctl call to be looked at. The I<h2ph> tool that
883comes with Perl tries to convert C include files to Perl code, which
884can be C<require>d. FIONREAD ends up defined as a function in the
885I<sys/ioctl.ph> file:
68dc0745 886
5a964f20 887 require 'sys/ioctl.ph';
68dc0745 888
5a964f20 889 $size = pack("L", 0);
890 ioctl(FH, FIONREAD(), $size) or die "Couldn't call ioctl: $!\n";
891 $size = unpack("L", $size);
68dc0745 892
5a964f20 893If I<h2ph> wasn't installed or doesn't work for you, you can
894I<grep> the include files by hand:
68dc0745 895
5a964f20 896 % grep FIONREAD /usr/include/*/*
897 /usr/include/asm/ioctls.h:#define FIONREAD 0x541B
68dc0745 898
5a964f20 899Or write a small C program using the editor of champions:
68dc0745 900
5a964f20 901 % cat > fionread.c
902 #include <sys/ioctl.h>
903 main() {
904 printf("%#08x\n", FIONREAD);
905 }
906 ^D
65acb1b1 907 % cc -o fionread fionread.c
5a964f20 908 % ./fionread
909 0x4004667f
910
8305e449 911And then hard code it, leaving porting as an exercise to your successor.
5a964f20 912
913 $FIONREAD = 0x4004667f; # XXX: opsys dependent
914
915 $size = pack("L", 0);
916 ioctl(FH, $FIONREAD, $size) or die "Couldn't call ioctl: $!\n";
917 $size = unpack("L", $size);
918
a6dd486b 919FIONREAD requires a filehandle connected to a stream, meaning that sockets,
5a964f20 920pipes, and tty devices work, but I<not> files.
68dc0745 921
922=head2 How do I do a C<tail -f> in perl?
923
924First try
925
926 seek(GWFILE, 0, 1);
927
928The statement C<seek(GWFILE, 0, 1)> doesn't change the current position,
929but it does clear the end-of-file condition on the handle, so that the
930next <GWFILE> makes Perl try again to read something.
931
932If that doesn't work (it relies on features of your stdio implementation),
933then you need something more like this:
934
935 for (;;) {
936 for ($curpos = tell(GWFILE); <GWFILE>; $curpos = tell(GWFILE)) {
937 # search for some stuff and put it into files
938 }
939 # sleep for a while
940 seek(GWFILE, $curpos, 0); # seek to where we had been
941 }
942
943If this still doesn't work, look into the POSIX module. POSIX defines
944the clearerr() method, which can remove the end of file condition on a
945filehandle. The method: read until end of file, clearerr(), read some
946more. Lather, rinse, repeat.
947
65acb1b1 948There's also a File::Tail module from CPAN.
949
68dc0745 950=head2 How do I dup() a filehandle in Perl?
951
952If you check L<perlfunc/open>, you'll see that several of the ways
953to call open() should do the trick. For example:
954
2359510d 955 open(LOG, ">>/foo/logfile");
68dc0745 956 open(STDERR, ">&LOG");
957
958Or even with a literal numeric descriptor:
959
960 $fd = $ENV{MHCONTEXTFD};
961 open(MHCONTEXT, "<&=$fd"); # like fdopen(3S)
962
c47ff5f1 963Note that "<&STDIN" makes a copy, but "<&=STDIN" make
5a964f20 964an alias. That means if you close an aliased handle, all
197aec24 965aliases become inaccessible. This is not true with
5a964f20 966a copied one.
967
968Error checking, as always, has been left as an exercise for the reader.
68dc0745 969
970=head2 How do I close a file descriptor by number?
971
972This should rarely be necessary, as the Perl close() function is to be
973used for things that Perl opened itself, even if it was a dup of a
a6dd486b 974numeric descriptor as with MHCONTEXT above. But if you really have
68dc0745 975to, you may be able to do this:
976
977 require 'sys/syscall.ph';
978 $rc = syscall(&SYS_close, $fd + 0); # must force numeric
979 die "can't sysclose $fd: $!" unless $rc == -1;
980
a6dd486b 981Or, just use the fdopen(3S) feature of open():
d92eb7b0 982
197aec24 983 {
984 local *F;
d92eb7b0 985 open F, "<&=$fd" or die "Cannot reopen fd=$fd: $!";
986 close F;
987 }
988
883f1635 989=head2 Why can't I use "C:\temp\foo" in DOS paths? Why doesn't `C:\temp\foo.exe` work?
68dc0745 990
991Whoops! You just put a tab and a formfeed into that filename!
992Remember that within double quoted strings ("like\this"), the
993backslash is an escape character. The full list of these is in
994L<perlop/Quote and Quote-like Operators>. Unsurprisingly, you don't
995have a file called "c:(tab)emp(formfeed)oo" or
65acb1b1 996"c:(tab)emp(formfeed)oo.exe" on your legacy DOS filesystem.
68dc0745 997
998Either single-quote your strings, or (preferably) use forward slashes.
46fc3d4c 999Since all DOS and Windows versions since something like MS-DOS 2.0 or so
68dc0745 1000have treated C</> and C<\> the same in a path, you might as well use the
a6dd486b 1001one that doesn't clash with Perl--or the POSIX shell, ANSI C and C++,
65acb1b1 1002awk, Tcl, Java, or Python, just to mention a few. POSIX paths
1003are more portable, too.
68dc0745 1004
1005=head2 Why doesn't glob("*.*") get all the files?
1006
1007Because even on non-Unix ports, Perl's glob function follows standard
46fc3d4c 1008Unix globbing semantics. You'll need C<glob("*")> to get all (non-hidden)
65acb1b1 1009files. This makes glob() portable even to legacy systems. Your
1010port may include proprietary globbing functions as well. Check its
1011documentation for details.
68dc0745 1012
1013=head2 Why does Perl let me delete read-only files? Why does C<-i> clobber protected files? Isn't this a bug in Perl?
1014
06a5f41f 1015This is elaborately and painstakingly described in the
1016F<file-dir-perms> article in the "Far More Than You Ever Wanted To
49d635f9 1017Know" collection in http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz .
68dc0745 1018
1019The executive summary: learn how your filesystem works. The
1020permissions on a file say what can happen to the data in that file.
1021The permissions on a directory say what can happen to the list of
1022files in that directory. If you delete a file, you're removing its
1023name from the directory (so the operation depends on the permissions
1024of the directory, not of the file). If you try to write to the file,
1025the permissions of the file govern whether you're allowed to.
1026
1027=head2 How do I select a random line from a file?
1028
1029Here's an algorithm from the Camel Book:
1030
1031 srand;
1032 rand($.) < 1 && ($line = $_) while <>;
1033
49d635f9 1034This has a significant advantage in space over reading the whole file
1035in. You can find a proof of this method in I<The Art of Computer
1036Programming>, Volume 2, Section 3.4.2, by Donald E. Knuth.
1037
1038You can use the File::Random module which provides a function
1039for that algorithm:
1040
1041 use File::Random qw/random_line/;
1042 my $line = random_line($filename);
1043
1044Another way is to use the Tie::File module, which treats the entire
1045file as an array. Simply access a random array element.
68dc0745 1046
65acb1b1 1047=head2 Why do I get weird spaces when I print an array of lines?
1048
1049Saying
1050
1051 print "@lines\n";
1052
1053joins together the elements of C<@lines> with a space between them.
1054If C<@lines> were C<("little", "fluffy", "clouds")> then the above
a6dd486b 1055statement would print
65acb1b1 1056
1057 little fluffy clouds
1058
1059but if each element of C<@lines> was a line of text, ending a newline
1060character C<("little\n", "fluffy\n", "clouds\n")> then it would print:
1061
1062 little
1063 fluffy
1064 clouds
1065
1066If your array contains lines, just print them:
1067
1068 print @lines;
1069
68dc0745 1070=head1 AUTHOR AND COPYRIGHT
1071
7678cced 1072Copyright (c) 1997-2005 Tom Christiansen, Nathan Torkington, and
1073other authors as noted. All rights reserved.
5a964f20 1074
5a7beb56 1075This documentation is free; you can redistribute it and/or modify it
1076under the same terms as Perl itself.
c8db1d39 1077
87275199 1078Irrespective of its distribution, all code examples here are in the public
c8db1d39 1079domain. You are permitted and encouraged to use this code and any
1080derivatives thereof in your own programs for fun or for profit as you
1081see fit. A simple comment in the code giving credit to the FAQ would
1082be courteous but is not required.