Commit | Line | Data |
68dc0745 |
1 | =head1 NAME |
2 | |
109f0441 |
3 | perlfaq5 - Files and Formats |
68dc0745 |
4 | |
5 | =head1 DESCRIPTION |
6 | |
7 | This section deals with I/O and the "f" issues: filehandles, flushing, |
8 | formats, and footers. |
9 | |
5a964f20 |
10 | =head2 How do I flush/unbuffer an output filehandle? Why must I do this? |
d74e8afc |
11 | X<flush> X<buffer> X<unbuffer> X<autoflush> |
68dc0745 |
12 | |
109f0441 |
13 | (contributed by brian d foy) |
5a964f20 |
14 | |
109f0441 |
15 | You might like to read Mark Jason Dominus's "Suffering From Buffering" |
16 | at http://perl.plover.com/FAQs/Buffering.html . |
68dc0745 |
17 | |
109f0441 |
18 | Perl normally buffers output so it doesn't make a system call for every |
19 | bit of output. By saving up output, it makes fewer expensive system calls. |
20 | For instance, in this little bit of code, you want to print a dot to the |
21 | screen for every line you process to watch the progress of your program. |
22 | Instead of seeing a dot for every line, Perl buffers the output and you |
23 | have a long wait before you see a row of 50 dots all at once: |
24 | |
25 | # long wait, then row of dots all at once |
26 | while( <> ) { |
27 | print "."; |
28 | print "\n" unless ++$count % 50; |
29 | |
30 | #... expensive line processing operations |
31 | } |
32 | |
33 | To get around this, you have to unbuffer the output filehandle, in this |
34 | case, C<STDOUT>. You can set the special variable C<$|> to a true value |
35 | (mnemonic: making your filehandles "piping hot"): |
36 | |
37 | $|++; |
38 | |
39 | # dot shown immediately |
40 | while( <> ) { |
41 | print "."; |
42 | print "\n" unless ++$count % 50; |
43 | |
44 | #... expensive line processing operations |
45 | } |
46 | |
47 | The C<$|> is one of the per-filehandle special variables, so each |
48 | filehandle has its own copy of its value. If you want to merge |
49 | standard output and standard error for instance, you have to unbuffer |
50 | each (although STDERR might be unbuffered by default): |
51 | |
52 | { |
53 | my $previous_default = select(STDOUT); # save previous default |
54 | $|++; # autoflush STDOUT |
55 | select(STDERR); |
56 | $|++; # autoflush STDERR, to be sure |
57 | select($previous_default); # restore previous default |
58 | } |
68dc0745 |
59 | |
109f0441 |
60 | # now should alternate . and + |
61 | while( 1 ) |
62 | { |
63 | sleep 1; |
64 | print STDOUT "."; |
65 | print STDERR "+"; |
66 | print STDOUT "\n" unless ++$count % 25; |
67 | } |
68 | |
69 | Besides the C<$|> special variable, you can use C<binmode> to give |
70 | your filehandle a C<:unix> layer, which is unbuffered: |
71 | |
72 | binmode( STDOUT, ":unix" ); |
68dc0745 |
73 | |
109f0441 |
74 | while( 1 ) { |
75 | sleep 1; |
76 | print "."; |
77 | print "\n" unless ++$count % 50; |
78 | } |
68dc0745 |
79 | |
109f0441 |
80 | For more information on output layers, see the entries for C<binmode> |
81 | and C<open> in L<perlfunc>, and the C<PerlIO> module documentation. |
68dc0745 |
82 | |
109f0441 |
83 | If you are using C<IO::Handle> or one of its subclasses, you can |
84 | call the C<autoflush> method to change the settings of the |
85 | filehandle: |
c195e131 |
86 | |
87 | use IO::Handle; |
109f0441 |
88 | open my( $io_fh ), ">", "output.txt"; |
89 | $io_fh->autoflush(1); |
90 | |
91 | The C<IO::Handle> objects also have a C<flush> method. You can flush |
92 | the buffer any time you want without auto-buffering |
c195e131 |
93 | |
109f0441 |
94 | $io_fh->flush; |
487af187 |
95 | |
e573f903 |
96 | =head2 How do I change, delete, or insert a line in a file, or append to the beginning of a file? |
d74e8afc |
97 | X<file, editing> |
68dc0745 |
98 | |
e573f903 |
99 | (contributed by brian d foy) |
100 | |
101 | The basic idea of inserting, changing, or deleting a line from a text |
102 | file involves reading and printing the file to the point you want to |
103 | make the change, making the change, then reading and printing the rest |
104 | of the file. Perl doesn't provide random access to lines (especially |
105 | since the record input separator, C<$/>, is mutable), although modules |
106 | such as C<Tie::File> can fake it. |
107 | |
108 | A Perl program to do these tasks takes the basic form of opening a |
109 | file, printing its lines, then closing the file: |
110 | |
111 | open my $in, '<', $file or die "Can't read old file: $!"; |
112 | open my $out, '>', "$file.new" or die "Can't write new file: $!"; |
113 | |
114 | while( <$in> ) |
115 | { |
116 | print $out $_; |
117 | } |
118 | |
119 | close $out; |
120 | |
121 | Within that basic form, add the parts that you need to insert, change, |
122 | or delete lines. |
123 | |
124 | To prepend lines to the beginning, print those lines before you enter |
125 | the loop that prints the existing lines. |
126 | |
127 | open my $in, '<', $file or die "Can't read old file: $!"; |
128 | open my $out, '>', "$file.new" or die "Can't write new file: $!"; |
129 | |
109f0441 |
130 | print $out "# Add this line to the top\n"; # <--- HERE'S THE MAGIC |
e573f903 |
131 | |
132 | while( <$in> ) |
133 | { |
134 | print $out $_; |
135 | } |
136 | |
137 | close $out; |
138 | |
139 | To change existing lines, insert the code to modify the lines inside |
140 | the C<while> loop. In this case, the code finds all lowercased |
141 | versions of "perl" and uppercases them. The happens for every line, so |
142 | be sure that you're supposed to do that on every line! |
143 | |
144 | open my $in, '<', $file or die "Can't read old file: $!"; |
145 | open my $out, '>', "$file.new" or die "Can't write new file: $!"; |
146 | |
109f0441 |
147 | print $out "# Add this line to the top\n"; |
e573f903 |
148 | |
149 | while( <$in> ) |
150 | { |
151 | s/\b(perl)\b/Perl/g; |
152 | print $out $_; |
153 | } |
154 | |
155 | close $out; |
156 | |
157 | To change only a particular line, the input line number, C<$.>, is |
ee891a00 |
158 | useful. First read and print the lines up to the one you want to |
159 | change. Next, read the single line you want to change, change it, and |
160 | print it. After that, read the rest of the lines and print those: |
e573f903 |
161 | |
ee891a00 |
162 | while( <$in> ) # print the lines before the change |
e573f903 |
163 | { |
e573f903 |
164 | print $out $_; |
ee891a00 |
165 | last if $. == 4; # line number before change |
e573f903 |
166 | } |
167 | |
ee891a00 |
168 | my $line = <$in>; |
169 | $line =~ s/\b(perl)\b/Perl/g; |
170 | print $out $line; |
171 | |
172 | while( <$in> ) # print the rest of the lines |
173 | { |
174 | print $out $_; |
175 | } |
109f0441 |
176 | |
e573f903 |
177 | To skip lines, use the looping controls. The C<next> in this example |
178 | skips comment lines, and the C<last> stops all processing once it |
179 | encounters either C<__END__> or C<__DATA__>. |
180 | |
181 | while( <$in> ) |
182 | { |
183 | next if /^\s+#/; # skip comment lines |
184 | last if /^__(END|DATA)__$/; # stop at end of code marker |
185 | print $out $_; |
186 | } |
187 | |
188 | Do the same sort of thing to delete a particular line by using C<next> |
189 | to skip the lines you don't want to show up in the output. This |
190 | example skips every fifth line: |
191 | |
192 | while( <$in> ) |
193 | { |
194 | next unless $. % 5; |
195 | print $out $_; |
196 | } |
197 | |
198 | If, for some odd reason, you really want to see the whole file at once |
199 | rather than processing line by line, you can slurp it in (as long as |
200 | you can fit the whole thing in memory!): |
201 | |
202 | open my $in, '<', $file or die "Can't read old file: $!" |
203 | open my $out, '>', "$file.new" or die "Can't write new file: $!"; |
204 | |
205 | my @lines = do { local $/; <$in> }; # slurp! |
206 | |
207 | # do your magic here |
208 | |
209 | print $out @lines; |
210 | |
211 | Modules such as C<File::Slurp> and C<Tie::File> can help with that |
212 | too. If you can, however, avoid reading the entire file at once. Perl |
213 | won't give that memory back to the operating system until the process |
214 | finishes. |
215 | |
216 | You can also use Perl one-liners to modify a file in-place. The |
217 | following changes all 'Fred' to 'Barney' in F<inFile.txt>, overwriting |
218 | the file with the new contents. With the C<-p> switch, Perl wraps a |
219 | C<while> loop around the code you specify with C<-e>, and C<-i> turns |
220 | on in-place editing. The current line is in C<$_>. With C<-p>, Perl |
221 | automatically prints the value of C<$_> at the end of the loop. See |
222 | L<perlrun> for more details. |
223 | |
224 | perl -pi -e 's/Fred/Barney/' inFile.txt |
225 | |
226 | To make a backup of C<inFile.txt>, give C<-i> a file extension to add: |
227 | |
228 | perl -pi.bak -e 's/Fred/Barney/' inFile.txt |
229 | |
230 | To change only the fifth line, you can add a test checking C<$.>, the |
231 | input line number, then only perform the operation when the test |
232 | passes: |
233 | |
234 | perl -pi -e 's/Fred/Barney/ if $. == 5' inFile.txt |
235 | |
236 | To add lines before a certain line, you can add a line (or lines!) |
237 | before Perl prints C<$_>: |
238 | |
239 | perl -pi -e 'print "Put before third line\n" if $. == 3' inFile.txt |
240 | |
241 | You can even add a line to the beginning of a file, since the current |
242 | line prints at the end of the loop: |
243 | |
244 | perl -pi -e 'print "Put before first line\n" if $. == 1' inFile.txt |
245 | |
246 | To insert a line after one already in the file, use the C<-n> switch. |
247 | It's just like C<-p> except that it doesn't print C<$_> at the end of |
248 | the loop, so you have to do that yourself. In this case, print C<$_> |
249 | first, then print the line that you want to add. |
250 | |
251 | perl -ni -e 'print; print "Put after fifth line\n" if $. == 5' inFile.txt |
252 | |
253 | To delete lines, only print the ones that you want. |
254 | |
255 | perl -ni -e 'print unless /d/' inFile.txt |
256 | |
257 | ... or ... |
258 | |
259 | perl -pi -e 'next unless /d/' inFile.txt |
68dc0745 |
260 | |
261 | =head2 How do I count the number of lines in a file? |
d74e8afc |
262 | X<file, counting lines> X<lines> X<line> |
68dc0745 |
263 | |
264 | One fairly efficient way is to count newlines in the file. The |
265 | following program uses a feature of tr///, as documented in L<perlop>. |
266 | If your text file doesn't end with a newline, then it's not really a |
267 | proper text file, so this may report one fewer line than you expect. |
268 | |
500071f4 |
269 | $lines = 0; |
270 | open(FILE, $filename) or die "Can't open `$filename': $!"; |
271 | while (sysread FILE, $buffer, 4096) { |
272 | $lines += ($buffer =~ tr/\n//); |
273 | } |
274 | close FILE; |
68dc0745 |
275 | |
5a964f20 |
276 | This assumes no funny games with newline translations. |
277 | |
4750257b |
278 | =head2 How can I use Perl's C<-i> option from within a program? |
d74e8afc |
279 | X<-i> X<in-place> |
4750257b |
280 | |
281 | C<-i> sets the value of Perl's C<$^I> variable, which in turn affects |
282 | the behavior of C<< <> >>; see L<perlrun> for more details. By |
283 | modifying the appropriate variables directly, you can get the same |
284 | behavior within a larger program. For example: |
285 | |
500071f4 |
286 | # ... |
287 | { |
288 | local($^I, @ARGV) = ('.orig', glob("*.c")); |
289 | while (<>) { |
290 | if ($. == 1) { |
291 | print "This line should appear at the top of each file\n"; |
292 | } |
293 | s/\b(p)earl\b/${1}erl/i; # Correct typos, preserving case |
294 | print; |
295 | close ARGV if eof; # Reset $. |
296 | } |
297 | } |
298 | # $^I and @ARGV return to their old values here |
4750257b |
299 | |
300 | This block modifies all the C<.c> files in the current directory, |
301 | leaving a backup of the original data from each file in a new |
302 | C<.c.orig> file. |
303 | |
7678cced |
304 | =head2 How can I copy a file? |
109f0441 |
305 | X<copy> X<file, copy> X<File::Copy> |
7678cced |
306 | |
307 | (contributed by brian d foy) |
308 | |
109f0441 |
309 | Use the C<File::Copy> module. It comes with Perl and can do a |
7678cced |
310 | true copy across file systems, and it does its magic in |
311 | a portable fashion. |
312 | |
313 | use File::Copy; |
314 | |
315 | copy( $original, $new_copy ) or die "Copy failed: $!"; |
316 | |
109f0441 |
317 | If you can't use C<File::Copy>, you'll have to do the work yourself: |
7678cced |
318 | open the original file, open the destination file, then print |
109f0441 |
319 | to the destination file as you read the original. You also have to |
320 | remember to copy the permissions, owner, and group to the new file. |
7678cced |
321 | |
68dc0745 |
322 | =head2 How do I make a temporary file name? |
d74e8afc |
323 | X<file, temporary> |
68dc0745 |
324 | |
7678cced |
325 | If you don't need to know the name of the file, you can use C<open()> |
109f0441 |
326 | with C<undef> in place of the file name. In Perl 5.8 or later, the |
327 | C<open()> function creates an anonymous temporary file: |
7678cced |
328 | |
329 | open my $tmp, '+>', undef or die $!; |
6670e5e7 |
330 | |
7678cced |
331 | Otherwise, you can use the File::Temp module. |
68dc0745 |
332 | |
500071f4 |
333 | use File::Temp qw/ tempfile tempdir /; |
a6dd486b |
334 | |
500071f4 |
335 | $dir = tempdir( CLEANUP => 1 ); |
336 | ($fh, $filename) = tempfile( DIR => $dir ); |
5a964f20 |
337 | |
500071f4 |
338 | # or if you don't need to know the filename |
5a964f20 |
339 | |
500071f4 |
340 | $fh = tempfile( DIR => $dir ); |
5a964f20 |
341 | |
16394a69 |
342 | The File::Temp has been a standard module since Perl 5.6.1. If you |
343 | don't have a modern enough Perl installed, use the C<new_tmpfile> |
344 | class method from the IO::File module to get a filehandle opened for |
345 | reading and writing. Use it if you don't need to know the file's name: |
5a964f20 |
346 | |
500071f4 |
347 | use IO::File; |
348 | $fh = IO::File->new_tmpfile() |
16394a69 |
349 | or die "Unable to make new temporary file: $!"; |
5a964f20 |
350 | |
a6dd486b |
351 | If you're committed to creating a temporary file by hand, use the |
352 | process ID and/or the current time-value. If you need to have many |
353 | temporary files in one process, use a counter: |
5a964f20 |
354 | |
500071f4 |
355 | BEGIN { |
68dc0745 |
356 | use Fcntl; |
16394a69 |
357 | my $temp_dir = -d '/tmp' ? '/tmp' : $ENV{TMPDIR} || $ENV{TEMP}; |
c195e131 |
358 | my $base_name = sprintf "%s/%d-%d-0000", $temp_dir, $$, time; |
500071f4 |
359 | |
68dc0745 |
360 | sub temp_file { |
500071f4 |
361 | local *FH; |
362 | my $count = 0; |
c195e131 |
363 | until( defined(fileno(FH)) || $count++ > 100 ) { |
364 | $base_name =~ s/-(\d+)$/"-" . (1 + $1)/e; |
365 | # O_EXCL is required for security reasons. |
366 | sysopen FH, $base_name, O_WRONLY|O_EXCL|O_CREAT; |
367 | } |
368 | |
369 | if( defined fileno(FH) ) { |
370 | return (*FH, $base_name); |
371 | } |
372 | else { |
373 | return (); |
374 | } |
500071f4 |
375 | } |
109f0441 |
376 | |
500071f4 |
377 | } |
68dc0745 |
378 | |
68dc0745 |
379 | =head2 How can I manipulate fixed-record-length files? |
d74e8afc |
380 | X<fixed-length> X<file, fixed-length records> |
68dc0745 |
381 | |
793f5136 |
382 | The most efficient way is using L<pack()|perlfunc/"pack"> and |
383 | L<unpack()|perlfunc/"unpack">. This is faster than using |
384 | L<substr()|perlfunc/"substr"> when taking many, many strings. It is |
385 | slower for just a few. |
5a964f20 |
386 | |
387 | Here is a sample chunk of code to break up and put back together again |
388 | some fixed-format input lines, in this case from the output of a normal, |
389 | Berkeley-style ps: |
68dc0745 |
390 | |
500071f4 |
391 | # sample input line: |
392 | # 15158 p5 T 0:00 perl /home/tchrist/scripts/now-what |
393 | my $PS_T = 'A6 A4 A7 A5 A*'; |
394 | open my $ps, '-|', 'ps'; |
395 | print scalar <$ps>; |
396 | my @fields = qw( pid tt stat time command ); |
397 | while (<$ps>) { |
398 | my %process; |
399 | @process{@fields} = unpack($PS_T, $_); |
793f5136 |
400 | for my $field ( @fields ) { |
500071f4 |
401 | print "$field: <$process{$field}>\n"; |
68dc0745 |
402 | } |
793f5136 |
403 | print 'line=', pack($PS_T, @process{@fields} ), "\n"; |
500071f4 |
404 | } |
68dc0745 |
405 | |
793f5136 |
406 | We've used a hash slice in order to easily handle the fields of each row. |
407 | Storing the keys in an array means it's easy to operate on them as a |
408 | group or loop over them with for. It also avoids polluting the program |
409 | with global variables and using symbolic references. |
5a964f20 |
410 | |
ac9dac7f |
411 | =head2 How can I make a filehandle local to a subroutine? How do I pass filehandles between subroutines? How do I make an array of filehandles? |
d74e8afc |
412 | X<filehandle, local> X<filehandle, passing> X<filehandle, reference> |
68dc0745 |
413 | |
c90536be |
414 | As of perl5.6, open() autovivifies file and directory handles |
415 | as references if you pass it an uninitialized scalar variable. |
416 | You can then pass these references just like any other scalar, |
417 | and use them in the place of named handles. |
68dc0745 |
418 | |
c90536be |
419 | open my $fh, $file_name; |
818c4caa |
420 | |
c90536be |
421 | open local $fh, $file_name; |
818c4caa |
422 | |
c90536be |
423 | print $fh "Hello World!\n"; |
818c4caa |
424 | |
c90536be |
425 | process_file( $fh ); |
68dc0745 |
426 | |
500071f4 |
427 | If you like, you can store these filehandles in an array or a hash. |
428 | If you access them directly, they aren't simple scalars and you |
ac9dac7f |
429 | need to give C<print> a little help by placing the filehandle |
500071f4 |
430 | reference in braces. Perl can only figure it out on its own when |
431 | the filehandle reference is a simple scalar. |
432 | |
433 | my @fhs = ( $fh1, $fh2, $fh3 ); |
ac9dac7f |
434 | |
500071f4 |
435 | for( $i = 0; $i <= $#fhs; $i++ ) { |
436 | print {$fhs[$i]} "just another Perl answer, \n"; |
437 | } |
438 | |
c90536be |
439 | Before perl5.6, you had to deal with various typeglob idioms |
440 | which you may see in older code. |
68dc0745 |
441 | |
c90536be |
442 | open FILE, "> $filename"; |
443 | process_typeglob( *FILE ); |
444 | process_reference( \*FILE ); |
818c4caa |
445 | |
c90536be |
446 | sub process_typeglob { local *FH = shift; print FH "Typeglob!" } |
447 | sub process_reference { local $fh = shift; print $fh "Reference!" } |
5a964f20 |
448 | |
c90536be |
449 | If you want to create many anonymous handles, you should |
450 | check out the Symbol or IO::Handle modules. |
5a964f20 |
451 | |
452 | =head2 How can I use a filehandle indirectly? |
d74e8afc |
453 | X<filehandle, indirect> |
5a964f20 |
454 | |
455 | An indirect filehandle is using something other than a symbol |
456 | in a place that a filehandle is expected. Here are ways |
a6dd486b |
457 | to get indirect filehandles: |
5a964f20 |
458 | |
500071f4 |
459 | $fh = SOME_FH; # bareword is strict-subs hostile |
460 | $fh = "SOME_FH"; # strict-refs hostile; same package only |
461 | $fh = *SOME_FH; # typeglob |
462 | $fh = \*SOME_FH; # ref to typeglob (bless-able) |
463 | $fh = *SOME_FH{IO}; # blessed IO::Handle from *SOME_FH typeglob |
5a964f20 |
464 | |
c90536be |
465 | Or, you can use the C<new> method from one of the IO::* modules to |
5a964f20 |
466 | create an anonymous filehandle, store that in a scalar variable, |
467 | and use it as though it were a normal filehandle. |
468 | |
500071f4 |
469 | use IO::Handle; # 5.004 or higher |
470 | $fh = IO::Handle->new(); |
5a964f20 |
471 | |
472 | Then use any of those as you would a normal filehandle. Anywhere that |
473 | Perl is expecting a filehandle, an indirect filehandle may be used |
474 | instead. An indirect filehandle is just a scalar variable that contains |
368c9434 |
475 | a filehandle. Functions like C<print>, C<open>, C<seek>, or |
c90536be |
476 | the C<< <FH> >> diamond operator will accept either a named filehandle |
5a964f20 |
477 | or a scalar variable containing one: |
478 | |
500071f4 |
479 | ($ifh, $ofh, $efh) = (*STDIN, *STDOUT, *STDERR); |
480 | print $ofh "Type it: "; |
481 | $got = <$ifh> |
482 | print $efh "What was that: $got"; |
5a964f20 |
483 | |
368c9434 |
484 | If you're passing a filehandle to a function, you can write |
5a964f20 |
485 | the function in two ways: |
486 | |
500071f4 |
487 | sub accept_fh { |
488 | my $fh = shift; |
489 | print $fh "Sending to indirect filehandle\n"; |
490 | } |
46fc3d4c |
491 | |
5a964f20 |
492 | Or it can localize a typeglob and use the filehandle directly: |
46fc3d4c |
493 | |
500071f4 |
494 | sub accept_fh { |
495 | local *FH = shift; |
496 | print FH "Sending to localized filehandle\n"; |
497 | } |
46fc3d4c |
498 | |
5a964f20 |
499 | Both styles work with either objects or typeglobs of real filehandles. |
500 | (They might also work with strings under some circumstances, but this |
501 | is risky.) |
502 | |
500071f4 |
503 | accept_fh(*STDOUT); |
504 | accept_fh($handle); |
5a964f20 |
505 | |
506 | In the examples above, we assigned the filehandle to a scalar variable |
a6dd486b |
507 | before using it. That is because only simple scalar variables, not |
508 | expressions or subscripts of hashes or arrays, can be used with |
509 | built-ins like C<print>, C<printf>, or the diamond operator. Using |
8305e449 |
510 | something other than a simple scalar variable as a filehandle is |
5a964f20 |
511 | illegal and won't even compile: |
512 | |
500071f4 |
513 | @fd = (*STDIN, *STDOUT, *STDERR); |
514 | print $fd[1] "Type it: "; # WRONG |
515 | $got = <$fd[0]> # WRONG |
516 | print $fd[2] "What was that: $got"; # WRONG |
5a964f20 |
517 | |
518 | With C<print> and C<printf>, you get around this by using a block and |
519 | an expression where you would place the filehandle: |
520 | |
500071f4 |
521 | print { $fd[1] } "funny stuff\n"; |
522 | printf { $fd[1] } "Pity the poor %x.\n", 3_735_928_559; |
523 | # Pity the poor deadbeef. |
5a964f20 |
524 | |
525 | That block is a proper block like any other, so you can put more |
526 | complicated code there. This sends the message out to one of two places: |
527 | |
500071f4 |
528 | $ok = -x "/bin/cat"; |
529 | print { $ok ? $fd[1] : $fd[2] } "cat stat $ok\n"; |
530 | print { $fd[ 1+ ($ok || 0) ] } "cat stat $ok\n"; |
5a964f20 |
531 | |
532 | This approach of treating C<print> and C<printf> like object methods |
533 | calls doesn't work for the diamond operator. That's because it's a |
534 | real operator, not just a function with a comma-less argument. Assuming |
535 | you've been storing typeglobs in your structure as we did above, you |
c90536be |
536 | can use the built-in function named C<readline> to read a record just |
c47ff5f1 |
537 | as C<< <> >> does. Given the initialization shown above for @fd, this |
c90536be |
538 | would work, but only because readline() requires a typeglob. It doesn't |
5a964f20 |
539 | work with objects or strings, which might be a bug we haven't fixed yet. |
540 | |
500071f4 |
541 | $got = readline($fd[0]); |
5a964f20 |
542 | |
543 | Let it be noted that the flakiness of indirect filehandles is not |
544 | related to whether they're strings, typeglobs, objects, or anything else. |
545 | It's the syntax of the fundamental operators. Playing the object |
546 | game doesn't help you at all here. |
46fc3d4c |
547 | |
68dc0745 |
548 | =head2 How can I set up a footer format to be used with write()? |
d74e8afc |
549 | X<footer> |
68dc0745 |
550 | |
54310121 |
551 | There's no builtin way to do this, but L<perlform> has a couple of |
68dc0745 |
552 | techniques to make it possible for the intrepid hacker. |
553 | |
554 | =head2 How can I write() into a string? |
d74e8afc |
555 | X<write, into a string> |
68dc0745 |
556 | |
c195e131 |
557 | See L<perlform/"Accessing Formatting Internals"> for an C<swrite()> function. |
68dc0745 |
558 | |
c195e131 |
559 | =head2 How can I open a filehandle to a string? |
109f0441 |
560 | X<string> X<open> X<IO::String> X<filehandle> |
c195e131 |
561 | |
562 | (contributed by Peter J. Holzer, hjp-usenet2@hjp.at) |
563 | |
109f0441 |
564 | Since Perl 5.8.0 a file handle referring to a string can be created by |
565 | calling open with a reference to that string instead of the filename. |
566 | This file handle can then be used to read from or write to the string: |
c195e131 |
567 | |
568 | open(my $fh, '>', \$string) or die "Could not open string for writing"; |
569 | print $fh "foo\n"; |
570 | print $fh "bar\n"; # $string now contains "foo\nbar\n" |
571 | |
572 | open(my $fh, '<', \$string) or die "Could not open string for reading"; |
573 | my $x = <$fh>; # $x now contains "foo\n" |
574 | |
575 | With older versions of Perl, the C<IO::String> module provides similar |
576 | functionality. |
487af187 |
577 | |
68dc0745 |
578 | =head2 How can I output my numbers with commas added? |
d74e8afc |
579 | X<number, commify> |
68dc0745 |
580 | |
b68463f7 |
581 | (contributed by brian d foy and Benjamin Goldberg) |
582 | |
583 | You can use L<Number::Format> to separate places in a number. |
584 | It handles locale information for those of you who want to insert |
585 | full stops instead (or anything else that they want to use, |
586 | really). |
587 | |
49d635f9 |
588 | This subroutine will add commas to your number: |
589 | |
590 | sub commify { |
500071f4 |
591 | local $_ = shift; |
592 | 1 while s/^([-+]?\d+)(\d{3})/$1,$2/; |
593 | return $_; |
594 | } |
49d635f9 |
595 | |
596 | This regex from Benjamin Goldberg will add commas to numbers: |
68dc0745 |
597 | |
500071f4 |
598 | s/(^[-+]?\d+?(?=(?>(?:\d{3})+)(?!\d))|\G\d{3}(?=\d))/$1,/g; |
68dc0745 |
599 | |
49d635f9 |
600 | It is easier to see with comments: |
68dc0745 |
601 | |
500071f4 |
602 | s/( |
603 | ^[-+]? # beginning of number. |
604 | \d+? # first digits before first comma |
605 | (?= # followed by, (but not included in the match) : |
606 | (?>(?:\d{3})+) # some positive multiple of three digits. |
607 | (?!\d) # an *exact* multiple, not x * 3 + 1 or whatever. |
608 | ) |
609 | | # or: |
610 | \G\d{3} # after the last group, get three digits |
611 | (?=\d) # but they have to have more digits after them. |
612 | )/$1,/xg; |
46fc3d4c |
613 | |
68dc0745 |
614 | =head2 How can I translate tildes (~) in a filename? |
d74e8afc |
615 | X<tilde> X<tilde expansion> |
68dc0745 |
616 | |
109f0441 |
617 | Use the E<lt>E<gt> (C<glob()>) operator, documented in L<perlfunc>. |
618 | Versions of Perl older than 5.6 require that you have a shell |
619 | installed that groks tildes. Later versions of Perl have this feature |
620 | built in. The C<File::KGlob> module (available from CPAN) gives more |
621 | portable glob functionality. |
68dc0745 |
622 | |
623 | Within Perl, you may use this directly: |
624 | |
625 | $filename =~ s{ |
626 | ^ ~ # find a leading tilde |
627 | ( # save this in $1 |
628 | [^/] # a non-slash character |
629 | * # repeated 0 or more times (0 means me) |
630 | ) |
631 | }{ |
632 | $1 |
633 | ? (getpwnam($1))[7] |
634 | : ( $ENV{HOME} || $ENV{LOGDIR} ) |
635 | }ex; |
636 | |
5a964f20 |
637 | =head2 How come when I open a file read-write it wipes it out? |
d74e8afc |
638 | X<clobber> X<read-write> X<clobbering> X<truncate> X<truncating> |
68dc0745 |
639 | |
640 | Because you're using something like this, which truncates the file and |
641 | I<then> gives you read-write access: |
642 | |
500071f4 |
643 | open(FH, "+> /path/name"); # WRONG (almost always) |
68dc0745 |
644 | |
645 | Whoops. You should instead use this, which will fail if the file |
197aec24 |
646 | doesn't exist. |
d92eb7b0 |
647 | |
500071f4 |
648 | open(FH, "+< /path/name"); # open for update |
d92eb7b0 |
649 | |
c47ff5f1 |
650 | Using ">" always clobbers or creates. Using "<" never does |
d92eb7b0 |
651 | either. The "+" doesn't change this. |
68dc0745 |
652 | |
5a964f20 |
653 | Here are examples of many kinds of file opens. Those using sysopen() |
654 | all assume |
68dc0745 |
655 | |
500071f4 |
656 | use Fcntl; |
68dc0745 |
657 | |
5a964f20 |
658 | To open file for reading: |
68dc0745 |
659 | |
500071f4 |
660 | open(FH, "< $path") || die $!; |
661 | sysopen(FH, $path, O_RDONLY) || die $!; |
5a964f20 |
662 | |
663 | To open file for writing, create new file if needed or else truncate old file: |
664 | |
500071f4 |
665 | open(FH, "> $path") || die $!; |
666 | sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT) || die $!; |
667 | sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT, 0666) || die $!; |
5a964f20 |
668 | |
669 | To open file for writing, create new file, file must not exist: |
670 | |
500071f4 |
671 | sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT) || die $!; |
672 | sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT, 0666) || die $!; |
5a964f20 |
673 | |
674 | To open file for appending, create if necessary: |
675 | |
500071f4 |
676 | open(FH, ">> $path") || die $!; |
677 | sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT) || die $!; |
678 | sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT, 0666) || die $!; |
5a964f20 |
679 | |
680 | To open file for appending, file must exist: |
681 | |
500071f4 |
682 | sysopen(FH, $path, O_WRONLY|O_APPEND) || die $!; |
5a964f20 |
683 | |
684 | To open file for update, file must exist: |
685 | |
500071f4 |
686 | open(FH, "+< $path") || die $!; |
687 | sysopen(FH, $path, O_RDWR) || die $!; |
5a964f20 |
688 | |
689 | To open file for update, create file if necessary: |
690 | |
500071f4 |
691 | sysopen(FH, $path, O_RDWR|O_CREAT) || die $!; |
692 | sysopen(FH, $path, O_RDWR|O_CREAT, 0666) || die $!; |
5a964f20 |
693 | |
694 | To open file for update, file must not exist: |
695 | |
500071f4 |
696 | sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT) || die $!; |
697 | sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT, 0666) || die $!; |
5a964f20 |
698 | |
699 | To open a file without blocking, creating if necessary: |
700 | |
500071f4 |
701 | sysopen(FH, "/foo/somefile", O_WRONLY|O_NDELAY|O_CREAT) |
2359510d |
702 | or die "can't open /foo/somefile: $!": |
5a964f20 |
703 | |
704 | Be warned that neither creation nor deletion of files is guaranteed to |
705 | be an atomic operation over NFS. That is, two processes might both |
a6dd486b |
706 | successfully create or unlink the same file! Therefore O_EXCL |
707 | isn't as exclusive as you might wish. |
68dc0745 |
708 | |
87275199 |
709 | See also the new L<perlopentut> if you have it (new for 5.6). |
65acb1b1 |
710 | |
04d666b1 |
711 | =head2 Why do I sometimes get an "Argument list too long" when I use E<lt>*E<gt>? |
d74e8afc |
712 | X<argument list too long> |
68dc0745 |
713 | |
c47ff5f1 |
714 | The C<< <> >> operator performs a globbing operation (see above). |
3a4b19e4 |
715 | In Perl versions earlier than v5.6.0, the internal glob() operator forks |
716 | csh(1) to do the actual glob expansion, but |
68dc0745 |
717 | csh can't handle more than 127 items and so gives the error message |
718 | C<Argument list too long>. People who installed tcsh as csh won't |
719 | have this problem, but their users may be surprised by it. |
720 | |
3a4b19e4 |
721 | To get around this, either upgrade to Perl v5.6.0 or later, do the glob |
d6260402 |
722 | yourself with readdir() and patterns, or use a module like File::KGlob, |
3a4b19e4 |
723 | one that doesn't use the shell to do globbing. |
68dc0745 |
724 | |
725 | =head2 Is there a leak/bug in glob()? |
d74e8afc |
726 | X<glob> |
68dc0745 |
727 | |
728 | Due to the current implementation on some operating systems, when you |
729 | use the glob() function or its angle-bracket alias in a scalar |
a6dd486b |
730 | context, you may cause a memory leak and/or unpredictable behavior. It's |
68dc0745 |
731 | best therefore to use glob() only in list context. |
732 | |
c47ff5f1 |
733 | =head2 How can I open a file with a leading ">" or trailing blanks? |
d74e8afc |
734 | X<filename, special characters> |
68dc0745 |
735 | |
b68463f7 |
736 | (contributed by Brian McCauley) |
68dc0745 |
737 | |
b68463f7 |
738 | The special two argument form of Perl's open() function ignores |
739 | trailing blanks in filenames and infers the mode from certain leading |
740 | characters (or a trailing "|"). In older versions of Perl this was the |
741 | only version of open() and so it is prevalent in old code and books. |
65acb1b1 |
742 | |
b68463f7 |
743 | Unless you have a particular reason to use the two argument form you |
744 | should use the three argument form of open() which does not treat any |
c195e131 |
745 | characters in the filename as special. |
58103a2e |
746 | |
881bdbd4 |
747 | open FILE, "<", " file "; # filename is " file " |
748 | open FILE, ">", ">file"; # filename is ">file" |
65acb1b1 |
749 | |
68dc0745 |
750 | =head2 How can I reliably rename a file? |
d74e8afc |
751 | X<rename> X<mv> X<move> X<file, rename> X<ren> |
68dc0745 |
752 | |
49d635f9 |
753 | If your operating system supports a proper mv(1) utility or its |
754 | functional equivalent, this works: |
68dc0745 |
755 | |
500071f4 |
756 | rename($old, $new) or system("mv", $old, $new); |
68dc0745 |
757 | |
d2321c93 |
758 | It may be more portable to use the File::Copy module instead. |
759 | You just copy to the new file to the new name (checking return |
760 | values), then delete the old one. This isn't really the same |
761 | semantically as a rename(), which preserves meta-information like |
68dc0745 |
762 | permissions, timestamps, inode info, etc. |
763 | |
d2321c93 |
764 | Newer versions of File::Copy export a move() function. |
5a964f20 |
765 | |
68dc0745 |
766 | =head2 How can I lock a file? |
d74e8afc |
767 | X<lock> X<file, lock> X<flock> |
68dc0745 |
768 | |
54310121 |
769 | Perl's builtin flock() function (see L<perlfunc> for details) will call |
68dc0745 |
770 | flock(2) if that exists, fcntl(2) if it doesn't (on perl version 5.004 and |
771 | later), and lockf(3) if neither of the two previous system calls exists. |
772 | On some systems, it may even use a different form of native locking. |
773 | Here are some gotchas with Perl's flock(): |
774 | |
775 | =over 4 |
776 | |
777 | =item 1 |
778 | |
779 | Produces a fatal error if none of the three system calls (or their |
780 | close equivalent) exists. |
781 | |
782 | =item 2 |
783 | |
784 | lockf(3) does not provide shared locking, and requires that the |
785 | filehandle be open for writing (or appending, or read/writing). |
786 | |
787 | =item 3 |
788 | |
d92eb7b0 |
789 | Some versions of flock() can't lock files over a network (e.g. on NFS file |
790 | systems), so you'd need to force the use of fcntl(2) when you build Perl. |
a6dd486b |
791 | But even this is dubious at best. See the flock entry of L<perlfunc> |
d92eb7b0 |
792 | and the F<INSTALL> file in the source distribution for information on |
793 | building Perl to do this. |
794 | |
795 | Two potentially non-obvious but traditional flock semantics are that |
a6dd486b |
796 | it waits indefinitely until the lock is granted, and that its locks are |
d92eb7b0 |
797 | I<merely advisory>. Such discretionary locks are more flexible, but |
798 | offer fewer guarantees. This means that files locked with flock() may |
799 | be modified by programs that do not also use flock(). Cars that stop |
800 | for red lights get on well with each other, but not with cars that don't |
801 | stop for red lights. See the perlport manpage, your port's specific |
802 | documentation, or your system-specific local manpages for details. It's |
803 | best to assume traditional behavior if you're writing portable programs. |
a6dd486b |
804 | (If you're not, you should as always feel perfectly free to write |
d92eb7b0 |
805 | for your own system's idiosyncrasies (sometimes called "features"). |
806 | Slavish adherence to portability concerns shouldn't get in the way of |
807 | your getting your job done.) |
68dc0745 |
808 | |
197aec24 |
809 | For more information on file locking, see also |
13a2d996 |
810 | L<perlopentut/"File Locking"> if you have it (new for 5.6). |
65acb1b1 |
811 | |
68dc0745 |
812 | =back |
813 | |
04d666b1 |
814 | =head2 Why can't I just open(FH, "E<gt>file.lock")? |
d74e8afc |
815 | X<lock, lockfile race condition> |
68dc0745 |
816 | |
817 | A common bit of code B<NOT TO USE> is this: |
818 | |
500071f4 |
819 | sleep(3) while -e "file.lock"; # PLEASE DO NOT USE |
820 | open(LCK, "> file.lock"); # THIS BROKEN CODE |
68dc0745 |
821 | |
822 | This is a classic race condition: you take two steps to do something |
823 | which must be done in one. That's why computer hardware provides an |
824 | atomic test-and-set instruction. In theory, this "ought" to work: |
825 | |
500071f4 |
826 | sysopen(FH, "file.lock", O_WRONLY|O_EXCL|O_CREAT) |
9b55d3ab |
827 | or die "can't open file.lock: $!"; |
68dc0745 |
828 | |
829 | except that lamentably, file creation (and deletion) is not atomic |
830 | over NFS, so this won't work (at least, not every time) over the net. |
65acb1b1 |
831 | Various schemes involving link() have been suggested, but |
c195e131 |
832 | these tend to involve busy-wait, which is also less than desirable. |
68dc0745 |
833 | |
fc36a67e |
834 | =head2 I still don't get locking. I just want to increment the number in the file. How can I do this? |
d74e8afc |
835 | X<counter> X<file, counter> |
68dc0745 |
836 | |
46fc3d4c |
837 | Didn't anyone ever tell you web-page hit counters were useless? |
5a964f20 |
838 | They don't count number of hits, they're a waste of time, and they serve |
a6dd486b |
839 | only to stroke the writer's vanity. It's better to pick a random number; |
840 | they're more realistic. |
68dc0745 |
841 | |
5a964f20 |
842 | Anyway, this is what you can do if you can't help yourself. |
68dc0745 |
843 | |
500071f4 |
844 | use Fcntl qw(:DEFAULT :flock); |
845 | sysopen(FH, "numfile", O_RDWR|O_CREAT) or die "can't open numfile: $!"; |
846 | flock(FH, LOCK_EX) or die "can't flock numfile: $!"; |
847 | $num = <FH> || 0; |
848 | seek(FH, 0, 0) or die "can't rewind numfile: $!"; |
849 | truncate(FH, 0) or die "can't truncate numfile: $!"; |
850 | (print FH $num+1, "\n") or die "can't write numfile: $!"; |
851 | close FH or die "can't close numfile: $!"; |
68dc0745 |
852 | |
46fc3d4c |
853 | Here's a much better web-page hit counter: |
68dc0745 |
854 | |
500071f4 |
855 | $hits = int( (time() - 850_000_000) / rand(1_000) ); |
68dc0745 |
856 | |
857 | If the count doesn't impress your friends, then the code might. :-) |
858 | |
f52f3be2 |
859 | =head2 All I want to do is append a small amount of text to the end of a file. Do I still have to use locking? |
d74e8afc |
860 | X<append> X<file, append> |
05caf3a7 |
861 | |
109f0441 |
862 | If you are on a system that correctly implements C<flock> and you use |
863 | the example appending code from "perldoc -f flock" everything will be |
864 | OK even if the OS you are on doesn't implement append mode correctly |
865 | (if such a system exists.) So if you are happy to restrict yourself to |
866 | OSs that implement C<flock> (and that's not really much of a |
867 | restriction) then that is what you should do. |
05caf3a7 |
868 | |
869 | If you know you are only going to use a system that does correctly |
109f0441 |
870 | implement appending (i.e. not Win32) then you can omit the C<seek> |
871 | from the code in the previous answer. |
872 | |
873 | If you know you are only writing code to run on an OS and filesystem |
874 | that does implement append mode correctly (a local filesystem on a |
875 | modern Unix for example), and you keep the file in block-buffered mode |
876 | and you write less than one buffer-full of output between each manual |
877 | flushing of the buffer then each bufferload is almost guaranteed to be |
878 | written to the end of the file in one chunk without getting |
879 | intermingled with anyone else's output. You can also use the |
880 | C<syswrite> function which is simply a wrapper around your system's |
881 | C<write(2)> system call. |
05caf3a7 |
882 | |
883 | There is still a small theoretical chance that a signal will interrupt |
109f0441 |
884 | the system level C<write()> operation before completion. There is also |
885 | a possibility that some STDIO implementations may call multiple system |
886 | level C<write()>s even if the buffer was empty to start. There may be |
887 | some systems where this probability is reduced to zero, and this is |
888 | not a concern when using C<:perlio> instead of your system's STDIO. |
05caf3a7 |
889 | |
68dc0745 |
890 | =head2 How do I randomly update a binary file? |
d74e8afc |
891 | X<file, binary patch> |
68dc0745 |
892 | |
893 | If you're just trying to patch a binary, in many cases something as |
894 | simple as this works: |
895 | |
500071f4 |
896 | perl -i -pe 's{window manager}{window mangler}g' /usr/bin/emacs |
68dc0745 |
897 | |
898 | However, if you have fixed sized records, then you might do something more |
899 | like this: |
900 | |
500071f4 |
901 | $RECSIZE = 220; # size of record, in bytes |
902 | $recno = 37; # which record to update |
903 | open(FH, "+<somewhere") || die "can't update somewhere: $!"; |
904 | seek(FH, $recno * $RECSIZE, 0); |
905 | read(FH, $record, $RECSIZE) == $RECSIZE || die "can't read record $recno: $!"; |
906 | # munge the record |
907 | seek(FH, -$RECSIZE, 1); |
908 | print FH $record; |
909 | close FH; |
68dc0745 |
910 | |
911 | Locking and error checking are left as an exercise for the reader. |
a6dd486b |
912 | Don't forget them or you'll be quite sorry. |
68dc0745 |
913 | |
68dc0745 |
914 | =head2 How do I get a file's timestamp in perl? |
d74e8afc |
915 | X<timestamp> X<file, timestamp> |
68dc0745 |
916 | |
881bdbd4 |
917 | If you want to retrieve the time at which the file was last |
918 | read, written, or had its meta-data (owner, etc) changed, |
a05e4845 |
919 | you use the B<-A>, B<-M>, or B<-C> file test operations as |
881bdbd4 |
920 | documented in L<perlfunc>. These retrieve the age of the |
921 | file (measured against the start-time of your program) in |
922 | days as a floating point number. Some platforms may not have |
923 | all of these times. See L<perlport> for details. To |
924 | retrieve the "raw" time in seconds since the epoch, you |
925 | would call the stat function, then use localtime(), |
926 | gmtime(), or POSIX::strftime() to convert this into |
927 | human-readable form. |
68dc0745 |
928 | |
929 | Here's an example: |
930 | |
500071f4 |
931 | $write_secs = (stat($file))[9]; |
932 | printf "file %s updated at %s\n", $file, |
c8db1d39 |
933 | scalar localtime($write_secs); |
68dc0745 |
934 | |
935 | If you prefer something more legible, use the File::stat module |
936 | (part of the standard distribution in version 5.004 and later): |
937 | |
500071f4 |
938 | # error checking left as an exercise for reader. |
939 | use File::stat; |
940 | use Time::localtime; |
941 | $date_string = ctime(stat($file)->mtime); |
942 | print "file $file updated at $date_string\n"; |
68dc0745 |
943 | |
65acb1b1 |
944 | The POSIX::strftime() approach has the benefit of being, |
945 | in theory, independent of the current locale. See L<perllocale> |
946 | for details. |
68dc0745 |
947 | |
948 | =head2 How do I set a file's timestamp in perl? |
d74e8afc |
949 | X<timestamp> X<file, timestamp> |
68dc0745 |
950 | |
951 | You use the utime() function documented in L<perlfunc/utime>. |
952 | By way of example, here's a little program that copies the |
953 | read and write times from its first argument to all the rest |
954 | of them. |
955 | |
500071f4 |
956 | if (@ARGV < 2) { |
957 | die "usage: cptimes timestamp_file other_files ...\n"; |
958 | } |
959 | $timestamp = shift; |
960 | ($atime, $mtime) = (stat($timestamp))[8,9]; |
961 | utime $atime, $mtime, @ARGV; |
68dc0745 |
962 | |
65acb1b1 |
963 | Error checking is, as usual, left as an exercise for the reader. |
68dc0745 |
964 | |
19a1cd16 |
965 | The perldoc for utime also has an example that has the same |
966 | effect as touch(1) on files that I<already exist>. |
967 | |
968 | Certain file systems have a limited ability to store the times |
969 | on a file at the expected level of precision. For example, the |
970 | FAT and HPFS filesystem are unable to create dates on files with |
971 | a finer granularity than two seconds. This is a limitation of |
972 | the filesystems, not of utime(). |
68dc0745 |
973 | |
974 | =head2 How do I print to more than one file at once? |
d74e8afc |
975 | X<print, to multiple files> |
68dc0745 |
976 | |
49d635f9 |
977 | To connect one filehandle to several output filehandles, |
978 | you can use the IO::Tee or Tie::FileHandle::Multiplex modules. |
68dc0745 |
979 | |
49d635f9 |
980 | If you only have to do this once, you can print individually |
981 | to each filehandle. |
68dc0745 |
982 | |
500071f4 |
983 | for $fh (FH1, FH2, FH3) { print $fh "whatever\n" } |
5a964f20 |
984 | |
49d635f9 |
985 | =head2 How can I read in an entire file all at once? |
d74e8afc |
986 | X<slurp> X<file, slurping> |
68dc0745 |
987 | |
49d635f9 |
988 | You can use the File::Slurp module to do it in one step. |
68dc0745 |
989 | |
49d635f9 |
990 | use File::Slurp; |
197aec24 |
991 | |
49d635f9 |
992 | $all_of_it = read_file($filename); # entire file in scalar |
109f0441 |
993 | @all_lines = read_file($filename); # one line per element |
d92eb7b0 |
994 | |
995 | The customary Perl approach for processing all the lines in a file is to |
996 | do so one line at a time: |
997 | |
500071f4 |
998 | open (INPUT, $file) || die "can't open $file: $!"; |
999 | while (<INPUT>) { |
1000 | chomp; |
1001 | # do something with $_ |
1002 | } |
1003 | close(INPUT) || die "can't close $file: $!"; |
d92eb7b0 |
1004 | |
1005 | This is tremendously more efficient than reading the entire file into |
1006 | memory as an array of lines and then processing it one element at a time, |
a6dd486b |
1007 | which is often--if not almost always--the wrong approach. Whenever |
d92eb7b0 |
1008 | you see someone do this: |
1009 | |
500071f4 |
1010 | @lines = <INPUT>; |
d92eb7b0 |
1011 | |
30852c57 |
1012 | you should think long and hard about why you need everything loaded at |
1013 | once. It's just not a scalable solution. You might also find it more |
1014 | fun to use the standard Tie::File module, or the DB_File module's |
1015 | $DB_RECNO bindings, which allow you to tie an array to a file so that |
1016 | accessing an element the array actually accesses the corresponding |
1017 | line in the file. |
d92eb7b0 |
1018 | |
f05bbc40 |
1019 | You can read the entire filehandle contents into a scalar. |
d92eb7b0 |
1020 | |
500071f4 |
1021 | { |
d92eb7b0 |
1022 | local(*INPUT, $/); |
1023 | open (INPUT, $file) || die "can't open $file: $!"; |
1024 | $var = <INPUT>; |
500071f4 |
1025 | } |
d92eb7b0 |
1026 | |
197aec24 |
1027 | That temporarily undefs your record separator, and will automatically |
d92eb7b0 |
1028 | close the file at block exit. If the file is already open, just use this: |
1029 | |
500071f4 |
1030 | $var = do { local $/; <INPUT> }; |
d92eb7b0 |
1031 | |
f05bbc40 |
1032 | For ordinary files you can also use the read function. |
1033 | |
1034 | read( INPUT, $var, -s INPUT ); |
1035 | |
1036 | The third argument tests the byte size of the data on the INPUT filehandle |
1037 | and reads that many bytes into the buffer $var. |
1038 | |
68dc0745 |
1039 | =head2 How can I read in a file by paragraphs? |
d74e8afc |
1040 | X<file, reading by paragraphs> |
68dc0745 |
1041 | |
65acb1b1 |
1042 | Use the C<$/> variable (see L<perlvar> for details). You can either |
68dc0745 |
1043 | set it to C<""> to eliminate empty paragraphs (C<"abc\n\n\n\ndef">, |
1044 | for instance, gets treated as two paragraphs and not three), or |
1045 | C<"\n\n"> to accept empty paragraphs. |
1046 | |
197aec24 |
1047 | Note that a blank line must have no blanks in it. Thus |
c4db748a |
1048 | S<C<"fred\n \nstuff\n\n">> is one paragraph, but C<"fred\n\nstuff\n\n"> is two. |
65acb1b1 |
1049 | |
68dc0745 |
1050 | =head2 How can I read a single character from a file? From the keyboard? |
d74e8afc |
1051 | X<getc> X<file, reading one character at a time> |
68dc0745 |
1052 | |
1053 | You can use the builtin C<getc()> function for most filehandles, but |
1054 | it won't (easily) work on a terminal device. For STDIN, either use |
a6dd486b |
1055 | the Term::ReadKey module from CPAN or use the sample code in |
68dc0745 |
1056 | L<perlfunc/getc>. |
1057 | |
65acb1b1 |
1058 | If your system supports the portable operating system programming |
1059 | interface (POSIX), you can use the following code, which you'll note |
1060 | turns off echo processing as well. |
68dc0745 |
1061 | |
500071f4 |
1062 | #!/usr/bin/perl -w |
1063 | use strict; |
1064 | $| = 1; |
1065 | for (1..4) { |
1066 | my $got; |
1067 | print "gimme: "; |
1068 | $got = getone(); |
1069 | print "--> $got\n"; |
1070 | } |
68dc0745 |
1071 | exit; |
1072 | |
500071f4 |
1073 | BEGIN { |
68dc0745 |
1074 | use POSIX qw(:termios_h); |
1075 | |
1076 | my ($term, $oterm, $echo, $noecho, $fd_stdin); |
1077 | |
1078 | $fd_stdin = fileno(STDIN); |
1079 | |
1080 | $term = POSIX::Termios->new(); |
1081 | $term->getattr($fd_stdin); |
1082 | $oterm = $term->getlflag(); |
1083 | |
1084 | $echo = ECHO | ECHOK | ICANON; |
1085 | $noecho = $oterm & ~$echo; |
1086 | |
1087 | sub cbreak { |
500071f4 |
1088 | $term->setlflag($noecho); |
1089 | $term->setcc(VTIME, 1); |
1090 | $term->setattr($fd_stdin, TCSANOW); |
1091 | } |
ac9dac7f |
1092 | |
68dc0745 |
1093 | sub cooked { |
500071f4 |
1094 | $term->setlflag($oterm); |
1095 | $term->setcc(VTIME, 0); |
1096 | $term->setattr($fd_stdin, TCSANOW); |
1097 | } |
68dc0745 |
1098 | |
1099 | sub getone { |
500071f4 |
1100 | my $key = ''; |
1101 | cbreak(); |
1102 | sysread(STDIN, $key, 1); |
1103 | cooked(); |
1104 | return $key; |
1105 | } |
68dc0745 |
1106 | |
500071f4 |
1107 | } |
68dc0745 |
1108 | |
500071f4 |
1109 | END { cooked() } |
68dc0745 |
1110 | |
a6dd486b |
1111 | The Term::ReadKey module from CPAN may be easier to use. Recent versions |
65acb1b1 |
1112 | include also support for non-portable systems as well. |
68dc0745 |
1113 | |
500071f4 |
1114 | use Term::ReadKey; |
1115 | open(TTY, "</dev/tty"); |
1116 | print "Gimme a char: "; |
1117 | ReadMode "raw"; |
1118 | $key = ReadKey 0, *TTY; |
1119 | ReadMode "normal"; |
1120 | printf "\nYou said %s, char number %03d\n", |
1121 | $key, ord $key; |
68dc0745 |
1122 | |
65acb1b1 |
1123 | =head2 How can I tell whether there's a character waiting on a filehandle? |
68dc0745 |
1124 | |
5a964f20 |
1125 | The very first thing you should do is look into getting the Term::ReadKey |
65acb1b1 |
1126 | extension from CPAN. As we mentioned earlier, it now even has limited |
1127 | support for non-portable (read: not open systems, closed, proprietary, |
1128 | not POSIX, not Unix, etc) systems. |
5a964f20 |
1129 | |
1130 | You should also check out the Frequently Asked Questions list in |
68dc0745 |
1131 | comp.unix.* for things like this: the answer is essentially the same. |
1132 | It's very system dependent. Here's one solution that works on BSD |
1133 | systems: |
1134 | |
500071f4 |
1135 | sub key_ready { |
1136 | my($rin, $nfd); |
1137 | vec($rin, fileno(STDIN), 1) = 1; |
1138 | return $nfd = select($rin,undef,undef,0); |
1139 | } |
68dc0745 |
1140 | |
65acb1b1 |
1141 | If you want to find out how many characters are waiting, there's |
1142 | also the FIONREAD ioctl call to be looked at. The I<h2ph> tool that |
1143 | comes with Perl tries to convert C include files to Perl code, which |
1144 | can be C<require>d. FIONREAD ends up defined as a function in the |
1145 | I<sys/ioctl.ph> file: |
68dc0745 |
1146 | |
500071f4 |
1147 | require 'sys/ioctl.ph'; |
68dc0745 |
1148 | |
500071f4 |
1149 | $size = pack("L", 0); |
1150 | ioctl(FH, FIONREAD(), $size) or die "Couldn't call ioctl: $!\n"; |
1151 | $size = unpack("L", $size); |
68dc0745 |
1152 | |
5a964f20 |
1153 | If I<h2ph> wasn't installed or doesn't work for you, you can |
1154 | I<grep> the include files by hand: |
68dc0745 |
1155 | |
500071f4 |
1156 | % grep FIONREAD /usr/include/*/* |
1157 | /usr/include/asm/ioctls.h:#define FIONREAD 0x541B |
68dc0745 |
1158 | |
5a964f20 |
1159 | Or write a small C program using the editor of champions: |
68dc0745 |
1160 | |
500071f4 |
1161 | % cat > fionread.c |
1162 | #include <sys/ioctl.h> |
1163 | main() { |
1164 | printf("%#08x\n", FIONREAD); |
1165 | } |
1166 | ^D |
1167 | % cc -o fionread fionread.c |
1168 | % ./fionread |
1169 | 0x4004667f |
5a964f20 |
1170 | |
8305e449 |
1171 | And then hard code it, leaving porting as an exercise to your successor. |
5a964f20 |
1172 | |
500071f4 |
1173 | $FIONREAD = 0x4004667f; # XXX: opsys dependent |
5a964f20 |
1174 | |
500071f4 |
1175 | $size = pack("L", 0); |
1176 | ioctl(FH, $FIONREAD, $size) or die "Couldn't call ioctl: $!\n"; |
1177 | $size = unpack("L", $size); |
5a964f20 |
1178 | |
a6dd486b |
1179 | FIONREAD requires a filehandle connected to a stream, meaning that sockets, |
5a964f20 |
1180 | pipes, and tty devices work, but I<not> files. |
68dc0745 |
1181 | |
1182 | =head2 How do I do a C<tail -f> in perl? |
ac9dac7f |
1183 | X<tail> X<IO::Handle> X<File::Tail> X<clearerr> |
68dc0745 |
1184 | |
1185 | First try |
1186 | |
500071f4 |
1187 | seek(GWFILE, 0, 1); |
68dc0745 |
1188 | |
1189 | The statement C<seek(GWFILE, 0, 1)> doesn't change the current position, |
1190 | but it does clear the end-of-file condition on the handle, so that the |
ac9dac7f |
1191 | next C<< <GWFILE> >> makes Perl try again to read something. |
68dc0745 |
1192 | |
1193 | If that doesn't work (it relies on features of your stdio implementation), |
1194 | then you need something more like this: |
1195 | |
1196 | for (;;) { |
1197 | for ($curpos = tell(GWFILE); <GWFILE>; $curpos = tell(GWFILE)) { |
1198 | # search for some stuff and put it into files |
1199 | } |
1200 | # sleep for a while |
1201 | seek(GWFILE, $curpos, 0); # seek to where we had been |
1202 | } |
1203 | |
ac9dac7f |
1204 | If this still doesn't work, look into the C<clearerr> method |
1205 | from C<IO::Handle>, which resets the error and end-of-file states |
1206 | on the handle. |
68dc0745 |
1207 | |
ac9dac7f |
1208 | There's also a C<File::Tail> module from CPAN. |
65acb1b1 |
1209 | |
68dc0745 |
1210 | =head2 How do I dup() a filehandle in Perl? |
d74e8afc |
1211 | X<dup> |
68dc0745 |
1212 | |
1213 | If you check L<perlfunc/open>, you'll see that several of the ways |
1214 | to call open() should do the trick. For example: |
1215 | |
500071f4 |
1216 | open(LOG, ">>/foo/logfile"); |
1217 | open(STDERR, ">&LOG"); |
68dc0745 |
1218 | |
1219 | Or even with a literal numeric descriptor: |
1220 | |
1221 | $fd = $ENV{MHCONTEXTFD}; |
1222 | open(MHCONTEXT, "<&=$fd"); # like fdopen(3S) |
1223 | |
c47ff5f1 |
1224 | Note that "<&STDIN" makes a copy, but "<&=STDIN" make |
5a964f20 |
1225 | an alias. That means if you close an aliased handle, all |
197aec24 |
1226 | aliases become inaccessible. This is not true with |
5a964f20 |
1227 | a copied one. |
1228 | |
1229 | Error checking, as always, has been left as an exercise for the reader. |
68dc0745 |
1230 | |
1231 | =head2 How do I close a file descriptor by number? |
ee891a00 |
1232 | X<file, closing file descriptors> X<POSIX> X<close> |
1233 | |
1234 | If, for some reason, you have a file descriptor instead of a |
1235 | filehandle (perhaps you used C<POSIX::open>), you can use the |
1236 | C<close()> function from the C<POSIX> module: |
68dc0745 |
1237 | |
ee891a00 |
1238 | use POSIX (); |
109f0441 |
1239 | |
ee891a00 |
1240 | POSIX::close( $fd ); |
109f0441 |
1241 | |
ac003c96 |
1242 | This should rarely be necessary, as the Perl C<close()> function is to be |
68dc0745 |
1243 | used for things that Perl opened itself, even if it was a dup of a |
ac003c96 |
1244 | numeric descriptor as with C<MHCONTEXT> above. But if you really have |
68dc0745 |
1245 | to, you may be able to do this: |
1246 | |
500071f4 |
1247 | require 'sys/syscall.ph'; |
1248 | $rc = syscall(&SYS_close, $fd + 0); # must force numeric |
1249 | die "can't sysclose $fd: $!" unless $rc == -1; |
68dc0745 |
1250 | |
ee891a00 |
1251 | Or, just use the fdopen(3S) feature of C<open()>: |
d92eb7b0 |
1252 | |
500071f4 |
1253 | { |
ee891a00 |
1254 | open my( $fh ), "<&=$fd" or die "Cannot reopen fd=$fd: $!"; |
1255 | close $fh; |
500071f4 |
1256 | } |
d92eb7b0 |
1257 | |
883f1635 |
1258 | =head2 Why can't I use "C:\temp\foo" in DOS paths? Why doesn't `C:\temp\foo.exe` work? |
d74e8afc |
1259 | X<filename, DOS issues> |
68dc0745 |
1260 | |
1261 | Whoops! You just put a tab and a formfeed into that filename! |
1262 | Remember that within double quoted strings ("like\this"), the |
1263 | backslash is an escape character. The full list of these is in |
1264 | L<perlop/Quote and Quote-like Operators>. Unsurprisingly, you don't |
1265 | have a file called "c:(tab)emp(formfeed)oo" or |
65acb1b1 |
1266 | "c:(tab)emp(formfeed)oo.exe" on your legacy DOS filesystem. |
68dc0745 |
1267 | |
1268 | Either single-quote your strings, or (preferably) use forward slashes. |
46fc3d4c |
1269 | Since all DOS and Windows versions since something like MS-DOS 2.0 or so |
68dc0745 |
1270 | have treated C</> and C<\> the same in a path, you might as well use the |
a6dd486b |
1271 | one that doesn't clash with Perl--or the POSIX shell, ANSI C and C++, |
65acb1b1 |
1272 | awk, Tcl, Java, or Python, just to mention a few. POSIX paths |
1273 | are more portable, too. |
68dc0745 |
1274 | |
1275 | =head2 Why doesn't glob("*.*") get all the files? |
d74e8afc |
1276 | X<glob> |
68dc0745 |
1277 | |
1278 | Because even on non-Unix ports, Perl's glob function follows standard |
46fc3d4c |
1279 | Unix globbing semantics. You'll need C<glob("*")> to get all (non-hidden) |
65acb1b1 |
1280 | files. This makes glob() portable even to legacy systems. Your |
1281 | port may include proprietary globbing functions as well. Check its |
1282 | documentation for details. |
68dc0745 |
1283 | |
1284 | =head2 Why does Perl let me delete read-only files? Why does C<-i> clobber protected files? Isn't this a bug in Perl? |
1285 | |
06a5f41f |
1286 | This is elaborately and painstakingly described in the |
1287 | F<file-dir-perms> article in the "Far More Than You Ever Wanted To |
49d635f9 |
1288 | Know" collection in http://www.cpan.org/misc/olddoc/FMTEYEWTK.tgz . |
68dc0745 |
1289 | |
1290 | The executive summary: learn how your filesystem works. The |
1291 | permissions on a file say what can happen to the data in that file. |
1292 | The permissions on a directory say what can happen to the list of |
1293 | files in that directory. If you delete a file, you're removing its |
1294 | name from the directory (so the operation depends on the permissions |
1295 | of the directory, not of the file). If you try to write to the file, |
1296 | the permissions of the file govern whether you're allowed to. |
1297 | |
1298 | =head2 How do I select a random line from a file? |
d74e8afc |
1299 | X<file, selecting a random line> |
68dc0745 |
1300 | |
109f0441 |
1301 | Short of loading the file into a database or pre-indexing the lines in |
1302 | the file, there are a couple of things that you can do. |
1303 | |
1304 | Here's a reservoir-sampling algorithm from the Camel Book: |
68dc0745 |
1305 | |
500071f4 |
1306 | srand; |
1307 | rand($.) < 1 && ($line = $_) while <>; |
68dc0745 |
1308 | |
49d635f9 |
1309 | This has a significant advantage in space over reading the whole file |
1310 | in. You can find a proof of this method in I<The Art of Computer |
1311 | Programming>, Volume 2, Section 3.4.2, by Donald E. Knuth. |
1312 | |
109f0441 |
1313 | You can use the C<File::Random> module which provides a function |
49d635f9 |
1314 | for that algorithm: |
1315 | |
1316 | use File::Random qw/random_line/; |
1317 | my $line = random_line($filename); |
1318 | |
109f0441 |
1319 | Another way is to use the C<Tie::File> module, which treats the entire |
49d635f9 |
1320 | file as an array. Simply access a random array element. |
68dc0745 |
1321 | |
65acb1b1 |
1322 | =head2 Why do I get weird spaces when I print an array of lines? |
1323 | |
109f0441 |
1324 | (contributed by brian d foy) |
1325 | |
1326 | If you are seeing spaces between the elements of your array when |
1327 | you print the array, you are probably interpolating the array in |
1328 | double quotes: |
1329 | |
1330 | my @animals = qw(camel llama alpaca vicuna); |
1331 | print "animals are: @animals\n"; |
65acb1b1 |
1332 | |
109f0441 |
1333 | It's the double quotes, not the C<print>, doing this. Whenever you |
1334 | interpolate an array in a double quote context, Perl joins the |
1335 | elements with spaces (or whatever is in C<$">, which is a space by |
1336 | default): |
65acb1b1 |
1337 | |
109f0441 |
1338 | animals are: camel llama alpaca vicuna |
65acb1b1 |
1339 | |
109f0441 |
1340 | This is different than printing the array without the interpolation: |
65acb1b1 |
1341 | |
109f0441 |
1342 | my @animals = qw(camel llama alpaca vicuna); |
1343 | print "animals are: ", @animals, "\n"; |
65acb1b1 |
1344 | |
109f0441 |
1345 | Now the output doesn't have the spaces between the elements because |
1346 | the elements of C<@animals> simply become part of the list to |
1347 | C<print>: |
65acb1b1 |
1348 | |
109f0441 |
1349 | animals are: camelllamaalpacavicuna |
1350 | |
1351 | You might notice this when each of the elements of C<@array> end with |
1352 | a newline. You expect to print one element per line, but notice that |
1353 | every line after the first is indented: |
1354 | |
1355 | this is a line |
1356 | this is another line |
1357 | this is the third line |
1358 | |
1359 | That extra space comes from the interpolation of the array. If you |
1360 | don't want to put anything between your array elements, don't use the |
1361 | array in double quotes. You can send it to print without them: |
65acb1b1 |
1362 | |
500071f4 |
1363 | print @lines; |
1364 | |
109f0441 |
1365 | =head2 How do I traverse a directory tree? |
1366 | |
1367 | (contributed by brian d foy) |
1368 | |
1369 | The C<File::Find> module, which comes with Perl, does all of the hard |
1370 | work to traverse a directory structure. It comes with Perl. You simply |
1371 | call the C<find> subroutine with a callback subroutine and the |
1372 | directories you want to traverse: |
1373 | |
1374 | use File::Find; |
1375 | |
1376 | find( \&wanted, @directories ); |
1377 | |
1378 | sub wanted { |
1379 | # full path in $File::Find::name |
1380 | # just filename in $_ |
1381 | ... do whatever you want to do ... |
1382 | } |
1383 | |
1384 | The C<File::Find::Closures>, which you can download from CPAN, provides |
1385 | many ready-to-use subroutines that you can use with C<File::Find>. |
1386 | |
1387 | The C<File::Finder>, which you can download from CPAN, can help you |
1388 | create the callback subroutine using something closer to the syntax of |
1389 | the C<find> command-line utility: |
1390 | |
1391 | use File::Find; |
1392 | use File::Finder; |
1393 | |
1394 | my $deep_dirs = File::Finder->depth->type('d')->ls->exec('rmdir','{}'); |
1395 | |
1396 | find( $deep_dirs->as_options, @places ); |
1397 | |
1398 | The C<File::Find::Rule> module, which you can download from CPAN, has |
1399 | a similar interface, but does the traversal for you too: |
1400 | |
1401 | use File::Find::Rule; |
1402 | |
1403 | my @files = File::Find::Rule->file() |
1404 | ->name( '*.pm' ) |
1405 | ->in( @INC ); |
1406 | |
1407 | =head2 How do I delete a directory tree? |
1408 | |
1409 | (contributed by brian d foy) |
1410 | |
1411 | If you have an empty directory, you can use Perl's built-in C<rmdir>. If |
1412 | the directory is not empty (so, no files or subdirectories), you either |
1413 | have to empty it yourself (a lot of work) or use a module to help you. |
1414 | |
1415 | The C<File::Path> module, which comes with Perl, has a C<rmtree> which |
1416 | can take care of all of the hard work for you: |
1417 | |
1418 | use File::Path qw(rmtree); |
1419 | |
1420 | rmtree( \@directories, 0, 0 ); |
1421 | |
1422 | The first argument to C<rmtree> is either a string representing a directory path |
1423 | or an array reference. The second argument controls progress messages, and the |
1424 | third argument controls the handling of files you don't have permissions to |
1425 | delete. See the C<File::Path> module for the details. |
1426 | |
1427 | =head2 How do I copy an entire directory? |
1428 | |
1429 | (contributed by Shlomi Fish) |
1430 | |
1431 | To do the equivalent of C<cp -R> (i.e. copy an entire directory tree |
1432 | recursively) in portable Perl, you'll either need to write something yourself |
1433 | or find a good CPAN module such as L<File::Copy::Recursive>. |
500071f4 |
1434 | =head1 REVISION |
1435 | |
109f0441 |
1436 | Revision: $Revision$ |
500071f4 |
1437 | |
109f0441 |
1438 | Date: $Date$ |
500071f4 |
1439 | |
1440 | See L<perlfaq> for source control details and availability. |
65acb1b1 |
1441 | |
68dc0745 |
1442 | =head1 AUTHOR AND COPYRIGHT |
1443 | |
109f0441 |
1444 | Copyright (c) 1997-2009 Tom Christiansen, Nathan Torkington, and |
7678cced |
1445 | other authors as noted. All rights reserved. |
5a964f20 |
1446 | |
5a7beb56 |
1447 | This documentation is free; you can redistribute it and/or modify it |
1448 | under the same terms as Perl itself. |
c8db1d39 |
1449 | |
87275199 |
1450 | Irrespective of its distribution, all code examples here are in the public |
c8db1d39 |
1451 | domain. You are permitted and encouraged to use this code and any |
1452 | derivatives thereof in your own programs for fun or for profit as you |
1453 | see fit. A simple comment in the code giving credit to the FAQ would |
1454 | be courteous but is not required. |