extra code in pp_concat, Take 2
[p5sagit/p5-mst-13.2.git] / pod / perlopentut.pod
CommitLineData
f8284313 1=head1 NAME
2
3perlopentut - tutorial on opening things in Perl
4
5=head1 DESCRIPTION
6
7Perl has two simple, built-in ways to open files: the shell way for
1a193132 8convenience, and the C way for precision. The shell way also has 2- and
93-argument forms, which have different semantics for handling the filename.
10The choice is yours.
f8284313 11
12=head1 Open E<agrave> la shell
13
14Perl's C<open> function was designed to mimic the way command-line
15redirection in the shell works. Here are some basic examples
16from the shell:
17
18 $ myprogram file1 file2 file3
19 $ myprogram < inputfile
20 $ myprogram > outputfile
21 $ myprogram >> outputfile
22 $ myprogram | otherprogram
23 $ otherprogram | myprogram
24
25And here are some more advanced examples:
26
27 $ otherprogram | myprogram f1 - f2
28 $ otherprogram 2>&1 | myprogram -
29 $ myprogram <&3
30 $ myprogram >&4
31
32Programmers accustomed to constructs like those above can take comfort
33in learning that Perl directly supports these familiar constructs using
34virtually the same syntax as the shell.
35
36=head2 Simple Opens
37
38The C<open> function takes two arguments: the first is a filehandle,
39and the second is a single string comprising both what to open and how
40to open it. C<open> returns true when it works, and when it fails,
1a193132 41returns a false value and sets the special variable C<$!> to reflect
f8284313 42the system error. If the filehandle was previously opened, it will
43be implicitly closed first.
44
45For example:
46
47 open(INFO, "datafile") || die("can't open datafile: $!");
48 open(INFO, "< datafile") || die("can't open datafile: $!");
49 open(RESULTS,"> runstats") || die("can't open runstats: $!");
50 open(LOG, ">> logfile ") || die("can't open logfile: $!");
51
52If you prefer the low-punctuation version, you could write that this way:
53
54 open INFO, "< datafile" or die "can't open datafile: $!";
55 open RESULTS,"> runstats" or die "can't open runstats: $!";
56 open LOG, ">> logfile " or die "can't open logfile: $!";
57
58A few things to notice. First, the leading less-than is optional.
59If omitted, Perl assumes that you want to open the file for reading.
60
1a193132 61Note also that the first example uses the C<||> logical operator, and the
62second uses C<or>, which has lower precedence. Using C<||> in the latter
63examples would effectively mean
64
65 open INFO, ( "< datafile" || die "can't open datafile: $!" );
66
67which is definitely not what you want.
68
f8284313 69The other important thing to notice is that, just as in the shell,
6b0ac556 70any whitespace before or after the filename is ignored. This is good,
f8284313 71because you wouldn't want these to do different things:
72
73 open INFO, "<datafile"
74 open INFO, "< datafile"
75 open INFO, "< datafile"
76
1a193132 77Ignoring surrounding whitespace also helps for when you read a filename
78in from a different file, and forget to trim it before opening:
f8284313 79
80 $filename = <INFO>; # oops, \n still there
81 open(EXTRA, "< $filename") || die "can't open $filename: $!";
82
83This is not a bug, but a feature. Because C<open> mimics the shell in
84its style of using redirection arrows to specify how to open the file, it
6b0ac556 85also does so with respect to extra whitespace around the filename itself
13a2d996 86as well. For accessing files with naughty names, see
87L<"Dispelling the Dweomer">.
f8284313 88
1a193132 89There is also a 3-argument version of C<open>, which lets you put the
90special redirection characters into their own argument:
91
92 open( INFO, ">", $datafile ) || die "Can't create $datafile: $!";
93
94In this case, the filename to open is the actual string in C<$datafile>,
95so you don't have to worry about C<$datafile> containing characters
96that might influence the open mode, or whitespace at the beginning of
97the filename that would be absorbed in the 2-argument version. Also,
98any reduction of unnecessary string interpolation is a good thing.
99
100=head2 Indirect Filehandles
101
102C<open>'s first argument can be a reference to a filehandle. As of
103perl 5.6.0, if the argument is uninitialized, Perl will automatically
104create a filehandle and put a reference to it in the first argument,
105like so:
106
107 open( my $in, $infile ) or die "Couldn't read $infile: $!";
108 while ( <$in> ) {
109 # do something with $_
110 }
111 close $in;
112
113Indirect filehandles make namespace management easier. Since filehandles
114are global to the current package, two subroutines trying to open
115C<INFILE> will clash. With two functions opening indirect filehandles
116like C<my $infile>, there's no clash and no need to worry about future
117conflicts.
118
119Another convenient behavior is that an indirect filehandle automatically
120closes when it goes out of scope or when you undefine it:
121
122 sub firstline {
123 open( my $in, shift ) && return scalar <$in>;
124 # no close() required
125 }
126
f8284313 127=head2 Pipe Opens
128
129In C, when you want to open a file using the standard I/O library,
130you use the C<fopen> function, but when opening a pipe, you use the
131C<popen> function. But in the shell, you just use a different redirection
132character. That's also the case for Perl. The C<open> call
133remains the same--just its argument differs.
134
f5daac4a 135If the leading character is a pipe symbol, C<open> starts up a new
1a193132 136command and opens a write-only filehandle leading into that command.
f8284313 137This lets you write into that handle and have what you write show up on
138that command's standard input. For example:
139
369c5433 140 open(PRINTER, "| lpr -Plp1") || die "can't run lpr: $!";
f8284313 141 print PRINTER "stuff\n";
142 close(PRINTER) || die "can't close lpr: $!";
143
144If the trailing character is a pipe, you start up a new command and open a
145read-only filehandle leading out of that command. This lets whatever that
146command writes to its standard output show up on your handle for reading.
147For example:
148
1a193132 149 open(NET, "netstat -i -n |") || die "can't fork netstat: $!";
f8284313 150 while (<NET>) { } # do something with input
151 close(NET) || die "can't close netstat: $!";
152
369c5433 153What happens if you try to open a pipe to or from a non-existent
154command? If possible, Perl will detect the failure and set C<$!> as
155usual. But if the command contains special shell characters, such as
156C<E<gt>> or C<*>, called 'metacharacters', Perl does not execute the
157command directly. Instead, Perl runs the shell, which then tries to
158run the command. This means that it's the shell that gets the error
159indication. In such a case, the C<open> call will only indicate
160failure if Perl can't even run the shell. See L<perlfaq8/"How can I
161capture STDERR from an external command?"> to see how to cope with
162this. There's also an explanation in L<perlipc>.
f8284313 163
164If you would like to open a bidirectional pipe, the IPC::Open2
13a2d996 165library will handle this for you. Check out
166L<perlipc/"Bidirectional Communication with Another Process">
f8284313 167
168=head2 The Minus File
169
170Again following the lead of the standard shell utilities, Perl's
171C<open> function treats a file whose name is a single minus, "-", in a
172special way. If you open minus for reading, it really means to access
173the standard input. If you open minus for writing, it really means to
174access the standard output.
175
40b7eeef 176If minus can be used as the default input or default output, what happens
f8284313 177if you open a pipe into or out of minus? What's the default command it
40b7eeef 178would run? The same script as you're currently running! This is actually
13a2d996 179a stealth C<fork> hidden inside an C<open> call. See
180L<perlipc/"Safe Pipe Opens"> for details.
f8284313 181
182=head2 Mixing Reads and Writes
183
184It is possible to specify both read and write access. All you do is
185add a "+" symbol in front of the redirection. But as in the shell,
186using a less-than on a file never creates a new file; it only opens an
187existing one. On the other hand, using a greater-than always clobbers
188(truncates to zero length) an existing file, or creates a brand-new one
189if there isn't an old one. Adding a "+" for read-write doesn't affect
190whether it only works on existing files or always clobbers existing ones.
191
192 open(WTMP, "+< /usr/adm/wtmp")
193 || die "can't open /usr/adm/wtmp: $!";
194
2359510d 195 open(SCREEN, "+> lkscreen")
196 || die "can't open lkscreen: $!";
f8284313 197
2359510d 198 open(LOGFILE, "+>> /var/log/applog"
199 || die "can't open /var/log/applog: $!";
f8284313 200
201The first one won't create a new file, and the second one will always
202clobber an old one. The third one will create a new file if necessary
203and not clobber an old one, and it will allow you to read at any point
204in the file, but all writes will always go to the end. In short,
205the first case is substantially more common than the second and third
206cases, which are almost always wrong. (If you know C, the plus in
207Perl's C<open> is historically derived from the one in C's fopen(3S),
208which it ultimately calls.)
209
210In fact, when it comes to updating a file, unless you're working on
211a binary file as in the WTMP case above, you probably don't want to
212use this approach for updating. Instead, Perl's B<-i> flag comes to
213the rescue. The following command takes all the C, C++, or yacc source
214or header files and changes all their foo's to bar's, leaving
1a193132 215the old version in the original filename with a ".orig" tacked
f8284313 216on the end:
217
218 $ perl -i.orig -pe 's/\bfoo\b/bar/g' *.[Cchy]
219
220This is a short cut for some renaming games that are really
221the best way to update textfiles. See the second question in
222L<perlfaq5> for more details.
223
224=head2 Filters
225
226One of the most common uses for C<open> is one you never
227even notice. When you process the ARGV filehandle using
c47ff5f1 228C<< <ARGV> >>, Perl actually does an implicit open
f8284313 229on each file in @ARGV. Thus a program called like this:
230
231 $ myprogram file1 file2 file3
232
233Can have all its files opened and processed one at a time
234using a construct no more complex than:
235
236 while (<>) {
237 # do something with $_
238 }
239
240If @ARGV is empty when the loop first begins, Perl pretends you've opened
241up minus, that is, the standard input. In fact, $ARGV, the currently
c47ff5f1 242open file during C<< <ARGV> >> processing, is even set to "-"
f8284313 243in these circumstances.
244
245You are welcome to pre-process your @ARGV before starting the loop to
246make sure it's to your liking. One reason to do this might be to remove
247command options beginning with a minus. While you can always roll the
1a193132 248simple ones by hand, the Getopts modules are good for this:
f8284313 249
250 use Getopt::Std;
251
252 # -v, -D, -o ARG, sets $opt_v, $opt_D, $opt_o
253 getopts("vDo:");
254
255 # -v, -D, -o ARG, sets $args{v}, $args{D}, $args{o}
256 getopts("vDo:", \%args);
257
258Or the standard Getopt::Long module to permit named arguments:
259
260 use Getopt::Long;
261 GetOptions( "verbose" => \$verbose, # --verbose
262 "Debug" => \$debug, # --Debug
263 "output=s" => \$output );
264 # --output=somestring or --output somestring
265
266Another reason for preprocessing arguments is to make an empty
267argument list default to all files:
268
269 @ARGV = glob("*") unless @ARGV;
270
271You could even filter out all but plain, text files. This is a bit
272silent, of course, and you might prefer to mention them on the way.
273
274 @ARGV = grep { -f && -T } @ARGV;
275
276If you're using the B<-n> or B<-p> command-line options, you
277should put changes to @ARGV in a C<BEGIN{}> block.
278
279Remember that a normal C<open> has special properties, in that it might
280call fopen(3S) or it might called popen(3S), depending on what its
281argument looks like; that's why it's sometimes called "magic open".
282Here's an example:
283
284 $pwdinfo = `domainname` =~ /^(\(none\))?$/
285 ? '< /etc/passwd'
286 : 'ypcat passwd |';
287
288 open(PWD, $pwdinfo)
289 or die "can't open $pwdinfo: $!";
290
291This sort of thing also comes into play in filter processing. Because
c47ff5f1 292C<< <ARGV> >> processing employs the normal, shell-style Perl C<open>,
f8284313 293it respects all the special things we've already seen:
294
295 $ myprogram f1 "cmd1|" - f2 "cmd2|" f3 < tmpfile
296
297That program will read from the file F<f1>, the process F<cmd1>, standard
298input (F<tmpfile> in this case), the F<f2> file, the F<cmd2> command,
299and finally the F<f3> file.
300
1a193132 301Yes, this also means that if you have files named "-" (and so on) in
302your directory, they won't be processed as literal files by C<open>.
303You'll need to pass them as "./-", much as you would for the I<rm> program,
304or you could use C<sysopen> as described below.
f8284313 305
306One of the more interesting applications is to change files of a certain
307name into pipes. For example, to autoprocess gzipped or compressed
308files by decompressing them with I<gzip>:
309
310 @ARGV = map { /^\.(gz|Z)$/ ? "gzip -dc $_ |" : $_ } @ARGV;
311
312Or, if you have the I<GET> program installed from LWP,
313you can fetch URLs before processing them:
314
315 @ARGV = map { m#^\w+://# ? "GET $_ |" : $_ } @ARGV;
316
c47ff5f1 317It's not for nothing that this is called magic C<< <ARGV> >>.
f8284313 318Pretty nifty, eh?
319
320=head1 Open E<agrave> la C
321
322If you want the convenience of the shell, then Perl's C<open> is
323definitely the way to go. On the other hand, if you want finer precision
1a193132 324than C's simplistic fopen(3S) provides you should look to Perl's
f8284313 325C<sysopen>, which is a direct hook into the open(2) system call.
326That does mean it's a bit more involved, but that's the price of
327precision.
328
329C<sysopen> takes 3 (or 4) arguments.
330
331 sysopen HANDLE, PATH, FLAGS, [MASK]
332
333The HANDLE argument is a filehandle just as with C<open>. The PATH is
334a literal path, one that doesn't pay attention to any greater-thans or
6b0ac556 335less-thans or pipes or minuses, nor ignore whitespace. If it's there,
f8284313 336it's part of the path. The FLAGS argument contains one or more values
337derived from the Fcntl module that have been or'd together using the
338bitwise "|" operator. The final argument, the MASK, is optional; if
339present, it is combined with the user's current umask for the creation
340mode of the file. You should usually omit this.
341
342Although the traditional values of read-only, write-only, and read-write
343are 0, 1, and 2 respectively, this is known not to hold true on some
344systems. Instead, it's best to load in the appropriate constants first
345from the Fcntl module, which supplies the following standard flags:
346
347 O_RDONLY Read only
348 O_WRONLY Write only
349 O_RDWR Read and write
350 O_CREAT Create the file if it doesn't exist
351 O_EXCL Fail if the file already exists
352 O_APPEND Append to the file
353 O_TRUNC Truncate the file
354 O_NONBLOCK Non-blocking access
355
ca6e1c26 356Less common flags that are sometimes available on some operating
357systems include C<O_BINARY>, C<O_TEXT>, C<O_SHLOCK>, C<O_EXLOCK>,
358C<O_DEFER>, C<O_SYNC>, C<O_ASYNC>, C<O_DSYNC>, C<O_RSYNC>,
359C<O_NOCTTY>, C<O_NDELAY> and C<O_LARGEFILE>. Consult your open(2)
360manpage or its local equivalent for details. (Note: starting from
1a193132 361Perl release 5.6 the C<O_LARGEFILE> flag, if available, is automatically
106325ad 362added to the sysopen() flags because large files are the default.)
f8284313 363
364Here's how to use C<sysopen> to emulate the simple C<open> calls we had
365before. We'll omit the C<|| die $!> checks for clarity, but make sure
366you always check the return values in real code. These aren't quite
6b0ac556 367the same, since C<open> will trim leading and trailing whitespace,
1a193132 368but you'll get the idea.
f8284313 369
370To open a file for reading:
371
372 open(FH, "< $path");
373 sysopen(FH, $path, O_RDONLY);
374
375To open a file for writing, creating a new file if needed or else truncating
376an old file:
377
378 open(FH, "> $path");
379 sysopen(FH, $path, O_WRONLY | O_TRUNC | O_CREAT);
380
381To open a file for appending, creating one if necessary:
382
383 open(FH, ">> $path");
384 sysopen(FH, $path, O_WRONLY | O_APPEND | O_CREAT);
385
386To open a file for update, where the file must already exist:
387
388 open(FH, "+< $path");
389 sysopen(FH, $path, O_RDWR);
390
391And here are things you can do with C<sysopen> that you cannot do with
1a193132 392a regular C<open>. As you'll see, it's just a matter of controlling the
f8284313 393flags in the third argument.
394
395To open a file for writing, creating a new file which must not previously
396exist:
397
398 sysopen(FH, $path, O_WRONLY | O_EXCL | O_CREAT);
399
400To open a file for appending, where that file must already exist:
401
402 sysopen(FH, $path, O_WRONLY | O_APPEND);
403
404To open a file for update, creating a new file if necessary:
405
406 sysopen(FH, $path, O_RDWR | O_CREAT);
407
408To open a file for update, where that file must not already exist:
409
410 sysopen(FH, $path, O_RDWR | O_EXCL | O_CREAT);
411
412To open a file without blocking, creating one if necessary:
413
414 sysopen(FH, $path, O_WRONLY | O_NONBLOCK | O_CREAT);
415
416=head2 Permissions E<agrave> la mode
417
418If you omit the MASK argument to C<sysopen>, Perl uses the octal value
4190666. The normal MASK to use for executables and directories should
420be 0777, and for anything else, 0666.
421
422Why so permissive? Well, it isn't really. The MASK will be modified
423by your process's current C<umask>. A umask is a number representing
424I<disabled> permissions bits; that is, bits that will not be turned on
425in the created files' permissions field.
426
427For example, if your C<umask> were 027, then the 020 part would
428disable the group from writing, and the 007 part would disable others
429from reading, writing, or executing. Under these conditions, passing
1a193132 430C<sysopen> 0666 would create a file with mode 0640, since C<0666 & ~027>
f8284313 431is 0640.
432
433You should seldom use the MASK argument to C<sysopen()>. That takes
434away the user's freedom to choose what permission new files will have.
435Denying choice is almost always a bad thing. One exception would be for
436cases where sensitive or private data is being stored, such as with mail
437folders, cookie files, and internal temporary files.
438
439=head1 Obscure Open Tricks
440
441=head2 Re-Opening Files (dups)
442
443Sometimes you already have a filehandle open, and want to make another
444handle that's a duplicate of the first one. In the shell, we place an
445ampersand in front of a file descriptor number when doing redirections.
c47ff5f1 446For example, C<< 2>&1 >> makes descriptor 2 (that's STDERR in Perl)
f8284313 447be redirected into descriptor 1 (which is usually Perl's STDOUT).
448The same is essentially true in Perl: a filename that begins with an
449ampersand is treated instead as a file descriptor if a number, or as a
450filehandle if a string.
451
452 open(SAVEOUT, ">&SAVEERR") || die "couldn't dup SAVEERR: $!";
453 open(MHCONTEXT, "<&4") || die "couldn't dup fd4: $!";
454
455That means that if a function is expecting a filename, but you don't
456want to give it a filename because you already have the file open, you
457can just pass the filehandle with a leading ampersand. It's best to
458use a fully qualified handle though, just in case the function happens
459to be in a different package:
460
461 somefunction("&main::LOGFILE");
462
463This way if somefunction() is planning on opening its argument, it can
464just use the already opened handle. This differs from passing a handle,
465because with a handle, you don't open the file. Here you have something
466you can pass to open.
467
468If you have one of those tricky, newfangled I/O objects that the C++
469folks are raving about, then this doesn't work because those aren't a
470proper filehandle in the native Perl sense. You'll have to use fileno()
471to pull out the proper descriptor number, assuming you can:
472
473 use IO::Socket;
474 $handle = IO::Socket::INET->new("www.perl.com:80");
475 $fd = $handle->fileno;
476 somefunction("&$fd"); # not an indirect function call
477
478It can be easier (and certainly will be faster) just to use real
479filehandles though:
480
481 use IO::Socket;
482 local *REMOTE = IO::Socket::INET->new("www.perl.com:80");
483 die "can't connect" unless defined(fileno(REMOTE));
484 somefunction("&main::REMOTE");
485
486If the filehandle or descriptor number is preceded not just with a simple
487"&" but rather with a "&=" combination, then Perl will not create a
488completely new descriptor opened to the same place using the dup(2)
489system call. Instead, it will just make something of an alias to the
490existing one using the fdopen(3S) library call This is slightly more
491parsimonious of systems resources, although this is less a concern
492these days. Here's an example of that:
493
494 $fd = $ENV{"MHCONTEXTFD"};
495 open(MHCONTEXT, "<&=$fd") or die "couldn't fdopen $fd: $!";
496
c47ff5f1 497If you're using magic C<< <ARGV> >>, you could even pass in as a
498command line argument in @ARGV something like C<"<&=$MHCONTEXTFD">,
f8284313 499but we've never seen anyone actually do this.
500
501=head2 Dispelling the Dweomer
502
503Perl is more of a DWIMmer language than something like Java--where DWIM
504is an acronym for "do what I mean". But this principle sometimes leads
505to more hidden magic than one knows what to do with. In this way, Perl
506is also filled with I<dweomer>, an obscure word meaning an enchantment.
507Sometimes, Perl's DWIMmer is just too much like dweomer for comfort.
508
509If magic C<open> is a bit too magical for you, you don't have to turn
510to C<sysopen>. To open a file with arbitrary weird characters in
511it, it's necessary to protect any leading and trailing whitespace.
512Leading whitespace is protected by inserting a C<"./"> in front of a
513filename that starts with whitespace. Trailing whitespace is protected
1a193132 514by appending an ASCII NUL byte (C<"\0">) at the end of the string.
f8284313 515
516 $file =~ s#^(\s)#./$1#;
517 open(FH, "< $file\0") || die "can't open $file: $!";
518
519This assumes, of course, that your system considers dot the current
520working directory, slash the directory separator, and disallows ASCII
521NULs within a valid filename. Most systems follow these conventions,
522including all POSIX systems as well as proprietary Microsoft systems.
523The only vaguely popular system that doesn't work this way is the
8e30f651 524"Classic" Macintosh system, which uses a colon where the rest of us
f8284313 525use a slash. Maybe C<sysopen> isn't such a bad idea after all.
526
c47ff5f1 527If you want to use C<< <ARGV> >> processing in a totally boring
f8284313 528and non-magical way, you could do this first:
529
530 # "Sam sat on the ground and put his head in his hands.
531 # 'I wish I had never come here, and I don't want to see
532 # no more magic,' he said, and fell silent."
533 for (@ARGV) {
534 s#^([^./])#./$1#;
535 $_ .= "\0";
536 }
537 while (<>) {
538 # now process $_
539 }
540
541But be warned that users will not appreciate being unable to use "-"
542to mean standard input, per the standard convention.
543
544=head2 Paths as Opens
545
546You've probably noticed how Perl's C<warn> and C<die> functions can
547produce messages like:
548
1761cee5 549 Some warning at scriptname line 29, <FH> line 7.
f8284313 550
551That's because you opened a filehandle FH, and had read in seven records
1a193132 552from it. But what was the name of the file, rather than the handle?
f8284313 553
1a193132 554If you aren't running with C<strict refs>, or if you've turned them off
f8284313 555temporarily, then all you have to do is this:
556
557 open($path, "< $path") || die "can't open $path: $!";
558 while (<$path>) {
559 # whatever
560 }
561
562Since you're using the pathname of the file as its handle,
563you'll get warnings more like
564
1761cee5 565 Some warning at scriptname line 29, </etc/motd> line 7.
f8284313 566
567=head2 Single Argument Open
568
569Remember how we said that Perl's open took two arguments? That was a
570passive prevarication. You see, it can also take just one argument.
571If and only if the variable is a global variable, not a lexical, you
572can pass C<open> just one argument, the filehandle, and it will
573get the path from the global scalar variable of the same name.
574
575 $FILE = "/etc/motd";
576 open FILE or die "can't open $FILE: $!";
577 while (<FILE>) {
578 # whatever
579 }
580
581Why is this here? Someone has to cater to the hysterical porpoises.
582It's something that's been in Perl since the very beginning, if not
583before.
584
585=head2 Playing with STDIN and STDOUT
586
587One clever move with STDOUT is to explicitly close it when you're done
588with the program.
589
590 END { close(STDOUT) || die "can't close stdout: $!" }
591
592If you don't do this, and your program fills up the disk partition due
593to a command line redirection, it won't report the error exit with a
594failure status.
595
596You don't have to accept the STDIN and STDOUT you were given. You are
597welcome to reopen them if you'd like.
598
599 open(STDIN, "< datafile")
600 || die "can't open datafile: $!";
601
602 open(STDOUT, "> output")
603 || die "can't open output: $!";
604
00dcde61 605And then these can be accessed directly or passed on to subprocesses.
f8284313 606This makes it look as though the program were initially invoked
607with those redirections from the command line.
608
609It's probably more interesting to connect these to pipes. For example:
610
611 $pager = $ENV{PAGER} || "(less || more)";
612 open(STDOUT, "| $pager")
613 || die "can't fork a pager: $!";
614
615This makes it appear as though your program were called with its stdout
616already piped into your pager. You can also use this kind of thing
617in conjunction with an implicit fork to yourself. You might do this
618if you would rather handle the post processing in your own program,
619just in a different process:
620
621 head(100);
622 while (<>) {
623 print;
624 }
625
626 sub head {
627 my $lines = shift || 20;
1eb83ea0 628 return if $pid = open(STDOUT, "|-"); # return if parent
f8284313 629 die "cannot fork: $!" unless defined $pid;
630 while (<STDIN>) {
f8284313 631 last if --$lines < 0;
1eb83ea0 632 print;
f8284313 633 }
634 exit;
635 }
636
637This technique can be applied to repeatedly push as many filters on your
638output stream as you wish.
639
640=head1 Other I/O Issues
641
642These topics aren't really arguments related to C<open> or C<sysopen>,
643but they do affect what you do with your open files.
644
645=head2 Opening Non-File Files
646
647When is a file not a file? Well, you could say when it exists but
648isn't a plain file. We'll check whether it's a symbolic link first,
649just in case.
650
651 if (-l $file || ! -f _) {
652 print "$file is not a plain file\n";
653 }
654
655What other kinds of files are there than, well, files? Directories,
656symbolic links, named pipes, Unix-domain sockets, and block and character
657devices. Those are all files, too--just not I<plain> files. This isn't
658the same issue as being a text file. Not all text files are plain files.
1a193132 659Not all plain files are text files. That's why there are separate C<-f>
f8284313 660and C<-T> file tests.
661
662To open a directory, you should use the C<opendir> function, then
663process it with C<readdir>, carefully restoring the directory
664name if necessary:
665
666 opendir(DIR, $dirname) or die "can't opendir $dirname: $!";
667 while (defined($file = readdir(DIR))) {
668 # do something with "$dirname/$file"
669 }
670 closedir(DIR);
671
672If you want to process directories recursively, it's better to use the
1a193132 673File::Find module. For example, this prints out all files recursively
674and adds a slash to their names if the file is a directory.
f8284313 675
676 @ARGV = qw(.) unless @ARGV;
677 use File::Find;
678 find sub { print $File::Find::name, -d && '/', "\n" }, @ARGV;
679
680This finds all bogus symbolic links beneath a particular directory:
681
682 find sub { print "$File::Find::name\n" if -l && !-e }, $dir;
683
684As you see, with symbolic links, you can just pretend that it is
685what it points to. Or, if you want to know I<what> it points to, then
686C<readlink> is called for:
687
688 if (-l $file) {
689 if (defined($whither = readlink($file))) {
690 print "$file points to $whither\n";
691 } else {
692 print "$file points nowhere: $!\n";
693 }
694 }
695
1a193132 696=head2 Opening Named Pipes
697
f8284313 698Named pipes are a different matter. You pretend they're regular files,
699but their opens will normally block until there is both a reader and
700a writer. You can read more about them in L<perlipc/"Named Pipes">.
701Unix-domain sockets are rather different beasts as well; they're
702described in L<perlipc/"Unix-Domain TCP Clients and Servers">.
703
1a193132 704When it comes to opening devices, it can be easy and it can be tricky.
f8284313 705We'll assume that if you're opening up a block device, you know what
706you're doing. The character devices are more interesting. These are
707typically used for modems, mice, and some kinds of printers. This is
708described in L<perlfaq8/"How do I read and write the serial port?">
709It's often enough to open them carefully:
710
711 sysopen(TTYIN, "/dev/ttyS1", O_RDWR | O_NDELAY | O_NOCTTY)
712 # (O_NOCTTY no longer needed on POSIX systems)
713 or die "can't open /dev/ttyS1: $!";
714 open(TTYOUT, "+>&TTYIN")
715 or die "can't dup TTYIN: $!";
716
717 $ofh = select(TTYOUT); $| = 1; select($ofh);
718
719 print TTYOUT "+++at\015";
720 $answer = <TTYIN>;
721
1a193132 722With descriptors that you haven't opened using C<sysopen>, such as
723sockets, you can set them to be non-blocking using C<fcntl>:
f8284313 724
725 use Fcntl;
21d1ba01 726 my $old_flags = fcntl($handle, F_GETFL, 0)
727 or die "can't get flags: $!";
728 fcntl($handle, F_SETFL, $old_flags | O_NONBLOCK)
f8284313 729 or die "can't set non blocking: $!";
730
731Rather than losing yourself in a morass of twisting, turning C<ioctl>s,
732all dissimilar, if you're going to manipulate ttys, it's best to
733make calls out to the stty(1) program if you have it, or else use the
734portable POSIX interface. To figure this all out, you'll need to read the
735termios(3) manpage, which describes the POSIX interface to tty devices,
736and then L<POSIX>, which describes Perl's interface to POSIX. There are
737also some high-level modules on CPAN that can help you with these games.
738Check out Term::ReadKey and Term::ReadLine.
739
1a193132 740=head2 Opening Sockets
741
f8284313 742What else can you open? To open a connection using sockets, you won't use
13a2d996 743one of Perl's two open functions. See
744L<perlipc/"Sockets: Client/Server Communication"> for that. Here's an
745example. Once you have it, you can use FH as a bidirectional filehandle.
f8284313 746
747 use IO::Socket;
748 local *FH = IO::Socket::INET->new("www.perl.com:80");
749
750For opening up a URL, the LWP modules from CPAN are just what
751the doctor ordered. There's no filehandle interface, but
752it's still easy to get the contents of a document:
753
754 use LWP::Simple;
6cecdcac 755 $doc = get('http://www.linpro.no/lwp/');
f8284313 756
757=head2 Binary Files
758
759On certain legacy systems with what could charitably be called terminally
760convoluted (some would say broken) I/O models, a file isn't a file--at
761least, not with respect to the C standard I/O library. On these old
762systems whose libraries (but not kernels) distinguish between text and
763binary streams, to get files to behave properly you'll have to bend over
764backwards to avoid nasty problems. On such infelicitous systems, sockets
765and pipes are already opened in binary mode, and there is currently no
766way to turn that off. With files, you have more options.
767
768Another option is to use the C<binmode> function on the appropriate
769handles before doing regular I/O on them:
770
771 binmode(STDIN);
772 binmode(STDOUT);
773 while (<STDIN>) { print }
774
775Passing C<sysopen> a non-standard flag option will also open the file in
776binary mode on those systems that support it. This is the equivalent of
1a193132 777opening the file normally, then calling C<binmode> on the handle.
f8284313 778
779 sysopen(BINDAT, "records.data", O_RDWR | O_BINARY)
780 || die "can't open records.data: $!";
781
782Now you can use C<read> and C<print> on that handle without worrying
1a193132 783about the non-standard system I/O library breaking your data. It's not
f8284313 784a pretty picture, but then, legacy systems seldom are. CP/M will be
785with us until the end of days, and after.
786
787On systems with exotic I/O systems, it turns out that, astonishingly
788enough, even unbuffered I/O using C<sysread> and C<syswrite> might do
789sneaky data mutilation behind your back.
790
791 while (sysread(WHENCE, $buf, 1024)) {
792 syswrite(WHITHER, $buf, length($buf));
793 }
794
795Depending on the vicissitudes of your runtime system, even these calls
796may need C<binmode> or C<O_BINARY> first. Systems known to be free of
e6f03d26 797such difficulties include Unix, the Mac OS, Plan 9, and Inferno.
f8284313 798
799=head2 File Locking
800
801In a multitasking environment, you may need to be careful not to collide
1a193132 802with other processes who want to do I/O on the same files as you
f8284313 803are working on. You'll often need shared or exclusive locks
804on files for reading and writing respectively. You might just
805pretend that only exclusive locks exist.
806
807Never use the existence of a file C<-e $file> as a locking indication,
808because there is a race condition between the test for the existence of
1a193132 809the file and its creation. It's possible for another process to create
810a file in the slice of time between your existence check and your attempt
811to create the file. Atomicity is critical.
f8284313 812
813Perl's most portable locking interface is via the C<flock> function,
1a193132 814whose simplicity is emulated on systems that don't directly support it
815such as SysV or Windows. The underlying semantics may affect how
f8284313 816it all works, so you should learn how C<flock> is implemented on your
817system's port of Perl.
818
819File locking I<does not> lock out another process that would like to
820do I/O. A file lock only locks out others trying to get a lock, not
821processes trying to do I/O. Because locks are advisory, if one process
822uses locking and another doesn't, all bets are off.
823
824By default, the C<flock> call will block until a lock is granted.
825A request for a shared lock will be granted as soon as there is no
d1be9408 826exclusive locker. A request for an exclusive lock will be granted as
f8284313 827soon as there is no locker of any kind. Locks are on file descriptors,
828not file names. You can't lock a file until you open it, and you can't
829hold on to a lock once the file has been closed.
830
831Here's how to get a blocking shared lock on a file, typically used
832for reading:
833
834 use 5.004;
835 use Fcntl qw(:DEFAULT :flock);
836 open(FH, "< filename") or die "can't open filename: $!";
837 flock(FH, LOCK_SH) or die "can't lock filename: $!";
838 # now read from FH
839
840You can get a non-blocking lock by using C<LOCK_NB>.
841
842 flock(FH, LOCK_SH | LOCK_NB)
843 or die "can't lock filename: $!";
844
845This can be useful for producing more user-friendly behaviour by warning
846if you're going to be blocking:
847
848 use 5.004;
849 use Fcntl qw(:DEFAULT :flock);
850 open(FH, "< filename") or die "can't open filename: $!";
851 unless (flock(FH, LOCK_SH | LOCK_NB)) {
852 $| = 1;
853 print "Waiting for lock...";
854 flock(FH, LOCK_SH) or die "can't lock filename: $!";
855 print "got it.\n"
856 }
857 # now read from FH
858
859To get an exclusive lock, typically used for writing, you have to be
860careful. We C<sysopen> the file so it can be locked before it gets
861emptied. You can get a nonblocking version using C<LOCK_EX | LOCK_NB>.
862
863 use 5.004;
864 use Fcntl qw(:DEFAULT :flock);
865 sysopen(FH, "filename", O_WRONLY | O_CREAT)
866 or die "can't open filename: $!";
867 flock(FH, LOCK_EX)
868 or die "can't lock filename: $!";
869 truncate(FH, 0)
870 or die "can't truncate filename: $!";
871 # now write to FH
872
873Finally, due to the uncounted millions who cannot be dissuaded from
874wasting cycles on useless vanity devices called hit counters, here's
875how to increment a number in a file safely:
876
877 use Fcntl qw(:DEFAULT :flock);
878
879 sysopen(FH, "numfile", O_RDWR | O_CREAT)
880 or die "can't open numfile: $!";
881 # autoflush FH
882 $ofh = select(FH); $| = 1; select ($ofh);
883 flock(FH, LOCK_EX)
884 or die "can't write-lock numfile: $!";
885
886 $num = <FH> || 0;
887 seek(FH, 0, 0)
888 or die "can't rewind numfile : $!";
889 print FH $num+1, "\n"
890 or die "can't write numfile: $!";
891
892 truncate(FH, tell(FH))
893 or die "can't truncate numfile: $!";
894 close(FH)
895 or die "can't close numfile: $!";
896
ae258fbb 897=head2 IO Layers
898
899In Perl 5.8.0 a new I/O framework called "PerlIO" was introduced.
900This is a new "plumbing" for all the I/O happening in Perl; for the
1a193132 901most part everything will work just as it did, but PerlIO also brought
902in some new features such as the ability to think of I/O as "layers".
ae258fbb 903One I/O layer may in addition to just moving the data also do
904transformations on the data. Such transformations may include
905compression and decompression, encryption and decryption, and transforming
906between various character encodings.
907
908Full discussion about the features of PerlIO is out of scope for this
909tutorial, but here is how to recognize the layers being used:
910
911=over 4
912
913=item *
914
1a193132 915The three-(or more)-argument form of C<open> is being used and the
ae258fbb 916second argument contains something else in addition to the usual
917C<< '<' >>, C<< '>' >>, C<< '>>' >>, C<< '|' >> and their variants,
918for example:
919
920 open(my $fh, "<:utf8", $fn);
921
922=item *
923
1a193132 924The two-argument form of C<binmode> is being used, for example
ae258fbb 925
926 binmode($fh, ":encoding(utf16)");
927
928=back
929
80fea0d2 930For more detailed discussion about PerlIO see L<PerlIO>;
ae258fbb 931for more detailed discussion about Unicode and I/O see L<perluniintro>.
932
f8284313 933=head1 SEE ALSO
934
1a193132 935The C<open> and C<sysopen> functions in perlfunc(1);
936the system open(2), dup(2), fopen(3), and fdopen(3) manpages;
f8284313 937the POSIX documentation.
938
939=head1 AUTHOR and COPYRIGHT
940
941Copyright 1998 Tom Christiansen.
942
5a7beb56 943This documentation is free; you can redistribute it and/or modify it
944under the same terms as Perl itself.
f8284313 945
946Irrespective of its distribution, all code examples in these files are
947hereby placed into the public domain. You are permitted and
948encouraged to use this code in your own programs for fun or for profit
949as you see fit. A simple comment in the code giving credit would be
950courteous but is not required.
951
952=head1 HISTORY
953
954First release: Sat Jan 9 08:09:11 MST 1999