For tied file handle calls, use PUSH* when we know that the stack has space.
[p5sagit/p5-mst-13.2.git] / lib / PerlIO.pm
CommitLineData
1141d9f8 1package PerlIO;
2
af048c18 3our $VERSION = '1.06';
8de1277c 4
1141d9f8 5# Map layer name to package that defines it
c1a61b17 6our %alias;
1141d9f8 7
8sub import
9{
10 my $class = shift;
11 while (@_)
12 {
13 my $layer = shift;
14 if (exists $alias{$layer})
15 {
16 $layer = $alias{$layer}
17 }
18 else
19 {
20 $layer = "${class}::$layer";
21 }
22 eval "require $layer";
23 warn $@ if $@;
24 }
25}
26
39f7a870 27sub F_UTF8 () { 0x8000 }
28
1141d9f8 291;
30__END__
b3d30bf7 31
32=head1 NAME
33
7d3b96bb 34PerlIO - On demand loader for PerlIO layers and root of PerlIO::* name space
b3d30bf7 35
36=head1 SYNOPSIS
37
a7845df8 38 open($fh,"<:crlf", "my.txt"); # support platform-native and CRLF text files
1cbfc93d 39
40 open($fh,"<","his.jpg"); # portably open a binary file for reading
41 binmode($fh);
7d3b96bb 42
43 Shell:
44 PERLIO=perlio perl ....
b3d30bf7 45
46=head1 DESCRIPTION
47
ec28694c 48When an undefined layer 'foo' is encountered in an C<open> or
49C<binmode> layer specification then C code performs the equivalent of:
b3d30bf7 50
51 use PerlIO 'foo';
52
53The perl code in PerlIO.pm then attempts to locate a layer by doing
54
55 require PerlIO::foo;
56
47bfe92f 57Otherwise the C<PerlIO> package is a place holder for additional
58PerlIO related functions.
b3d30bf7 59
7d3b96bb 60The following layers are currently defined:
b3d30bf7 61
7d3b96bb 62=over 4
63
3d897973 64=item :unix
7d3b96bb 65
3d897973 66Lowest level layer which provides basic PerlIO operations in terms of
67UNIX/POSIX numeric file descriptor calls
68(open(), read(), write(), lseek(), close()).
7d3b96bb 69
3d897973 70=item :stdio
7d3b96bb 71
47bfe92f 72Layer which calls C<fread>, C<fwrite> and C<fseek>/C<ftell> etc. Note
73that as this is "real" stdio it will ignore any layers beneath it and
9ec269cb 74go straight to the operating system via the C library as usual.
7d3b96bb 75
3d897973 76=item :perlio
7d3b96bb 77
3d897973 78A from scratch implementation of buffering for PerlIO. Provides fast
79access to the buffer for C<sv_gets> which implements perl's readline/E<lt>E<gt>
80and in general attempts to minimize data copying.
7d3b96bb 81
3d897973 82C<:perlio> will insert a C<:unix> layer below itself to do low level IO.
7d3b96bb 83
3d897973 84=item :crlf
7d3b96bb 85
3d897973 86A layer that implements DOS/Windows like CRLF line endings. On read
87converts pairs of CR,LF to a single "\n" newline character. On write
88converts each "\n" to a CR,LF pair. Note that this layer likes to be
89one of its kind: it silently ignores attempts to be pushed into the
90layer stack more than once.
91
92It currently does I<not> mimic MS-DOS as far as treating of Control-Z
93as being an end-of-file marker.
94
95(Gory details follow) To be more exact what happens is this: after
96pushing itself to the stack, the C<:crlf> layer checks all the layers
97below itself to find the first layer that is capable of being a CRLF
98layer but is not yet enabled to be a CRLF layer. If it finds such a
99layer, it enables the CRLFness of that other deeper layer, and then
100pops itself off the stack. If not, fine, use the one we just pushed.
101
102The end result is that a C<:crlf> means "please enable the first CRLF
103layer you can find, and if you can't find one, here would be a good
104spot to place a new one."
105
106Based on the C<:perlio> layer.
107
108=item :mmap
109
110A layer which implements "reading" of files by using C<mmap()> to
9ec269cb 111make a (whole) file appear in the process's address space, and then
3d897973 112using that as PerlIO's "buffer". This I<may> be faster in certain
113circumstances for large files, and may result in less physical memory
114use when multiple processes are reading the same file.
115
116Files which are not C<mmap()>-able revert to behaving like the C<:perlio>
9ec269cb 117layer. Writes also behave like the C<:perlio> layer, as C<mmap()> for write
3d897973 118needs extra house-keeping (to extend the file) which negates any advantage.
119
9ec269cb 120The C<:mmap> layer will not exist if the platform does not support C<mmap()>.
3d897973 121
122=item :utf8
7d3b96bb 123
2575c402 124Declares that the stream accepts perl's I<internal> encoding of
47bfe92f 125characters. (Which really is UTF-8 on ASCII machines, but is
126UTF-EBCDIC on EBCDIC machines.) This allows any character perl can
127represent to be read from or written to the stream. The UTF-X encoding
128is chosen to render simple text parts (i.e. non-accented letters,
129digits and common punctuation) human readable in the encoded file.
130
131Here is how to write your native data out using UTF-8 (or UTF-EBCDIC)
132and then read it back in.
133
134 open(F, ">:utf8", "data.utf");
135 print F $out;
136 close(F);
137
138 open(F, "<:utf8", "data.utf");
139 $in = <F>;
140 close(F);
7d3b96bb 141
740d4bb2 142Note that this layer does not validate byte sequences. For reading
9ec269cb 143input, using C<:encoding(utf8)> instead of bare C<:utf8> is strongly
740d4bb2 144recommended.
145
3d897973 146=item :bytes
c1a61b17 147
9ec269cb 148This is the inverse of the C<:utf8> layer. It turns off the flag
c1a61b17 149on the layer below so that data read from it is considered to
9ec269cb 150be "octets" i.e. characters in the range 0..255 only. Likewise
c1a61b17 151on output perl will warn if a "wide" character is written
152to a such a stream.
153
3d897973 154=item :raw
7d3b96bb 155
0226bbdb 156The C<:raw> layer is I<defined> as being identical to calling
9ec269cb 157C<binmode($fh)> - the stream is made suitable for passing binary data,
18aba96f 158i.e. each byte is passed as-is. The stream will still be
3d897973 159buffered.
160
161In Perl 5.6 and some books the C<:raw> layer (previously sometimes also
162referred to as a "discipline") is documented as the inverse of the
163C<:crlf> layer. That is no longer the case - other layers which would
9ec269cb 164alter the binary nature of the stream are also disabled. If you want UNIX
3d897973 165line endings on a platform that normally does CRLF translation, but still
9ec269cb 166want UTF-8 or encoding defaults, the appropriate thing to do is to add
167C<:perlio> to the PERLIO environment variable.
1cbfc93d 168
0226bbdb 169The implementation of C<:raw> is as a pseudo-layer which when "pushed"
170pops itself and then any layers which do not declare themselves as suitable
171for binary data. (Undoing :utf8 and :crlf are implemented by clearing
39f7a870 172flags rather than popping layers but that is an implementation detail.)
01e6739c 173
9ec269cb 174As a consequence of the fact that C<:raw> normally pops layers,
39f7a870 175it usually only makes sense to have it as the only or first element in
176a layer specification. When used as the first element it provides
0226bbdb 177a known base on which to build e.g.
7d3b96bb 178
0226bbdb 179 open($fh,":raw:utf8",...)
7d3b96bb 180
0226bbdb 181will construct a "binary" stream, but then enable UTF-8 translation.
b3d30bf7 182
3d897973 183=item :pop
4ec2216f 184
185A pseudo layer that removes the top-most layer. Gives perl code
186a way to manipulate the layer stack. Should be considered
187as experimental. Note that C<:pop> only works on real layers
188and will not undo the effects of pseudo layers like C<:utf8>.
189An example of a possible use might be:
190
191 open($fh,...)
192 ...
193 binmode($fh,":encoding(...)"); # next chunk is encoded
194 ...
3c4b39be 195 binmode($fh,":pop"); # back to un-encoded
4ec2216f 196
197A more elegant (and safer) interface is needed.
198
3d897973 199=item :win32
200
9ec269cb 201On Win32 platforms this I<experimental> layer uses the native "handle" IO
202rather than the unix-like numeric file descriptor layer. Known to be
3d897973 203buggy as of perl 5.8.2.
204
7d3b96bb 205=back
206
39f7a870 207=head2 Custom Layers
208
209It is possible to write custom layers in addition to the above builtin
210ones, both in C/XS and Perl. Two such layers (and one example written
211in Perl using the latter) come with the Perl distribution.
212
213=over 4
214
215=item :encoding
216
217Use C<:encoding(ENCODING)> either in open() or binmode() to install
9ec269cb 218a layer that transparently does character set and encoding transformations,
e76300d6 219for example from Shift-JIS to Unicode. Note that under C<stdio>
220an C<:encoding> also enables C<:utf8>. See L<PerlIO::encoding>
221for more information.
39f7a870 222
223=item :via
224
225Use C<:via(MODULE)> either in open() or binmode() to install a layer
226that does whatever transformation (for example compression /
227decompression, encryption / decryption) to the filehandle.
228See L<PerlIO::via> for more information.
229
230=back
231
01e6739c 232=head2 Alternatives to raw
233
0226bbdb 234To get a binary stream an alternate method is to use:
01e6739c 235
0226bbdb 236 open($fh,"whatever")
01e6739c 237 binmode($fh);
238
9ec269cb 239this has the advantage of being backward compatible with how such things have
01e6739c 240had to be coded on some platforms for years.
01e6739c 241
9ec269cb 242To get an unbuffered stream specify an unbuffered layer (e.g. C<:unix>)
0226bbdb 243in the open call:
01e6739c 244
245 open($fh,"<:unix",$path)
246
7d3b96bb 247=head2 Defaults and how to override them
248
ec28694c 249If the platform is MS-DOS like and normally does CRLF to "\n"
250translation for text files then the default layers are :
7d3b96bb 251
252 unix crlf
253
47bfe92f 254(The low level "unix" layer may be replaced by a platform specific low
255level layer.)
7d3b96bb 256
9ec269cb 257Otherwise if C<Configure> found out how to do "fast" IO using the system's
046e4a6a 258stdio, then the default layers are:
7d3b96bb 259
260 unix stdio
261
262Otherwise the default layers are
263
264 unix perlio
265
266These defaults may change once perlio has been better tested and tuned.
267
47bfe92f 268The default can be overridden by setting the environment variable
39f7a870 269PERLIO to a space separated list of layers (C<unix> or platform low
270level layer is always pushed first).
47bfe92f 271
7d3b96bb 272This can be used to see the effect of/bugs in the various layers e.g.
273
274 cd .../perl/t
275 PERLIO=stdio ./perl harness
276 PERLIO=perlio ./perl harness
277
9ec269cb 278For the various values of PERLIO see L<perlrun/PERLIO>.
3b0db4f9 279
4c11337c 280=head2 Querying the layers of filehandles
39f7a870 281
282The following returns the B<names> of the PerlIO layers on a filehandle.
283
9d569fce 284 my @layers = PerlIO::get_layers($fh); # Or FH, *FH, "FH".
39f7a870 285
286The layers are returned in the order an open() or binmode() call would
f0fd62e2 287use them. Note that the "default stack" depends on the operating
cc83745d 288system and on the Perl version, and both the compile-time and
289runtime configurations of Perl.
79d9a4d7 290
79d9a4d7 291The following table summarizes the default layers on UNIX-like and
9ec269cb 292DOS-like platforms and depending on the setting of C<$ENV{PERLIO}>:
79d9a4d7 293
f0fd62e2 294 PERLIO UNIX-like DOS-like
a7845df8 295 ------ --------- --------
f0fd62e2 296 unset / "" unix perlio / stdio [1] unix crlf
297 stdio unix perlio / stdio [1] stdio
298 perlio unix perlio unix perlio
299 mmap unix mmap unix mmap
39f7a870 300
f0fd62e2 301 # [1] "stdio" if Configure found out how to do "fast stdio" (depends
302 # on the stdio implementation) and in Perl 5.8, otherwise "unix perlio"
046e4a6a 303
9ec269cb 304By default the layers from the input side of the filehandle are
305returned; to get the output side, use the optional C<output> argument:
39f7a870 306
2ae85e59 307 my @layers = PerlIO::get_layers($fh, output => 1);
39f7a870 308
309(Usually the layers are identical on either side of a filehandle but
2ae85e59 310for example with sockets there may be differences, or if you have
311been using the C<open> pragma.)
39f7a870 312
92a3e63c 313There is no set_layers(), nor does get_layers() return a tied array
314mirroring the stack, or anything fancy like that. This is not
315accidental or unintentional. The PerlIO layer stack is a bit more
316complicated than just a stack (see for example the behaviour of C<:raw>).
317You are supposed to use open() and binmode() to manipulate the stack.
318
39f7a870 319B<Implementation details follow, please close your eyes.>
320
9ec269cb 321The arguments to layers are by default returned in parentheses after
39f7a870 322the name of the layer, and certain layers (like C<utf8>) are not real
9ec269cb 323layers but instead flags on real layers; to get all of these returned
324separately, use the optional C<details> argument:
39f7a870 325
2ae85e59 326 my @layer_and_args_and_flags = PerlIO::get_layers($fh, details => 1);
39f7a870 327
328The result will be up to be three times the number of layers:
329the first element will be a name, the second element the arguments
330(unspecified arguments will be C<undef>), the third element the flags,
331the fourth element a name again, and so forth.
332
333B<You may open your eyes now.>
334
7d3b96bb 335=head1 AUTHOR
336
337Nick Ing-Simmons E<lt>nick@ing-simmons.netE<gt>
338
339=head1 SEE ALSO
340
39f7a870 341L<perlfunc/"binmode">, L<perlfunc/"open">, L<perlunicode>, L<perliol>,
342L<Encode>
7d3b96bb 343
344=cut