Populate perl571delta.

[p5sagit/p5-mst-13.2.git] / pod / perl571delta.pod

=head1 NAME

perl571delta - what's new for perl v5.7.1

=head1 DESCRIPTION

This document describes differences between the 5.7.0 release and the
5.7.1 release.  

(To view the differences between the 5.6.0 release and the 5.7.0
release, see L<perl570delta>).

=head1 Security Vulnerability Closed

(This change was already made in 5.7.0 but bears repeating here.)

A potential security vulnerability in the optional suidperl component
of Perl was identified in August 2000.  suidperl is neither built nor
installed by default.  As of April 2001 the only known vulnerable
platform is Linux, most likely all Linux distributions.  CERT and
various vendors and distributors have been alerted about the vulnerability.
See http://www.cpan.org/src/5.0/sperl-2000-08-05/sperl-2000-08-05.txt
for more information.

The problem was caused by Perl trying to report a suspected security
exploit attempt using an external program, /bin/mail.  On Linux
platforms the /bin/mail program had an undocumented feature which
when combined with suidperl gave access to a root shell, resulting in
a serious compromise instead of reporting the exploit attempt.  If you
don't have /bin/mail, or if you have 'safe setuid scripts', or if
suidperl is not installed, you are safe.

The exploit attempt reporting feature has been completely removed from
all the Perl 5.7 releases (and will be gone also from the maintenance
release 5.6.1), so that particular vulnerability isn't there anymore.
However, further security vulnerabilities are, unfortunately, always
possible.  The suidperl code is being reviewed and if deemed too risky
to continue to be supported, it may be completely removed from future
releases.  In any case, suidperl should only be used by security
experts who know exactly what they are doing and why they are using
suidperl instead of some other solution such as sudo (see
http://www.courtesan.com/sudo/).

=head1 Incompatible Changes

=over 4

=item *

Although "you shouldn't do that", it was possible to write code that
depends on Perl's hashed key order (Data::Dumper does this).  The new
algorithm "One-at-a-Time" produces a different hashed key order.
More details are in L</"Performance Enhancements">.

=item *

The list of filenames from glob() (or <...>) is now by default sorted
alphabetically to be csh-compliant.  (bsd_glob() does still sort platform
natively, ASCII or EBCDIC, unless GLOB_ALPHASORT is specified.)

=back

=head1 Retired Features

The use of chop() is now deprecated.  This has been made easier by
rewriting all the examples in the documentation to use chomp() instead.

=head1 Core Enhancements

=over 4

=item *

AUTOLOAD is now lvaluable.

=item *

IO is now by default done via PerlIO rather than system's "stdio".
PerlIO allows "layers" to be "pushed" onto a file handle to alter the
handle's behaviour.  Layers can be specified at open time via 3-arg
form of open:

   open($fh,'>:crlf :utf8', $path) || ...

or on already opened handles via extended C<binmode>:

   binmode($fh,':encoding(iso-8859-7)');

The built-in layers are: unix (low level read/write), stdio (as in
previous Perls), perlio (re-implementation of stdio buffering in a
portable manner), crlf (does CRLF <=> "\n" translation as on Win32,
but available on any platform).  A mmap layer may be available if
platform supports it (mostly UNIXes).

Layers to be applied by default may be specified via the 'open' pragma.

See L</"Installation and Configuration Improvements"> for the effects
of PerlIO on your architecture name.

=item *

File handles can be marked as accepting Perl's internal encoding of Unicode
(UTF-8 or UTF-EBCDIC depending on platfrom) by a pseudo layer ":utf8" :

   open($fh,">:utf8","Uni.txt");

=item *

File handles can translate character encodings from/to Perl's internal
Unicode form on read/write via the ":encoding()" layer.

=item *

File handles can be opened to "in memory" files held in Perl scalars via:

   open($fh,'>', \$variable) || ...

=item *

Anonymous temporary files are available without need to
'use FileHandle' or other module via

   open($fh,"+>", undef) || ...

That is a literal undef, not an undefined value.

=item *

The list form of C<open> is now implemented for pipes (at least on UNIX):

   open($fh,"-|", 'cat', '/etc/motd')

creates a pipe, and runs the equivalent of exec('cat', '/etc/motd') in
the child process.

=item *

The following keywords are now overrideable: chop, chomp, each, keys,
pop, push, shift, splice, unshift.

=item *

Formats now support zero-padded decimal fields.

=item *

Perl now tries internally to use integer values in numeric conversions
and basic arithmetics (+ - * /) if the arguments are integers, and
tries also to keep the results stored internally as integers.
This change leads into often slightly faster and always less lossy
arithmetics (previously Perl always preferred floating point numbers
in its math)

=item *
 
The printf and sprintf now support parameter reordering using the
C<%\d+\$> and C<*\d+\$> syntaxes.

=item *

Unicode in general should be now much more usable.  Unicode can be
used in hash keys, Unicode in regular expressions should work now,
Unicode in tr/// should work now (though tr/// seems to be a
particularly tricky to get right, so you have been warned)

=item *

The Unicode Character Database coming with Perl has been upgraded
to Unicode 3.1.

=item *

The Unicode character classes \p{Blank} and \p{SpacePerl} have been
added.  "Blank" is like C isblank(), that is, it contains only
"horizontal whitespace" (the space character is, the newline isn't),
and the "SpacePerl" is the Unicode equivalent of C<\s> (\p{Space}
isn't, since that includes the vertical tabulator character, whereas
C<\s> doesn't.)

=back

=head2 Modules and Pragmata

=head2 New Modules

=over 4

=item *

B::Concise is a new compiler backend for walking the Perl syntax tree,
printing concise info about ops.  The output is highly customizable,
so customizable that B::Terse has been reimplemented in terms of
B::Concise.

=item *

Class::ISA for reporting the search path for a class's ISA tree,
from Sean Burke, has been added.

=item *

Cwd has now a split personality: if possible, an extension is used,
(this will hopefully be both faster and more secure and robust) but
if not possible, the familiar Perl library implementation is used.

=item *

Digest, a frontend module for calculating digests (checksums),
from Gisle Aas, has been added.

=item *

Digest::MD5 for calculating MD5 digests (checksums), from Gisle Aas,
has been added.

NOTE: the MD5 backward compatibility module is purposefully not
included since its use is discouraged.

=item *

Encode provides a mechanism to translate between different character
encodings.  Support for Unicode, ISO-8859-*, ASCII, CP*, KOI8-R, and
three variants of EBCDIC are compiled in to the module.  Several other
encodings (like Japanese, Chinese, and MacIntosh encodings) are
included and will be loaded at runtime.

Any encoding supported by Encode module is also available to the
":encoding()" layer if PerlIO is used.

=item *

Filter::Simple is an easy-to-use frontend to Filter::Util::Call,
from Damian Conway.

=item *

Filter::Util::Call, from Paul Marquess, provides you with the
framework to write I<Source Filters> in Perl.  For most uses
the frontend Filter::Simple is to be preferred.

=item *

Locale::Constants, Locale::Country, Locale::Currency, and Locale::Language,
from Neil Bowers, have been added.  They provide the codes for various
locale standards, such as "fr" for France, "usd" for US Dollar, and
"jp" for Japanese.

=item *

MIME::Base64, from Gisle Aas, allows you to encode data in base64.

=item *

MIME::QuotedPrint, from Gisle Aas, allows you to encode data in
quoted-printable encoding.

MIME::QuotedPrint has been enhanced to provide the basic methods
necessary to use it with PerlIO::Via as in :

 use MIME::QuotedPrint;
 open($fh,">Via(MIME::QuotedPrint)",$path)

=item *

PerlIO::Scalar provides the IO to "in memory" perl scalars discussed
above.  It also serves as an example of a loadable layer.

=item *

PerlIO::Via acts as a PerlIO layer and wraps PerlIO layer
functionality provided by a class (typically implemented in
perl code).

=item *

Pod::Text::Overstrike, from Joe Smith, has been added.
It converts POD data to formatted overstrike text.

=item *

Switch from Damian Conway has been added.  After

        use Switch;

you have switch() and case() in Perl.

=item *

Text::Balanced from Damian Conway has been added, for
extracting delimited text sequences from strings.

=item *

Tie::RefHash::Nestable, from Edward Avis, allows storing hash references
(unlike the standard Tie::Refhash)

=item *

XS::Typemap is a test extension that exercizes XS typemaps.
Nothing gets installed but for extension writers the code is
worth studying.

=back

=head2 Updated And Improved Modules and Pragmata

=over 4

=item *

B::Deparse should be now more robust (still far from providing a full
roundtrip for any random piece of Perl code).

=item *

Class::Struct has now compile-time features.

=item *

Math::BigFloat has undergone much fixing.

=item *

Devel::Peek now has an interface for the Perl memory statistics
(this works only if you are using perl's malloc, and if you have
compiled with debugging).

=item *

IO::Socket has now atmark() method, which returns true if the socket
is positioned at the out-of-band mark.  The method is also exportable
as a sockatmark() function.

=item

IO::Socket::INET has support for ReusePort option (if your platform
suppport it).  The Reuse option has now an alias, ReuseAddr.

=item *

Net::Ping has been greatly enhanced.

=item *

The C<open> pragma allows layers other than ":raw" and ":crlf" when
using PerlIO.

=item *

POSIX::sigaction() is now much more flexible and robust.

=item *

The Test module has been significantly enhanced.  Its use is
greatly recommended for module writers.

=item *

The utf8:: name space (as in the pragma) provides various
Perl-callable functions to provide low level access to Perl's
internal Unicode representation.  At the moment only length()
has been implemented.

=back

The following modules have been upgraded from CPAN: CPAN, CGI, DB::File,
Getopt::Long, Pod::Man, Pod::Text, Storable, Text-Tabs+Wrap.

=head1 Performance Enhancements

=over 4

=item *

Hashes now use Bob Jenkins "One-at-a-Time" hashing key algorithm
(http://burtleburtle.net/bob/hash/doobs.html).
This algorithm is reasonably fast while producing a much better spread
of values.  Hash values output from the algorithm on a hash of all
3-char printable ASCII keys comes much closer to passing the DIEHARD
random number generation tests.  According to perlbench, this change
has not affected the overall speed of Perl.

=item *

unshift() should now be noticeably faster.

=back

=head1 Utility Changes

=over 4

=item *

h2xs now produces template README.

=item *

s2p has been completely rewritten in Perl.  (It is in fact a full
implementation of sed in Perl.)

=item *

xsubpp now supports OUT keyword.

=back

=head1 New Documentation

=head2 perlclib

Internal replacements for standard C library functions.

=head2 perliol

Internals of PerlIO with layers.

=head2 README.aix

Documentation on compiling Perl on AIX has been added.  AIX has
several different C compilers and getting the right patchlevel
is essential.  On install README.aix will be installed as L<perlaix>.

=head2 README.bs2000

Documentation on compling Perl on the POSIX-BC platform (an EBCDIC
mainframe environment) has been added.

This was formerly known as README.posix-bc but the name was considered
to be too confusing (it has nothing to do with the POSIX module or the
POSIX standard).  On install README.bs2000 will be installed as L<perlbs2000>.

=head2 README.macos

In perl 5.7.1 (and in the 5.6.1) the MacPerl sources have been
synchronized with the standard Perl sources.  To compile MacPerl
some additional steps are required, and this file documents those
steps.  On install README.macos will be installed as L<perlmacos>.

=head2 README.mpeix

The README.mpeix has been podified, which means that this information
about compiling and using Perl on the MPE/iX miniframe platform will
on install be installed as L<perlmpeix>.

=head2 README.solaris

README.solaris has been created and Solaris wisdom from elsewhere
in the Perl documentation has been collected there.  On install
README.solaris will be installed as L<perlsolaris>.

=head2 README.vos

The README.vos has been podified, which means that this information
about compiling and using Perl on the Stratus VOS miniframe platform will
on install be installed as L<perlvos>.

=head2 Porting/repository.pod

Documentation on how to use the Perl source repository has been added.

=head1 Performance Enhancements

=head1 Installation and Configuration Improvements

Because PerlIO is now the default on most platforms, "-perlio" doesn't
get appended to the $Config{archname} (also known as $^O) anymore.
Instead, if you explicitly choose not to use perlio (Configure command
line option -Uuseperlio), you will get "-stdio" appended.

Another change related to the architecture name is that "-64all"
(-Duse64bitall, or "maximally 64-bit") is appended only if your pointers
are 64 bits wide.

APPLLIB_EXP, a less-know configuration-time definition, has been
documented.  It can be used to prepend site-specific directories
to Perl's default search path (@INC), see INSTALL for information.

Building Berkeley DB3 for compatibility modes for DB, NDBM, and ODBM
has been documented in INSTALL.

If you are on IRIX or Tru64 platforms, new profiling/debugging options
have been added, see L</perlhack> for more information about pixie and
Third Degree.

=head2 New Or Improved Platforms

=over 4

=item *

AIX dynamic loading should be now better supported.

=item *

AmigaOS has been verified to be happy with Perl after a long pause.

=item *

MacOS Classic (MacPerl has of course been available since
perl 5.004 but now the source code bases of standard Perl
and MacPerl have been synchronised)

=item *

NCR MP-RAS

=item *

NonStop-UX

=item *

Amdahl UTS

=item *

z/OS (formerly known as OS/390, formerly known as MVS OE) has now
support for dynamic loading.  This is not selected by default,
however, you must specify -Dusedl in the arguments of Configure.

=back

=head2 Generic Improvements

=over 4

=item *

Configure no longer includes the DBM libraries (dbm, gdbm, db, ndbm)
when building the Perl binary.  The only exception to this is SunOS 4.x,
which needs them.

=item *

Some new Configure symbols, useful for extension writers: d_cmsghdr
(for struct cmsgdhr), d_fcntl_can_lock (whether fcntl() can be used
for file locking), d_fsync, d_getitimer(), d_getpagsz (for getpagesize(),
though you should prefer POSIX::sysconf(_SC_PAGE_SIZE)), d_msghdr_s
(for struct msgdhr), need_va_copy (whether one needs to use Perl_va_copy()
to copy varargs), d_readv, d_recvmsg, d_sendmsg, sig_size (the number
of elements in an array needed to hold all the available signals),
d_sockatmark, d_strtoq, d_u32align (whether one needs to access
character data aligned by U32 sized pointers), d_ualarm, d_usleep,
d_writev.

=item *

Removed Configure symbols: the PDP-11 memory model settings: huge,
large, medium, models.

=item *

SOCKS support is now much more robust.

=item *

If your file system supports symbolic links you can build Perl outside
of the source directory by
        
        mkdir /tmp/perl/build/directory
        cd /tmp/perl/build/directory
        sh /path/to/perl/source/Configure -Dmksymlinks ...

This will create in /tmp/perl/build/directory a tree of symbolic links
pointing to files in /path/to/perl/source.  The original files are left
unaffected.  After Configure has finished you can just say

        make all test

and Perl will be built and tested, all in /tmp/perl/build/directory.

=back

=head1 Selected Bug Fixes

Numerous memory leaks have been hunted down.  Most importantly anonymous
subs used to leak quite a bit.

=over 4

=item *

Small unpredictactabilities in the order of DESTROYS have been
even small.

=item *

mkdir() now ignores trailing slashes in the directory name,
as mandated by POSIX.

=item *

The PERL5OPT environment variable didn't really work before.

=item *

All but the first argument of the IO syswrite() method are now optional.

=item *

Tie::ARRAY SPLICE method was broken.

=item *

vec() now tries to work with characters <= 255 when possible, but it leaves
higher character values in place.  In that case, if vec() was used to modify
the string, it is no longer considered to be utf8-encoded.

=back

=head2 Platform Specific Changes and Fixes

=over 4

=item *

Linux previously had problems related to sockaddrlen when using
accept(), revcfrom() (in Perl: recv()), getpeername(), and getsockname().

=item *

Previously DYNIX/ptx had problems in its Configure probe for
non-blocking I/O.

=back

=head1 New or Changed Diagnostics

Two new debugging options have been added: if you have compiled your
Perl with debugging, you can use the -DT and -DR options to trace
tokenizing and to add reference counts to displaying variables,
respectively.

=over 4

=item *

If an attempt to use a (non-blessed) reference as an array element
is made, a warning is given.

=item *

C<push @a;> and C<unshift @a;> (with no values to push or unshift)
now give a warning.  This may be a problem for generated and evaled
code.

=back

=head1 Changed Internals

=over 4

=item *

Some new internal APIs: ptr_table_clear, ptr_table_free, sv_setref_uv.
For the full list see L<perlapi>.

=item *

dTHR and djSP have been obsoleted; the former removed and the latter
replaced with dSP.

=item *

Perl now uses system malloc instead of Perl malloc in all 64-bit platforms.

=back

=head1 Known Problems

=head2 lib/b test 19

The test fails in various platforms (PA64 and IA64 are known), but the
exact cause is still being investigated.

=head2 Localizing a Tied Variable Leaks Memory

    use Tie::Hash;
    tie my %tie_hash => 'Tie::StdHash';

    ...

    local($tie_hash{Foo}) = 1; # leaks

Code like the above is known to leak memory every time the local()
is executed.

=head2 sigaction test 13 in VMS

The test is known to fail, whether it's because of VMS of because
of faulty test, is not known.

=head2 sprintf tests 129 and 130

The op/sprintf tests 129 and 130 are known to fail in some platforms.
Examples include any platform using sfio, and Tandem's NonStop-UX.
The failing platforms do not comply with the ANSI C Standard, line
19ff on page 134 of ANSI X3.159 1989 to be exact.  (They produce
something else than "1" and "-1" when formatting 0.6 and -0.6 using
the printf format "%.0f", most often they produce "0" and "-0".)

=head2 Self-tying of Arrays and Hashes Is Forbidden

Self-tying of arrays and hashes is broken in rather deep and
hard-to-fix ways.  As a stop-gap measure to avoid people from getting
frustrated at the mysterious results (core dumps, most often) it is
for now forbidden (you will get a fatal error even from an attempt).

=head1 Reporting Bugs

If you find what you think is a bug, you might check the articles
recently posted to the comp.lang.perl.misc newsgroup and the perl
bug database at http://bugs.perl.org.  There may also be
information at http://www.perl.com/perl/, the Perl Home Page.

If you believe you have an unreported bug, please run the B<perlbug>
program included with your release.  Be sure to trim your bug down
to a tiny but sufficient test case.  Your bug report, along with the
output of C<perl -V>, will be sent off to perlbug@perl.org to be
analysed by the Perl porting team.

=head1 SEE ALSO

The F<Changes> file for exhaustive details on what changed.

The F<INSTALL> file for how to build Perl.

The F<README> file for general stuff.

The F<Artistic> and F<Copying> files for copyright information.

=head1 HISTORY

Written by Jarkko Hietaniemi <F<jhi@iki.fi>>, with many contributions
from The Perl Porters and Perl Users submitting feedback and patches.

Send omissions or corrections to <F<perlbug@perl.org>>.

=cut