Quickie documentation of the C UTF-8 API.

[p5sagit/p5-mst-13.2.git] / pod / perluniintro.pod

diff --git a/pod/perluniintro.pod b/pod/perluniintro.pod
index 55f8d56..0ecfba0 100644 (file)
--- a/pod/perluniintro.pod
+++ b/pod/perluniintro.pod
@@ -170,11 +170,12 @@ At run-time you can use C<chr()>:
 
 Naturally, C<ord()> will do the reverse: turn a character to a code point.
 
-Note that C<\x..>, C<\x{..}> and C<chr(...)> for arguments less than
-0x100 (decimal 256) will generate an eight-bit character for backward
-compatibility with older Perls.  For arguments of 0x100 or more,
-Unicode will always be produced.  If you want UTF-8 always, use
-C<pack("U", ...)> instead of C<\x..>, C<\x{..}>, or C<chr()>.
+Note that C<\x..> (no C<{}> and only two hexadecimal digits), C<\x{...}>
+and C<chr(...)> for arguments less than 0x100 (decimal 256) will
+generate an eight-bit character for backward compatibility with older
+Perls.  For arguments of 0x100 or more, Unicode will always be
+produced.  If you want UTF-8 always, use C<pack("U", ...)> instead of
+C<\x..>, C<\x{...}>, or C<chr()>.
 
 You can also use the C<charnames> pragma to invoke characters
 by name in doublequoted strings:
@@ -187,6 +188,10 @@ characters:
 
    my $georgian_an  = pack("U", 0x10a0);
 
+Note that both C<\x{...}> and C<\N{...}> are compile-time string
+constants: you cannot use variables in them.  if you want similar
+run-time functionality, use C<chr()> and C<charnames::vianame()>.
+
 =head2 Handling Unicode
 
 Handling Unicode is for the most part transparent: just use the
@@ -269,8 +274,8 @@ the C<open> pragma; see L<open>:
 
 With the C<open> pragma you can use the C<:locale> discipline
 
-    $ENV{LANG} = 'ru_RU.KOI8-R';
-    # the :locale will probe the locale environment variables like LANG
+    $ENV{LC_ALL} = $ENV{LANG} = 'ru_RU.KOI8-R';
+    # the :locale will probe the locale environment variables like LC_ALL
     use open OUT => ':locale'; # russki parusski
     open(O, ">koi8");
     print O chr(0x430); # Unicode CYRILLIC SMALL LETTER A = KOI8-R 0xc1
@@ -298,10 +303,10 @@ streams, use explicit disciplines directly in the C<open()> call.
 You can switch encodings on an already opened stream by using
 C<binmode()>, see L<perlfunc/binmode>.
 
-The C<:locale> does not currently work with C<open()> and
-C<binmode()>, only with the C<open> pragma.  The C<:utf8> and
-C<:encoding(...)> do work with all of C<open()>, C<binmode()>,
-and the C<open> pragma.
+The C<:locale> does not currently (as of Perl 5.8.0) work with
+C<open()> and C<binmode()>, only with the C<open> pragma.  The
+C<:utf8> and C<:encoding(...)> do work with all of C<open()>,
+C<binmode()>, and the C<open> pragma.
 
 Similarly, you may use these I/O disciplines on input streams to
 automatically convert data from the specified encoding when it is
@@ -340,6 +345,32 @@ If you run this code twice, the contents of the F<file> will be twice
 UTF-8 encoded.  A C<use open ':utf8'> would have avoided the bug, or
 explicitly opening also the F<file> for input as UTF-8.
 
+B<NOTE>: the C<:utf8> and C<:encoding> features work only if your
+Perl has been built with the new "perlio" feature.  Almost all 
+Perl 5.8 platforms do use "perlio", though: you can see whether
+yours is by running "perl -V" and looking for C<useperlio=define>.
+
+=head2 Displaying Unicode As Text
+
+Sometimes you might want to display Perl scalars containing Unicode as
+simple ASCII (or EBCDIC) text.  The following subroutine will convert
+its argument so that Unicode characters with code points greater than
+255 are displayed as "\x{...}", control characters (like "\n") are
+displayed as "\x..", and the rest of the characters as themselves.
+
+sub nice_string {
+   join("",
+       map { $_ > 255 ?                        # if wide character...
+                 sprintf("\\x{%x}", $_) :      # \x{...}
+                 chr($_) =~ /[[:cntrl:]]/ ?    # else if control character ...
+                     sprintf("\\x%02x", $_) :  # \x..
+                      chr($_) }                        # else as themselves
+            unpack("U*", $_[0]));              # unpack Unicode characters
+}
+
+For example, C<nice_string("foo\x{100}bar\n")> will return
+C<"foo\x{100}bar\x0a">.
+
 =head2 Special Cases
 
 =over 4
@@ -680,6 +711,14 @@ the C<Unicode::UCD> module.
 
 =back
 
+=head1 UNICODE IN OLDER PERLS
+
+If you cannot upgrade your Perl to 5.8.0 or later, you can still
+do some Unicode processing by using the modules C<Unicode::String>,
+C<Unicode::Map8>, and C<Unicode::Map>, available from CPAN.
+If you have the GNU recode installed, you can also use the
+Perl frontend C<Convert::Recode> for character conversions.
+
 =head1 SEE ALSO
 
 L<perlunicode>, L<Encode>, L<encoding>, L<open>, L<utf8>, L<bytes>,