Naturally, C<ord()> will do the reverse: turn a character to a code point.
-Note that C<\x..>, C<\x{..}> and C<chr(...)> for arguments less than
-0x100 (decimal 256) will generate an eight-bit character for backward
-compatibility with older Perls. For arguments of 0x100 or more,
-Unicode will always be produced. If you want UTF-8 always, use
-C<pack("U", ...)> instead of C<\x..>, C<\x{..}>, or C<chr()>.
+Note that C<\x..> (no C<{}> and only two hexadecimal digits), C<\x{...}>
+and C<chr(...)> for arguments less than 0x100 (decimal 256) will
+generate an eight-bit character for backward compatibility with older
+Perls. For arguments of 0x100 or more, Unicode will always be
+produced. If you want UTF-8 always, use C<pack("U", ...)> instead of
+C<\x..>, C<\x{...}>, or C<chr()>.
You can also use the C<charnames> pragma to invoke characters
by name in doublequoted strings:
my $georgian_an = pack("U", 0x10a0);
+Note that both C<\x{...}> and C<\N{...}> are compile-time string
+constants: you cannot use variables in them. if you want similar
+run-time functionality, use C<chr()> and C<charnames::vianame()>.
+
=head2 Handling Unicode
Handling Unicode is for the most part transparent: just use the
With the C<open> pragma you can use the C<:locale> discipline
- $ENV{LANG} = 'ru_RU.KOI8-R';
- # the :locale will probe the locale environment variables like LANG
+ $ENV{LC_ALL} = $ENV{LANG} = 'ru_RU.KOI8-R';
+ # the :locale will probe the locale environment variables like LC_ALL
use open OUT => ':locale'; # russki parusski
open(O, ">koi8");
print O chr(0x430); # Unicode CYRILLIC SMALL LETTER A = KOI8-R 0xc1
You can switch encodings on an already opened stream by using
C<binmode()>, see L<perlfunc/binmode>.
-The C<:locale> does not currently work with C<open()> and
-C<binmode()>, only with the C<open> pragma. The C<:utf8> and
-C<:encoding(...)> do work with all of C<open()>, C<binmode()>,
-and the C<open> pragma.
+The C<:locale> does not currently (as of Perl 5.8.0) work with
+C<open()> and C<binmode()>, only with the C<open> pragma. The
+C<:utf8> and C<:encoding(...)> do work with all of C<open()>,
+C<binmode()>, and the C<open> pragma.
Similarly, you may use these I/O disciplines on input streams to
automatically convert data from the specified encoding when it is
UTF-8 encoded. A C<use open ':utf8'> would have avoided the bug, or
explicitly opening also the F<file> for input as UTF-8.
+=head2 Displaying Unicode As Text
+
+Sometimes you might want to display Perl scalars containing Unicode as
+simple ASCII (or EBCDIC) text. The following subroutine will convert
+its argument so that Unicode characters with code points greater than
+255 are displayed as "\x{...}", control characters (like "\n") are
+displayed as "\x..", and the rest of the characters as themselves.
+
+sub nice_string {
+ join("",
+ map { $_ > 255 ? # if wide character...
+ sprintf("\\x{%x}", $_) : # \x{...}
+ chr($_) =~ /[[:cntrl:]]/ ? # else if control character ...
+ sprintf("\\x%02x", $_) : # \x..
+ chr($_) } # else as themselves
+ unpack("U*", $_[0])); # unpack Unicode characters
+}
+
+For example, C<nice_string("foo\x{100}bar\n")> will return
+C<"foo\x{100}bar\x0a">.
+
=head2 Special Cases
=over 4
The question of string equivalence turns somewhat complicated
in Unicode: what do you mean by equal?
- Is C<LATIN CAPITAL LETTER A WITH ACUTE> equal to
- C<LATIN CAPITAL LETTER A>?
+(Is C<LATIN CAPITAL LETTER A WITH ACUTE> equal to
+C<LATIN CAPITAL LETTER A>?)
The short answer is that by default Perl compares equivalence
(C<eq>, C<ne>) based only on code points of the characters.
People like to see their strings nicely sorted, or as Unicode
parlance goes, collated. But again, what do you mean by collate?
- Does C<LATIN CAPITAL LETTER A WITH ACUTE> come before or after
- C<LATIN CAPITAL LETTER A WITH GRAVE>?
+(Does C<LATIN CAPITAL LETTER A WITH ACUTE> come before or after
+C<LATIN CAPITAL LETTER A WITH GRAVE>?)
The short answer is that by default Perl compares strings (C<lt>,
C<le>, C<cmp>, C<ge>, C<gt>) based only on the code points of the
=back
+=head1 UNICODE IN OLDER PERLS
+
+If you cannot upgrade your Perl to 5.8.0 or later, you can still
+do some Unicode processing by using the modules C<Unicode::String>,
+C<Unicode::Map8>, and C<Unicode::Map>, available from CPAN.
+If you have the GNU recode installed, you can also use the
+Perl frontend C<Convert::Recode> for character conversions.
+
=head1 SEE ALSO
L<perlunicode>, L<Encode>, L<encoding>, L<open>, L<utf8>, L<bytes>,