X-Git-Url: http://git.shadowcat.co.uk/gitweb/gitweb.cgi?a=blobdiff_plain;f=pod%2Fperlunicode.pod;h=6606ecdc86249c28b34ac229a736cec9b0d1b531;hb=888aee597441568824c1835285c8012bab253529;hp=2fca71454a7f9c7b696c3178ef21174c7925748d;hpb=29bdacb8f1686adf0d6e73a4e2fd7fb9becf6eab;p=p5sagit%2Fp5-mst-13.2.git diff --git a/pod/perlunicode.pod b/pod/perlunicode.pod index 2fca714..6606ecd 100644 --- a/pod/perlunicode.pod +++ b/pod/perlunicode.pod @@ -694,6 +694,31 @@ length (1 to 6 bytes, current character allocations require 4 bytes), byteorder independent encoding. For ASCII, UTF-8 is transparent (and we really do mean 7-bit ASCII, not any 8-bit encoding). +The following table is from Unicode 3.1. + + Code Points 1st Byte 2nd Byte 3rd Byte 4th Byte + + U+0000..U+007F 00..7F    + U+0080..U+07FF C2..DF 80..BF    + U+0800..U+0FFF E0 A0..BF 80..BF   + U+1000..U+FFFF E1..EF 80..BF 80..BF   + U+10000..U+3FFFF F0 90..BF 80..BF 80..BF + U+40000..U+FFFFF F1..F3 80..BF 80..BF 80..BF + U+100000..U+10FFFF F4 80..8F 80..BF 80..BF + +Or, another way to look at it, as bits: + + Code Points 1st Byte 2nd Byte 3rd Byte 4th Byte + + 0aaaaaaa 0aaaaaaa + 00000bbbbbaaaaaa 110bbbbb 10aaaaaa + ccccbbbbbbaaaaaa 1110cccc 10bbbbbb 10aaaaaa + 00000dddccccccbbbbbbaaaaaa 11110ddd 10cccccc 10bbbbbb 10aaaaaa + +As you can see, the continuation bytes all begin with C<10>, and the +leading bits of the start byte tells how many bytes the are in the +encoded character. + =item UTF-16, UTF-16BE, UTF16-LE, Surrogates, and BOMs (Byte Order Marks) UTF-16 is a 2 or 4 byte encoding. The Unicode code points @@ -782,7 +807,7 @@ for more discussion of the issues. =head1 SEE ALSO -L, L, L, L, L, L, -L +L, L, L, L, L, L, +L, L =cut