Integrate mainline

[p5sagit/p5-mst-13.2.git] / pod / perlunicode.pod

diff --git a/pod/perlunicode.pod b/pod/perlunicode.pod
index 9ba32ee..d636700 100644 (file)
--- a/pod/perlunicode.pod
+++ b/pod/perlunicode.pod
@@ -20,6 +20,11 @@ Other encodings can be converted to perl's encoding on input, or from
 perl's encoding on output by use of the ":encoding(...)" layer.
 See L<open>.
 
+In some filesystems (for example Microsoft NTFS and Apple HFS+) the
+filenames are in UTF-8 .  By using opendir() and File::Glob you can
+make readdir() and glob() to return the filenames as Unicode, see
+L<perlfunc/opendir> and L<File::Glob> for details.
+
 To mark the Perl source itself as being in a particular encoding,
 see L<encoding>.
 
@@ -280,6 +285,7 @@ C<\p{Latin}> or \p{Cyrillic>, are as follows:
     Armenian
     Bengali
     Bopomofo
+    Buhid
     CanadianAboriginal
     Cherokee
     Cyrillic
@@ -293,6 +299,7 @@ C<\p{Latin}> or \p{Cyrillic>, are as follows:
     Gurmukhi
     Han
     Hangul
+    Hanunoo
     Hebrew
     Hiragana
     Inherited
@@ -310,6 +317,8 @@ C<\p{Latin}> or \p{Cyrillic>, are as follows:
     Runic
     Sinhala
     Syriac
+    Tagalog
+    Tagbanwa
     Tamil
     Telugu
     Thaana
@@ -320,21 +329,32 @@ C<\p{Latin}> or \p{Cyrillic>, are as follows:
 There are also extended property classes that supplement the basic
 properties, defined by the F<PropList> Unicode database:
 
-    ASCII_Hex_Digit
+    ASCIIHexDigit
     BidiControl
     Dash
+    Deprecated
     Diacritic
     Extender
+    GraphemeLink
     HexDigit
     Hyphen
     Ideographic
+    IDSBinaryOperator
+    IDSTrinaryOperator
     JoinControl
+    LogicalOrderException
     NoncharacterCodePoint
     OtherAlphabetic
+    OtherDefaultIgnorableCodePoint
+    OtherGraphemeExtend
     OtherLowercase
     OtherMath
     OtherUppercase
     QuotationMark
+    Radical
+    SoftDotted
+    TerminalPunctuation
+    UnifiedIdeograph
     WhiteSpace
 
 and further derived properties:
@@ -384,102 +404,116 @@ to avoid confusion.
 
 These block names are supported:
 
-   InAlphabeticPresentationForms
-   InArabicBlock
-   InArabicPresentationFormsA
-   InArabicPresentationFormsB
-   InArmenianBlock
-   InArrows
-   InBasicLatin
-   InBengaliBlock
-   InBlockElements
-   InBopomofoBlock
-   InBopomofoExtended
-   InBoxDrawing
-   InBraillePatterns
-   InByzantineMusicalSymbols
-   InCJKCompatibility
-   InCJKCompatibilityForms
-   InCJKCompatibilityIdeographs
-   InCJKCompatibilityIdeographsSupplement
-   InCJKRadicalsSupplement
-   InCJKSymbolsAndPunctuation
-   InCJKUnifiedIdeographs
-   InCJKUnifiedIdeographsExtensionA
-   InCJKUnifiedIdeographsExtensionB
-   InCherokeeBlock
-   InCombiningDiacriticalMarks
-   InCombiningHalfMarks
-   InCombiningMarksForSymbols
-   InControlPictures
-   InCurrencySymbols
-   InCyrillicBlock
-   InDeseretBlock
-   InDevanagariBlock
-   InDingbats
-   InEnclosedAlphanumerics
-   InEnclosedCJKLettersAndMonths
-   InEthiopicBlock
-   InGeneralPunctuation
-   InGeometricShapes
-   InGeorgianBlock
-   InGothicBlock
-   InGreekBlock
-   InGreekExtended
-   InGujaratiBlock
-   InGurmukhiBlock
-   InHalfwidthAndFullwidthForms
-   InHangulCompatibilityJamo
-   InHangulJamo
-   InHangulSyllables
-   InHebrewBlock
-   InHighPrivateUseSurrogates
-   InHighSurrogates
-   InHiraganaBlock
-   InIPAExtensions
-   InIdeographicDescriptionCharacters
-   InKanbun
-   InKangxiRadicals
-   InKannadaBlock
-   InKatakanaBlock
-   InKhmerBlock
-   InLaoBlock
-   InLatin1Supplement
-   InLatinExtendedAdditional
-   InLatinExtended-A
-   InLatinExtended-B
-   InLetterlikeSymbols
-   InLowSurrogates
-   InMalayalamBlock
-   InMathematicalAlphanumericSymbols
-   InMathematicalOperators
-   InMiscellaneousSymbols
-   InMiscellaneousTechnical
-   InMongolianBlock
-   InMusicalSymbols
-   InMyanmarBlock
-   InNumberForms
-   InOghamBlock
-   InOldItalicBlock
-   InOpticalCharacterRecognition
-   InOriyaBlock
-   InPrivateUse
-   InRunicBlock
-   InSinhalaBlock
-   InSmallFormVariants
-   InSpacingModifierLetters
-   InSpecials
-   InSuperscriptsAndSubscripts
-   InSyriacBlock
-   InTags
-   InTamilBlock
-   InTeluguBlock
-   InThaanaBlock
-   InThaiBlock
-   InTibetanBlock
-   InUnifiedCanadianAboriginalSyllabics
-   InYiRadicals
-   InYiSyllables
+    InAlphabeticPresentationForms
+    InArabic
+    InArabicPresentationFormsA
+    InArabicPresentationFormsB
+    InArmenian
+    InArrows
+    InBasicLatin
+    InBengali
+    InBlockElements
+    InBopomofo
+    InBopomofoExtended
+    InBoxDrawing
+    InBraillePatterns
+    InBuhid
+    InByzantineMusicalSymbols
+    InCJKCompatibility
+    InCJKCompatibilityForms
+    InCJKCompatibilityIdeographs
+    InCJKCompatibilityIdeographsSupplement
+    InCJKRadicalsSupplement
+    InCJKSymbolsAndPunctuation
+    InCJKUnifiedIdeographs
+    InCJKUnifiedIdeographsExtensionA
+    InCJKUnifiedIdeographsExtensionB
+    InCherokee
+    InCombiningDiacriticalMarks
+    InCombiningDiacriticalMarksforSymbols
+    InCombiningHalfMarks
+    InControlPictures
+    InCurrencySymbols
+    InCyrillic
+    InCyrillicSupplementary
+    InDeseret
+    InDevanagari
+    InDingbats
+    InEnclosedAlphanumerics
+    InEnclosedCJKLettersAndMonths
+    InEthiopic
+    InGeneralPunctuation
+    InGeometricShapes
+    InGeorgian
+    InGothic
+    InGreekExtended
+    InGreekAndCoptic
+    InGujarati
+    InGurmukhi
+    InHalfwidthAndFullwidthForms
+    InHangulCompatibilityJamo
+    InHangulJamo
+    InHangulSyllables
+    InHanunoo
+    InHebrew
+    InHighPrivateUseSurrogates
+    InHighSurrogates
+    InHiragana
+    InIPAExtensions
+    InIdeographicDescriptionCharacters
+    InKanbun
+    InKangxiRadicals
+    InKannada
+    InKatakana
+    InKatakanaPhoneticExtensions
+    InKhmer
+    InLao
+    InLatin1Supplement
+    InLatinExtendedA
+    InLatinExtendedAdditional
+    InLatinExtendedB
+    InLetterlikeSymbols
+    InLowSurrogates
+    InMalayalam
+    InMathematicalAlphanumericSymbols
+    InMathematicalOperators
+    InMiscellaneousMathematicalSymbolsA
+    InMiscellaneousMathematicalSymbolsB
+    InMiscellaneousSymbols
+    InMiscellaneousTechnical
+    InMongolian
+    InMusicalSymbols
+    InMyanmar
+    InNumberForms
+    InOgham
+    InOldItalic
+    InOpticalCharacterRecognition
+    InOriya
+    InPrivateUseArea
+    InRunic
+    InSinhala
+    InSmallFormVariants
+    InSpacingModifierLetters
+    InSpecials
+    InSuperscriptsAndSubscripts
+    InSupplementalArrowsA
+    InSupplementalArrowsB
+    InSupplementalMathematicalOperators
+    InSupplementaryPrivateUseAreaA
+    InSupplementaryPrivateUseAreaB
+    InSyriac
+    InTagalog
+    InTagbanwa
+    InTags
+    InTamil
+    InTelugu
+    InThaana
+    InThai
+    InTibetan
+    InUnifiedCanadianAboriginalSyllabics
+    InVariationSelectors
+    InYiRadicals
+    InYiSyllables
 
 =over 4
 
@@ -636,8 +670,8 @@ For example, what TR18 might write as
 
 in Perl can be written as:
 
-    (?!\p{Unassigned})\p{InGreek}
-    (?=\p{Assigned})\p{InGreek}
+    (?!\p{Unassigned})\p{InGreekAndCoptic}
+    (?=\p{Assigned})\p{InGreekAndCoptic}
 
 But in this particular example, you probably really want
 
@@ -708,6 +742,11 @@ The following table is from Unicode 3.2.
 
 Note the A0..BF in U+0800..U+0FFF, the 80..9F in U+D000...U+D7FF,
 the 90..BF in U+10000..U+3FFFF, and the 80...8F in U+100000..U+10FFFF.
+The "gaps" are caused by legal UTF-8 avoiding non-shortest encodings:
+it is technically possible to UTF-8-encode a single code point in different
+ways, but that is explicitly forbidden, and the shortest possible encoding
+should always be used (and that is what Perl does).
+
 Or, another way to look at it, as bits:
 
  Code Points                    1st Byte   2nd Byte  3rd Byte  4th Byte
@@ -750,7 +789,7 @@ are the range 0xDC00..0xDFFFF.  The surrogate encoding is
 
 and the decoding is
 
-       $uni = 0x10000 + ($hi - 0xD8000) * 0x400 + ($lo - 0xDC00);
+       $uni = 0x10000 + ($hi - 0xD800) * 0x400 + ($lo - 0xDC00);
 
 If you try to generate surrogates (for example by using chr()), you
 will get a warning if warnings are turned on (C<-w> or C<use