2 # Time-stamp: "2004-01-11 18:35:34 AST"
6 Locale::Maketext - framework for localization
13 # ...which inherits from Locale::Maketext
14 my $lh = MyProgram::L10N->get_handle() || die "What language?";
16 # And then any messages your program emits, like:
17 warn $lh->maketext( "Can't open file [_1]: [_2]\n", $f, $! );
22 It is a common feature of applications (whether run directly,
23 or via the Web) for them to be "localized" -- i.e., for them
24 to a present an English interface to an English-speaker, a German
25 interface to a German-speaker, and so on for all languages it's
26 programmed with. Locale::Maketext
27 is a framework for software localization; it provides you with the
28 tools for organizing and accessing the bits of text and text-processing
29 code that you need for producing localized applications.
31 In order to make sense of Maketext and how all its
32 components fit together, you should probably
33 go read L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13>, and
34 I<then> read the following documentation.
36 You may also want to read over the source for C<File::Findgrep>
37 and its constituent modules -- they are a complete (if small)
38 example application that uses Maketext.
42 The basic design of Locale::Maketext is object-oriented, and
43 Locale::Maketext is an abstract base class, from which you
44 derive a "project class".
45 The project class (with a name like "TkBocciBall::Localize",
46 which you then use in your module) is in turn the base class
47 for all the "language classes" for your project
48 (with names "TkBocciBall::Localize::it",
49 "TkBocciBall::Localize::en",
50 "TkBocciBall::Localize::fr", etc.).
53 a class containing a lexicon of phrases as class data,
54 and possibly also some methods that are of use in interpreting
55 phrases in the lexicon, or otherwise dealing with text in that
58 An object belonging to a language class is called a "language
59 handle"; it's typically a flyweight object.
61 The normal course of action is to call:
63 use TkBocciBall::Localize; # the localization project class
64 $lh = TkBocciBall::Localize->get_handle();
65 # Depending on the user's locale, etc., this will
66 # make a language handle from among the classes available,
67 # and any defaults that you declare.
68 die "Couldn't make a language handle??" unless $lh;
70 From then on, you use the C<maketext> function to access
71 entries in whatever lexicon(s) belong to the language handle
74 print $lh->maketext("You won!"), "\n";
76 ...emits the right text for this language. If the object
77 in C<$lh> belongs to class "TkBocciBall::Localize::fr" and
78 %TkBocciBall::Localize::fr::Lexicon contains C<("You won!"
79 =E<gt> "Tu as gagnE<eacute>!")>, then the above
80 code happily tells the user "Tu as gagnE<eacute>!".
84 Locale::Maketext offers a variety of methods, which fall
85 into three categories:
91 Methods to do with constructing language handles.
95 C<maketext> and other methods to do with accessing %Lexicon data
96 for a given language handle.
100 Methods that you may find it handy to use, from routines of
101 yours that you put in %Lexicon entries.
105 These are covered in the following section.
107 =head2 Construction Methods
109 These are to do with constructing a language handle:
115 $lh = YourProjClass->get_handle( ...langtags... ) || die "lg-handle?";
117 This tries loading classes based on the language-tags you give (like
118 C<("en-US", "sk", "kon", "es-MX", "ja", "i-klingon")>, and for the first class
119 that succeeds, returns YourProjClass::I<language>->new().
121 If it runs thru the entire given list of language-tags, and finds no classes
122 for those exact terms, it then tries "superordinate" language classes.
123 So if no "en-US" class (i.e., YourProjClass::en_us)
124 was found, nor classes for anything else in that list, we then try
125 its superordinate, "en" (i.e., YourProjClass::en), and so on thru
126 the other language-tags in the given list: "es".
127 (The other language-tags in our example list:
128 happen to have no superordinates.)
130 If none of those language-tags leads to loadable classes, we then
131 try classes derived from YourProjClass->fallback_languages() and
132 then if nothing comes of that, we use classes named by
133 YourProjClass->fallback_language_classes(). Then in the (probably
134 quite unlikely) event that that fails, we just return undef.
138 $lh = YourProjClass->get_handleB<()> || die "lg-handle?";
140 When C<get_handle> is called with an empty parameter list, magic happens:
142 If C<get_handle> senses that it's running in program that was
143 invoked as a CGI, then it tries to get language-tags out of the
144 environment variable "HTTP_ACCEPT_LANGUAGE", and it pretends that
145 those were the languages passed as parameters to C<get_handle>.
147 Otherwise (i.e., if not a CGI), this tries various OS-specific ways
148 to get the language-tags for the current locale/language, and then
149 pretends that those were the value(s) passed to C<get_handle>.
151 Currently this OS-specific stuff consists of looking in the environment
152 variables "LANG" and "LANGUAGE"; and on MSWin machines (where those
153 variables are typically unused), this also tries using
154 the module Win32::Locale to get a language-tag for whatever language/locale
155 is currently selected in the "Regional Settings" (or "International"?)
156 Control Panel. I welcome further
157 suggestions for making this do the Right Thing under other operating
158 systems that support localization.
160 If you're using localization in an application that keeps a configuration
161 file, you might consider something like this in your project class:
163 sub get_handle_via_config {
165 my $chosen_language = $Config_settings{'language'};
167 if($chosen_language) {
168 $lh = $class->get_handle($chosen_language)
169 || die "No language handle for \"$chosen_language\" or the like";
171 # Config file missing, maybe?
172 $lh = $class->get_handle()
173 || die "Can't get a language handle";
180 $lh = YourProjClass::langname->new();
182 This constructs a language handle. You usually B<don't> call this
183 directly, but instead let C<get_handle> find a language class to C<use>
184 and to then call ->new on.
190 This is called by ->new to initialize newly-constructed language handles.
191 If you define an init method in your class, remember that it's usually
192 considered a good idea to call $lh->SUPER::init in it (presumably at the
193 beginning), so that all classes get a chance to initialize a new object
194 however they see fit.
198 YourProjClass->fallback_languages()
200 C<get_handle> appends the return value of this to the end of
201 whatever list of languages you pass C<get_handle>. Unless
202 you override this method, your project class
203 will inherit Locale::Maketext's C<fallback_languages>, which
204 currently returns C<('i-default', 'en', 'en-US')>.
205 ("i-default" is defined in RFC 2277).
207 This method (by having it return the name
208 of a language-tag that has an existing language class)
209 can be used for making sure that
210 C<get_handle> will always manage to construct a language
211 handle (assuming your language classes are in an appropriate
212 @INC directory). Or you can use the next method:
216 YourProjClass->fallback_language_classes()
218 C<get_handle> appends the return value of this to the end
219 of the list of classes it will try using. Unless
220 you override this method, your project class
221 will inherit Locale::Maketext's C<fallback_language_classes>,
222 which currently returns an empty list, C<()>.
223 By setting this to some value (namely, the name of a loadable
224 language class), you can be sure that
225 C<get_handle> will always manage to construct a language
230 =head2 The "maketext" Method
232 This is the most important method in Locale::Maketext:
234 $text = $lh->maketext(I<key>, ...parameters for this phrase...);
236 This looks in the %Lexicon of the language handle
237 $lh and all its superclasses, looking
238 for an entry whose key is the string I<key>. Assuming such
239 an entry is found, various things then happen, depending on the
242 If the value is a scalarref, the scalar is dereferenced and returned
243 (and any parameters are ignored).
245 If the value is a coderef, we return &$value($lh, ...parameters...).
247 If the value is a string that I<doesn't> look like it's in Bracket Notation,
248 we return it (after replacing it with a scalarref, in its %Lexicon).
250 If the value I<does> look like it's in Bracket Notation, then we compile
251 it into a sub, replace the string in the %Lexicon with the new coderef,
252 and then we return &$new_sub($lh, ...parameters...).
254 Bracket Notation is discussed in a later section. Note
255 that trying to compile a string into Bracket Notation can throw
256 an exception if the string is not syntactically valid (say, by not
257 balancing brackets right.)
259 Also, calling &$coderef($lh, ...parameters...) can throw any sort of
260 exception (if, say, code in that sub tries to divide by zero). But
261 a very common exception occurs when you have Bracket
262 Notation text that says to call a method "foo", but there is no such
263 method. (E.g., "You have [quaB<tn>,_1,ball]." will throw an exception
264 on trying to call $lh->quaB<tn>($_[1],'ball') -- you presumably meant
265 "quant".) C<maketext> catches these exceptions, but only to make the
266 error message more readable, at which point it rethrows the exception.
268 An exception I<may> be thrown if I<key> is not found in any
269 of $lh's %Lexicon hashes. What happens if a key is not found,
270 is discussed in a later section, "Controlling Lookup Failure".
272 Note that you might find it useful in some cases to override
273 the C<maketext> method with an "after method", if you want to
274 translate encodings, or even scripts:
276 package YrProj::zh_cn; # Chinese with PRC-style glyphs
277 use base ('YrProj::zh_tw'); # Taiwan-style
279 my $self = shift(@_);
280 my $value = $self->maketext(@_);
281 return Chineeze::taiwan2mainland($value);
284 Or you may want to override it with something that traps
285 any exceptions, if that's critical to your program:
288 my($lh, @stuff) = @_;
290 eval { $out = $lh->SUPER::maketext(@stuff) };
291 return $out unless $@;
292 ...otherwise deal with the exception...
295 Other than those two situations, I don't imagine that
296 it's useful to override the C<maketext> method. (If
297 you run into a situation where it is useful, I'd be
298 interested in hearing about it.)
302 =item $lh->fail_with I<or> $lh->fail_with(I<PARAM>)
304 =item $lh->failure_handler_auto
306 These two methods are discussed in the section "Controlling
311 =head2 Utility Methods
313 These are methods that you may find it handy to use, generally
314 from %Lexicon routines of yours (whether expressed as
315 Bracket Notation or not).
319 =item $language->quant($number, $singular)
321 =item $language->quant($number, $singular, $plural)
323 =item $language->quant($number, $singular, $plural, $negative)
325 This is generally meant to be called from inside Bracket Notation
326 (which is discussed later), as in
328 "Your search matched [quant,_1,document]!"
330 It's for I<quantifying> a noun (i.e., saying how much of it there is,
331 while giving the correct form of it). The behavior of this method is
332 handy for English and a few other Western European languages, and you
333 should override it for languages where it's not suitable. You can feel
334 free to read the source, but the current implementation is basically
335 as this pseudocode describes:
337 if $number is 0 and there's a $negative,
340 return "1 $singular";
341 elsif there's a $plural,
342 return "$number $plural";
344 return "$number " . $singular . "s";
346 # ...except that we actually call numf to
347 # stringify $number before returning it.
349 So for English (with Bracket Notation)
350 C<"...[quant,_1,file]..."> is fine (for 0 it returns "0 files",
351 for 1 it returns "1 file", and for more it returns "2 files", etc.)
353 But for "directory", you'd want C<"[quant,_1,directory,directories]">
354 so that our elementary C<quant> method doesn't think that the
355 plural of "directory" is "directorys". And you might find that the
356 output may sound better if you specify a negative form, as in:
358 "[quant,_1,file,files,No files] matched your query.\n"
360 Remember to keep in mind verb agreement (or adjectives too, in
361 other languages), as in:
363 "[quant,_1,document] were matched.\n"
365 Because if _1 is one, you get "1 document B<were> matched".
366 An acceptable hack here is to do something like this:
368 "[quant,_1,document was, documents were] matched.\n"
370 =item $language->numf($number)
372 This returns the given number formatted nicely according to
373 this language's conventions. Maketext's default method is
374 mostly to just take the normal string form of the number
375 (applying sprintf "%G" for only very large numbers), and then
376 to add commas as necessary. (Except that
377 we apply C<tr/,./.,/> if $language->{'numf_comma'} is true;
378 that's a bit of a hack that's useful for languages that express
379 two million as "2.000.000" and not as "2,000,000").
381 If you want anything fancier, consider overriding this with something
382 that uses L<Number::Format|Number::Format>, or does something else
385 Note that numf is called by quant for stringifying all quantifying
388 =item $language->sprintf($format, @items)
390 This is just a wrapper around Perl's normal C<sprintf> function.
391 It's provided so that you can use "sprintf" in Bracket Notation:
393 "Couldn't access datanode [sprintf,%10x=~[%s~],_1,_2]!\n"
397 Couldn't access datanode Stuff=[thangamabob]!
399 =item $language->language_tag()
401 Currently this just takes the last bit of C<ref($language)>, turns
402 underscores to dashes, and returns it. So if $language is
403 an object of class Hee::HOO::Haw::en_us, $language->language_tag()
404 returns "en-us". (Yes, the usual representation for that language
405 tag is "en-US", but case is I<never> considered meaningful in
406 language-tag comparison.)
408 You may override this as you like; Maketext doesn't use it for
411 =item $language->encoding()
413 Currently this isn't used for anything, but it's provided
414 (with default value of
415 C<(ref($language) && $language-E<gt>{'encoding'})) or "iso-8859-1">
416 ) as a sort of suggestion that it may be useful/necessary to
417 associate encodings with your language handles (whether on a
418 per-class or even per-handle basis.)
422 =head2 Language Handle Attributes and Internals
424 A language handle is a flyweight object -- i.e., it doesn't (necessarily)
425 carry any data of interest, other than just being a member of
426 whatever class it belongs to.
428 A language handle is implemented as a blessed hash. Subclasses of yours
429 can store whatever data you want in the hash. Currently the only hash
430 entry used by any crucial Maketext method is "fail", so feel free to
431 use anything else as you like.
433 B<Remember: Don't be afraid to read the Maketext source if there's
434 any point on which this documentation is unclear.> This documentation
435 is vastly longer than the module source itself.
441 =head1 LANGUAGE CLASS HIERARCHIES
443 These are Locale::Maketext's assumptions about the class
444 hierarchy formed by all your language classes:
450 You must have a project base class, which you load, and
451 which you then use as the first argument in
452 the call to YourProjClass->get_handle(...). It should derive
453 (whether directly or indirectly) from Locale::Maketext.
454 It B<doesn't matter> how you name this class, although assuming this
455 is the localization component of your Super Mega Program,
456 good names for your project class might be
457 SuperMegaProgram::Localization, SuperMegaProgram::L10N,
458 SuperMegaProgram::I18N, SuperMegaProgram::International,
459 or even SuperMegaProgram::Languages or SuperMegaProgram::Messages.
463 Language classes are what YourProjClass->get_handle will try to load.
464 It will look for them by taking each language-tag (B<skipping> it
465 if it doesn't look like a language-tag or locale-tag!), turning it to
466 all lowercase, turning dashes to underscores, and appending it
467 to YourProjClass . "::". So this:
469 $lh = YourProjClass->get_handle(
470 'en-US', 'fr', 'kon', 'i-klingon', 'i-klingon-romanized'
473 will try loading the classes
474 YourProjClass::en_us (note lowercase!), YourProjClass::fr,
476 YourProjClass::i_klingon
477 and YourProjClass::i_klingon_romanized. (And it'll stop at the
478 first one that actually loads.)
482 I assume that each language class derives (directly or indirectly)
483 from your project class, and also defines its @ISA, its %Lexicon,
484 or both. But I anticipate no dire consequences if these assumptions
489 Language classes may derive from other language classes (although they
490 should have "use I<Thatclassname>" or "use base qw(I<...classes...>)").
491 They may derive from the project
492 class. They may derive from some other class altogether. Or via
493 multiple inheritance, it may derive from any mixture of these.
497 I foresee no problems with having multiple inheritance in
498 your hierarchy of language classes. (As usual, however, Perl will
499 complain bitterly if you have a cycle in the hierarchy: i.e., if
500 any class is its own ancestor.)
504 =head1 ENTRIES IN EACH LEXICON
506 A typical %Lexicon entry is meant to signify a phrase,
507 taking some number (0 or more) of parameters. An entry
508 is meant to be accessed by via
509 a string I<key> in $lh->maketext(I<key>, ...parameters...),
510 which should return a string that is generally meant for
511 be used for "output" to the user -- regardless of whether
512 this actually means printing to STDOUT, writing to a file,
513 or putting into a GUI widget.
515 While the key must be a string value (since that's a basic
516 restriction that Perl places on hash keys), the value in
517 the lexicon can currently be of several types:
518 a defined scalar, scalarref, or coderef. The use of these is
519 explained above, in the section 'The "maketext" Method', and
520 Bracket Notation for strings is discussed in the next section.
522 While you can use arbitrary unique IDs for lexicon keys
523 (like "_min_larger_max_error"), it is often
524 useful for if an entry's key is itself a valid value, like
525 this example error message:
527 "Minimum ([_1]) is larger than maximum ([_2])!\n",
529 Compare this code that uses an arbitrary ID...
531 die $lh->maketext( "_min_larger_max_error", $min, $max )
534 ...to this code that uses a key-as-value:
537 "Minimum ([_1]) is larger than maximum ([_2])!\n",
541 The second is, in short, more readable. In particular, it's obvious
542 that the number of parameters you're feeding to that phrase (two) is
543 the number of parameters that it I<wants> to be fed. (Since you see
544 _1 and a _2 being used in the key there.)
546 Also, once a project is otherwise
547 complete and you start to localize it, you can scrape together
548 all the various keys you use, and pass it to a translator; and then
549 the translator's work will go faster if what he's presented is this:
551 "Minimum ([_1]) is larger than maximum ([_2])!\n",
552 => "", # fill in something here, Jacques!
554 rather than this more cryptic mess:
556 "_min_larger_max_error"
557 => "", # fill in something here, Jacques
559 I think that keys as lexicon values makes the completed lexicon
560 entries more readable:
562 "Minimum ([_1]) is larger than maximum ([_2])!\n",
563 => "Le minimum ([_1]) est plus grand que le maximum ([_2])!\n",
565 Also, having valid values as keys becomes very useful if you set
566 up an _AUTO lexicon. _AUTO lexicons are discussed in a later
569 I almost always use keys that are themselves
570 valid lexicon values. One notable exception is when the value is
571 quite long. For example, to get the screenful of data that
572 a command-line program might return when given an unknown switch,
573 I often just use a brief, self-explanatory key such as "_USAGE_MESSAGE". At that point I then go
574 and immediately to define that lexicon entry in the
575 ProjectClass::L10N::en lexicon (since English is always my "project
578 '_USAGE_MESSAGE' => <<'EOSTUFF',
579 ...long long message...
582 and then I can use it as:
584 getopt('oDI', \%opts) or die $lh->maketext('_USAGE_MESSAGE');
587 note that each class's C<%Lexicon> inherits-and-extends
588 the lexicons in its superclasses. This is not because these are
589 special hashes I<per se>, but because you access them via the
590 C<maketext> method, which looks for entries across all the
591 C<%Lexicon> hashes in a language class I<and> all its ancestor classes.
592 (This is because the idea of "class data" isn't directly implemented
593 in Perl, but is instead left to individual class-systems to implement
596 Note that you may have things stored in a lexicon
597 besides just phrases for output: for example, if your program
598 takes input from the keyboard, asking a "(Y/N)" question,
599 you probably need to know what the equivalent of "Y[es]/N[o]" is
600 in whatever language. You probably also need to know what
601 the equivalents of the answers "y" and "n" are. You can
602 store that information in the lexicon (say, under the keys
603 "~answer_y" and "~answer_n", and the long forms as
604 "~answer_yes" and "~answer_no", where "~" is just an ad-hoc
605 character meant to indicate to programmers/translators that
606 these are not phrases for output).
608 Or instead of storing this in the language class's lexicon,
609 you can (and, in some cases, really should) represent the same bit
610 of knowledge as code in a method in the language class. (That
611 leaves a tidy distinction between the lexicon as the things we
612 know how to I<say>, and the rest of the things in the lexicon class
613 as things that we know how to I<do>.) Consider
614 this example of a processor for responses to French "oui/non"
618 return undef unless defined $_[1] and length $_[1];
619 my $answer = lc $_[1]; # smash case
620 return 1 if $answer eq 'o' or $answer eq 'oui';
621 return 0 if $answer eq 'n' or $answer eq 'non';
625 ...which you'd then call in a construct like this:
628 until(defined $response) {
629 print $lh->maketext("Open the pod bay door (y/n)? ");
630 $response = $lh->y_or_n( get_input_from_keyboard_somehow() );
632 if($response) { $pod_bay_door->open() }
633 else { $pod_bay_door->leave_closed() }
635 Other data worth storing in a lexicon might be things like
636 filenames for language-targetted resources:
640 => "/styles/en_us/main_splash.png",
641 "_main_splash_imagemap"
642 => "/styles/en_us/main_splash.incl",
643 "_general_graphics_path"
646 => "/styles/en_us/hey_there.wav",
650 => "right_arrow.png",
651 # In some other languages, left equals
652 # BACKwards, and right is FOREwards.
655 You might want to do the same thing for expressing key bindings
656 or the like (since hardwiring "q" as the binding for the function
657 that quits a screen/menu/program is useful only if your language
658 happens to associate "q" with "quit"!)
660 =head1 BRACKET NOTATION
662 Bracket Notation is a crucial feature of Locale::Maketext. I mean
663 Bracket Notation to provide a replacement for the use of sprintf formatting.
664 Everything you do with Bracket Notation could be done with a sub block,
665 but bracket notation is meant to be much more concise.
667 Bracket Notation is a like a miniature "template" system (in the sense
668 of L<Text::Template|Text::Template>, not in the sense of C++ templates),
669 where normal text is passed thru basically as is, but text in special
670 regions is specially interpreted. In Bracket Notation, you use square brackets ("[...]"),
671 not curly braces ("{...}") to note sections that are specially interpreted.
673 For example, here all the areas that are taken literally are underlined with
674 a "^", and all the in-bracket special regions are underlined with an X:
676 "Minimum ([_1]) is larger than maximum ([_2])!\n",
677 ^^^^^^^^^ XX ^^^^^^^^^^^^^^^^^^^^^^^^^^ XX ^^^^
679 When that string is compiled from bracket notation into a real Perl sub,
680 it's basically turned into:
688 ") is larger than maximum (",
692 # to be called by $lh->maketext(KEY, params...)
694 In other words, text outside bracket groups is turned into string
695 literals. Text in brackets is rather more complex, and currently follows
702 Bracket groups that are empty, or which consist only of whitespace,
703 are ignored. (Examples: "[]", "[ ]", or a [ and a ] with returns
704 and/or tabs and/or spaces between them.
706 Otherwise, each group is taken to be a comma-separated group of items,
707 and each item is interpreted as follows:
711 An item that is "_I<digits>" or "_-I<digits>" is interpreted as
712 $_[I<value>]. I.e., "_1" becomes with $_[1], and "_-3" is interpreted
713 as $_[-3] (in which case @_ should have at least three elements in it).
714 Note that $_[0] is the language handle, and is typically not named
719 An item "_*" is interpreted to mean "all of @_ except $_[0]".
720 I.e., C<@_[1..$#_]>. Note that this is an empty list in the case
721 of calls like $lh->maketext(I<key>) where there are no
722 parameters (except $_[0], the language handle).
726 Otherwise, each item is interpreted as a string literal.
730 The group as a whole is interpreted as follows:
736 If the first item in a bracket group looks like a method name,
737 then that group is interpreted like this:
739 $lh->that_method_name(
740 ...rest of items in this group...
745 If the first item in a bracket group is "*", it's taken as shorthand
746 for the so commonly called "quant" method. Similarly, if the first
747 item in a bracket group is "#", it's taken to be shorthand for
752 If the first item in a bracket group is the empty-string, or "_*"
753 or "_I<digits>" or "_-I<digits>", then that group is interpreted
754 as just the interpolation of all its items:
757 ...rest of items in this group...
760 Examples: "[_1]" and "[,_1]", which are synonymous; and
761 "C<[,ID-(,_4,-,_2,)]>", which compiles as
762 C<join "", "ID-(", $_[4], "-", $_[2], ")">.
766 Otherwise this bracket group is invalid. For example, in the group
767 "[!@#,whatever]", the first item C<"!@#"> is neither the empty-string,
768 "_I<number>", "_-I<number>", "_*", nor a valid method name; and so
769 Locale::Maketext will throw an exception of you try compiling an
770 expression containing this bracket group.
774 Note, incidentally, that items in each group are comma-separated,
775 not C</\s*,\s*/>-separated. That is, you might expect that this
778 "Hoohah [foo, _1 , bar ,baz]!"
780 would compile to this:
786 $lh->foo( $_[1], "bar", "baz"),
790 But it actually compiles as this:
796 $lh->foo(" _1 ", " bar ", "baz"), # note the <space> in " bar "
800 In the notation discussed so far, the characters "[" and "]" are given
801 special meaning, for opening and closing bracket groups, and "," has
802 a special meaning inside bracket groups, where it separates items in the
803 group. This begs the question of how you'd express a literal "[" or
804 "]" in a Bracket Notation string, and how you'd express a literal
805 comma inside a bracket group. For this purpose I've adopted "~" (tilde)
806 as an escape character: "~[" means a literal '[' character anywhere
807 in Bracket Notation (i.e., regardless of whether you're in a bracket
808 group or not), and ditto for "~]" meaning a literal ']', and "~," meaning
809 a literal comma. (Altho "," means a literal comma outside of
810 bracket groups -- it's only inside bracket groups that commas are special.)
812 And on the off chance you need a literal tilde in a bracket expression,
813 you get it with "~~".
815 Currently, an unescaped "~" before a character
816 other than a bracket or a comma is taken to mean just a "~" and that
817 character. I.e., "~X" means the same as "~~X" -- i.e., one literal tilde,
818 and then one literal "X". However, by using "~X", you are assuming that
819 no future version of Maketext will use "~X" as a magic escape sequence.
820 In practice this is not a great problem, since first off you can just
821 write "~~X" and not worry about it; second off, I doubt I'll add lots
822 of new magic characters to bracket notation; and third off, you
823 aren't likely to want literal "~" characters in your messages anyway,
824 since it's not a character with wide use in natural language text.
826 Brackets must be balanced -- every openbracket must have
827 one matching closebracket, and vice versa. So these are all B<invalid>:
829 "I ate [quant,_1,rhubarb pie."
830 "I ate [quant,_1,rhubarb pie[."
831 "I ate quant,_1,rhubarb pie]."
832 "I ate quant,_1,rhubarb pie[."
834 Currently, bracket groups do not nest. That is, you B<cannot> say:
836 "Foo [bar,baz,[quux,quuux]]\n";
838 If you need a notation that's that powerful, use normal Perl:
846 $lh->bar('baz', $lh->quux('quuux')),
852 Or write the "bar" method so you don't need to pass it the
853 output from calling quux.
855 I do not anticipate that you will need (or particularly want)
856 to nest bracket groups, but you are welcome to email me with
857 convincing (real-life) arguments to the contrary.
861 If maketext goes to look in an individual %Lexicon for an entry
862 for I<key> (where I<key> does not start with an underscore), and
863 sees none, B<but does see> an entry of "_AUTO" => I<some_true_value>,
864 then we actually define $Lexicon{I<key>} = I<key> right then and there,
865 and then use that value as if it had been there all
866 along. This happens before we even look in any superclass %Lexicons!
868 (This is meant to be somewhat like the AUTOLOAD mechanism in
869 Perl's function call system -- or, looked at another way,
870 like the L<AutoLoader|AutoLoader> module.)
872 I can picture all sorts of circumstances where you just
873 do not want lookup to be able to fail (since failing
874 normally means that maketext throws a C<die>, although
875 see the next section for greater control over that). But
876 here's one circumstance where _AUTO lexicons are meant to
877 be I<especially> useful:
879 As you're writing an application, you decide as you go what messages
880 you need to emit. Normally you'd go to write this:
883 go_process_file($filename)
885 print qq{Couldn't find file "$filename"!\n};
888 but since you anticipate localizing this, you write:
890 use ThisProject::I18N;
891 my $lh = ThisProject::I18N->get_handle();
892 # For the moment, assume that things are set up so
893 # that we load class ThisProject::I18N::en
894 # and that that's the class that $lh belongs to.
897 go_process_file($filename)
900 qq{Couldn't find file "[_1]"!\n}, $filename
904 Now, right after you've just written the above lines, you'd
905 normally have to go open the file
906 ThisProject/I18N/en.pm, and immediately add an entry:
908 "Couldn't find file \"[_1]\"!\n"
909 => "Couldn't find file \"[_1]\"!\n",
911 But I consider that somewhat of a distraction from the work
912 of getting the main code working -- to say nothing of the fact
913 that I often have to play with the program a few times before
914 I can decide exactly what wording I want in the messages (which
915 in this case would require me to go changing three lines of code:
916 the call to maketext with that key, and then the two lines in
917 ThisProject/I18N/en.pm).
919 However, if you set "_AUTO => 1" in the %Lexicon in,
920 ThisProject/I18N/en.pm (assuming that English (en) is
921 the language that all your programmers will be using for this
922 project's internal message keys), then you don't ever have to
923 go adding lines like this
925 "Couldn't find file \"[_1]\"!\n"
926 => "Couldn't find file \"[_1]\"!\n",
928 to ThisProject/I18N/en.pm, because if _AUTO is true there,
929 then just looking for an entry with the key "Couldn't find
930 file \"[_1]\"!\n" in that lexicon will cause it to be added,
933 Note that the reason that keys that start with "_"
934 are immune to _AUTO isn't anything generally magical about
935 the underscore character -- I just wanted a way to have most
936 lexicon keys be autoable, except for possibly a few, and I
937 arbitrarily decided to use a leading underscore as a signal
938 to distinguish those few.
940 =head1 CONTROLLING LOOKUP FAILURE
942 If you call $lh->maketext(I<key>, ...parameters...),
943 and there's no entry I<key> in $lh's class's %Lexicon, nor
944 in the superclass %Lexicon hash, I<and> if we can't auto-make
945 I<key> (because either it starts with a "_", or because none
946 of its lexicons have C<_AUTO =E<gt> 1,>), then we have
947 failed to find a normal way to maketext I<key>. What then
948 happens in these failure conditions, depends on the $lh object's
951 If the language handle has no "fail" attribute, maketext
952 will simply throw an exception (i.e., it calls C<die>, mentioning
953 the I<key> whose lookup failed, and naming the line number where
954 the calling $lh->maketext(I<key>,...) was.
956 If the language handle has a "fail" attribute whose value is a
957 coderef, then $lh->maketext(I<key>,...params...) gives up and calls:
959 return $that_subref->($lh, $key, @params);
961 Otherwise, the "fail" attribute's value should be a string denoting
962 a method name, so that $lh->maketext(I<key>,...params...) can
965 return $lh->$that_method_name($phrase, @params);
967 The "fail" attribute can be accessed with the C<fail_with> method:
970 $lh->fail_with( \&failure_handler );
972 # Set to a method name:
973 $lh->fail_with( 'failure_method' );
975 # Set to nothing (i.e., so failure throws a plain exception)
976 $lh->fail_with( undef );
978 # Get the current value
979 $handler = $lh->fail_with();
981 Now, as to what you may want to do with these handlers: Maybe you'd
982 want to log what key failed for what class, and then die. Maybe
983 you don't like C<die> and instead you want to send the error message
984 to STDOUT (or wherever) and then merely C<exit()>.
986 Or maybe you don't want to C<die> at all! Maybe you could use a
989 # Make all lookups fall back onto an English value,
990 # but only after we log it for later fingerpointing.
991 my $lh_backup = ThisProject->get_handle('en');
992 open(LEX_FAIL_LOG, ">>wherever/lex.log") || die "GNAARGH $!";
994 my($failing_lh, $key, $params) = @_;
995 print LEX_FAIL_LOG scalar(localtime), "\t",
996 ref($failing_lh), "\t", $key, "\n";
997 return $lh_backup->maketext($key,@params);
1000 Some users have expressed that they think this whole mechanism of
1001 having a "fail" attribute at all, seems a rather pointless complication.
1002 But I want Locale::Maketext to be usable for software projects of I<any>
1003 scale and type; and different software projects have different ideas
1004 of what the right thing is to do in failure conditions. I could simply
1005 say that failure always throws an exception, and that if you want to be
1006 careful, you'll just have to wrap every call to $lh->maketext in an
1007 S<eval { }>. However, I want programmers to reserve the right (via
1008 the "fail" attribute) to treat lookup failure as something other than
1009 an exception of the same level of severity as a config file being
1010 unreadable, or some essential resource being inaccessible.
1012 One possibly useful value for the "fail" attribute is the method name
1013 "failure_handler_auto". This is a method defined in the class
1014 Locale::Maketext itself. You set it with:
1016 $lh->fail_with('failure_handler_auto');
1018 Then when you call $lh->maketext(I<key>, ...parameters...) and
1019 there's no I<key> in any of those lexicons, maketext gives up with
1021 return $lh->failure_handler_auto($key, @params);
1023 But failure_handler_auto, instead of dying or anything, compiles
1026 $lh->{'failure_lex'}{$key} = $complied
1028 and then calls the compiled value, and returns that. (I.e., if
1029 $key looks like bracket notation, $compiled is a sub, and we return
1030 &{$compiled}(@params); but if $key is just a plain string, we just
1033 The effect of using "failure_auto_handler"
1034 is like an AUTO lexicon, except that it 1) compiles $key even if
1035 it starts with "_", and 2) you have a record in the new hashref
1036 $lh->{'failure_lex'} of all the keys that have failed for
1037 this object. This should avoid your program dying -- as long
1038 as your keys aren't actually invalid as bracket code, and as
1039 long as they don't try calling methods that don't exist.
1041 "failure_auto_handler" may not be exactly what you want, but I
1042 hope it at least shows you that maketext failure can be mitigated
1043 in any number of very flexible ways. If you can formalize exactly
1044 what you want, you should be able to express that as a failure
1045 handler. You can even make it default for every object of a given
1046 class, by setting it in that class's init:
1049 my $lh = $_[0]; # a newborn handle
1051 $lh->fail_with('my_clever_failure_handler');
1054 sub my_clever_failure_handler {
1055 ...you clever things here...
1058 =head1 HOW TO USE MAKETEXT
1060 Here is a brief checklist on how to use Maketext to localize
1067 Decide what system you'll use for lexicon keys. If you insist,
1068 you can use opaque IDs (if you're nostalgic for C<catgets>),
1069 but I have better suggestions in the
1070 section "Entries in Each Lexicon", above. Assuming you opt for
1071 meaningful keys that double as values (like "Minimum ([_1]) is
1072 larger than maximum ([_2])!\n"), you'll have to settle on what
1073 language those should be in. For the sake of argument, I'll
1074 call this English, specifically American English, "en-US".
1078 Create a class for your localization project. This is
1079 the name of the class that you'll use in the idiom:
1082 my $lh = Projname::L10N->get_handle(...) || die "Language?";
1084 Assuming you call your class Projname::L10N, create a class
1085 consisting minimally of:
1087 package Projname::L10N;
1088 use base qw(Locale::Maketext);
1089 ...any methods you might want all your languages to share...
1091 # And, assuming you want the base class to be an _AUTO lexicon,
1092 # as is discussed a few sections up:
1098 Create a class for the language your internal keys are in. Name
1099 the class after the language-tag for that language, in lowercase,
1100 with dashes changed to underscores. Assuming your project's first
1101 language is US English, you should call this Projname::L10N::en_us.
1102 It should consist minimally of:
1104 package Projname::L10N::en_us;
1105 use base qw(Projname::L10N);
1111 (For the rest of this section, I'll assume that this "first
1112 language class" of Projname::L10N::en_us has
1117 Go and write your program. Everywhere in your program where
1120 print "Foobar $thing stuff\n";
1122 instead do it thru maketext, using no variable interpolation in
1125 print $lh->maketext("Foobar [_1] stuff\n", $thing);
1127 If you get tired of constantly saying C<print $lh-E<gt>maketext>,
1128 consider making a functional wrapper for it, like so:
1132 $lh = Projname::L10N->get_handle(...) || die "Language?";
1133 sub pmt (@) { print( $lh->maketext(@_)) }
1134 # "pmt" is short for "Print MakeText"
1136 # so if maketext fails, we see made the call to pmt
1138 Besides whole phrases meant for output, anything language-dependent
1139 should be put into the class Projname::L10N::en_us,
1140 whether as methods, or as lexicon entries -- this is discussed
1141 in the section "Entries in Each Lexicon", above.
1145 Once the program is otherwise done, and once its localization for
1146 the first language works right (via the data and methods in
1147 Projname::L10N::en_us), you can get together the data for translation.
1148 If your first language lexicon isn't an _AUTO lexicon, then you already
1149 have all the messages explicitly in the lexicon (or else you'd be
1150 getting exceptions thrown when you call $lh->maketext to get
1151 messages that aren't in there). But if you were (advisedly) lazy and are
1152 using an _AUTO lexicon, then you've got to make a list of all the phrases
1153 that you've so far been letting _AUTO generate for you. There are very
1154 many ways to assemble such a list. The most straightforward is to simply
1155 grep the source for every occurrence of "maketext" (or calls
1156 to wrappers around it, like the above C<pmt> function), and to log the
1161 You may at this point want to consider whether your base class
1162 (Projname::L10N), from which all lexicons inherit from (Projname::L10N::en,
1163 Projname::L10N::es, etc.), should be an _AUTO lexicon. It may be true
1164 that in theory, all needed messages will be in each language class;
1165 but in the presumably unlikely or "impossible" case of lookup failure,
1166 you should consider whether your program should throw an exception,
1167 emit text in English (or whatever your project's first language is),
1168 or some more complex solution as described in the section
1169 "Controlling Lookup Failure", above.
1173 Submit all messages/phrases/etc. to translators.
1175 (You may, in fact, want to start with localizing to I<one> other language
1176 at first, if you're not sure that you've properly abstracted the
1177 language-dependent parts of your code.)
1179 Translators may request clarification of the situation in which a
1180 particular phrase is found. For example, in English we are entirely happy
1181 saying "I<n> files found", regardless of whether we mean "I looked for files,
1182 and found I<n> of them" or the rather distinct situation of "I looked for
1183 something else (like lines in files), and along the way I saw I<n>
1184 files." This may involve rethinking things that you thought quite clear:
1185 should "Edit" on a toolbar be a noun ("editing") or a verb ("to edit")? Is
1186 there already a conventionalized way to express that menu option, separate
1187 from the target language's normal word for "to edit"?
1189 In all cases where the very common phenomenon of quantification
1190 (saying "I<N> files", for B<any> value of N)
1191 is involved, each translator should make clear what dependencies the
1192 number causes in the sentence. In many cases, dependency is
1193 limited to words adjacent to the number, in places where you might
1194 expect them ("I found the-?PLURAL I<N>
1195 empty-?PLURAL directory-?PLURAL"), but in some cases there are
1196 unexpected dependencies ("I found-?PLURAL ..."!) as well as long-distance
1197 dependencies "The I<N> directory-?PLURAL could not be deleted-?PLURAL"!).
1199 Remind the translators to consider the case where N is 0:
1200 "0 files found" isn't exactly natural-sounding in any language, but it
1201 may be unacceptable in many -- or it may condition special
1202 kinds of agreement (similar to English "I didN'T find ANY files").
1204 Remember to ask your translators about numeral formatting in their
1205 language, so that you can override the C<numf> method as
1206 appropriate. Typical variables in number formatting are: what to
1207 use as a decimal point (comma? period?); what to use as a thousands
1208 separator (space? nonbreaking space? comma? period? small
1209 middot? prime? apostrophe?); and even whether the so-called "thousands
1210 separator" is actually for every third digit -- I've heard reports of
1211 two hundred thousand being expressible as "2,00,000" for some Indian
1212 (Subcontinental) languages, besides the less surprising "S<200 000>",
1213 "200.000", "200,000", and "200'000". Also, using a set of numeral
1214 glyphs other than the usual ASCII "0"-"9" might be appreciated, as via
1215 C<tr/0-9/\x{0966}-\x{096F}/> for getting digits in Devanagari script
1216 (for Hindi, Konkani, others).
1218 The basic C<quant> method that Locale::Maketext provides should be
1219 good for many languages. For some languages, it might be useful
1220 to modify it (or its constituent C<numerate> method)
1221 to take a plural form in the two-argument call to C<quant>
1222 (as in "[quant,_1,files]") if
1223 it's all-around easier to infer the singular form from the plural, than
1224 to infer the plural form from the singular.
1226 But for other languages (as is discussed at length
1227 in L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13>), simple
1228 C<quant>/C<numerify> is not enough. For the particularly problematic
1229 Slavic languages, what you may need is a method which you provide
1230 with the number, the citation form of the noun to quantify, and
1231 the case and gender that the sentence's syntax projects onto that
1232 noun slot. The method would then be responsible for determining
1233 what grammatical number that numeral projects onto its noun phrase,
1234 and what case and gender it may override the normal case and gender
1235 with; and then it would look up the noun in a lexicon providing
1236 all needed inflected forms.
1240 You may also wish to discuss with the translators the question of
1241 how to relate different subforms of the same language tag,
1242 considering how this reacts with C<get_handle>'s treatment of
1243 these. For example, if a user accepts interfaces in "en, fr", and
1244 you have interfaces available in "en-US" and "fr", what should
1245 they get? You may wish to resolve this by establishing that "en"
1246 and "en-US" are effectively synonymous, by having one class
1247 zero-derive from the other.
1249 For some languages this issue may never come up (Danish is rarely
1250 expressed as "da-DK", but instead is just "da"). And for other
1251 languages, the whole concept of a "generic" form may verge on
1252 being uselessly vague, particularly for interfaces involving voice
1253 media in forms of Arabic or Chinese.
1257 Once you've localized your program/site/etc. for all desired
1258 languages, be sure to show the result (whether live, or via
1259 screenshots) to the translators. Once they approve, make every
1260 effort to have it then checked by at least one other speaker of
1261 that language. This holds true even when (or especially when) the
1262 translation is done by one of your own programmers. Some
1263 kinds of systems may be harder to find testers for than others,
1264 depending on the amount of domain-specific jargon and concepts
1265 involved -- it's easier to find people who can tell you whether
1266 they approve of your translation for "delete this message" in an
1267 email-via-Web interface, than to find people who can give you
1268 an informed opinion on your translation for "attribute value"
1269 in an XML query tool's interface.
1275 I recommend reading all of these:
1277 L<Locale::Maketext::TPJ13|Locale::Maketext::TPJ13> -- my I<The Perl
1278 Journal> article about Maketext. It explains many important concepts
1279 underlying Locale::Maketext's design, and some insight into why
1280 Maketext is better than the plain old approach of having
1281 message catalogs that are just databases of sprintf formats.
1283 L<File::Findgrep|File::Findgrep> is a sample application/module
1284 that uses Locale::Maketext to localize its messages. For a larger
1285 internationalized system, see also L<Apache::MP3>.
1287 L<I18N::LangTags|I18N::LangTags>.
1289 L<Win32::Locale|Win32::Locale>.
1291 RFC 3066, I<Tags for the Identification of Languages>,
1292 as at http://sunsite.dk/RFC/rfc/rfc3066.html
1294 RFC 2277, I<IETF Policy on Character Sets and Languages>
1295 is at http://sunsite.dk/RFC/rfc/rfc2277.html -- much of it is
1296 just things of interest to protocol designers, but it explains
1297 some basic concepts, like the distinction between locales and
1300 The manual for GNU C<gettext>. The gettext dist is available in
1301 C<ftp://prep.ai.mit.edu/pub/gnu/> -- get
1302 a recent gettext tarball and look in its "doc/" directory, there's
1303 an easily browsable HTML version in there. The
1304 gettext documentation asks lots of questions worth thinking
1305 about, even if some of their answers are sometimes wonky,
1306 particularly where they start talking about pluralization.
1308 The Locale/Maketext.pm source. Obverse that the module is much
1309 shorter than its documentation!
1311 =head1 COPYRIGHT AND DISCLAIMER
1313 Copyright (c) 1999-2004 Sean M. Burke. All rights reserved.
1315 This library is free software; you can redistribute it and/or modify
1316 it under the same terms as Perl itself.
1318 This program is distributed in the hope that it will be useful, but
1319 without any warranty; without even the implied warranty of
1320 merchantability or fitness for a particular purpose.
1324 Sean M. Burke C<sburke@cpan.org>