994 lines
39 KiB
Plaintext
994 lines
39 KiB
Plaintext
=head1 NAME
|
|
|
|
perllocale - Perl locale handling (internationalization and localization)
|
|
|
|
=head1 DESCRIPTION
|
|
|
|
Perl supports language-specific notions of data such as "is this
|
|
a letter", "what is the uppercase equivalent of this letter", and
|
|
"which of these letters comes first". These are important issues,
|
|
especially for languages other than English--but also for English: it
|
|
would be naE<iuml>ve to imagine that C<A-Za-z> defines all the "letters"
|
|
needed to write in English. Perl is also aware that some character other
|
|
than '.' may be preferred as a decimal point, and that output date
|
|
representations may be language-specific. The process of making an
|
|
application take account of its users' preferences in such matters is
|
|
called B<internationalization> (often abbreviated as B<i18n>); telling
|
|
such an application about a particular set of preferences is known as
|
|
B<localization> (B<l10n>).
|
|
|
|
Perl can understand language-specific data via the standardized (ISO C,
|
|
XPG4, POSIX 1.c) method called "the locale system". The locale system is
|
|
controlled per application using one pragma, one function call, and
|
|
several environment variables.
|
|
|
|
B<NOTE>: This feature is new in Perl 5.004, and does not apply unless an
|
|
application specifically requests it--see L<Backward compatibility>.
|
|
The one exception is that write() now B<always> uses the current locale
|
|
- see L<"NOTES">.
|
|
|
|
=head1 PREPARING TO USE LOCALES
|
|
|
|
If Perl applications are to understand and present your data
|
|
correctly according a locale of your choice, B<all> of the following
|
|
must be true:
|
|
|
|
=over 4
|
|
|
|
=item *
|
|
|
|
B<Your operating system must support the locale system>. If it does,
|
|
you should find that the setlocale() function is a documented part of
|
|
its C library.
|
|
|
|
=item *
|
|
|
|
B<Definitions for locales that you use must be installed>. You, or
|
|
your system administrator, must make sure that this is the case. The
|
|
available locales, the location in which they are kept, and the manner
|
|
in which they are installed all vary from system to system. Some systems
|
|
provide only a few, hard-wired locales and do not allow more to be
|
|
added. Others allow you to add "canned" locales provided by the system
|
|
supplier. Still others allow you or the system administrator to define
|
|
and add arbitrary locales. (You may have to ask your supplier to
|
|
provide canned locales that are not delivered with your operating
|
|
system.) Read your system documentation for further illumination.
|
|
|
|
=item *
|
|
|
|
B<Perl must believe that the locale system is supported>. If it does,
|
|
C<perl -V:d_setlocale> will say that the value for C<d_setlocale> is
|
|
C<define>.
|
|
|
|
=back
|
|
|
|
If you want a Perl application to process and present your data
|
|
according to a particular locale, the application code should include
|
|
the S<C<use locale>> pragma (see L<The use locale pragma>) where
|
|
appropriate, and B<at least one> of the following must be true:
|
|
|
|
=over 4
|
|
|
|
=item *
|
|
|
|
B<The locale-determining environment variables (see L<"ENVIRONMENT">)
|
|
must be correctly set up> at the time the application is started, either
|
|
by yourself or by whoever set up your system account.
|
|
|
|
=item *
|
|
|
|
B<The application must set its own locale> using the method described in
|
|
L<The setlocale function>.
|
|
|
|
=back
|
|
|
|
=head1 USING LOCALES
|
|
|
|
=head2 The use locale pragma
|
|
|
|
By default, Perl ignores the current locale. The S<C<use locale>>
|
|
pragma tells Perl to use the current locale for some operations:
|
|
|
|
=over 4
|
|
|
|
=item *
|
|
|
|
B<The comparison operators> (C<lt>, C<le>, C<cmp>, C<ge>, and C<gt>) and
|
|
the POSIX string collation functions strcoll() and strxfrm() use
|
|
C<LC_COLLATE>. sort() is also affected if used without an
|
|
explicit comparison function, because it uses C<cmp> by default.
|
|
|
|
B<Note:> C<eq> and C<ne> are unaffected by locale: they always
|
|
perform a byte-by-byte comparison of their scalar operands. What's
|
|
more, if C<cmp> finds that its operands are equal according to the
|
|
collation sequence specified by the current locale, it goes on to
|
|
perform a byte-by-byte comparison, and only returns I<0> (equal) if the
|
|
operands are bit-for-bit identical. If you really want to know whether
|
|
two strings--which C<eq> and C<cmp> may consider different--are equal
|
|
as far as collation in the locale is concerned, see the discussion in
|
|
L<Category LC_COLLATE: Collation>.
|
|
|
|
=item *
|
|
|
|
B<Regular expressions and case-modification functions> (uc(), lc(),
|
|
ucfirst(), and lcfirst()) use C<LC_CTYPE>
|
|
|
|
=item *
|
|
|
|
B<The formatting functions> (printf(), sprintf() and write()) use
|
|
C<LC_NUMERIC>
|
|
|
|
=item *
|
|
|
|
B<The POSIX date formatting function> (strftime()) uses C<LC_TIME>.
|
|
|
|
=back
|
|
|
|
C<LC_COLLATE>, C<LC_CTYPE>, and so on, are discussed further in L<LOCALE
|
|
CATEGORIES>.
|
|
|
|
The default behavior is restored with the S<C<no locale>> pragma, or
|
|
upon reaching the end of block enclosing C<use locale>.
|
|
|
|
The string result of any operation that uses locale
|
|
information is tainted, as it is possible for a locale to be
|
|
untrustworthy. See L<"SECURITY">.
|
|
|
|
=head2 The setlocale function
|
|
|
|
You can switch locales as often as you wish at run time with the
|
|
POSIX::setlocale() function:
|
|
|
|
# This functionality not usable prior to Perl 5.004
|
|
require 5.004;
|
|
|
|
# Import locale-handling tool set from POSIX module.
|
|
# This example uses: setlocale -- the function call
|
|
# LC_CTYPE -- explained below
|
|
use POSIX qw(locale_h);
|
|
|
|
# query and save the old locale
|
|
$old_locale = setlocale(LC_CTYPE);
|
|
|
|
setlocale(LC_CTYPE, "fr_CA.ISO8859-1");
|
|
# LC_CTYPE now in locale "French, Canada, codeset ISO 8859-1"
|
|
|
|
setlocale(LC_CTYPE, "");
|
|
# LC_CTYPE now reset to default defined by LC_ALL/LC_CTYPE/LANG
|
|
# environment variables. See below for documentation.
|
|
|
|
# restore the old locale
|
|
setlocale(LC_CTYPE, $old_locale);
|
|
|
|
The first argument of setlocale() gives the B<category>, the second the
|
|
B<locale>. The category tells in what aspect of data processing you
|
|
want to apply locale-specific rules. Category names are discussed in
|
|
L<LOCALE CATEGORIES> and L<"ENVIRONMENT">. The locale is the name of a
|
|
collection of customization information corresponding to a particular
|
|
combination of language, country or territory, and codeset. Read on for
|
|
hints on the naming of locales: not all systems name locales as in the
|
|
example.
|
|
|
|
If no second argument is provided and the category is something else
|
|
than LC_ALL, the function returns a string naming the current locale
|
|
for the category. You can use this value as the second argument in a
|
|
subsequent call to setlocale().
|
|
|
|
If no second argument is provided and the category is LC_ALL, the
|
|
result is implementation-dependent. It may be a string of
|
|
concatenated locales names (separator also implementation-dependent)
|
|
or a single locale name. Please consult your L<setlocale(3)> for
|
|
details.
|
|
|
|
If a second argument is given and it corresponds to a valid locale,
|
|
the locale for the category is set to that value, and the function
|
|
returns the now-current locale value. You can then use this in yet
|
|
another call to setlocale(). (In some implementations, the return
|
|
value may sometimes differ from the value you gave as the second
|
|
argument--think of it as an alias for the value you gave.)
|
|
|
|
As the example shows, if the second argument is an empty string, the
|
|
category's locale is returned to the default specified by the
|
|
corresponding environment variables. Generally, this results in a
|
|
return to the default that was in force when Perl started up: changes
|
|
to the environment made by the application after startup may or may not
|
|
be noticed, depending on your system's C library.
|
|
|
|
If the second argument does not correspond to a valid locale, the locale
|
|
for the category is not changed, and the function returns I<undef>.
|
|
|
|
For further information about the categories, consult L<setlocale(3)>.
|
|
|
|
=head2 Finding locales
|
|
|
|
For locales available in your system, consult also L<setlocale(3)> to
|
|
see whether it leads to the list of available locales (search for the
|
|
I<SEE ALSO> section). If that fails, try the following command lines:
|
|
|
|
locale -a
|
|
|
|
nlsinfo
|
|
|
|
ls /usr/lib/nls/loc
|
|
|
|
ls /usr/lib/locale
|
|
|
|
ls /usr/lib/nls
|
|
|
|
ls /usr/share/locale
|
|
|
|
and see whether they list something resembling these
|
|
|
|
en_US.ISO8859-1 de_DE.ISO8859-1 ru_RU.ISO8859-5
|
|
en_US.iso88591 de_DE.iso88591 ru_RU.iso88595
|
|
en_US de_DE ru_RU
|
|
en de ru
|
|
english german russian
|
|
english.iso88591 german.iso88591 russian.iso88595
|
|
english.roman8 russian.koi8r
|
|
|
|
Sadly, even though the calling interface for setlocale() has been
|
|
standardized, names of locales and the directories where the
|
|
configuration resides have not been. The basic form of the name is
|
|
I<language_territory>B<.>I<codeset>, but the latter parts after
|
|
I<language> are not always present. The I<language> and I<country>
|
|
are usually from the standards B<ISO 3166> and B<ISO 639>, the
|
|
two-letter abbreviations for the countries and the languages of the
|
|
world, respectively. The I<codeset> part often mentions some B<ISO
|
|
8859> character set, the Latin codesets. For example, C<ISO 8859-1>
|
|
is the so-called "Western European codeset" that can be used to encode
|
|
most Western European languages adequately. Again, there are several
|
|
ways to write even the name of that one standard. Lamentably.
|
|
|
|
Two special locales are worth particular mention: "C" and "POSIX".
|
|
Currently these are effectively the same locale: the difference is
|
|
mainly that the first one is defined by the C standard, the second by
|
|
the POSIX standard. They define the B<default locale> in which
|
|
every program starts in the absence of locale information in its
|
|
environment. (The I<default> default locale, if you will.) Its language
|
|
is (American) English and its character codeset ASCII.
|
|
|
|
B<NOTE>: Not all systems have the "POSIX" locale (not all systems are
|
|
POSIX-conformant), so use "C" when you need explicitly to specify this
|
|
default locale.
|
|
|
|
=head2 LOCALE PROBLEMS
|
|
|
|
You may encounter the following warning message at Perl startup:
|
|
|
|
perl: warning: Setting locale failed.
|
|
perl: warning: Please check that your locale settings:
|
|
LC_ALL = "En_US",
|
|
LANG = (unset)
|
|
are supported and installed on your system.
|
|
perl: warning: Falling back to the standard locale ("C").
|
|
|
|
This means that your locale settings had LC_ALL set to "En_US" and
|
|
LANG exists but has no value. Perl tried to believe you but could not.
|
|
Instead, Perl gave up and fell back to the "C" locale, the default locale
|
|
that is supposed to work no matter what. This usually means your locale
|
|
settings were wrong, they mention locales your system has never heard
|
|
of, or the locale installation in your system has problems (for example,
|
|
some system files are broken or missing). There are quick and temporary
|
|
fixes to these problems, as well as more thorough and lasting fixes.
|
|
|
|
=head2 Temporarily fixing locale problems
|
|
|
|
The two quickest fixes are either to render Perl silent about any
|
|
locale inconsistencies or to run Perl under the default locale "C".
|
|
|
|
Perl's moaning about locale problems can be silenced by setting the
|
|
environment variable PERL_BADLANG to a zero value, for example "0".
|
|
This method really just sweeps the problem under the carpet: you tell
|
|
Perl to shut up even when Perl sees that something is wrong. Do not
|
|
be surprised if later something locale-dependent misbehaves.
|
|
|
|
Perl can be run under the "C" locale by setting the environment
|
|
variable LC_ALL to "C". This method is perhaps a bit more civilized
|
|
than the PERL_BADLANG approach, but setting LC_ALL (or
|
|
other locale variables) may affect other programs as well, not just
|
|
Perl. In particular, external programs run from within Perl will see
|
|
these changes. If you make the new settings permanent (read on), all
|
|
programs you run see the changes. See L<ENVIRONMENT> for for
|
|
the full list of relevant environment variables and L<USING LOCALES>
|
|
for their effects in Perl. Effects in other programs are
|
|
easily deducible. For example, the variable LC_COLLATE may well affect
|
|
your B<sort> program (or whatever the program that arranges `records'
|
|
alphabetically in your system is called).
|
|
|
|
You can test out changing these variables temporarily, and if the
|
|
new settings seem to help, put those settings into your shell startup
|
|
files. Consult your local documentation for the exact details. For in
|
|
Bourne-like shells (B<sh>, B<ksh>, B<bash>, B<zsh>):
|
|
|
|
LC_ALL=en_US.ISO8859-1
|
|
export LC_ALL
|
|
|
|
This assumes that we saw the locale "en_US.ISO8859-1" using the commands
|
|
discussed above. We decided to try that instead of the above faulty
|
|
locale "En_US"--and in Cshish shells (B<csh>, B<tcsh>)
|
|
|
|
setenv LC_ALL en_US.ISO8859-1
|
|
|
|
If you do not know what shell you have, consult your local
|
|
helpdesk or the equivalent.
|
|
|
|
=head2 Permanently fixing locale problems
|
|
|
|
The slower but superior fixes are when you may be able to yourself
|
|
fix the misconfiguration of your own environment variables. The
|
|
mis(sing)configuration of the whole system's locales usually requires
|
|
the help of your friendly system administrator.
|
|
|
|
First, see earlier in this document about L<Finding locales>. That tells
|
|
how to find which locales are really supported--and more importantly,
|
|
installed--on your system. In our example error message, environment
|
|
variables affecting the locale are listed in the order of decreasing
|
|
importance (and unset variables do not matter). Therefore, having
|
|
LC_ALL set to "En_US" must have been the bad choice, as shown by the
|
|
error message. First try fixing locale settings listed first.
|
|
|
|
Second, if using the listed commands you see something B<exactly>
|
|
(prefix matches do not count and case usually counts) like "En_US"
|
|
without the quotes, then you should be okay because you are using a
|
|
locale name that should be installed and available in your system.
|
|
In this case, see L<Permanently fixing system locale configuration>.
|
|
|
|
=head2 Permanently fixing your locale configuration
|
|
|
|
This is when you see something like:
|
|
|
|
perl: warning: Please check that your locale settings:
|
|
LC_ALL = "En_US",
|
|
LANG = (unset)
|
|
are supported and installed on your system.
|
|
|
|
but then cannot see that "En_US" listed by the above-mentioned
|
|
commands. You may see things like "en_US.ISO8859-1", but that isn't
|
|
the same. In this case, try running under a locale
|
|
that you can list and which somehow matches what you tried. The
|
|
rules for matching locale names are a bit vague because
|
|
standardization is weak in this area. See again the L<Finding
|
|
locales> about general rules.
|
|
|
|
=head2 Fixing system locale configuration
|
|
|
|
Contact a system administrator (preferably your own) and report the exact
|
|
error message you get, and ask them to read this same documentation you
|
|
are now reading. They should be able to check whether there is something
|
|
wrong with the locale configuration of the system. The L<Finding locales>
|
|
section is unfortunately a bit vague about the exact commands and places
|
|
because these things are not that standardized.
|
|
|
|
=head2 The localeconv function
|
|
|
|
The POSIX::localeconv() function allows you to get particulars of the
|
|
locale-dependent numeric formatting information specified by the current
|
|
C<LC_NUMERIC> and C<LC_MONETARY> locales. (If you just want the name of
|
|
the current locale for a particular category, use POSIX::setlocale()
|
|
with a single parameter--see L<The setlocale function>.)
|
|
|
|
use POSIX qw(locale_h);
|
|
|
|
# Get a reference to a hash of locale-dependent info
|
|
$locale_values = localeconv();
|
|
|
|
# Output sorted list of the values
|
|
for (sort keys %$locale_values) {
|
|
printf "%-20s = %s\n", $_, $locale_values->{$_}
|
|
}
|
|
|
|
localeconv() takes no arguments, and returns B<a reference to> a hash.
|
|
The keys of this hash are variable names for formatting, such as
|
|
C<decimal_point> and C<thousands_sep>. The values are the
|
|
corresponding, er, values. See L<POSIX (3)/localeconv> for a longer
|
|
example listing the categories an implementation might be expected to
|
|
provide; some provide more and others fewer. You don't need an
|
|
explicit C<use locale>, because localeconv() always observes the
|
|
current locale.
|
|
|
|
Here's a simple-minded example program that rewrites its command-line
|
|
parameters as integers correctly formatted in the current locale:
|
|
|
|
# See comments in previous example
|
|
require 5.004;
|
|
use POSIX qw(locale_h);
|
|
|
|
# Get some of locale's numeric formatting parameters
|
|
my ($thousands_sep, $grouping) =
|
|
@{localeconv()}{'thousands_sep', 'grouping'};
|
|
|
|
# Apply defaults if values are missing
|
|
$thousands_sep = ',' unless $thousands_sep;
|
|
|
|
# grouping and mon_grouping are packed lists
|
|
# of small integers (characters) telling the
|
|
# grouping (thousand_seps and mon_thousand_seps
|
|
# being the group dividers) of numbers and
|
|
# monetary quantities. The integers' meanings:
|
|
# 255 means no more grouping, 0 means repeat
|
|
# the previous grouping, 1-254 means use that
|
|
# as the current grouping. Grouping goes from
|
|
# right to left (low to high digits). In the
|
|
# below we cheat slightly by never using anything
|
|
# else than the first grouping (whatever that is).
|
|
if ($grouping) {
|
|
@grouping = unpack("C*", $grouping);
|
|
} else {
|
|
@grouping = (3);
|
|
}
|
|
|
|
# Format command line params for current locale
|
|
for (@ARGV) {
|
|
$_ = int; # Chop non-integer part
|
|
1 while
|
|
s/(\d)(\d{$grouping[0]}($|$thousands_sep))/$1$thousands_sep$2/;
|
|
print "$_";
|
|
}
|
|
print "\n";
|
|
|
|
=head1 LOCALE CATEGORIES
|
|
|
|
The following subsections describe basic locale categories. Beyond these,
|
|
some combination categories allow manipulation of more than one
|
|
basic category at a time. See L<"ENVIRONMENT"> for a discussion of these.
|
|
|
|
=head2 Category LC_COLLATE: Collation
|
|
|
|
In the scope of S<C<use locale>>, Perl looks to the C<LC_COLLATE>
|
|
environment variable to determine the application's notions on collation
|
|
(ordering) of characters. For example, 'b' follows 'a' in Latin
|
|
alphabets, but where do 'E<aacute>' and 'E<aring>' belong? And while
|
|
'color' follows 'chocolate' in English, what about in Spanish?
|
|
|
|
The following collations all make sense and you may meet any of them
|
|
if you "use locale".
|
|
|
|
A B C D E a b c d e
|
|
A a B b C c D d D e
|
|
a A b B c C d D e E
|
|
a b c d e A B C D E
|
|
|
|
Here is a code snippet to tell what alphanumeric
|
|
characters are in the current locale, in that locale's order:
|
|
|
|
use locale;
|
|
print +(sort grep /\w/, map { chr() } 0..255), "\n";
|
|
|
|
Compare this with the characters that you see and their order if you
|
|
state explicitly that the locale should be ignored:
|
|
|
|
no locale;
|
|
print +(sort grep /\w/, map { chr() } 0..255), "\n";
|
|
|
|
This machine-native collation (which is what you get unless S<C<use
|
|
locale>> has appeared earlier in the same block) must be used for
|
|
sorting raw binary data, whereas the locale-dependent collation of the
|
|
first example is useful for natural text.
|
|
|
|
As noted in L<USING LOCALES>, C<cmp> compares according to the current
|
|
collation locale when C<use locale> is in effect, but falls back to a
|
|
byte-by-byte comparison for strings that the locale says are equal. You
|
|
can use POSIX::strcoll() if you don't want this fall-back:
|
|
|
|
use POSIX qw(strcoll);
|
|
$equal_in_locale =
|
|
!strcoll("space and case ignored", "SpaceAndCaseIgnored");
|
|
|
|
$equal_in_locale will be true if the collation locale specifies a
|
|
dictionary-like ordering that ignores space characters completely and
|
|
which folds case.
|
|
|
|
If you have a single string that you want to check for "equality in
|
|
locale" against several others, you might think you could gain a little
|
|
efficiency by using POSIX::strxfrm() in conjunction with C<eq>:
|
|
|
|
use POSIX qw(strxfrm);
|
|
$xfrm_string = strxfrm("Mixed-case string");
|
|
print "locale collation ignores spaces\n"
|
|
if $xfrm_string eq strxfrm("Mixed-casestring");
|
|
print "locale collation ignores hyphens\n"
|
|
if $xfrm_string eq strxfrm("Mixedcase string");
|
|
print "locale collation ignores case\n"
|
|
if $xfrm_string eq strxfrm("mixed-case string");
|
|
|
|
strxfrm() takes a string and maps it into a transformed string for use
|
|
in byte-by-byte comparisons against other transformed strings during
|
|
collation. "Under the hood", locale-affected Perl comparison operators
|
|
call strxfrm() for both operands, then do a byte-by-byte
|
|
comparison of the transformed strings. By calling strxfrm() explicitly
|
|
and using a non locale-affected comparison, the example attempts to save
|
|
a couple of transformations. But in fact, it doesn't save anything: Perl
|
|
magic (see L<perlguts/Magic Variables>) creates the transformed version of a
|
|
string the first time it's needed in a comparison, then keeps this version around
|
|
in case it's needed again. An example rewritten the easy way with
|
|
C<cmp> runs just about as fast. It also copes with null characters
|
|
embedded in strings; if you call strxfrm() directly, it treats the first
|
|
null it finds as a terminator. don't expect the transformed strings
|
|
it produces to be portable across systems--or even from one revision
|
|
of your operating system to the next. In short, don't call strxfrm()
|
|
directly: let Perl do it for you.
|
|
|
|
Note: C<use locale> isn't shown in some of these examples because it isn't
|
|
needed: strcoll() and strxfrm() exist only to generate locale-dependent
|
|
results, and so always obey the current C<LC_COLLATE> locale.
|
|
|
|
=head2 Category LC_CTYPE: Character Types
|
|
|
|
In the scope of S<C<use locale>>, Perl obeys the C<LC_CTYPE> locale
|
|
setting. This controls the application's notion of which characters are
|
|
alphabetic. This affects Perl's C<\w> regular expression metanotation,
|
|
which stands for alphanumeric characters--that is, alphabetic and
|
|
numeric characters. (Consult L<perlre> for more information about
|
|
regular expressions.) Thanks to C<LC_CTYPE>, depending on your locale
|
|
setting, characters like 'E<aelig>', 'E<eth>', 'E<szlig>', and
|
|
'E<oslash>' may be understood as C<\w> characters.
|
|
|
|
The C<LC_CTYPE> locale also provides the map used in transliterating
|
|
characters between lower and uppercase. This affects the case-mapping
|
|
functions--lc(), lcfirst, uc(), and ucfirst(); case-mapping
|
|
interpolation with C<\l>, C<\L>, C<\u>, or C<\U> in double-quoted strings
|
|
and C<s///> substitutions; and case-independent regular expression
|
|
pattern matching using the C<i> modifier.
|
|
|
|
Finally, C<LC_CTYPE> affects the POSIX character-class test
|
|
functions--isalpha(), islower(), and so on. For example, if you move
|
|
from the "C" locale to a 7-bit Scandinavian one, you may find--possibly
|
|
to your surprise--that "|" moves from the ispunct() class to isalpha().
|
|
|
|
B<Note:> A broken or malicious C<LC_CTYPE> locale definition may result
|
|
in clearly ineligible characters being considered to be alphanumeric by
|
|
your application. For strict matching of (mundane) letters and
|
|
digits--for example, in command strings--locale-aware applications
|
|
should use C<\w> inside a C<no locale> block. See L<"SECURITY">.
|
|
|
|
=head2 Category LC_NUMERIC: Numeric Formatting
|
|
|
|
In the scope of S<C<use locale>>, Perl obeys the C<LC_NUMERIC> locale
|
|
information, which controls an application's idea of how numbers should
|
|
be formatted for human readability by the printf(), sprintf(), and
|
|
write() functions. String-to-numeric conversion by the POSIX::strtod()
|
|
function is also affected. In most implementations the only effect is to
|
|
change the character used for the decimal point--perhaps from '.' to ','.
|
|
These functions aren't aware of such niceties as thousands separation and
|
|
so on. (See L<The localeconv function> if you care about these things.)
|
|
|
|
Output produced by print() is B<never> affected by the
|
|
current locale: it is independent of whether C<use locale> or C<no
|
|
locale> is in effect, and corresponds to what you'd get from printf()
|
|
in the "C" locale. The same is true for Perl's internal conversions
|
|
between numeric and string formats:
|
|
|
|
use POSIX qw(strtod);
|
|
use locale;
|
|
|
|
$n = 5/2; # Assign numeric 2.5 to $n
|
|
|
|
$a = " $n"; # Locale-independent conversion to string
|
|
|
|
print "half five is $n\n"; # Locale-independent output
|
|
|
|
printf "half five is %g\n", $n; # Locale-dependent output
|
|
|
|
print "DECIMAL POINT IS COMMA\n"
|
|
if $n == (strtod("2,5"))[0]; # Locale-dependent conversion
|
|
|
|
=head2 Category LC_MONETARY: Formatting of monetary amounts
|
|
|
|
The C standard defines the C<LC_MONETARY> category, but no function
|
|
that is affected by its contents. (Those with experience of standards
|
|
committees will recognize that the working group decided to punt on the
|
|
issue.) Consequently, Perl takes no notice of it. If you really want
|
|
to use C<LC_MONETARY>, you can query its contents--see L<The localeconv
|
|
function>--and use the information that it returns in your application's
|
|
own formatting of currency amounts. However, you may well find that
|
|
the information, voluminous and complex though it may be, still does not
|
|
quite meet your requirements: currency formatting is a hard nut to crack.
|
|
|
|
=head2 LC_TIME
|
|
|
|
Output produced by POSIX::strftime(), which builds a formatted
|
|
human-readable date/time string, is affected by the current C<LC_TIME>
|
|
locale. Thus, in a French locale, the output produced by the C<%B>
|
|
format element (full month name) for the first month of the year would
|
|
be "janvier". Here's how to get a list of long month names in the
|
|
current locale:
|
|
|
|
use POSIX qw(strftime);
|
|
for (0..11) {
|
|
$long_month_name[$_] =
|
|
strftime("%B", 0, 0, 0, 1, $_, 96);
|
|
}
|
|
|
|
Note: C<use locale> isn't needed in this example: as a function that
|
|
exists only to generate locale-dependent results, strftime() always
|
|
obeys the current C<LC_TIME> locale.
|
|
|
|
=head2 Other categories
|
|
|
|
The remaining locale category, C<LC_MESSAGES> (possibly supplemented
|
|
by others in particular implementations) is not currently used by
|
|
Perl--except possibly to affect the behavior of library functions called
|
|
by extensions outside the standard Perl distribution.
|
|
|
|
=head1 SECURITY
|
|
|
|
Although the main discussion of Perl security issues can be found in
|
|
L<perlsec>, a discussion of Perl's locale handling would be incomplete
|
|
if it did not draw your attention to locale-dependent security issues.
|
|
Locales--particularly on systems that allow unprivileged users to
|
|
build their own locales--are untrustworthy. A malicious (or just plain
|
|
broken) locale can make a locale-aware application give unexpected
|
|
results. Here are a few possibilities:
|
|
|
|
=over 4
|
|
|
|
=item *
|
|
|
|
Regular expression checks for safe file names or mail addresses using
|
|
C<\w> may be spoofed by an C<LC_CTYPE> locale that claims that
|
|
characters such as "E<gt>" and "|" are alphanumeric.
|
|
|
|
=item *
|
|
|
|
String interpolation with case-mapping, as in, say, C<$dest =
|
|
"C:\U$name.$ext">, may produce dangerous results if a bogus LC_CTYPE
|
|
case-mapping table is in effect.
|
|
|
|
=item *
|
|
|
|
If the decimal point character in the C<LC_NUMERIC> locale is
|
|
surreptitiously changed from a dot to a comma, C<sprintf("%g",
|
|
0.123456e3)> produces a string result of "123,456". Many people would
|
|
interpret this as one hundred and twenty-three thousand, four hundred
|
|
and fifty-six.
|
|
|
|
=item *
|
|
|
|
A sneaky C<LC_COLLATE> locale could result in the names of students with
|
|
"D" grades appearing ahead of those with "A"s.
|
|
|
|
=item *
|
|
|
|
An application that takes the trouble to use information in
|
|
C<LC_MONETARY> may format debits as if they were credits and vice versa
|
|
if that locale has been subverted. Or it might make payments in US
|
|
dollars instead of Hong Kong dollars.
|
|
|
|
=item *
|
|
|
|
The date and day names in dates formatted by strftime() could be
|
|
manipulated to advantage by a malicious user able to subvert the
|
|
C<LC_DATE> locale. ("Look--it says I wasn't in the building on
|
|
Sunday.")
|
|
|
|
=back
|
|
|
|
Such dangers are not peculiar to the locale system: any aspect of an
|
|
application's environment which may be modified maliciously presents
|
|
similar challenges. Similarly, they are not specific to Perl: any
|
|
programming language that allows you to write programs that take
|
|
account of their environment exposes you to these issues.
|
|
|
|
Perl cannot protect you from all possibilities shown in the
|
|
examples--there is no substitute for your own vigilance--but, when
|
|
C<use locale> is in effect, Perl uses the tainting mechanism (see
|
|
L<perlsec>) to mark string results that become locale-dependent, and
|
|
which may be untrustworthy in consequence. Here is a summary of the
|
|
tainting behavior of operators and functions that may be affected by
|
|
the locale:
|
|
|
|
=over 4
|
|
|
|
=item B<Comparison operators> (C<lt>, C<le>, C<ge>, C<gt> and C<cmp>):
|
|
|
|
Scalar true/false (or less/equal/greater) result is never tainted.
|
|
|
|
=item B<Case-mapping interpolation> (with C<\l>, C<\L>, C<\u> or C<\U>)
|
|
|
|
Result string containing interpolated material is tainted if
|
|
C<use locale> is in effect.
|
|
|
|
=item B<Matching operator> (C<m//>):
|
|
|
|
Scalar true/false result never tainted.
|
|
|
|
Subpatterns, either delivered as a list-context result or as $1 etc.
|
|
are tainted if C<use locale> is in effect, and the subpattern regular
|
|
expression contains C<\w> (to match an alphanumeric character), C<\W>
|
|
(non-alphanumeric character), C<\s> (white-space character), or C<\S>
|
|
(non white-space character). The matched-pattern variable, $&, $`
|
|
(pre-match), $' (post-match), and $+ (last match) are also tainted if
|
|
C<use locale> is in effect and the regular expression contains C<\w>,
|
|
C<\W>, C<\s>, or C<\S>.
|
|
|
|
=item B<Substitution operator> (C<s///>):
|
|
|
|
Has the same behavior as the match operator. Also, the left
|
|
operand of C<=~> becomes tainted when C<use locale> in effect
|
|
if modified as a result of a substitution based on a regular
|
|
expression match involving C<\w>, C<\W>, C<\s>, or C<\S>; or of
|
|
case-mapping with C<\l>, C<\L>,C<\u> or C<\U>.
|
|
|
|
=item B<In-memory formatting function> (sprintf()):
|
|
|
|
Result is tainted if C<use locale> is in effect.
|
|
|
|
=item B<Output formatting functions> (printf() and write()):
|
|
|
|
Success/failure result is never tainted.
|
|
|
|
=item B<Case-mapping functions> (lc(), lcfirst(), uc(), ucfirst()):
|
|
|
|
Results are tainted if C<use locale> is in effect.
|
|
|
|
=item B<POSIX locale-dependent functions> (localeconv(), strcoll(),
|
|
strftime(), strxfrm()):
|
|
|
|
Results are never tainted.
|
|
|
|
=item B<POSIX character class tests> (isalnum(), isalpha(), isdigit(),
|
|
isgraph(), islower(), isprint(), ispunct(), isspace(), isupper(),
|
|
isxdigit()):
|
|
|
|
True/false results are never tainted.
|
|
|
|
=back
|
|
|
|
Three examples illustrate locale-dependent tainting.
|
|
The first program, which ignores its locale, won't run: a value taken
|
|
directly from the command line may not be used to name an output file
|
|
when taint checks are enabled.
|
|
|
|
#/usr/local/bin/perl -T
|
|
# Run with taint checking
|
|
|
|
# Command line sanity check omitted...
|
|
$tainted_output_file = shift;
|
|
|
|
open(F, ">$tainted_output_file")
|
|
or warn "Open of $untainted_output_file failed: $!\n";
|
|
|
|
The program can be made to run by "laundering" the tainted value through
|
|
a regular expression: the second example--which still ignores locale
|
|
information--runs, creating the file named on its command line
|
|
if it can.
|
|
|
|
#/usr/local/bin/perl -T
|
|
|
|
$tainted_output_file = shift;
|
|
$tainted_output_file =~ m%[\w/]+%;
|
|
$untainted_output_file = $&;
|
|
|
|
open(F, ">$untainted_output_file")
|
|
or warn "Open of $untainted_output_file failed: $!\n";
|
|
|
|
Compare this with a similar but locale-aware program:
|
|
|
|
#/usr/local/bin/perl -T
|
|
|
|
$tainted_output_file = shift;
|
|
use locale;
|
|
$tainted_output_file =~ m%[\w/]+%;
|
|
$localized_output_file = $&;
|
|
|
|
open(F, ">$localized_output_file")
|
|
or warn "Open of $localized_output_file failed: $!\n";
|
|
|
|
This third program fails to run because $& is tainted: it is the result
|
|
of a match involving C<\w> while C<use locale> is in effect.
|
|
|
|
=head1 ENVIRONMENT
|
|
|
|
=over 12
|
|
|
|
=item PERL_BADLANG
|
|
|
|
A string that can suppress Perl's warning about failed locale settings
|
|
at startup. Failure can occur if the locale support in the operating
|
|
system is lacking (broken) in some way--or if you mistyped the name of
|
|
a locale when you set up your environment. If this environment
|
|
variable is absent, or has a value that does not evaluate to integer
|
|
zero--that is, "0" or ""-- Perl will complain about locale setting
|
|
failures.
|
|
|
|
B<NOTE>: PERL_BADLANG only gives you a way to hide the warning message.
|
|
The message tells about some problem in your system's locale support,
|
|
and you should investigate what the problem is.
|
|
|
|
=back
|
|
|
|
The following environment variables are not specific to Perl: They are
|
|
part of the standardized (ISO C, XPG4, POSIX 1.c) setlocale() method
|
|
for controlling an application's opinion on data.
|
|
|
|
=over 12
|
|
|
|
=item LC_ALL
|
|
|
|
C<LC_ALL> is the "override-all" locale environment variable. If
|
|
set, it overrides all the rest of the locale environment variables.
|
|
|
|
=item LANGUAGE
|
|
|
|
B<NOTE>: C<LANGUAGE> is a GNU extension, it affects you only if you
|
|
are using the GNU libc. This is the case if you are using e.g. Linux.
|
|
If you are using "commercial" UNIXes you are most probably I<not>
|
|
using GNU libc and you can ignore C<LANGUAGE>.
|
|
|
|
However, in the case you are using C<LANGUAGE>: it affects the
|
|
language of informational, warning, and error messages output by
|
|
commands (in other words, it's like C<LC_MESSAGES>) but it has higher
|
|
priority than L<LC_ALL>. Moreover, it's not a single value but
|
|
instead a "path" (":"-separated list) of I<languages> (not locales).
|
|
See the GNU C<gettext> library documentation for more information.
|
|
|
|
=item LC_CTYPE
|
|
|
|
In the absence of C<LC_ALL>, C<LC_CTYPE> chooses the character type
|
|
locale. In the absence of both C<LC_ALL> and C<LC_CTYPE>, C<LANG>
|
|
chooses the character type locale.
|
|
|
|
=item LC_COLLATE
|
|
|
|
In the absence of C<LC_ALL>, C<LC_COLLATE> chooses the collation
|
|
(sorting) locale. In the absence of both C<LC_ALL> and C<LC_COLLATE>,
|
|
C<LANG> chooses the collation locale.
|
|
|
|
=item LC_MONETARY
|
|
|
|
In the absence of C<LC_ALL>, C<LC_MONETARY> chooses the monetary
|
|
formatting locale. In the absence of both C<LC_ALL> and C<LC_MONETARY>,
|
|
C<LANG> chooses the monetary formatting locale.
|
|
|
|
=item LC_NUMERIC
|
|
|
|
In the absence of C<LC_ALL>, C<LC_NUMERIC> chooses the numeric format
|
|
locale. In the absence of both C<LC_ALL> and C<LC_NUMERIC>, C<LANG>
|
|
chooses the numeric format.
|
|
|
|
=item LC_TIME
|
|
|
|
In the absence of C<LC_ALL>, C<LC_TIME> chooses the date and time
|
|
formatting locale. In the absence of both C<LC_ALL> and C<LC_TIME>,
|
|
C<LANG> chooses the date and time formatting locale.
|
|
|
|
=item LANG
|
|
|
|
C<LANG> is the "catch-all" locale environment variable. If it is set, it
|
|
is used as the last resort after the overall C<LC_ALL> and the
|
|
category-specific C<LC_...>.
|
|
|
|
=back
|
|
|
|
=head1 NOTES
|
|
|
|
=head2 Backward compatibility
|
|
|
|
Versions of Perl prior to 5.004 B<mostly> ignored locale information,
|
|
generally behaving as if something similar to the C<"C"> locale were
|
|
always in force, even if the program environment suggested otherwise
|
|
(see L<The setlocale function>). By default, Perl still behaves this
|
|
way for backward compatibility. If you want a Perl application to pay
|
|
attention to locale information, you B<must> use the S<C<use locale>>
|
|
pragma (see L<The use locale pragma>) to instruct it to do so.
|
|
|
|
Versions of Perl from 5.002 to 5.003 did use the C<LC_CTYPE>
|
|
information if available; that is, C<\w> did understand what
|
|
were the letters according to the locale environment variables.
|
|
The problem was that the user had no control over the feature:
|
|
if the C library supported locales, Perl used them.
|
|
|
|
=head2 I18N:Collate obsolete
|
|
|
|
In versions of Perl prior to 5.004, per-locale collation was possible
|
|
using the C<I18N::Collate> library module. This module is now mildly
|
|
obsolete and should be avoided in new applications. The C<LC_COLLATE>
|
|
functionality is now integrated into the Perl core language: One can
|
|
use locale-specific scalar data completely normally with C<use locale>,
|
|
so there is no longer any need to juggle with the scalar references of
|
|
C<I18N::Collate>.
|
|
|
|
=head2 Sort speed and memory use impacts
|
|
|
|
Comparing and sorting by locale is usually slower than the default
|
|
sorting; slow-downs of two to four times have been observed. It will
|
|
also consume more memory: once a Perl scalar variable has participated
|
|
in any string comparison or sorting operation obeying the locale
|
|
collation rules, it will take 3-15 times more memory than before. (The
|
|
exact multiplier depends on the string's contents, the operating system
|
|
and the locale.) These downsides are dictated more by the operating
|
|
system's implementation of the locale system than by Perl.
|
|
|
|
=head2 write() and LC_NUMERIC
|
|
|
|
Formats are the only part of Perl that unconditionally use information
|
|
from a program's locale; if a program's environment specifies an
|
|
LC_NUMERIC locale, it is always used to specify the decimal point
|
|
character in formatted output. Formatted output cannot be controlled by
|
|
C<use locale> because the pragma is tied to the block structure of the
|
|
program, and, for historical reasons, formats exist outside that block
|
|
structure.
|
|
|
|
=head2 Freely available locale definitions
|
|
|
|
There is a large collection of locale definitions at
|
|
C<ftp://dkuug.dk/i18n/WG15-collection>. You should be aware that it is
|
|
unsupported, and is not claimed to be fit for any purpose. If your
|
|
system allows installation of arbitrary locales, you may find the
|
|
definitions useful as they are, or as a basis for the development of
|
|
your own locales.
|
|
|
|
=head2 I18n and l10n
|
|
|
|
"Internationalization" is often abbreviated as B<i18n> because its first
|
|
and last letters are separated by eighteen others. (You may guess why
|
|
the internalin ... internaliti ... i18n tends to get abbreviated.) In
|
|
the same way, "localization" is often abbreviated to B<l10n>.
|
|
|
|
=head2 An imperfect standard
|
|
|
|
Internationalization, as defined in the C and POSIX standards, can be
|
|
criticized as incomplete, ungainly, and having too large a granularity.
|
|
(Locales apply to a whole process, when it would arguably be more useful
|
|
to have them apply to a single thread, window group, or whatever.) They
|
|
also have a tendency, like standards groups, to divide the world into
|
|
nations, when we all know that the world can equally well be divided
|
|
into bankers, bikers, gamers, and so on. But, for now, it's the only
|
|
standard we've got. This may be construed as a bug.
|
|
|
|
=head1 BUGS
|
|
|
|
=head2 Broken systems
|
|
|
|
In certain systems, the operating system's locale support
|
|
is broken and cannot be fixed or used by Perl. Such deficiencies can
|
|
and will result in mysterious hangs and/or Perl core dumps when the
|
|
C<use locale> is in effect. When confronted with such a system,
|
|
please report in excruciating detail to <F<perlbug@perl.com>>, and
|
|
complain to your vendor: bug fixes may exist for these problems
|
|
in your operating system. Sometimes such bug fixes are called an
|
|
operating system upgrade.
|
|
|
|
=head1 SEE ALSO
|
|
|
|
L<POSIX (3)/isalnum>
|
|
|
|
L<POSIX (3)/isalpha>
|
|
|
|
L<POSIX (3)/isdigit>
|
|
|
|
L<POSIX (3)/isgraph>
|
|
|
|
L<POSIX (3)/islower>
|
|
|
|
L<POSIX (3)/isprint>,
|
|
|
|
L<POSIX (3)/ispunct>
|
|
|
|
L<POSIX (3)/isspace>
|
|
|
|
L<POSIX (3)/isupper>,
|
|
|
|
L<POSIX (3)/isxdigit>
|
|
|
|
L<POSIX (3)/localeconv>
|
|
|
|
L<POSIX (3)/setlocale>,
|
|
|
|
L<POSIX (3)/strcoll>
|
|
|
|
L<POSIX (3)/strftime>
|
|
|
|
L<POSIX (3)/strtod>,
|
|
|
|
L<POSIX (3)/strxfrm>
|
|
|
|
=head1 HISTORY
|
|
|
|
Jarkko Hietaniemi's original F<perli18n.pod> heavily hacked by Dominic
|
|
Dunlop, assisted by the perl5-porters. Prose worked over a bit by
|
|
Tom Christiansen.
|
|
|
|
Last update: Thu Jun 11 08:44:13 MDT 1998
|