Subject: Locale Definition

From: [email-masked] (Chang Hyeoungkyu)
Date: Thu, 14 Nov 91 2:12:45 JST

Dear members,

In the included memo, I want to say for the definition of locale.
Also I want to argue the need of locale and application specific
locale sensitive functions.

I'm cc'ing this to i18n mailing list. 

If I'm going to wrong direction, please let me know.
I'll greatly appreciate your comments.

Best Regards,
Chang Hyeoungkyu

-- 
Chang Hyeoungkyu  -  [email-masked]
GUI Consortium / SA Lab., CS Dept., KAIST, KOREA

--------------------------------------------------------------------

Locale
------

The term 'locale' is defined as the combination of language, cultural
data and coded character set by International POSIX and X/Open --
Uniforum Joint Internationalization Group. And the POSIX model of
internationalization allows a specific version of a locale modified
for an application type. Because locale sensitive processing is
missing from the definition of locale, locale sensitive processing
should be done in locale and application neutral way by the definition
of locale. That is to say, a locale sensitive function should operate
for all locales including the modified locales by application
modifiers by switching locale information data. The examples of locale
sensitive functions are atof(), printf(), scanf(), strcoll() and so on.

I don't think that locale sensitive processing can be language,
cultural data and application neutral. So I define a 'locale' as a
combination of language, cultural data, coded character set and their
processing. By the term 'processing', I mean only the locale and
application sensitive processing and it can be achieved by locale and
application specific locale sensitive functions, while common (locale
and application neutral) locale sensitive functions are used by
International POSIX and X/Open -- Uniforum Joint Internationalization
Group. Below I present the examples for collating sequences with which
we can't do language, cultural data and application neutral
processing. The examples are chosen from the book titled 'Digital
Guide to Developing International Software' and the e-mail 'Guideline
for producing a national POSIX locale' of Mr. Keld Simonsen.

Collating symbols
-----------------

Numbers, punctuation, and additional symbols can be treated in a
variety of ways when producing ordered lists. It may be a requirement
to allow for different ways of treating them if the software is to be
used in different applicaton domains. For example, a space between
characters is ignored for some applications but observed for others.
If the space is ignored, the resulting list would be

    Daniels
    Da Silva
    Dauxois

However, if the space is not ignored, the resulting list would be

    Da Silva
    Daniels
    Dauxois

Collating Danish characters
---------------------------

There are many levels of complication for collation of Danish as
defined in the official collating standard DS 377. For example, on the
telephone level, Mc is the same as Mac, numbers are spelled out, and
certain words like ``the'' are ignored or moved to the end. Another
level is the phonetic level - soundex, which is a little less
complicated. A third level is transcripted characters, as the
librarians use when they see a greek alpha and order that as a normal
``a''.

Collating Arabic characters
---------------------------

Arabic is a single-case language, so the problems of collating
uppercase and lowercase characters do not occur. The following
guidelines apply to the Arabic collating sequence:

    -     The Arabic connecting character, the ``tatweel'' has no
	  significance in a word and should be excluded during
	  collation.

    -     Words are first sorted in code order with the Arabic vowels
	  characters excluded.

    -     Groups of words having the same consonants are then sorted
	  in code order including the vowel characters.

Collating ideographic characters
--------------------------------

Collating ideographic characters is more complex than collating Latin
characters. The following three different methods of collating are
used:

    -     By radicals

	  Radicals are the root forms of a character that give the
	  character its basic meaning. The radical collating sequence
	  sorts according to the radicals that make up the character.
	  If there is more than one character with the same radical,
	  then these similar characters are further sorted by the
	  number of strokes that make up the character.

    -     By number of strokes

	  Characters are sorted by the number of strokes that make up
	  the character. If more than one character has the same
	  number of strokes, these characters are further sorted by
	  radicals.

    -     By phonetic sequence

	  Characters are sorted according to the sequence in which
	  they appear in a phonetic alphabet. In this phonetic
	  alphabet, the characters are organized according to their
	  romanized (western) spelling.

Locale and application specific locale sensitive functions
----------------------------------------------------------

If common locale sensitive functions are used, a locale sensitive
function should accommodate all locales by switching locale
information data. For example, the function ``strcoll()'' should be
able to compare two strings from all locales. If it could be done by
switching collation table for each locale and application combination,
we would have no problem. As shown in the examples above, however,
multi-path algorithm is used for some locales and other locale and
application combinations need different processing to collate two
strings. Locale and application specific locale sensitive functions
are the means to give clients locale and application sensitive
processing.