Re: localization, internationalization and Caml

From: skaller (skaller@maxtal.com.au)
Date: Sun Oct 17 1999 - 13:27:09 MET DST


Date: Sun, 17 Oct 1999 21:27:09 +1000
From: skaller <skaller@maxtal.com.au>
To: Francis Dupont <Francis.Dupont@inria.fr>
Subject: Re: localization, internationalization and Caml

Francis Dupont wrote:
>
> In your previous mail you wrote:
>
> The current 'support' for 8 bit characters in ocaml should be
> deprecated immediately. It is an extremely bad thing to have, since
> Latin-1 et al are archaic 8 bit standards incompatible with the
> international standard for ISO10646 communication, namely
> the UTF-8 encoding.
>
> => there is a rather strong opposition against UTF-8 in France
> because it is not a natural encoding (ie. if ASCII maps to ASCII
> it is not the case for ISO 8859-* characters, imagine a new UTF-X
> encoding maps ASCII to strange things and you'd be able to understand
> our concern).

        I do understand the concern, but the decision on
the International Standards has been made. The transition
for ISO 8859-x clients will involve some pain. Better to
start going through the pain now :-(

> Yes, I know Latin-1 is useful now for French.
>
> => it is more than useful, Latin-1 (soon ISO IS 8859-15) is necessary
> if you need really readable texts in French.

        No, what you mean is that with _current technology_
there is plenty of support for 8 bit characters, using code pages,
so that Latin-1 is well supported.

        For example, there are a lot of text editors that
accept 8 bit characters, and even permit switching code pages.
There are almost none that work with ISO10646 or unicode,
let alone accept UTF-8 encoding. (Yudit is the only one I know of).

> The way forward may well be to provide an input filter to convert
> Latin-1 (or any other encoding) to UTF8, and have ocaml process that.
>
> => my problem is the output of the filter will be no more readable when
> I've put too much French in the program (in comments for instance).

        You will have no problem with the right tools: the difference
will be transparent. Of course, you will need the right tools.
For example, you will need a browser like Internet Explorer 5,
which processes UTF-8 encoding correctly.

        I agree that this is a problem, but supporting
Latin-1, or any other archaic standard, is not going
to help move forward. It is bad enough that most vendors
only support Unicode, which is a small, almost filled,
16 bit subset of the full 31 bit ISO-10646 Standard.
 
> This requires almost no changes to the compiler: the design should
> open the set of characters acceptable in identifiers, probably
> to some subset of the set recommended in one of the ISO10646 related
> documents; the other change required is to accept \uXXXX and \UXXXXXXXX
> escapes in strings. String processing functions should generally
> continue to be 8 bit [per octet]: full internationalisation of client
> string handling functions is a very complex, non-trivial, task]
>
> => I believe internationalization should not be done by countries
> where English is the only used language: this is at least awkward...

        I believe people with international concerns can work
together no matter what their native language. Some English
speakers may be concerned, some, like me, are somewhat
embarrased to be non-fluent in _any_ other language.
[I speak a smattering of high school German]

        However, Australia, where I live, has migrants
from all over the world and support for many languages
is an important issue here. Particularly Asian languages.
And ISO-8859-x is not much help there :-)

-- 
John Skaller, mailto:skaller@maxtal.com.au
1/10 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
downloads: http://www.triode.net.au/~skaller



This archive was generated by hypermail 2b29 : Sun Jan 02 2000 - 11:58:27 MET