Subject: Re: (i18n 74) Re: paper presented to WG11 (#20)

From: Erik M. van der Poel <[email-masked]>
Date: Tue, 19 Feb 91 14:55:36 +0900

> 	char *str = "s\<o/>ndag";
>  [...]
> 	3. Compile the \< token into an otherwise unused character
> 	   code and use that at run-time to trigger "on the fly"
> 	   substitution in all STANDARD routines operating on strings,
> 	   e.g. strcpy and printf.
> 
> Kim F. Storm

We should try to resist the temptation of attaching special meanings
to "otherwise unused" codes. One of the versions of the cpp program
used the code 0xfe for something special. This caused core dumps when
used with Japanese characters. (Why have *another* special code? Why
not just use `\'?) [Of course, if the combination `\<' causes a core
dump, we're still up the creek...]

It is of course easy to modify e.g. strcpy and printf to handle the
new escape sequences, but frequently the programmer needs to process
strings within his/her own program, rather than within a "standard"
routine. Simple operations like counting characters become rather
complex when escape sequences are present. So, in this sense, it would
be easier to use a fixed-width, stateless code like 10646's 4-byte
form. E.g.:

	wchar_t	*ws = L"s\<o/>ndag";

Even when the programmer insists on using `char' instead of `wchar_t',
we need to make sure that strings read in from a file and strings
compiled into a program are encoded consistently. This could be done
by having a compiler that compiles strings into, say, 10646, and
having files in 10646 too. Or we could leave the strings as they are
and store files in a <o/>-like format, but I don't think this is
Keld's intention.

Of course, interpreting the strings at compile-time will hard-code
info from a particular charmap, but then that is one of the reasons
for having a universal coded character set. :-)

Erik M. van der Poel                                      [email-masked]
Software Research Associates, Inc., Tokyo, Japan     TEL +81-3-3234-2692