Subject: (i18n.179) plan 9 and 10646

From: [email-masked] (Glenn Adams)
Date: Wed, 23 Dec 92 00:38:53 EST

>From: [email-masked]
   >
   >firstly, we can answer what FEFF is. it is not a character as such
   >(in fact, it and FFEF are defined as never being characters).

Actually, FEFF *is* a character.  It is called ZERO-WIDTH NO-BREAK SPACE. 
It also may be used as a byte order mark.  The only codepoints of 10646BMP
defined as *not a character* are FFFE and FFFF.  [FFEF in Andrew's message
should be read as FFFE, the byte swapped form of FEFF, known both as
ZWNBSP and BOM.]

I append below the text of the Unicode 1.0.1 technical report which deals
with this a little more.

   B. Byte Order Mark
   U+FEFF       ZERO WIDTH NO-BREAK SPACE

   In addition to the meaning of BYTE ORDER MARK, as defined in Volume 1 of 
   the Unicode standard, the code value U+FEFF may now also be used as ZERO 
   WIDTH NO-BREAK SPACE (ZWNBSP). For convenience in discussion, it can 
   also be referred to by this name (which is the ISO 10646/Unicode 1.1 
   name for U+FEFF).

   ZWNBSP behaves like a U+00A0 NO-BREAK SPACE in that it indicates the 
   absence of word boundaries; however, ZWNBSP has no width. For example, 
   this character can be inserted after the fourth character in the text 
   "base+delta" to indicate that there should be no line break between the 
   "e" and the "+" (for more information, see Volume 2, pp. 6-7).

Glenn Adams