Subject: Re: (TC304.190) Full-text searching: don't keep it simple and stupi

From: [email-masked]
Date: 17 Jun 94 15:19:00 -0500

----------
The Danish or other cultural requirements are totally included in the model I am
developing.  The algorithm is multi-level and will work on a LOCALE (with
further syntax to care about multi-script properties and some extra features),
which, for ISO/IEC 10646 level 1 conformance, is POSIX-syntax conformant (the
extra script statements can be processed as comments in this case).

And the model also has tailorability features which allow to change the
definition of say, <AE> for Danish or for French/English, alternatively.

So in the message I sent, this was of course implicit, but I did not copy the
full text of the draft.

Alain

Message original:
==============================================================================
         A:
            RNET ([email-masked], [email-masked], [email-masked],),
            RNET ([email-masked], [email-masked]), ALB
        CC: RNET ([email-masked], [email-masked], [email-masked])
        De: RNET ([email-masked])
     Objet: Re: (TC304.190) Full-text searching: don't keep it simple and stupi
      Date: ven 17 jui 94
     Heure: 05:10 TU
      Type: Mail
 Livraison: Reguliere
==============================================================================
[email-masked] writes:

> Subject  : Full-text search: don't keep it simple and stupid
>
> >Keld, my company (which produces a full text search product) is
> >attempting to establish character classes for various European
> >languages.  For most such languages, our users prefer that we
> >ignore case and accents.
> >
> >However, Danish seems to have some exceptions to this.  An 'O'
> >with a slash is treated as a separate letter.  Are there others?
> >For example, would users be upset if a search for "angstrom"
> >ignored the ring, or conversely, would they be upset if a search
> >with the ring did NOT find ones without (and vice-versa)?
> >What is normal practice in Denmark?
>
> Keld answered, legitimately and correctly:
>
> >In Denmark, the letters O WITH STROKE, AE and A WITH RING are genuine
> >letters and people would be very upset if it is not handled as such.
>
> Now I think for French (and perhaps German and other languages too), the answer
> is unfortunately not as simple.

I agree with Alain, that a number of parameters should be available,
so different searches (for example with regards to precision) are
possible.

The point in my above comment was that a cultural requirement is also
needed as a parameter, and that is not listed in Alain's model.
Or maybe you could say it is implicitely included, as the comparison
is done on a sorting algoritm - which may be cultural dependent,
as per the different national POSIX locales available.

Keld