Subject: Re: (TC304.190) Full-text searching: don't keep it simple and stupid!

From: [email-masked] (Keld J|rn Simonsen)
Date: Fri, 17 Jun 1994 07:03:02 +0200

[email-masked] writes:

> Subject  : Full-text search: don't keep it simple and stupid
> 
> >Keld, my company (which produces a full text search product) is
> >attempting to establish character classes for various European
> >languages.  For most such languages, our users prefer that we
> >ignore case and accents.
> >
> >However, Danish seems to have some exceptions to this.  An 'O'
> >with a slash is treated as a separate letter.  Are there others?
> >For example, would users be upset if a search for "angstrom"
> >ignored the ring, or conversely, would they be upset if a search
> >with the ring did NOT find ones without (and vice-versa)?
> >What is normal practice in Denmark?
> 
> Keld answered, legitimately and correctly:
> 
> >In Denmark, the letters O WITH STROKE, AE and A WITH RING are genuine
> >letters and people would be very upset if it is not handled as such.
> 
> Now I think for French (and perhaps German and other languages too), the answer
> is unfortunately not as simple.

I agree with Alain, that a number of parameters should be available,
so different searches (for example with regards to precision) are
possible. 

The point in my above comment was that a cultural requirement is also
needed as a parameter, and that is not listed in Alain's model.
Or maybe you could say it is implicitely included, as the comparison
is done on a sorting algoritm - which may be cultural dependent,
as per the different national POSIX locales available.

Keld