From rinehuls@access.digex.net Mon May 18 20:20:01 1998 Received: from access1.digex.net (qlrhmEbBUV1EY@access1.digex.net [205.197.245.192]) by dkuug.dk (8.6.12/8.6.12) with ESMTP id UAA12986 for ; Mon, 18 May 1998 20:19:14 +0200 Received: from localhost (rinehuls@localhost) by access1.digex.net (8.8.4/8.8.4) with SMTP id OAA24755 for ; Mon, 18 May 1998 14:19:09 -0400 (EDT) Date: Mon, 18 May 1998 14:19:08 -0400 (EDT) From: "william c. rinehuls" To: sc22docs@dkuug.dk Subject: SC22 N2719 - Voting Summary for FCD 14651 - Method for Comparing Character Strings and Description of the Common Template Tailorable Ordering Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Transfer-Encoding: QUOTED-PRINTABLE _________________ beginning of title page ____________________________ ISO/IEC JTC 1/SC22 Programming languages, their environments and system software interfaces Secretariat: U.S.A. (ANSI) ISO/IEC JTC 1/SC22 N2719 TITLE: Summary of Voting on FCD Approval of FCD 14651 - Information technology - International String Ordering and Comparison - Method for Comparing Character Strings and Description of the Common Template Tailorable Ordering DATE ASSIGNED: 1998-05-18 SOURCE: Secretariat, ISO/IEC JTC 1/SC22 BACKWARD POINTER: N/A DOCUMENT TYPE: Summary of Voting PROJECT NUMBER: JTC 1.22.30.02.02 STATUS: WG20 is requested to prepare a Disposition of Comments Report and a recommendation on the further processing of the FCD. ACTION IDENTIFIER: FYI to SC22 Member Bodies ACT to WG20 DUE DATE: N/A DISTRIBUTION: Text CROSS REFERENCE: SC22 N2607 DISTRIBUTION FORM: Def Address reply to: ISO/IEC JTC 1/SC22 Secretariat William C. Rinehuls 8457 Rushing Creek Court Springfield, VA 22153 USA Telephone: +1 (703) 912-9680 Fax: +1 (703) 912-2973 email: rinehuls@access.digex.net ______________ end of title page; beginning of overall summary ________ SUMMARY OF VOTING ON Letter Ballot Reference No: SC22 N2607 Circulated by: JTC 1/SC22 Circulation Date: 1997-12-12 Closing Date: 1998-04-30 Closing Date Extended to: 1998-05-15 (at the reqwest of Australia) SUBJECT: FCD Approval for FCD 14651 - Information technology - International String Ordering and Comparison - Method for Comparing Character Strings and Description of the Common Template Tailorable Ordering ---------------------------------------------------------------------- The following responses have been received on the subject of approval: "P" Members supporting approval without comments 10 "P" Members supporting approval with comments 2 "P" Members not supporting approval 5 "P" Members abstaining 3 "P" Members not voting 2 "O" Members supporting approval without comments 1 "P" Members not supporting approval 1 --------------------------------------------------------------------- Secretariat Action: WG20 is requested to prepare a Disposition of Comments Report and make a recommendation on the further processing of the FCD. The comments accompanying the affirmative votes from Canada and Ireland, and the negative votes from Denmark, Japan, Netherlands, Sweden, United Kingdom and the USA are attached. The comment accompanying the abstention vote from Australia was: "Not enough time to consider this proposal due to the late arrival of the document." The comment accompanying the abstention vote from Austria was: "Lack of expert resources." The comment accompanying the abstention vote from Germany was "There is no national rapporteur."=20 Secretariat comment: The document has been available to all SC22 Member Bodies either electronically or in paper form since December 1997. ___________ end of overall summary; beginning of detail summary _______ ISO/IEC JTC1/SC22 LETTER BALLOT SUMMARY =20 PROJECT NO: JTC 1.22.30.02.02 SUBJECT: =20 FCD Approval for FCD 14651 - Information technology - International String Ordering and Comparison - Method for Comparing Character Strings and Description of the Common Template Tailorable Ordering =20 Reference Document No: N2607 Ballot Document No: N2607 Circulation Date: 12-12-1997 =20 Closing Date: 1998-04-30=20 Closing Date Extended to 1998-05-15 (at the request of Australia) = =20 =20 Circulated To: SC22 P, O, L Circulated By: Secretariat ------------------------------------------------------------------------ SUMMARY OF VOTING AND COMMENTS RECEIVED Approve Disapprove Abstain Comments Not Voting 'P' Members Australia ( ) ( ) (X) (X) ( ) Austria ( ) ( ) (X) (X) ( ) Belgium (X) ( ) ( ) ( ) ( ) Brazil (X) ( ) ( ) ( ) ( ) =20 Canada (X) ( ) ( ) (X) ( ) China (X) ( ) ( ) ( ) ( ) Czech Republic (X) ( ) ( ) ( ) ( ) Denmark ( ) (X) ( ) (X) ( ) Egypt ( ) ( ) ( ) ( ) (X) Finland (X) ( ) ( ) ( ) ( ) France (X) ( ) ( ) ( ) ( ) Germany ( ) ( ) (X) (X) ( ) Ireland (X) ( ) ( ) (X) ( ) Japan ( ) (X) ( ) (X) ( ) Netherlands ( ) (X) ( ) (X) ( ) Norway (X) ( ) ( ) ( ) ( ) Romania (X) ( ) ( ) ( ) ( ) Russian Federation (X) ( ) ( ) ( ) ( ) Slovenia ( ) ( ) ( ) ( ) (X) UK ( ) (X) ( ) (X) ( ) Ukraine (X) ( ) ( ) ( ) ( ) USA ( ) (X) ( ) (X) ( ) 'O' Members Voting Korea Republic (X) ( ) ( ) ( ) ( ) Sweden ( ) (X) ( ) (X) ( ) ___________ end of detail summary; beginnin of Canada comments _______ Document Number FCD14651 (22N2607) Canada APPROVES WITH THE FOLLOWING COMMENTS: Canada votes YES on FCD ballot on 14651, with the following comments: 1. On the condition that normalizing characters to either precomposed=20 sequences or decomposed sequences for characters which have double=20 coding (called canonical equivalences according to the Unicode=20 standard) is made mandatory, Canada is of opinion that combining marks should have a second level equivalent to the precomposed forms' second level (ex. precomposed character =E9 [E ACUTE] has a weight for the acute accent at the second level. The character COMBINING ACUTE should have the same second level weight). This comment is to solve the problem raised by the Unicode consortium about canonical equivalences. However we insist that this scheme technically works only if=20 normalization of character sequences is mandated, a direction to which the Unicode consortium now agreed and added/specified in consequence=20 in its own algorithm. 2. The syntactic mistakes of the table should be fixed and passed through an existing commercial locale parser such as the one of AIX for final checking. 3. Reestablish the COLL_WEIGHT_MAX=3D4 after the LC_COLLATE statement. We= =20 believe that this parameter should be clarified in 14652 as being a=20 declaration of the maximum number of levels used in the table. This is= =20 very useful for not having to parse the table twice, just to determine= =20 storage allocation before processing the table. A comment to that=20 effect will also be done for the ballot on 14652. 4. The annex I of the previous CD should be reintroduced, however with=20 nonrelevant parameters removed from the example as per our previous=20 Canadian comment. ________ end of Canada comments; beginning of Denmark comments _______ Vote from Danish Standards Association on FCD 14651 - Sorting - SC22 N2607. The Danish member body vote is "no" with comments. If the technical comments can be solved satisfactorily, the Danish "no" vote can be changed to "yes", unless other significant changes be made to the standard in an unsatisfactory way. An example of an unsatisfactory change to the standard is the technical annex in SC22 N2608 on an example user interface, which is out of scope of the standard, and does not explain the relation to the APIs and specifications in the standard, which is not trivial. The user interface assumes that a number of tailored sorting specifications be available at runtime, presumably in compiled binary form, and that selecting mechanisms be available. Technical comments. 1. On page 120, an "order_end" and an "END LC_COLLATE" statement is missing. 2. A number of scripts should have a specified ordering. 3. It needs to be explained that it is not necessary to convert into binary sorting strings to do the comparison. The comparison can be done character by character on the fly, and this may often resultin a difference within the first few characters, thus avoiding costly complete conversion of both the strings for comparison. This should at least be explained for COMPCAR. 4. It needs to be explained that the binary strings are locale dependent and therefore not adequate for storing, in databases and the like. A note on this should be available for CHARABIN. 5. As a number of national specifications have been developed using specific naming of weights different from the names in the standard it is necessary to support this existing practise to add these weights via the "symbol-equivalence" statement of 14652. The weights in question include: collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol collating-symbol 6. It is wrong to define weights two times. Weights etc. on page 44-51 needs to be deleted. Editorial comments: 1. The standard should use established weight names such as the ones in ISO/IEC 9945-2 annex G and many POSIX locales, and not reinvent the wheel. 2. The annexes should use normal ISO numbering, and normative annexes should be using numbers "A" and "B" etc. 3. a reference should be made to ISO/IEC 9899 programming language C, possibly in the normative references. This concludes the DS ballot text on SC22 N2607. ----------- end of Denmark comments; beginning of Ireland comments ____ VOTE ON COMMITTEE DRAFT: FCD 14651 Date of Circulation: 12-12-1997 Reference Number: Closing Date for Voting: 04-30-1998 ISO/IEC JTC 1/SC22 N2607 ------------------------------------------------------------------------- P-members have an obligation to vote ------------------------------------------------------------------------- CD FCD 14651 TITLE: Information technology - International String Ordering and Comparison - Method for Comparing Character Strings and Description of the Common Template Tailorable Ordering -------------------------------------------------------------------------- Please put an "X" on the appropriate line(s) Approval of the draft: _____ as presented _____ with comments attached ****** Ireland approves with the following comments _____ general ****** Ireland believes that this standards is important and timely. However, we see a need for harmonization with work done in this area by the Unicode Consortium where possible and practicable, in order to increase the likelihood that standard will actually be used. _____ technical ****** Normalization of characters to either recomposed sequences or decomposed sequences should be made mandatory. ****** Errors in the specification table (for instance, in Cyrillic and elsewhere) should be corrected. ****** A version of informative Annex I of the previous CD should be put back into the standard. _____ editorial ****** The entire English text needs to be revised for language. _____ Disapproval of the draft for reasons attached _____ Acceptance of these reasons and appropriate changes in the text will change our vote to approval _____ Abstention (for reasons below) National Standards Authority of Ireland P-member voting Date: 30 April 1998 Signature (if mailed) Dr. I.J. Cowan ______________ end of Ireland comments; beginning of Japan comments ____ From=20gotoushi@gm.soft.hitachi.co.jp Thu Apr 2 12:39:23 1998 Subject: Japan's vote on FCD 14651 (N2607) SC 22 N 2607: FCD 14651 Method for Comparing Character Strings and the Description of Common Template Tailorable Ordering. (X) Disapproved National Body: Japan Date: 1998-02-17 Signature: KATSUHIKO KAKEHI -------------------------------------------------------------- The National Body of Japan disapproves FCD 14651 for the reasons below.=20 If the comments are satisfactorily resolved, Japan will change its vote to approval.=20 J.1) Considering the high dependency of this FCD on ISO/IEC 14652,=20 Japan requests to wait and synchronize the review of FCD 14651 until=20 FCD 14652 is approved. J.2) The concept of the common template is not consistent in this draft=20 as follows: a) on one hand, the draft defines that the common template is not=20 a default ordering, in the following text: a.1) the text in 7.1 Data specification --=20 =09Although the data in table 1 defines a specific ordering,=20 =09it should not be considered as an internationally acceptable=20 =09default ordering but only as a starting point for the=20 =09tailoring process prescribed in the next paragraph. a.2) the content of the common template in Annex 1 is designed so as to= =20 invoke an intentional mistake if it is parsed without tailoring -- defining some toggle switches. b) on the other hand, the draft describes as if the common template were a default ordering, in the following places: b.1) the text at the beginning of Clause 5 Requirements -- =09Hence the first conformance level (conformance level A) is=20 =09limited to a comparison API using the fixed common template=20 =09specified in annex 1 (table 1) with the result precision fixed;=20 b.2) the descriptions of the parameter "table" in the procedures COMPCAR and CARABIN --=20 =09 - table: the name of the ordering table to be used to=20 =09make the comparison. This is only required for conformance =09level C. If this parameter is not provided, the common template=20 =09will be assumed. ... Japan's proposals for the problem: c.1) keep the a.1 and a.2 as it is (except for some detail), =20 c.2) change b.1 as to delete the first conformance level c.3) change b.2 as to invoke the implementation dependent table =09if the parameter "table" is not provided. J.3) The relation to ISO/IEC 14652 is not well described.=20 The last sentence in the first bullet of the Clause 1 Scope should be=20 changed from=20 =09This interface uses transformation tables derived from either the=20 =09common template provided or from its tailoring. to the two sentences =09This interface uses transformation tables which conforms to =09the specifications for LC_COLLATE in ISO/IEC 14652. =09The table shall be derived from either the common template provided or from its tailoring as defined in this standard. J.4) The term "prehandling" should be applied to a process handling=20 character strings to be compared as defined in Clause 3. Japan's proposal: 1) Change the title of Subclause 5.1 from "prehandling phase" to "preprocessing phase" or "preparation phase" 2) Change the title of Subclause 5.1.1 from "prehandling of the symbolic=20 table data" to "preparation of the symbolic table data". 3) Change the first sentence of the fourth paragraph of 5.1.1 from =09As prehandling can be done on many different tables in a given=20 =09application environment, each one shall be identified by a name in =09this environment to=20 =09As preparation may be done to generate more than one table in a given application environment, each one shall be identified by a name in this environment J.5) In splitting the functionality of COMPCAR into COMPBIN and CARABIN, the concept of "precision" is introduced in COMBIN in this draft. But adding the parameter "precision" also in CARABIN will produce a=20 comparable bit string shorter than the one for full precision and those shorter strings are preferable in many situations. J.6) The definition=20 =09collating symbol a symbol used to specify weights assigned=20 =09to a character in a symbolic fashion rather than absolutely is not understandable. What word does the word "absolutely" correspond? J.7) The definitions for "first order token", "second order token",=20 "third order token", and "fourth order token" should be merged to =09n-th order token: an absolute number used as a comparison element =09corresponding to the n-th column of a line in the table. =09In the typical case of Latin characters, there are=20 =09four levels of tokens and the first three levels of token=20 =09describes a character (note that some=20 =09characters, such as ligatures, may lead to more than one token=20 =09for a character at one given level) while the fourth order token=20 =09describes a special character, or specifying the position of the=20 =09special character represented in the original string;=20 =09tokens of the fourth order level are always in pairs, the first=20 =09token being a position,=20 =09the second one being a weight for the character represented. J.8) The symbols and abbreviations introduced in Clause 4 seems to appear only in Annex 1 and the second, fourth and fifth paragraphs are not=20 appropriate for "Symbols and abbreviations". It is recommended to put=20 the material into subclause 5.4 or Annex 1. J.9) The sentence in Clause 4 =09If a character outside of the standard repertoire of ISO/IEC 10646=20 =09is to be used in tailored ordering tables, it is recommended that=20 =09the code-independent symbol identifying this character use the form=20 =09 for documentary purposes indicating its nonstandard=20 =09nature is not understandable for many readers and apt to be upset by future ISO/IEC 10646 revisions. We propose to use the form =20 where '!U' means "not 10646". J.10) Subclause 5.1 is not in the scope of the standard. It should be=20 moved to a new informative annex. J.11) The first sentence of the second paragraph of subclause 5.2.1: "consists in" -> "consists of" J.12) 5.2.1.1, Input parameters: The description =09 - string1: referenced character string that is to be compared=20 =09to the reference string (string2) =09 - string2: reference character string that is to be compared=20 =09to the referenced string (string1) should be changed to=20 =09 - string1: referenced character string that is to be compared=20 =09to the reference string (string2) =09 - string2: reference character string -- string serves as a base =09reference for comparison J.13) 5.2.1.1, Output Parameter: The name of the parameter "return" should be changed to "result". J.14) 5.2.1.2: The changes similar to J.12 and J.13 should be done. J.15) 6. Conformance: The definition of the conformance level here is different from the one in= =20 Clause 5. It should be aligned. J.16) 7.1, Data Specification, 2nd para. The sentence=20 =09Although the data in table 1 defines a specific ordering, it=20 =09should not be considered as an internationally acceptable default=20 =09ordering but only as a starting point for the tailoring process=20 =09prescribed in the next paragraph. should be changed to =09The data in table 1 does not define a specific ordering. It =09should not be considered as an internationally acceptable default=20 =09ordering but only as a starting point for the tailoring process=20 =09prescribed in the next paragraph. J.17) Annex 2=20 It should be changed to informative with changing the first sentence from =09The benchmark shall be tested in using the default table of=20 =09annex 1, adapted in defining the following toggles: PRIMNBSP,=20 =09ACCENTSONLATIN, PRIMIN to=20 =09The benchmark should be tested when the default table of=20 =09annex 1 is tailored by defining the following toggles: PRIMNBSP,=20 =09ACCENTSONLATIN, PRIMIN.=20 J.18) (not on FCD 14651) Japan supports the decision at Quebec meeting=20 to eliminate the informative annex I in the first CD draft because the annex misleads readers to think tailoring is a very complex process that=20 almost always needs a visual interface. _____________ end of Japan comments; beginning of Netherlands comments ___ The NNI votes NO on this FCD 14651 for the reasons detailed hereafter: -1- The NNI is of the opinion that this document shows insufficient maturity to warrant an _F_CD. -2- The structure, contents and wording of this document is considered to be below standards. This is considered unacceptable. -3- The editor has ignored the comments on the CD from the NNI.=20 This is equally unacceptable. The NNI will change its vote into a yes vote only when all three issues mentioned above and detailed hereafter will have been resolved satisfactorily. =20 re 1: Given the number of comments on the previous version of this document, the NNI considers the stability of content and form of this document insufficient to warrant its circulation as a FINAL CD. Given the comments detailed hereafter, the NNI requests that at least one other CD be circulated before another FCD can be attempted. re 2: The algorithm described has been partially detailed in the definition clauses and in _in_formative annexes. The normative body of the document does not contain a sufficient description of the algorithm. The annexes have been formulated as examples only. Although it will be not easy to construct an appropriate abstract definition of the algorithm, the document is considered incomplete without this. The document contains English text that clearly has been translated from French. Many remains from the French text are still visible. These remains make the document difficult to understand. re 3: In our comments on the CD the NNI stated that: "The text of this standard leaves much to be desired regarding precision of definition, clarity of presentation ... The NNI cannot give detailed comments here, nor offer replacement text as doing so would require rewriting more than half of the document for which we have no resources available. The NNI already gave some directions with its vote on the CD-registratio= n, but found alas almost no improvement in this CD" The Disposition of Comments sc22n2466 completely ignored our comment by stating " This is purely ITTF matter ...." However, sharpness of definition and clarity of presentation is not an ITTF matter at all. It is the main task of a responsible editor. The NNI feels very disappointed that apparently the editor of this document is unable to write clearly (see below), is unfamiliar with programming=20 language terminology and is unfamiliar with the mathematics behind=20 programming languages but is even more so unable to read and=20 understand the comments (not only ours, as can be seen clearly from the DOC). As we felt these shortcomings in definitions and presentation to=20 be clearly visible to the intelligent reader, we refrained from enumerating them. This seems however not to have worked, so this time we will give a _few_ examples from the clauses 3 and 5 of the document. Please note that there is no use in making local repairs to the document. The document should be totally checked and revised. Comments will be given in-line with the text using question marks or on separate lines preceded by a hash mark and a number. Before starting with our comments on clauses 3 and 5 it should be noted that 7 out of the 28 notions defined in clause 3 have not been used in the document at all. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3Dlet's start her= e=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 3 Definitions API Application Program Interface defined as a=20 standardized application process describing the=20 specifications of different procedures ... #001# How can an API be defined as a STANDARDIZED APPLICATION PROCESS? It seems the editor is unfamiliar with the meaning of API and/or=20 APPLICATION. #002# What is a process here; what is the difference between process=20 and procedure? character string=20 a series of characters concatenated in=20 logical sequence #003# It is unclear what makes a logical sequence; does the logical sequence of characters in a string differ from their physical sequence? #004# This definition uses concatenation as if it was defined with the following signature: =20 Concatenation : Char x Char x Char x Char ... -> CharString Thus is VERY unusual. collating symbol=20 a symbol used to specify weights assigned to a=20 character in a symbolic fashion rather than absolutely #005# Absolute contrasts relative; symbolic contrasts real. #006# In SC22 terminology "to assign" means that variables will be involved; this however, seems not to be the case. #007# What does it mean to "assign something in a symbolic fashion"? #008# What does it mean to "assign something absolutely"? collating element=20 the smallest entity used to determine the logical ordering of strings. #009# What is a logical ordering of strings? how does it differ from their physical ordering? It normally consists of either a single character or two or more characters collating as a single entity #010# This seems to mean that two strings can be ordered by one=20 single character? #011# The "entity" at the beginning of this sentence seems to be of a different kind than the "entity" at the end of the sentence. #012# What is the "not-normal" situation? concatenation =20 logical operation which consists in adding an element at the end of a string to consider the result as a new string, longer than the first one. #013# What is a "logical" operation? Does it involve the logical connectives 'and', 'or', 'not'? #014# In what sense is an "operation" different from a "process"? #015# Concatenation has here been defined as of type: Concatenation : X-string x X -> X-String Instantiating X by Char gives a definition different from the earlier one. equivalence =20 in a comparison between two character strings, a form of partial equality between the two strings. After decomposition of the two strings in different levels according to the ordering table ... #016# It is unclear that the ordering table describes a decomposition #017# The ordering table is called elsewhere in this document:=20 - common template ordering table - Common Template order description - Common Template table - common template - International Common Template Table .... The typical case ... #018# It is believed that examples should not be used in a definition clause field for the purposes of this International Standard, a single character string or any other data type which may be ordered alone(?) or in conjunction with other fields of a record, each field of a record being compared to the same field of another record; #019# Is "a single character string" the same as=20 "a string containing one character" or=20 "a single string containing characters"? #020# It is unclear how such a string can be ordered "alone" #021# It is unclear what a "record" is; see later comment #022# Precisely the "same" field? Perhaps corresponding? #023# A data type (in SC22 terminology) can not be ordered; values of a data type may perhaps be ordered. =20 in case of absolute equality of two equivalent fields, other fields of the records .... #024# Yet another definition of equivalent?=20 #025# What is absolute equality? #026# It is believed that is not proper to distribute the description of the algorithm throughout the document. first order token=20 an absolute number used as a comparison element, obtained out of tables for the first level that describes a character; ... #028# What kind of element is this? #029# Absolute number; does this have the mathematical meaning of |x| ? #030# It is unclear to what part of the sentence "that" refers to; the number, the element or the level? #031# Does a level "describe" a character? fourth order token=20 an absolute number ....., obtained either out of, or specifying the position ....; tokens of the fourth order level are always in pairs the first token being a position, the second one being a weight ... #032# The first sub-sentence states that a fourth order token consists=20 of "one" absolute number; the second sub-sentence states that=20 a fourth order token is a pair. #033# The first sub-sentence states that a fourth order token is an=20 "either-or" the second sub-sentence states that it is a pair. level =20 Whenever used ....level .."key level" .."precision level" ... conformance level... normally ... #034# Why such a difficult terminology? Why not just use precision level and conformance level throughout? #035# What is the abnormal situation? #036# It is again believed that it is not appropriate to detail the=20 algorithm here. numeric relative value=20 the relative .. in its final numeric and processable form #037# Are there non-final forms? #038# Are there unprocessable forms? ordering key =20 a series of bits, the numerical value of which determines its order; to a character a ..... characters #040# The second sub-sentence seems unconnected to the first one and is again discussing the algorithm posthandling =20 a process in which ... internally ... straightforwa= rd comparisons ... #041# What is a process? #042# What is meant by internally? Internal to what? #043# Why are the comparisons done according to this standard "straightforward"? record =20 the exhaustive structured set of fields that=20 form a monolithic block in a file .... #044# What is exhaustive here? #045# In mathematics sets are unstructured. What is the structuring criterion? #046# What is meant by monolithic? #047# What is a block? #048# What has a file to do with all this? reference string ............ a base reference .. #049# Are there also non-base references? string =20 a series of individual elements which form a whole, when they are concatenated ... #050# It seems that this definition disallows character strings consisting of a single character transformation =20 A(a?)n operation performed prior to comparison or ordering ... #051# What is the difference between comparison or ordering? This definition helps to have a more general discussion on the=20 terminology used in this standard. There are three verbs that are central to this standard: to order, to compare and to collate. Looking up the meanings of these verbs in this definition clause and= =20 in a dictionary does not bring much light in the question of what the= =20 precise definition used here is. Looking ahead in the document one finds that the API only allows for= =20 comparisons to be done.=20 The mechanism is such that first strings are mapped to unique values (isomorphic to integers; what is called an injection in mathematics). These values are then compared using the underlying ordering of=20 the set of integers. The fact that the strings can then be (re)ordered seems to be outside the scope of this standard. Collation has not defined here at all. Looking up its meaning in Webster's gives again 'to compare' and 'to order', but does not give any insight. 5.1 Prehandling phase ... This requirement shall be met for conformance levels greater than 1 #052# This requirement -> The requirement specified in this clause ... #053# Conformance requirements have been indicated by letters! It is recommended .. #054# This recommendation conflicts with "shall" in the first sentence. #055# But, this whole sentence seems misplaced here. The symbolic table, ... #056# No symbolic table has been provided in this document shall be provided in a numeric form=20 #057# What kind of numeric form? floating point numbers? =2E... each element of the matrix being a numerical token indicating=20 one or more relative weights. #058# Numerical token has not been defined #059# Relative weight has not been defined #060# Can "one" token indicate "more than one" weights? #061# What is the meaning of "to indicate a weight"? As a balance? #062# Generally speaking, it seems somewhat of an overspecification to require that the table be built in a particular(matrix) form (n *m array). As long as the outcome of the comparison of two strings conforms to the=20 requirements of this standard, the internal form used for data seems not significant. However(,?) the values used shall respect the order specified in the=20 symbolic table data. #063# No symbolic table can be found in this document. Conformance level C requires that the API shall allow to choose a table built at a preceding time #064# What is "a preceding time"? Earlier than prehandling?=20 It may be necessary to transform a field of a record=20 #065# Into what? =2E.. or to transform unstructured records #066# What are "unstructured" records? Please note the definition of record= s=20 in clause 3 =2E.. into structured fields=20 #067# Given the definitions of record and field in clause 3 this=20 definition is incomprehensible =2E.. before the actual process can begin. #068# Yet another process? The implementer is responsible for ensuring ... #068# The implementer of this standard? How can that be? This is a global operation #069# What is meant by global? #070# So an operation is a process? that may involve "exploding" records before ordering them #071# How does one explode an "exhaustive structured set" or=20 a "monolithic block"? #072# Are these records ordered, or are the strings ordered? Therefore, the prehandling phase ... #073# Why a conclusion here? Thus, prehandling is not part of the comparison operation API. #074# Why a conclusion here? We already knew that. The comparison API will not contain any default method related to prehandling. #075# What is a default method? #076# Isn't this derivable from the previous sentence? =2E..for allowing the use of this IS in higher layers of the application? #077# It seems inappropriate for this standard to state anything about the structuring of an application using this standard. The prehandling phase ... transform the actual coded characters used on input #078# In the previous section it was said that prehandling operated on records? =2E...Then the prehandling phase can correspond to the empty process #079# So a phase is equal to a process? Ideally, all control characters #080# Ideally =3D shall/recommended/optional ? #081# Elsewhere, control characters have been named control functions =2E. in case of absolute homography ... #082# Homography between ? and ? #083# Homography means "equal in writing"; what is absolute equal in writing? =2E.. in the common template #084# No common template can be found in this document 5.2.1 .... The general interface consists in (of?) three ... #085# Is there a specific interface too? #086# Several typos here #087# It is unclear why these procedures have been numbered #088# It is unclear how CARABIN can _produce_ a _PRE_fabricated bit string. #089# It is unclear whether there are bit strings that are=20 processable indirectly. The implementer may choose appropriate values for the application. #090# The implementer of this API chooses values for an application=20 he does not know anything about? #091# General remark: As far as understood, all three 'procedures' are=20 effectively functions (ie they have no side-effect). It is unclear why they are called procedures then. 5.2.1.1 ..... table .... If this parameter is not provided, the common template=20 will be assumed. #092# It is unclear how this should be implemented in a language=20 that requires all the parameters being present. COMPCAR process: This procedure shall be processed to give results equivalent to the following: This procedure shall be processed to give results equivalent to the following: #093# Two times the same sentence #094# Yet another use of equivalent #095# Process or procedure? Submit character strings string1 and string 2 and table 'table' to procdure CARABIN ... #096# CARABIN does not have such parameters. After having looked up the definitions, it is clear what has been=20 intended, why not describe it precisely? Execute procedure .... #097# Perhaps process the procedure? A minimum interface for conformance level A or B can very well(?) use=20 the standard C-language function model of ... #098# Why "minimum"? #099# The C-language has no notion of function model This limited interface ... #100# Is a limited interface the same as a minimum interface? COMPBIN process: =2E... coded character strings .. #101# Undefined notion =2E.. octet by octet ... #102# Unclear what octets have to do here 5.2.1.3. =2E...to a comparable bit strings #103# All bit strings are comparable, aren't they? =2E.. not to compile this table 'table' every time this function is called. #104# It is unclear what compilation model is used here.=20 Is this compilation the the same as the earlier mentioned prehandling? =20 ____________ end of Netherlands comments; beginning of Sweden comments ___ Swedish ballot comments on ISO/IEC FCD 14651.2 (SC22 N 2607) As an O-member Sweden submits following comments for consideration. If not the comments are not considered. Sweden may vote NO on the FDIS. MAJOR comments Each of the following points must be satisfactorily resolved, and implemented in the next draft, in order to turn a possible NO vote at FDIS level into a yes vote. All API requirements MUST be completely removed. It must be possible to follow the International String Ordering standard, or base national string ordering standards on it, WITHOUT requiring implementations to provide a particular API. Secondly, before standardising on a new API for this there should be considerable experience and consensus on how it should work. This is not yet achieved. In either case a string ordering API must be moved to another later standard, if standardised at all. All FILE FORMAT (14652 or other) requirements MUST be completely removed. It must be possible to follow the International String Ordering standard, or base national string ordering standards on it, WITHOUT requiring implementations to provide or use a particular file format. All other dependencies on ISO/IEC 14652 must also be removed, including the normative reference. Those wishing to follow 14651 should NOT be forced to also follow 14652. Combining non-spacing characters, as well as other canonical equivalences (in the Unicode 2.0 sense) in general, MUST be handled properly. I.e. character sub-sequences that are canonically equivalent according to Unicode 2.1 MUST be regarded as equivalent at all "levels" (in the 14651 sense) when compared as text (i.e. character) strings according to ISO/IEC 14651. The easiest way to do this is to do full canonical decomposition and combining marks canonical reordering according to Unicode 2.0. An appropriate table of (full) canonical decompositions is publicly available from the Unicode consortium, and should be a normative table in an annex of 14651. This must include the algorithmic decomposition of Hangul syllables into conjoining Hangul Jamo. (See also comment 9 below on Japanese.) Compatibility equivalent (in the Unicode 2.0 sense) sub-sequences must be regarded as different ONLY at the least significant "level" ("level 4"), NOT at any more significant level. E.g. level 3 should only handle "case" variants, no other kind of variation (like full-width/half-width) which is not based on orthographic needs, only on backward compatibility needs. An appropriate table of (full) compatibility decompositions is publicly available from the Unicode consortium, and should be a normative table in an annex of 14651. 14651 must cover all scripts and characters currently covered by Unicode 2.1 and 10646 including all amendments to date. The current draft misses many scripts. It is our understanding that the Unicode consortium can provide fuller tables, covering all scripts currently in 10646. Arabic: The suggestion (annex C, and annex 1 p. 94) that the Arabic presentation forms are stored in a different order than the nominal characters for Arabic must be removed (it is false). Thai and Lao: The standard should, however, take into account the "visual", rather than "logical", storage order for Thai and Lao characters, and require pre-processing to order them "logically" before key construction. Japanese: The half-width voiced marks must be considered combining with the preceding half-width Katakana character (to get correct collation). This affects the mapping from compatibility characters to nominal characters. Prolonged sound marks: These take on different weights depending on the preceding kana letter. Repeat/iteration marks: Again, our understanding is that these should take on different weights depending on the preceding kana letter.=20 Kanji digits (including "fraud-proof forms"), 0-9: These should be handled = like other digits. However, handling of ideographs for "ten", "hundred", etc., need not be done by defa= ult. Make certain "old" Katakana sort together with "modern" spellings (va, ve, = vi, vo). The standard should explicitly allow for "transliterative" collation. I.e. the collation is primarily done on transliterated strings (whether manually, "pronunciation field", semi-automatically, or automatically derived), and only to a lesser degree depend on the direct input/output strings (which should not be completely ignored). This would be useful for "pronunciation" collation often used for Korean and Japanese, or indeed even for "phonebook" collation. I.e., levels 1-3 would be levels 1-3 on the transliterated string, and levels 4-6 would be levels 1-3 on the original string, level 7 would be level 4 of the original string. Sequences of (base ten) digits, up to a settable length (n) which is at least five, in the same script must by default be pre-processed to include leading zeroes so that the length of the digit sequence is n digits long. By default, we limit this to natural numbers written as ordinary base ten numerals. A tailored version may do something more fancy. Each digit is given a weight that in some way corresponds to its (individual) decimal value, irrespective of script. The weight is such that all digits are collated after all alphabetical or ideographic characters). Periods and digits are to be collated after spaces (if spaces are not ignored). =20 We would then, without any tailoring, get the expected ordering of strings like these: 1. Intro 1.1 bbbb 2. dsfsdfsdf 2.1 fsdfds 2.2 sdfsfsd ... 2.11 dfsd 2.11.1 vxvxcv 3 vxcvxc 3.1 dfsf ... 10. cvsdf 11. sdfsda Many things are sequence numbered (and sub-sequence numbered) just like this. It would be a bad idea if they by default came out in the wrong order, and additional effort had to be made every time such strings needed to be properly ordered. 14651 must make ordering compromises ONLY if there is a need to compromise among existing nationally/linguistically acceptable orderings (transliteration aside). This means that for many scripts, one would not, if this principle were followed, need to specify any national variation. The normative reference to the programming language C must be removed. There must be no requirement that one must use C as the programming language. Many programming languages can be used and there is no reason to single out C, even it is used in an example (which should be deleted). Indeed, with the deletion of the API part, there is no need to refer to any programming language at all, even for examples. The notion of "posthandling" must be removed since no posthandling is, or need be, specified in this standard. All non-empty sequences of space characters (there are about 10-20 different character codes), paragraph separators, line separators, C0, and C1 "control characters" must be regarded as equivalent at all but the last "level", and regarded as either a single space or be ignored. Even if not in general ignored, they should be ignored for collation purposes at all but the last level if they occur in the very beginning of a string. (This can be handled by the pre-processing.) There is no normative mathematical description on how the comparison keys for strings are constructed from the (symbolic) weights for individual characters. =20 There is an informal description (annex C), but it is very incomplete, imprecise and based on some strange but weak implementation experience. The current draft is far from satisfactory in describing the string ordering and how it depends on the weight tables and other data. What is needed is a mathematically precise specification, so that we have a firm semantics for "multi-level" string collation. Map each (pre-processed) string to a value in R (the mathematical "real" numbers), in such a way that the values mapped to are ordered, in R, in the same way that the strings should be ordered. The standard should allow implementations to use any strictly monotonically increasing mapping from the specified values in R to other values in R that can be more compactly represented. The standard need not normatively specify any such mapping, but can give a few as examples (it is our understanding that the Unicode consortium can provide a few representations). The Swedish NB can help with the detailed specifications of the mapping from strings to R (using tabulated character weights, whether symbolic, as in the current annex 1, or direct), and example strictly monotonous mappings from R to R. Parsing strings into "lines"/"records" and "fields" (or any other structure) must be considered out of scope for this standard. If done, it would be done prior to, not included with, collation proper. The major comments above, and the minor comments below, strongly indicate the need to rewrite the entire text from scratch. We see no way in which minor changes here and there can correct the collection of errors, omissions, and misdirected requirements contained in the current text. Suggested table of contents: 1. Scope [see below] 2. Conformity [see below] 3. Normative references [only 10646] 4. Symbols and Definitions [see below] 5. Multi-level string ordering 5.1 Included pre-processing [canonical equiv., digit seq., etc.] 5.2 The levels 5.3 The weights 5.4 Symbolic weights 5.5 Basic key construction from weights [mathematical description] 5.6 Strictly monotonic mappings of keys [allowance, not details] 6. Tailoring of the string ordering [how it would work; what can be done] 7. Transliterative string ordering 8. Additional explanation about the collation tables [format of tables not normative except for the standard itself] Annex A (normative): Combining non-spacing marks. Annex B (normative): Canonical decomposition, and canonical reordering. Annex C (normative): Compatibility decomposition. [+special for half-width voiced marks] Annex D (normative): Thai and Lao reordering for collation. Annex E (normative): Multi-level character weights. Annex F (informative): Rationale for multi-level string collation. Annex G (informative): Example collations. [NOT normative] Annex H (informative): Example suitable monotonically increasing mappings. [NOT normative] Annex J (informative): Example collation tailorings. [at least one complete example] It is our understanding that the US NB may present other major comments, e.g. regarding such things as the principles by which the weight tables are constructed, that we too may support as major. Minor comments The comments in this section are recommendations for change, but do not one by one constitute reason for a no vote. Taken together, however, they strongly indicate the need to completely rewrite the entire text of 14651 from scratch. Since we do suggest a complete rewrite, there are no references to specific sentences in the comments below. Rather, it is a number of items for the editor to keep in mind when writing the new text. Change the title. Suggestion: "International string ordering _ Multi-level collation method and international compromise string order". Section 1: Please make two fairly detailed lists: one with items that are in scope for this standard, and one with items that are, or could be seen as, related, but that are out of scope. The first list (in scope) would include items such as=20 An internationally acceptable collation order for strings that include characters that are normally not used in a particular region. Compromises between (expected) national collation orders for strings with scripts used for many languages and/or conventions of collation. Certain pre-processing (internal to the collation) to obtain actual strings to compare. The pre-processing can reorder certain characters, and replace sub-strings by equivalent, but canonical, sub-strings. Proper handling of combining sequences of characters. Transliterative collation, where the transliteration itself is external to the collation. The second list (out of scope) would include items such as:=20 Application programming interface (API) for invoking string comparison or collation. File formats for specifying collation order to an actual implementation. Parsing of a string to obtain sub-strings to collate. Escape-sequence or control code interpretation. National variations from the compromise string ordering (national standards are expected to cover these). There should be an informative annex indicating at least one complete, medium complex example national tailoring (which of course is/are not normative even for the nation(s) taken as examples). All examples and side remarks must be made into "Note"s, to clarify their non-normative nature.= =20 (Since we suggest a complete rewrite, we see no point in listing the ones in the current text here, but the editor should keep this in mind when writing the new text.) Section 3: "API" is out of scope and should not occur in the definition list. Section 3: "character string"; strange definition, in what sense "logical"? Section 3: "concatenation"; strange definition, in what sense "logical"? Section 3: "equivalence"; the text there constitutes a lot of ranting about string comparison, but contains no definition of 'equivalence'. Section 3: "field"; this is out of scope for this standard. Section 3: "posthandling"; no such thing is specified, nor need be specified, by this standard. Section 3: "record"; this is out of scope for this standard. Section 3: "reference/referenced string"; bad terminology, please do not use it. Section 3: "procedure"; this is out of scope for this standard. Section 3: "transformation"; the text there constitutes a lot of ranting about string comparison, but contains no definition of 'transformation'. Section 5: OUT OF SCOPE, DELETE. See above. Section 6: Rewrite completely, current one refers only to out-of scope material. Other sections: see major comment 16. B.t.w.: We can do without the cryptic symbolic weight/script names (like "Hy" for Armenian, and "a8" for "a's weight" (having a very American pun)). ______ end of Sweden comments; beginning of UK comments _______________ From: Suzanne Davison % tonos collating-symbol % psili =2E.. tonos normally near the end of the sequence, not at the beginning collating-symbol collating-symbol collating-symbol collating-symbol % oxia collating-symbol collating-symbol collating-symbol % perispomeni collating-symbol collating-symbol % ypogegrammeni collating-symbol % prosgegrammeni collating-symbol % dialytika collating-symbol collating-symbol =2E.. Dialytika is normally the initial grouping, not secondary grouping. Some of the names above also reflect this inconsistency. =2E.. TONOS is normally near the end of the seqeunce, not at the beginning =2E.. DIALYTIKA normally the initial grouping, not secondary grouping % Accents Several of these characters are dual-purpose Latin/Greek combining accents. Could this lead to any complications in mixed Latin/Greek text? IGNORE;IGNORE;IGNORE; % ACUTE ACCENT IGNORE;IGNORE;IGNORE; % MODIFIER LETTER ACUTE ACCENT IGNORE;IGNORE;IGNORE; % COMBINING ACUTE ACCENT (Oxia) IGNORE;IGNORE;IGNORE; % GRAVE ACCENT IGNORE;IGNORE;IGNORE; % MODIFIER LETTER GRAVE ACCENT IGNORE;IGNORE;IGNORE; % COMBINING GRAVE ACCENT (Varia) IGNORE;IGNORE;IGNORE; % BREVE IGNORE;IGNORE;IGNORE; % COMBINING BREVE (Vrachy) IGNORE;IGNORE;IGNORE; % DIAERESIS IGNORE;IGNORE;IGNORE; % COMBINING DIAERESIS (Dialytika) IGNORE;IGNORE;IGNORE; % MIDDLE DOT IGNORE;IGNORE;IGNORE; % GREEK PSILI IGNORE;IGNORE;IGNORE; % COMBINING COMMA ABOVE (Psili) IGNORE;IGNORE;IGNORE; % CYRILLIC COMBINING PSILI PNEUMATA IGNORE;IGNORE;IGNORE; % GREEK PSILI AND VARIA IGNORE;IGNORE;IGNORE; % GREEK PSILI AND OXIA IGNORE;IGNORE;IGNORE; % GREEK PSILI AND PERISPOMENI IGNORE;IGNORE;IGNORE; % GREEK DASIA IGNORE;IGNORE;IGNORE; % COMBINING REVERSED COMMA ABOVE (Dasia) IGNORE;IGNORE;IGNORE; % CYRILLIC COMBINING DASIA PNEUMATA IGNORE;IGNORE;IGNORE; % GREEK DASIA AND VARIA IGNORE;IGNORE;IGNORE; % GREEK DASIA AND OXIA IGNORE;IGNORE;IGNORE; % GREEK DASIA AND PERISPOMENI IGNORE;IGNORE;IGNORE; % GREEK VARIA IGNORE;IGNORE;IGNORE; % GREEK OXIA IGNORE;IGNORE;IGNORE; % GREEK TONOS =2E.. In most accented characters below, TONOS follows PERISPOMENI. Here, just above, it appears out of order. IGNORE;IGNORE;IGNORE; % COMBINING VERTICAL LINE ABOVE (Tonos) IGNORE;IGNORE;IGNORE; % GREEK PERISPOMENI IGNORE;IGNORE;IGNORE; % GREEK PROSGEGRAMMENI IGNORE;IGNORE;IGNORE; % GREEK YPOGEGRAMMENI IGNORE;IGNORE;IGNORE; % GREEK DIALYTIKA AND VARIA IGNORE;IGNORE;IGNORE; % GREEK DIALYTIKA AND OXIA IGNORE;IGNORE;IGNORE; % GREEK DIALYTIKA TONOS In most accented characters below, DIALYTIKA AND TONOS follows DIALYTIKA AND PERISPOMENI. Here, just above, it appears out of order. IGNORE;IGNORE;IGNORE; % GREEK DIALYTIKA AND PERISPOMENI IGNORE;IGNORE;IGNORE; % MODIFIER LETTER TRIANGULAR COLON % order_start ;forward;forward;forward;forward,position % This section below, and accented letters listed under other vowels, appears to show the correct ordering. ;;;IGNORE % GREEK CAPITAL LETTER ALPHA ;;;IGNORE % GREEK SMALL LETTER ALPHA ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH VRACHY ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH VRACHY ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH MACRON ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH MACRON ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PSILI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PSILI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PSILI AND VARIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PSILI AND VARIA ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PSILI AND VARIA AND PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PSILI AND VARIA AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PSILI AND OXIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PSILI AND OXIA ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PSILI AND OXIA AND PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PSILI AND OXIA AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PSILI AND PERISPOMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PSILI AND PERISPOMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PSILI AND PERISPOMENI AND PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PSILI AND PERISPOMENI AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PSILI AND PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PSILI AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH DASIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH DASIA ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH DASIA AND VARIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH DASIA AND VARIA ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH DASIA AND VARIA AND PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH DASIA AND VARIA AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH DASIA AND OXIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH DASIA AND OXIA ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH DASIA AND OXIA AND PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH DASIA AND OXIA AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH DASIA AND PERISPOMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH DASIA AND PERISPOMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH DASIA AND PERISPOMENI AND PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH DASIA AND PERISPOMENI AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH DASIA AND PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH DASIA AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH VARIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH VARIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH VARIA AND YPOGEGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH OXIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH OXIA ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH OXIA AND YPOGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PERISPOMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH PERISPOMENI AND YPOGRAMMENI ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH TONOS ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH TONOS ;;;IGNORE % GREEK CAPITAL LETTER ALPHA WITH PROSGEGRAMMENI ;;;IGNORE % GREEK SMALL LETTER ALPHA WITH YPOGEGRAMM ENI =2E.. ____________ end of UK comments; beginning of USA comments _________ The US National Body votes to Disapprove SO/IEC FCD 14651, Information technology - International String Ordering and Comparison - Method for Comparing Character Strings and Description of the Common Template Tailorable Ordering. =20 =20 Comments: =20 The Conformance clause of 14651 "must not mandate: - more than 3 customizable levels - the use of the precise data or tailoring format specified - the use of the precise API specified" =20 =20 The International Common Template Table must incorporate all of the corrections for syntax errors and other problems reported by Ken Whistler to the Editor. These corrections are required in order for the Table to even be machine processed at all. =20 Furthermore, the U.S. strongly prefers that the content of the =20 International Common Template Table be inclusive of the *entire* content of ISO/IEC 10646-1 through Amendment 7 (Collection 30), rather than an arbitrary and otherwise unidentified subset of 10646. The best way to accomplish this is to replace the content of Annex 1 with the symbolic information in the symdump2.txt table provided by Ken Whistler to the Editor. This would have the additional advantag of aligning the content of the International Common Template Table with the default table in use by vendors which implement the Unicode Collation Algorithm. =20 The International Common Template Table must be published in machine-readable format to be usable for implementation. =20 An expert contribution to WG20 from the US is being prepared=20 for the WG20 meeting in June 1998. _________________ end of SC22 N2719 __________________________________ = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = = =20 = = = =20 = = = ! = = = =20 = = =20 [ Part 2: "Attached Text" ] The NNI votes NO on this FCD 14651 for the reasons detailed hereafter: -1- The NNI is of the opinion that this document shows insufficient maturity to warrant an _F_CD. -2- The structure, contents and wording of this document is considered to be below standards. This is considered unacceptable. -3- The editor has ignored the comments on the CD from the NNI.=20 This is equally unacceptable. The NNI will change its vote into a yes vote only when all three issues mentioned above and detailed hereafter will have been resolved satisfactori= ly. =20 re 1: Given the number of comments on the previous version of this document, the NNI considers the stability of content and form of this document insufficient to warrant its circulation as a FINAL CD. Given the comments detailed hereafter, the NNI requests that at least one other CD be circulated before another FCD can be attempted. re 2: The algorithm described has been partially detailed in the=20 definition clauses and in _in_formative annexes.=20 The normative body of the document does not contain a sufficient descriptio= n=20 of the algorithm. The annexes have been formulated as examples only. Although it will be not easy to construct an appropriate abstract definition of the algorithm, the document is considered incomplete without this. The document contains English text that clearly has been translated from French. Many remains from the French text are still visible. These remains make the document difficult to understand. re 3: In our comments on the CD the NNI stated that: "The text of this standard leaves much to be desired regarding precision of definition, clarity of presentation ... The NNI cannot give detailed comments here, nor offer replacement text as doing so would require rewriting more than half of the document for which we have no resources available. The NNI already gave some directions with its vote on the CD-registratio= n, but found alas almost no improvement in this CD" The Disposition of Comments sc22n2466 completely ignored our comment by stating " This is purely ITTF matter ...." However, sharpness of definition and clarity of presentation is not an ITTF matter at all. It is the main task of a responsible editor. The NNI feels very disappointed that apparently the editor of this document is unable to write clearly (see below), is unfamiliar with programming=20 language terminology and is unfamiliar with the mathematics behind=20 programming languages but is even more so unable to read and=20 understand the comments (not only ours, as can be seen clearly from the DOC= ). As we felt these shortcomings in definitions and presentation to=20 be clearly visible to the intelligent reader, we refrained from enumerating them. This seems however not to have worked, so this time we will give a _few_ examples from the clauses 3 and 5 of the document. Please note that there is no use in making local repairs to the document. The document should be totally checked and revised. Comments will be given in-line with the text using question marks or on separate lines preceded by a hash mark and a number. Before starting with our comments on clauses 3 and 5 it should be noted that 7 out of the 28 notions defined in clause 3 have not been used in the document at all. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3Dlet's start her= e=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 3 Definitions API Application Program Interface defined as a=20 standardized application process describing the=20 specifications of different procedures ... #001# How can an API be defined as a STANDARDIZED APPLICATION PROCESS? It seems the editor is unfamiliar with the meaning of API and/or=20 APPLICATION. #002# What is a process here; what is the difference between process=20 and procedure? character string=20 a series of characters concatenated in=20 logical sequence #003# It is unclear what makes a logical sequence; does the logical sequenc= e=20 of characters in a string differ from their physical sequence? #004# This definition uses concatenation as if it was defined with the following signature: =20 Concatenation : Char x Char x Char x Char ... -> CharString Thus is VERY unusual. collating symbol=20 a symbol used to specify weights assigned to a=20 character in a symbolic fashion rather than absolut= ely #005# Absolute contrasts relative; symbolic contrasts real. #006# In SC22 terminology "to assign" means that variables will be involved= ;=20 this however, seems not to be the case. #007# What does it mean to "assign something in a symbolic fashion"? #008# What does it mean to "assign something absolutely"? collating element=20 the smallest entity used to determine the logical ordering of strings. #009# What is a logical ordering of strings? how does it differ from their physical ordering? It normally consists of either a single character or two or more characters collating as a single ent= ity #010# This seems to mean that two strings can be ordered by one=20 single character? #011# The "entity" at the beginning of this sentence seems to be of a diffe= rent kind than the "entity" at the end of the sentence. #012# What is the "not-normal" situation? concatenation =20 logical operation which consists in adding an eleme= nt=20 at the end of a string to consider the result as a = new string, longer than the first one. #013# What is a "logical" operation? Does it involve the logical connective= s 'and', 'or', 'not'? #014# In what sense is an "operation" different from a "process"? #015# Concatenation has here been defined as of type: Concatenation : X-string x X -> X-String Instantiating X by Char gives a definition different from the earlier= one. equivalence =20 in a comparison between two character strings, a form of partial equality between the two strings. After decomposition of the two strings in different levels according to the ordering table ... #016# It is unclear that the ordering table describes a decomposition #017# The ordering table is called elsewhere in this document:=20 - common template ordering table - Common Template order description - Common Template table - common template - International Common Template Table .... The typical case ... #018# It is believed that examples should not be used in a definition claus= e field for the purposes of this International Standard, a single character string or any other data type which may be ordered alone(?) or in conjunction with other fields of a record, each field of a reco= rd being compared to the same field of another record; #019# Is "a single character string" the same as=20 "a string containing one character" or=20 "a single string containing characters"? #020# It is unclear how such a string can be ordered "alone" #021# It is unclear what a "record" is; see later comment #022# Precisely the "same" field? Perhaps corresponding? #023# A data type (in SC22 terminology) can not be ordered; values of a dat= a type may perhaps be ordered. in case of absolute equality of two equivalent fiel= ds, other fields of the records .... #024# Yet another definition of equivalent?=20 #025# What is absolute equality? #026# It is believed that is not proper to distribute the description of the algorithm throughout the document. first order token=20 an absolute number used as a comparison element, obtained out of tables for the first level that describes a character; ... #028# What kind of element is this? #029# Absolute number; does this have the mathematical meaning of |x| ? #030# It is unclear to what part of the sentence "that" refers to; the number, the element or the level? #031# Does a level "describe" a character? fourth order token=20 an absolute number ....., obtained either out of, or specifying the position ....; tokens of the fourth order level are always in pair= s the first token being a position, the second one=20 being a weight ... #032# The first sub-sentence states that a fourth order token consists=20 of "one" absolute number; the second sub-sentence states that=20 a fourth order token is a pair. #033# The first sub-sentence states that a fourth order token is an=20 "either-or" the second sub-sentence states that it is a pair. level =20 Whenever used ....level .."key level" .."precision level" ... conformance level... normally ... #034# Why such a difficult terminology? Why not just use precision level and conformance level throughout? #035# What is the abnormal situation? #036# It is again believed that it is not appropriate to detail the=20 algorithm here. numeric relative value=20 the relative .. in its final numeric and processabl= e form #037# Are there non-final forms? #038# Are there unprocessable forms? ordering key =20 a series of bits, the numerical value of which determines its order; to a character a ..... characters #040# The second sub-sentence seems unconnected to the first one and is aga= in=20 discussing the algorithm posthandling =20 a process in which ... internally ... straightforwa= rd comparisons ... #041# What is a process? #042# What is meant by internally? Internal to what? #043# Why are the comparisons done according to this standard "straightforw= ard"? record =20 the exhaustive structured set of fields that=20 form a monolithic block in a file .... #044# What is exhaustive here? #045# In mathematics sets are unstructured. What is the structuring criteri= on? #046# What is meant by monolithic? #047# What is a block? #048# What has a file to do with all this? reference string ............ a base reference .. #049# Are there also non-base references? string =20 a series of individual elements which form a whole, when they are concatenated ... #050# It seems that this definition disallows character strings consisting= =20 of a single character transformation =20 A(a?)n operation performed prior to comparison or ordering ... #051# What is the difference between comparison or ordering? This definition helps to have a more general discussion on the=20 terminology used in this standard. There are three verbs that are central to this standard: to order, to compare and to collate. Looking up the meanings of these verbs in this definition clause and= =20 in a dictionary does not bring much light in the question of what the= =20 precise definition used here is. Looking ahead in the document one finds that the API only allows for= =20 comparisons to be done.=20 The mechanism is such that first strings are mapped to unique values (isomorphic to integers; what is called an injection in mathematics). These values are then compared using the underlying ordering of=20 the set of integers. The fact that the strings can then be (re)ordered seems to be outside the scope of this standard. Collation has not defined here at all. Looking up its meaning in Webs= ter's gives again 'to compare' and 'to order', but does not give any insigh= t. 5.1 Prehandling phase ... This requirement shall be met for conformance levels greater than 1 #052# This requirement -> The requirement specified in this clause ... #053# Conformance requirements have been indicated by letters! It is recommended .. #054# This recommendation conflicts with "shall" in the first sentence. #055# But, this whole sentence seems misplaced here. The symbolic table, ... #056# No symbolic table has been provided in this document shall be provided in a numeric form=20 #057# What kind of numeric form? floating point numbers? =2E... each element of the matrix being a numerical token indicating=20 one or more relative weights. #058# Numerical token has not been defined #059# Relative weight has not been defined #060# Can "one" token indicate "more than one" weights? #061# What is the meaning of "to indicate a weight"? As a balance? #062# Generally speaking, it seems somewhat of an overspecification to requ= ire that the table be built in a particular(matrix) form (n *m array). As long as the outcome of the comparison of two strings conforms to t= he requirements of this standard, the internal form used for data seems not significant. However(,?) the values used shall respect the order specified in the=20 symbolic table data. #063# No symbolic table can be found in this document. Conformance level C requires that the API shall allow to choose a table built at a preceding time #064# What is "a preceding time"? Earlier than prehandling?=20 It may be necessary to transform a field of a record=20 #065# Into what? =2E.. or to transform unstructured records #066# What are "unstructured" records? Please note the definition of record= s=20 in clause 3 =2E.. into structured fields=20 #067# Given the definitions of record and field in clause 3 this=20 definition is incomprehensible =2E.. before the actual process can begin. #068# Yet another process? The implementer is responsible for ensuring ... #068# The implementer of this standard? How can that be? This is a global operation #069# What is meant by global? #070# So an operation is a process? that may involve "exploding" records before ordering them #071# How does one explode an "exhaustive structured set" or=20 a "monolithic block"? #072# Are these records ordered, or are the strings ordered? Therefore, the prehandling phase ... #073# Why a conclusion here? Thus, prehandling is not part of the comparison operation API. #074# Why a conclusion here? We already knew that. The comparison API will not contain any default method related to prehandli= ng. #075# What is a default method? #076# Isn't this derivable from the previous sentence? =2E..for allowing the use of this IS in higher layers of the application? #077# It seems inappropriate for this standard to state anything about the structuring of an application using this standard. The prehandling phase ... transform the actual coded characters used on inp= ut #078# In the previous section it was said that prehandling operated on reco= rds? =2E...Then the prehandling phase can correspond to the empty process #079# So a phase is equal to a process? Ideally, all control characters #080# Ideally =3D shall/recommended/optional ? #081# Elsewhere, control characters have been named control functions =2E. in case of absolute homography ... #082# Homography between ? and ? #083# Homography means "equal in writing"; what is absolute equal in writin= g? =2E.. in the common template #084# No common template can be found in this document 5.2.1 .... The general interface consists in (of?) three ... #085# Is there a specific interface too? #086# Several typos here #087# It is unclear why these procedures have been numbered #088# It is unclear how CARABIN can _produce_ a _PRE_fabricated bit string. #089# It is unclear whether there are bit strings that are=20 processable indirectly. The implementer may choose appropriate values for the application. #090# The implementer of this API chooses values for an application=20 he does not know anything about? #091# General remark: As far as understood, all three 'procedures' are=20 effectively functions (ie they have no side-effect). It is unclear why they are called procedures then. 5.2.1.1 ..... table .... If this parameter is not provided, the common template=20 will be assumed. #092# It is unclear how this should be implemented in a language=20 that requires all the parameters being present. COMPCAR process: This procedure shall be processed to give results equivalent to the followi= ng: This procedure shall be processed to give results equivalent to the followi= ng: #093# Two times the same sentence #094# Yet another use of equivalent #095# Process or procedure? Submit character strings string1 and string 2 and table 'table' to procdure CARABIN ... #096# CARABIN does not have such parameters. After having looked up the definitions, it is clear what has been=20 intended, why not describe it precisely? Execute procedure .... #097# Perhaps process the procedure? A minimum interface for conformance level A or B can very well(?) use=20 the standard C-language function model of ... #098# Why "minimum"? #099# The C-language has no notion of function model This limited interface ... #100# Is a limited interface the same as a minimum interface? COMPBIN process: =2E... coded character strings .. #101# Undefined notion =2E.. octet by octet ... #102# Unclear what octets have to do here 5.2.1.3. =2E...to a comparable bit strings #103# All bit strings are comparable, aren't they? =2E.. not to compile this table 'table' every time this function is called. #104# It is unclear what compilation model is used here.=20 Is this compilation the the same as the earlier mentioned prehandling= ? =20