Subject: Document SC22 N2244

From: "william c. rinehuls" <[email-masked]>
Date: Mon, 19 Aug 1996 12:54:38 -0400 (EDT)

ISO/IEC JTC 1/SC22
Programming languages, their environments and system software interfaces
Secretariat:  U.S.A.  (ANSI)

ISO/IEC JTC 1/SC22
N2244

August 1996

TITLE:              United Kingdom Contribution to SC22 Concerning
                    SC22 N2111

SOURCE:             Secretariat, ISO/IEC JTC 1/SC22

WORK ITEM:          N/A

STATUS:             N/A

CROSS REFERENCE:    SC22 N2111

DOCUMENT TYPE:      United Kingdom Contribution

ACTION:             To SC22 Member Bodies for information.

                    THIS DOCUMENT WILL BE DISCUSSED UNDER AGENDA ITEM
                    11.4 AT THE SEPTEMBER 1996 JTC 1/SC22 PLENARY.

Address reply to:
ISO IEC JTC 1/SC22 Secretariat
William C. Rinehuls
8457 Rushing Creek Court
Springfield, VA 22153  USA
Tel:  +1 (703) 912-9680
Fax:  +1 (703) 912-2973
email:  [email-masked]

___________________________________________________________________________
From: Jean Stride <[email-masked]>

Subject: UK contribution to SC22 concerning N2111 (JTC1 N3826)

UK contribution to SC22 concerning N2111 (JTC1 N3826)

N2111 (JTC1 N3826) is the "recommended request for action 
plans and information from SCs and SGFS on implementation of
JTC1 policies on conformity assessment and interoperability".
This UK contribution to SC22's consideration of this document 
confines itself mainly to language standards, though a 
contribution from a UK expert relating to PCTE is appended as an 
Annex.  We believe that responses relating to Posix, PCTE and 
internationalization would come best from the working groups 
concerned in the first instance (though of course IST/5 reserves 
the right to comment).

In considering these matters it should be noted that language 
standards relate to two different kinds of conforming product:  
programs, and language processors such as compilers.  Conformity 
requirements on programs are aimed at portability:  ensuring that 
a conforming program will run, subject to resource limitations, on 
any conforming processor and deliver equivalent results (note:  
not necessarily identical results).  Interoperability is not an 
issue.  Conformity requirements on processors are also aimed 
primarily at portability:  ensuring that a conforming processor 
will, subject to resource limitations, accept any conforming 
program and process it correctly.  Interoperability is again not 
an issue;  the only other issues apart from portability that may 
arise fall loosely under the categories "usability" or "safety", 
e.g. requiring that exceptions be detected and reported.

With that in mind it is possible to give some answers to the 
request for information in section 3.1 of N2111/N3826.

(a) What is the practice and experience with regard to 
conformity to standards and interoperability of IT systems?

Conformity of programs can, from the user's point of view, most 
readily be tested and confirmed by the processor being used.  
SC22 guidelines recommend that a conforming processor should be 
required to have a mode of operation in which use of non-standard 
features by a program is rejected or at least flagged.  (Usually, 
conforming processors are allowed to define and accept 
non-standard features provided these do not conflict with the 
standard.)  Whether or not this is a conformity requirement, many 
processors do offer that capability.

Conformity of processors is normally tested and confirmed through 
use of validation suites.  However, for any language, there is an 
infinity of possible conforming programs of arbitrary complexity, 
so designing an effective validation suite is a non-trivial task 
and applying one to test a processor is a lengthy procedure needing 
substantial resources.  There is considerable literature on the 
subject, but the provision of validation suites has not been 
regarded as a suitable matter for standardisation.

SC22 has, however, produced two Technical Reports giving guidelines 
on specifying conformity rules in language standards (TR 10034) and 
on developing test methods (TR 9547).  SC22 WGs working on language 
standards are expected to make use of these.

Interoperability, as already noted, does not feature in language 
standards.  Interoperability between languages is something which 
has long been recognised as a need, but the great majority of the 
language communities concerned with the language standards have no 
interest in it:  their concern is for their language and that 
language alone.  Hence interoperability has always been given very 
low priority in language standard committees, and indeed usually 
ignored altogether.  

The Ada 95 Standard ISO/IEC 8652:1995 is an exception in that it 
makes a positive attempt to provide a level of interoperability with 
other languages. In its normative Annex B, Interface to Other 
Languages, it defines general import and export facilities for both 
data and specifying calling conventions, and it provides specific 
packages, including datatype declarations, for interfacing with C, 
Cobol and Fortran.

In addition, an Ada/SQL interface is in draft (DIS 12227), a project 
has recently started to allow Fortran programs to access C libraries, 
and the Modula-2 working group is considering something similar for 
that language though an NP has yet to be submitted.  More generally 
SC22 has recently published two standards, on language independent 
datatypes (ISO/IEC 11404) and on language independent procedure 
calling (ISO/IEC 13886), expressly as "enabling standards" to support 
interoperability between languages.  Hence the material exists to 
develop interoperability, for those that wish to use it.

(b)  To what extent is there a need to differentiate between 
conformity assessment and interoperability assessment?

Totally.  As already noted, conformity can and should address many 
other issues besides interoperability, and in some cases need not 
address it at all.  Furthermore, even when it is addressed, conformity 
may not guarantee interoperability, whether unintentionally (e.g. 
through faulty or inadequate conformity rules) or by design (e.g. 
through allowing options which in fact restrict interoperability by 
confining it to particular compatible environments).  Trying to lump 
the two together is not just unhelpful, it is positively misleading 
and confusing.

(c)  Are reference implementations of standards used?  If so, 
     how does their use relate to conformity and/or 
     interoperability assessment?

The UK regards the use of reference implementations in this area to 
be inappropriate as far as standards are concerned.  Some providers 
of validation suites may use "model implementations" as yardsticks 
for their suites, but that is all.

(d)  Are formal specifications of standards used?  If so, 
     how does their use relate to conformity and/or 
     interoperability assessment?

Most language standards use a syntactic metalanguage to define the 
syntax of a conforming program, "syntax" here meaning context-free 
syntax.  (Context dependencies, for example what is often called 
"static semantics" can be defined using a two-level metalanguage, but 
in practice are usually handled in a different way, usually employing 
natural-language definitions.)  The metalanguage used for a 
particular language is usually one inherited from the original 
language designer, which has become accepted in the language 
community concerned before the standards process began, and hence is 
not standard across languages, though a UK-originated standard 
metalanguage (ISO/IEC 14977, forthcoming) has recently been 
approved by fast-track.

Language processors commonly use syntax analysers that accept the 
relevant syntactic metalanguage.  Stand-alone syntax analysers are 
also available for various languages.

SC22 is currently producing standards for formal specification 
languages (VDM-SL, CD13817, and Z Notation, CD 13568) the first of 
which is already being used for another language standard (Modula-2, 
DIS 10514).  VDM-SL and Z can be used to express semantics, on the basis 
of a concrete syntax defined in a metalanguage such as ISO/IEC 14977.
Ultimately, formal specifications written in these languages will be 
able to be used for conformity assessment.

(e)  What methodologies and tools (including formal techniques) 
     already exist and are in use for assessment of conformity and 
     interoperability?

There is nothing to add to what has already been described.  As noted 
above, SC22 is not directly involved in assessment of conformity.

(f)  What is the standardization status of each of these 
     assessment methodologies and tools?

Again, see above.

(g)  To what extent and in what manner is there a need for 
     conformity assessment to be complemented by 
     interoperability assessment?

Very little, if the priorities of the language communities are any 
guide.  On the other hand, it is clear that there is a minority need 
which the majority interests are frustrating.  The trouble is that 
it is very hard to assess the size or importance of this minority need.

(h)  What, if any, additional standardization needed for 
     interoperability assessment?

The new standards on language independent datatypes and language 
independent procedure calling need language bindings for each language 
they are to be used with.  Production of such bindings are currently 
given very low priority in language standard committees, for the same 
reasons as explained above.

(i)  To what extent and in what manner is it possible to rely 
     upon and to recognise, transpose or reference existing 
     material from sources outside the SC/SGFS, e.g. publicly 
     available specifications for testing methods or test suites?

In theory it would be possible.  The UK does not believe that in 
this area there would be any benefit in practice in so doing.

It is possible, however, that the two SC22 Technical Reports 
referred to above (guidelines on specifying conformity rules in 
language standards and on developing test methods) might, with 
suitable adaptation, interpretation and "lateral thinking", be 
of help to other standards groups.

(j)  Are there areas which are important for current and 
     future market needs, for which conformity assessment
     and/or interoperability assessment methodologies need to 
     be developed and/or standardized, and if so which are the 
     high priority ones?

The UK believes that conformity assessment methodologies for 
VDM-SL and Z form such an area.

ANNEX

UK expert contribution on application of SC22 N2111 
(JTC1 N3826) to PCTE.

PCTE (ISO/IEC 13719) is a public tool interface for software 
engineering tools, based on a distributed network of workstations.  
It is defined as an abstract (i.e. programming-language-independent) 
specification plus dependent language bindings, e.g. for Ada and C.  
Various kinds of conformity arise:

 (1) conformity of implementation (to abstract specification plus a 
     binding);
 (2) conformity of tools (similarly);
 (3) conformity of a language binding to the abstract specification.

(3) is of interest only to standards groups producing new bindings, 
and is not considered further here.  (1) and (2) are analogous to 
conformity of language processors and of programs, respectively, 
but there are important differences.

The PCTE definition is modular and various levels of conformity of 
implementation are defined.  There is no definition of conformity of 
tools, as the variety of tools is too wide.

Interworking of tools is a primary design aim of PCTE, but again the 
variety of tools is too wide for a uniform approach to assessment.  
Interoperability of PCTE implementations is an important issue, 
covering (in order of increasing difficulty) portability of tools, 
data transfer and full tool interoperability (which requires a common 
view of the semantics of the data), and interworking of different 
implementations on the workstations (of the same or different types) 
in a single installation (heterogeneous distribution).

(a) What is the practice and experience with regard to conformity to 
    standards and interoperability of IT systems?

Conformity of implementations is the subject of a European Commission 
project under the CTS/5 programme, which will establish a conformity 
testing service.  This will ensure portability of tools across 
validated implementations, within the usual resource and modular 
implementation constraints.  There are no known plans for conformity 
assessment of tools.

Data transfer and full tool interoperability is the subject of a 
separate standardization exercise in conjunction with SC7/WG11 and 
CDIF.  Heterogeneous distribution is at present beyond the state of 
the art.

(b)  To what extent is there a need to differentiate between conformity 
     assessment and interoperability assessment?

Conformity assessment of implementations implies portability of tools, 
as noted above, but other aspects of interoperability are distinct.

(c)  Are reference implementations of standards used?  If so, 
     how does their use relate to conformity and/or 
     interoperability assessment?

There are no reference implementations of PCTE; this is seen as 
inappropriate for conformity assessment of implementations, though 
a possible approach to conformity assessment of tools.

(d)  Are formal specifications of standards used?  If so, 
     how does their use relate to conformity and/or 
     interoperability assessment?

Unfortunately there is no formal definition of the semantics of PCTE 
(though one exists of a predecessor).

(e)  What methodologies and tools (including formal techniques) 
     already exist and are in use for assessment of conformity and 
     interoperability?

See answer to (a).

(f)  What is the standardization status of each of these 
     assessment methodologies and tools?

It is intended that the results of CTS/5 - PCTE be presented to an 
appropriate standardization body (probably SC22/WG22).

(g)  To what extent and in what manner is there a need for 
     conformity assessment to be complemented by 
     interoperability assessment?

It might be useful but is probably too expensive to be practical.

(h)  What, if any, additional standardization needed for 
     interoperability assessment?

Standardization of schemas for data transfer and full tool 
interoperability (see (a)), based on standardization of the PCTE 
Data Description Language and of CDIF language and subject areas.

(i)  To what extent and in what manner is it possible to rely 
     upon and to recognise, transpose or reference existing 
     material from sources outside the SC/SGFS, e.g. publicly 
     available specifications for testing methods or test suites?

The CTS/5 - PCTE validation tests are originally based on the test 
suites of PCTE implementors, but are expanded and transposed into 
a general framework.

(j)  Are there areas which are important for current and 
     future market needs, for which conformity assessment
     and/or interoperability assessment methodologies need to 
     be developed and/or standardized, and if so which are the 
     high priority ones?

No comment.

_________________________end of document SC22 N2244 ____________________