Subject: N2970 - REVISED Draft SC22 Response on Metrics of JTC 1 Accomplishments - PLENARY AGENDA ITEM

From: William Rinehuls <[email-masked]>
Date: Wed, 4 Aug 1999 13:28:01 -0400 (EDT)

____________________ beginning of title page __________________________
ISO/IEC JTC 1/SC22
Programming languages, their environments and system software interfaces
Secretariat:  U.S.A.  (ANSI)

ISO/IEC JTC 1/SC22
N2970

SUBJECT:
REVISED Draft Response to ISO/IEC JTC 1 on Request for Metrics of JTC 1
Accomplishments - PLENARY AGENDA ITEM

DATE ASSIGNED:
1999-08-03

SOURCE:
Secretariat, ISO/IEC JTC 1/SC22

BACKWARD POINTER:
N/A

DOCUMENT TYPE:
Other

PROJECT NUMBER:
N/A

STATUS:
Discussion of this document and final approval of an SC22 response will be
an agenda item for the September 1999 JTC 1/SC22 Plenary.

ACTION IDENTIFIER:
FYI

DUE DATE:
N/A

DISTRIBUTION:
Text

CROSS REFERENCE:
N2885
Supersedes N2967

DISTRIBUTION FORM:
Def

Address reply to:
ISO/IEC JTC 1/SC22 Secretariat
William C. Rinehuls
8457 Rushing Creek Court
Springfield, VA 22153 USA
Telephone:  +1 (703) 912-9680
Fax:  +1 (703) 912-2973
email:  [email-masked]

_________________ end of title page; beginning of document ____________

                             DRAFT RESPONSE

Ms. Lisa Rajchel
Secretariat, ISO/IEC JTC 1
American National Standards Institute
11 West 42nd Street
New York, NY 10036

SUBJECT:  JTC 1 Request for Metrics of JTC 1 Accomplishments

Dear Lisa:

Reference is made to Rio Resolution 24 in which ISO/IEC JTC 1 requests
its Subcommittees to provide contributions detailing methods for measuring
the effectiveness of JTC 1 accomplishments.

General Comment.

JTC 1/SC22 notes that press releases are not a very useful measurement
of the significance or usefulness of an International standard or its
implementation.  While the number of press releases gives a hint of how
much work ISO/IEC JTC 1 is accomplishing over a certain period of time,
it does not measure the quality or effectiveness of these accomplishments.
To be a significant measurement tool, the number of press releases should
be matched against the media impact set forth in the objectives of the
Business Plan.

It is suggested that the objectives laid out in the Business Plan be
measurable to the largest extent practicable.  Few measures can be applied
otherwise.  

Additional Metrics

a.  A number of JTC 1/SC22 projects result in standards which undergo
    conformity assessment such as "compiler validation".  This assessment 
    is made to ensure that products developed which claim compliance with
    International standards developed within JTC 1/SC22 are, in fact,
    compliant.

    The records of the results of product validation are publicly
    maintained and these records may be a useful metric in determining the
    usefulness of the associated International standard.
 
b.  Provided the Business Plan assigns some corresponding objectives, the
    following can be measured:

    1) Growth of the market covered by the standard 
    2) Growth of the market share relying on the standard
    3) Timeliness of completion of the standard compared to its planned
       completion date
    4) Number of visitors to the JTC 1 web page interested in this
       standard

JTC 1/SC22 offers these suggestion as possible additional measurements of
the effectiveness of JTC 1 accomplishments.
 
Sincerely

WILLIAM C. RINEHULS
Secretariat, JTC 1/SC22

___________________ end of SC22 N2970 __________________________________