Subject: Apparent slow production of standards in SC24 (and SC22)

From: Chris Cartledge <[email-masked]>
Date: Mon, 29 Nov 1993 17:21:44 +0100

Colleagues,

Apologies for being so slow to give further information, but I have now
got the paper on which the original information was based.  This is
ISO/TECHNICAL BOARD 399 dated September 93 (page 205 onwards), titled ISO 
PROJECT MANGEMENT.  It is a large document mostly consisting of tables of 
statistics for Technical Committees and Sub-Committees.

The statistics are calculated as follows:

The time for production of document is calculated as follows:

the mean of 

   the number of standards produced in a "1 year" snapshot (supposedly
   93-01 to 93-12 but since this document was produced on 1993-09-15, I fail
   to understand this bit of information)

   and the average rate of production of standards (taking the life time of
   the committee). 

is divided into the current number in progress to give an estimate of the 
time taken. 

This would be OK if the work of a committee were constant in terms of
number of work items over its lifetime (and deciding on an accurate
committee lifespan - not easy for SC24).  It also depends on the snapshot
being a good reflection of committee output.  In general, neither of 
these, is reasonable. 

In the case of SC24, it is claimed that we take 5.95 years to produce 
an IS from DIS; 5.56 years to produce a DIS from CD and 3.57 to produce a 
CD from NWI.  This suggests that the figure used for the lifetime of SC24 
approaches infinity (is about 800 years!) so that the 34 standards we have
produced over the life of SC24 have almost no weight and the rate of 
production of standards depends solely on the 5 produced in the review 
period.

This is not as bad as it might sound - in the crazy world of ISO
statistics, the equivalent figures for SC22 are 800, 466.67 and 3066.67
respectively.  As far as I can tell, these suggest that SC22 has been
working for 599 years.  Now I know standards take a long time but even
FORTRAN progresses quicker than this... 

These statistics are both misleading and inaccurate - they effectively 
constitute a libel on the performance of committees - ISO must withdraw 
them forthwith.  TCs are being asked to confirm the figures before March 
1994 - JTC1 must dispute them.  

The intention that under-performing committees (and JTC1/SC22 is one)
should be selected for further study is ludicrous.  ISO need to radically
rethink their performance indicators. They need to take into account the
agreed targets for standards procuction and committee's success in meeting
those targets. 

I will be drafting a letter along these lines as Chair of BSI IST/31,
Computer Graphics to BSI management.  Jean will be copying the relevant
pages to National Bodies.  You should ensure that similar comments go to
your respective member bodies. 

I think from this that there is some evidence that the SC24 practice of 
listing work items where an editor has yet to be appointed should stop.  
Under the circumstances, no believable timescales for completion can be 
given.  Such timescales are likely to be important in a reasonable set of 
performance indicators. 

Regards

Chris