Software Fault Tolerance : Achievement and Assessment Strategies

This volume summarizes the results obtained by the group working on softwarefault tolerance within the REQUEST (Reliability and Quality of European Software Technology) project of the ESPRIT programme of the European Communities. It should be read by anyone with a professional interest in safety-critical and fault-tolerant computing. A generic model is developed for evaluating the reliability of fault-tolerant software systems.Emphasis is put on identification of problem areas in the development and assessment of fault-tolerant software systems and in the components.Examples of crucial failures are those of diverse versions due to a common cause, or failures in the adjudicator which acts on outputs of diverse versions. The causes for common failures of versions are similarities in the solutions of specified problems. Methods were developed to determine similarity among versions by means of well-known software engineering methods. Concerning adjudicators, the influences of several factors on failure detection capability are discussed and guidelines are given for optimal design. A methodology is developed to determine dissimilarity on the level of diverse specifications. Cost-based support is given for deciding whether diversity should be used in a software system or a single program shouldbe enhanced by additional verification effort.

1 Introduction.- 2 Overview.- 2.1 The Concept of Software Fault-tolerance.- 2.2 Failure Dependence.- 2.2.1 The Problem of Failure Dependence.- 2.2.2 Reduction of Failure Dependence.- 2.2.2.1 Forced Diversity.- 2.2.2.2 Functional Diversity.- 2.2.3 Measurement of Failure Dependence.- 2.2.3.1 Measurement by Statistical Inference from Past Failure Data.- 2.2.3.2 Measurement by Static Analysis.- 2.2.3.3 Measurement by Dynamic Analysis.- 2.2.3.4 A Pattern Matching Approach.- 2.2.3.5 An Expert System Approach.- 2.2.3.6 Measurement of Functional Diversity.- 2.3 Evaluation of Reliability of Fault Tolerant Software.- 2.3.1 General Considerations.- 2.3.2 Model Application to Functionally Diverse Software.- 2.4 Adjudication Mechanisms.- 2.4.1 Voting Systems.- 2.4.2 Acceptance Tests.- 2.4.3 Location of Checkpoints.- 2.5 Conclusion.- References.- 3 Considerations on Software Diversity on the Basis of Experimental and Theoretical Work.- 3.1 The Different Failure Sets of a Two-fold Diverse System.- 3.2 Experimental Approach.- 3.3 Theoretical Approach.- 3.4 Additional Requirements.- 3.5 Comparison Between Single and Diverse Use of Programs.- 3.6 Conclusion.- References.- 4 The Impact of Forced Diversity on the Failure Behaviour of Multiversion Software.- 4.1 Introduction.- 4.2 Common Failure Behaviour of Forced and Unforced Diverse Systems w. r. t. the Voter Majority.- 4.2.1 Theoretical Results of Littlewood and Miller.- 4.2.2 Experimental Results of Kelly and Avizienis.- 4.3 Common Failure Behaviour of Forced and Unforced Diverse Systems w. r. t. the Voter Granularity.- 4.3.1 Theoretical Results.- 4.3.2 Experimental Results of PODS and STEM.- 4.4 Conclusion.- References.- 5 Functional Diversity.- 5.1 Introduction.- 5.2 Limitations of Normal Diversity.- 5.3 Description of Functional Diversity Methodology.- 5.4 Advantages of Functional with respect to Normal Diversity.- 5.5 Disadvantages of Functional Diversity.- 5.6 Application Fields.- 5.7 Choice of the Modelling Approach for Functional Diversity.- 5.8 Classical Semantic Approach.- 5.8.1 Operational Semantics.- 5.8.2 Denotational Semantics.- 5.9 Functional Semantics.- 5.10 Semantic Modelling of Functional Diversity.- 5.11 Functional Diversity Metrication.- 5.12 Definition of Functional Diversity Metrics.- 5.12.1 The EFF_WOR Metric.- 5.12.2 The IND_WOR and IND_AVE Metrics.- 5.12.3 The VER WOR and VER_AVE Metrics.- 5.12.4 The GLO_REL Metric.- 5.13 Classification of the Metrics.- 5.14 Reliability Analysis for Functionally Diverse Systems.- 5.15 Static Specification Analysis.- 5.16 Reliability Evaluation.- 5.16.1 One Version Reliability Evaluation.- 5.16.2 System Reliability Evaluation.- 5.17 Semantic Specification Language.- 5.17.1 Specification Language Characteristics for Functionally Diverse Systems.- 5.17.2 Guidelines for a Semantic Specification Language Definition.- 5.17.2.1 Declaration Block.- 5.17.2.2 Specification Body.- 5.17.3 Specification Structure.- 5.18 Semantic Specification Analysis Methodology.- 5.18.1 Static Specification Analysis.- 5.18.1.1 Diversity Degree Assessment.- 5.18.1.2 Reliability Evaluation.- References.- 6 Estimation of Failure Correlation in Diverse Software Systems with Dependent Components.- 6.1 Introduction.- 6.2 Evaluation of the Inaccuracy Resulting from the Independence Assumption.- 6.3 The Case of Available Failure Observations.- 6.4 The Case of No Available Failure Observations.- 6.5 Conclusion.- References.- 7 Measurement of Diversity Degree by Quantification of Dissimilarity in the Input Partition.- 7.1 Input Partition and Coverage Diversity.- 7.2 Partition Diversity during the Testing Phase.- 7.3 Conclusion.- References.- 8 Comparison of Mnemonics for Software Diversity Assessment.- 8.1 The Initial Prototype Investigation.- 8.1.1 Initial Tests and Results.- 8.1.2 Shortcomings of the Prototype Technique.- 8.1.2.1 Length of Programs.- 8.1.2.2 Suitability of Trial Data.- 8.1.2.3 Matching Algorithm.- 8.1.2.4 Programming Style.- 8.1.2.5 Lack of Automation.- 8.1.2.6 Assessment of Results.- 8.2 Enhancement of the Prototype.- 8.2.1 Improvements to Overcome Identified Shortcomings.- 8.2.1.1 Automation of Mnemonic Code File Generation.- 8.2.1.2 Selection of Trial Data.- 8.2.1.3 Reducing the Effect of Noise.- 8.2.2 Tests with Improved Technique.- 8.3 Further Improvements to Technique.- 8.3.1 Selection of a Better Set of Test Data.- 8.3.2 Mathematical Comparison of Results and Presentation.- 8.3.3 Testing of Further Improvements.- 8.3.4 Results.- 8.4 Conclusions.- References.- 9 The FRIL Model Approach for Software Diversity Assessment.- 9.1 Software Attributes Affecting Diversity.- 9.1.1 Process Attributes.- 9.1.1.1 Process Character.- 9.1.1.2 Process Profile.- 9.1.1.3 Tools.- 9.1.1.4 Personnel.- 9.1.1.5 Machines.- 9.1.2 Product Attributes.- 9.1.2.1 Product Character.- 9.1.2.2 Product Profile.- 9.2 Measuring Diversity.- 9.2.1 Measurement of Attributes.- 9.2.2 Tools to Aid in Measuring Attributes.- 9.2.2.1 Compilers.- 9.2.2.2 Static Analysers.- 9.2.3 Measuring Process Attributes.- 9.2.3.1 Process Character.- 9.2.3.2 Process Profile.- 9.2.3.3 Tools.- 9.2.3.4 Personnel.- 9.2.3.5 Machines.- 9.2.4 Measuring Product Attributes.- 9.2.4.1 Product Character.- 9.2.4.2 Product Profile.- 9.2.4.3 Product Use.- 9.3 The FRIL Model for Software Diversity Assessment.- 9.3.1 Description of Model.- 9.3.2 Design of the FRIL Program.- 9.4 Extension of the Work.- 9.4.1 Prototype Development.- 9.4.1.1 The Attributes.- 9.4.1.2 Model Development.- 9.4.1.3 The Rules and Inference.- 9.4.1.4 The Interface.- 9.4.2 The Results and Future.- References.- 10 Reliability Evaluation.- 10.1 Introduction.- 10.2 State of The Art of Reliability Models for Fault Tolerant Software.- 10.3 System States of Fault Tolerant Architectures.- 10.4 Analysis of System Sub-states.- 10.5 Modelling Approach.- 10.6 Modelling Methods.- 10.6.1 The Special Purpose Method.- 10.6.2 The General Purpose Method.- 10.6.3 Implementation Choice.- 10.7 Evaluation of the Equations.- 10.7.1 Single Component Test.- 10.7.2 Fault Tolerant System Integration Test.- References.- 11 The Impact of Voter Granularity in Fault-Tolerant Software on System Reliability and Availability.- 11.1 Definition of System States.- 11.2 Effect of Voter Granularity on System States.- 11.3 Examples.- 11.3.1 The UCLA Versions from the NASA Four-University-Experiment.- 11.3.2 The PODS Experiment.- 11.4 Strategic Choice of Optimal Granularity.- 11.5 Mixed Solutions.- 11.6 Conclusion.- References.- 12 A Theoretical Evaluation of the Acceptance Test in Recovery Block Programming.- 12.1 Introduction.- 12.2 General Features and Examples of Acceptance Tests.- 12.3 Formal Definition of Acceptance Test Characteristics.- 12.4 An Error Model for the Acceptance Test Behaviour.- 12.5 Conclusion.- References.- 13 Location of Checkpoints by Considering Information Reduction.- 13.1 Introduction.- 13.2 Failure Masking.- 13.3 Function Classes Reducing Information.- 13.4 Impact of Information Reduction on Failure Dependence.- 13.5 Information Reduction for Binary Values.- 13.6 Location of Checkpoints.- 13.7 Example.- 13.8 Conclusion.- References.- 14 Conclusions.- 14.1 Hardware Failure vs. Software Failure.- 14.2 Diversity and the Design of Fault-tolerant Software Systems.- 14.3 Assessment of Software Fault-tolerance.- 14.4 Prospect.