Fagan inspection

Fagan inspection refers to a structured process of trying to find defects in development documents such as programming code, specifications, designs and others during various phases of the software development process. It is named after Michael Fagan who is credited with being the inventor of formal software inspections.

Software Testing portal

Definition

Fagan Inspection is a group review method used to evaluate output of a given process.

Fagan Inspection defines a process as a certain activity with a pre-specified entry and exit criteria. In every activity or operation for which entry and exit criteria are specified Fagan Inspections can be used to validate if the output of the process complies with the exit criteria specified for the process.

Examples of activities for which Fagan Inspection can be used are:

• Requirement specification
• Software/Information System architecture (for example DYA)
• Programming (for example for iterations in XP or DSDM)
• Software testing (for example when creating test scripts)

Usage

The software development process is a typical application of Fagan Inspection; software development process is a series of operations which will deliver a certain end product and consists of operations like requirements definition, design, coding up to testing and maintenance. As the costs to remedy a defect are up to 10-100 times less in the early operations compared to fixing a defect in the maintenance phase it is essential to find defects as close to the point of insertion as possible. This is done by inspecting the output of each operation and comparing that to the output requirements, or exit-criteria of that operation.

Criteria

Entry criteria are the criteria or requirements which must be met to enter a specific process^[1]. For example for Fagan inspections the high- and low-level documents must comply with specific entry-criteria before they can be used for a formal inspection process.

Exit criteria are the criteria or requirements which must be met to complete a specific process. For example for Fagan inspections the low-level document must comply with specific exit-criteria (as specified in the high-level document) before the development process can be taken to the next phase.

The exit-criteria are specified in a high-level document, which is then used as the standard to compare the operation result (low-level document) to during the inspections. Deviations of the low-level document from the requirements specified in the high-level document are called defects and can be categorized in Major Defects and Minor Defects.

Defects

According to M.E. Fagan, “A defect is an instance in which a requirement is not satisfied.”^[1]

In the process of software inspection the defects which are found are categorized in two categories: major and minor defects (often many more categories are used). The defects which are incorrect or even missing functionality or specifications can be classified as major defects: the software will not function correctly when these defects are not being solved.

In contrast to major defects, minor defects do not threaten the correct functioning of the software, but are mostly small errors like spelling mistakes in documents or optical issues like incorrect positioning of controls in a program interface.

Typical operations

In a typical Fagan inspection the inspection process consists of the following operations^[1]:

• Planning
  • Preparation of materials
  • Arranging of participants
  • Arranging of meeting place
• Overview
  • Group education of participants on the materials under review
  • Assignment of roles
• Preparation
  • The participants review the item to be inspected and supporting material to prepare for the meeting noting any questions or possible defects
  • The participants prepare their roles
• Inspection meeting
  • Actual finding of defect
• Rework
  • Rework is the step in software inspection in which the defects found during the inspection meeting are resolved by the author, designer or programmer. On the basis of the list of defects the low-level document is corrected until the requirements in the high-level document are met.
• Follow-up
  • In the follow-up phase of software inspections all defects found in the inspection meeting should be corrected (as they have been fixed in the rework phase). The moderator is responsible for verifying that this is indeed the case. He should verify if all defects are fixed and no new defects are inserted while trying to fix the initial defects. It is crucial that all defects are corrected as the costs of fixing them in a later phase of the project will be 10 to 100 times higher compared to the current costs.

Figure 1: Fagan inspection basic model

Follow-up

In the follow-up phase of a Fagan Inspection, defects fixed in the rework phase should be verified. The moderator is usually responsible for verifying rework. Sometimes fixed work can be accepted without being verified, such as when the defect was trivial. In non-trivial cases, a full re-inspection is performed by the inspection team (not only the moderator).

If verification fails, go back to the rework process.

Roles

The participants of the inspection process are normally just members of the team that is performing the project. The participants fulfill different roles within the inspection process^[2][3]:

• Author/Designer/Coder: the person who wrote the low-level document
• Reader: paraphrases the document
• Reviewers: reviews the document from a testing standpoint
• Moderator: responsible for the inspection session, functions as a coach

Benefits and results

By using inspections the number of errors in the final product can significantly decrease, creating a higher quality product. In the future the team will even be able to avoid errors as the inspection sessions give them insight in the most frequently made errors in both design and coding providing avoidance of error at the root of their occurrence. By continuously improving the inspection process these insights can even further be used [Fagan, 1986].

Together with the qualitative benefits mentioned above major "cost improvements" can be reached as the avoidance and earlier detection of errors will reduce the amount of resources needed for debugging in later phases of the project.

In practice very positive results have been reported by large corporations like IBM indicating that 80-90% of defects can be found and savings in resources up to 25% can be reached [Fagan, 1986]...

Improvements

Although the Fagan Inspection method has proved to be very effective, improvements have been suggested by multiple researchers. Genuchten for example has been researching the usage of an Electronic Meeting System (EMS) to improve the productivity of the meetings with positive results [Genuchten, 1997].

Other researchers propose the usage of software that keeps a database of detected errors and automatically scans program code for these common errors [Doolan,1992]. This again should result in improved productivity.

Example

In the diagram a very simple example is given of an inspection process in which a two-line piece of code is inspected on the basis on a high-level document with a single requirement.

As can be seen in the high-level document for this project is specified that in all software code produced variables should be declared ‘strong typed’. On the basis of this requirement the low-level document is checked for defects. Unfortunately a defect is found on line 1, as a variable is not declared ‘strong typed’. The defect found is then reported in the list of defects found and categorized according to the categorizations specified in the high-level document.

References

1. ^ ^{a b c} Fagan, M.E., Advances in Software Inspections, July 1986, IEEE Transactions on Software Engineering, Vol. SE-12, No. 7, Page 744-751 [1]
2. ^ M.E., Fagan (1976). "Design and Code inspections to reduce errors in program development". IBM Systems Journal 15 (3): pp. 182–211.  [2]
3. ^ Eickelmann, Nancy S, Ruffolo, Francesca, Baik, Jongmoon, Anant, A, 2003 An Empirical Study of Modifying the Fagan Inspection Process and the Resulting Main Effects and Interaction Effects Among Defects Found, Effort Required, Rate of Preparation and Inspection, Number of Team Members and Product 1st Pass Quality, Proceedings of the 27th Annual NASA Goddard/IEEE Software Engineering Workshop

1. [Laitenberger, 1999] Laitenberger,O, DeBaud, J.M, 1999 An encompassing life cycle centric survey of software inspection, Journal of Systems and Software 50 (2000), Page 5-31
2. [So, 1995] So, S, Lim, Y, Cha, S.D., Kwon, Y,J, 1995 An Empirical Study on Software Error Detection: Voting, Instrumentation, and Fagan Inspection *, Proceedings of the 1995 Asia Pacific Software Engineering Conference (APSEC '95), Page 345-351
3. [Doolan,1992] Doolan, E.P.. 1992 Experience with Fagan’s Inspection Method, SOFTWARE—PRACTICE AND EXPERIENCE, (FEBRUARY 1992) Vol. 22(2), Page 173–182
4. [Genuchten, 1997] Genuchten, M, Cornelissen, W, Van Dijk, C, 1997 Supporting Inspections with an Electronic Meeting System, Journal of Management Information Systems, Winter 1997-98/Volume 14, No. 3, Page 165-179