Quality assurance has become particularly important because of the increasing reliance of business on software. Software is of good quality if it at least:
1) meets user needs
2) is reliable (i.e., does not fail)
3) is easy to maintain (i.e., to correct or change)
An important aspect of quality is measurement. In software engineering, measures are termed as metrics. A metric is a measure such as, cost or size, that can be applied to software. Metrics can be applied throughout the software development process. Metrics address issues as:
1) how big the software will be
2) how complex the component is
3) how reliable the software is
4) how much the software will cost
5) how much effort the software will need to meet changed needs
Metrics are applied in various ways to software. The purposes of metrics in software engineering include:
1) predicting qualities of developing software
2) making judgment on the quality of a piece of developing software
3) monitoring and improving the software development process
4) comparing and assessing different software development approaches
In this chapter, we first discuss metrics and then the application of quality assurance to software.
The simplest measure of software is its size. Two basic metrics are the size in bytes and size in the number of statements. The size in statements is termed as LOCs (lines of codes) or SLOCs (source lines of codes). The size in bytes is dependent on the main memory and disk space requirements and affects performance. The size in statements relates to development effort and maintenance costs. A longer program may not take a longer time to develop than a shorter program because what really matters is the complexity of software.
The second major technique is person months, a measure of developer effort. Since people’s time is a major factor in software development, person months usually determine cost. If an organization measures the development time for components, the information can be used to determine the time of future developments. It can also be used to verify the effectiveness of new techniques used.
The third basic metric is the number of bugs. As a component is being developed, a log can be kept of the bugs that are found. This helps predict how many bugs will remain at the end of the development. These figures also help to determine how good the new techniques are.
In the early days of programming, main memory was small and processors slow. The effort was more on making programs efficient. Nowadays the pressure is on reducing the development time of programs and easing the burden of maintenance. Therefore, it becomes necessary to write clear and simple programs so that checking, understanding and editing can be carried out conveniently.
What accounts for simplicity?
1) It is quicker to debug simple software
2) It is quicker to test simple software
3) Simple software is likely to be reliable
4) It is quicker to modify simple software
In the world of design engineering, a good designer maintains a complete understanding and control over every aspect of their project. The more difficult a project, the more they insist on simplicity because without it no one can gain full understanding.
Many software engineers and programmers strive to make their software as clear and simple as possible. When a programmer finishes a program, he is satisfied that it both works correctly and is clearly written.
What are actually the factors that affect program complexity? The program length, the number of alternative paths through the program, and the number of references to the data are all measures of complexity.
Among many attempts to measure software’s complexity is Mc Cabe’s cyclomatic complexity. This complexity shows that it does not depend on the number of statements in the program but rather the decision structure of the program for instance, if, while and other similar conditions. To measure the cyclomatic complexity of the program, we have to count the number of conditions and add one. Consider the following program segment:
Fig. 6.1: A Program Segment
This program segment has a cyclomatic complexity of 2 because it has one if condition and we add 1 to it give the complexity of 2.
Note that if a program has only a sequence of statements, it has a cyclomatic complexity of 1, however long it is. So the smallest value of this metric is 1.
A valid complexity measure can help software developers to a great extent in the following ways:
A human error in developing software causes a fault (or bug) resulting in a failure of the system (or several different failures). Testing every single piece of software gives rise to faults. If software developers develop software and they are able to tell clients professionally and honestly how many errors or estimated bugs there are, the clients are able to get some idea of the expected reliability.
A commonly used metric for faults is the fault density, which is the estimated number of faults per 1000 lines of code (or kilo lines of code, KLOC). Faults are detected during the verification process and during normal use after the software has been put into productive use. Some faults are corrected beforehand and do not count towards the fault density. In addition to known and existing faults, there are faults that are present, yet undetected.
Fault density figures are between 2 and 50 per KLOC. A figure of 2 is rated highly creditable.
Experimental studies show that majority of faults rarely cause system failures while a small number of faults contribute to system failure. Thus, if this small figure can be traced and fixed, it would be most cost-effective.
A major disadvantage of the fault density metric, as we have just seen, is that some bugs are more significant than others, which may also be difficult to detect. A more useful metric for this purpose could be Mean Time To Failure (MTTF). This is the average time for a system to perform without failing. This can be determined by logging the times at which the system failures occur and calculating the average time between successive failures. This then allows to predict the future failure rate of the system.
How do you assert that you have developed good, quality software? There are two ways to identify it. Here they go:
We can put in a commonly used definition of quality as follows:
A product which fulfills and continues to fulfill the purpose for which it was developed is a quality product.
There is an alternative course of action. That is, at each stage of the development process, everything is in order. At each stage, we can correct a fault that has been done incorrectly.
Here is a list of desirable software qualities:
Not every one of the above attributes may be required by any piece of software. Therefore, for each project, there arises a need to identify the required attributes before development can proceed.
The above list of quality factors can be used in one or more of the following situations:
1) At the initiation of software development for clarifying goals.
2) During development to guiding the development process towards the goals.
3) On completion, to assess the completed piece of software.
The above qualitative attributes are only qualitative and not quantitative. Usually the purpose of metrics is to quantify desirable qualities. Thus, a quality measure such as, McCabe’s can used to measure maintainability. Reliability can be measured as MTTF. However, for many of these attributes it is hard to judge accurately and therefore, a subjective guess, with all its uncertainties, should be made.
Quality assurance means ensuring a software system meets its quality goals. The goals differ from one object to another. They must be clear and can be selected from the list of quality factors. To achieve its goals, a project must use effective tools and methods. Also checks during the development process must be carried out at every available opportunity to ensure that the process is being carried out correctly.
To make sure all the effective tools and methods are being used, the organization keeps track of its best practices and documents them in a quality manual. This is like a library of all the effective tools, methods and notations. On the overall, this manual describes all the standards and procedures that are available to be used.
A standard defines a range, limit, tolerance or norm of some measurable attribute against which compliance is judged. For instance, during white box testing, every source code statement should be executed at least once.
A procedure defines a way of doing things (rules, steps, guidelines, plans). For instance, black box testing, white box testing and a walkthrough must be used to verify each component of the software.
To be effective, quality assurance must be planned ahead along with the planning of all other aspects of a software project. The project manager:
1) Decides which quality factors are important for a project e.g., reliability, maintenance etc.
2) Selects standards and procedures from the quality manual to meet quality goals of the project e.g., use of complexity metrics to check maintenance.
3) Combines 1) and 2) into a quality assurance plan. It describes what procedures and standards should be used for the project, when they should be completed and who will execute them.
An organization must not only use sound methods but also be seen to be using them. Therefore, a quality plan contains a number of quality controls. A quality control is an activity that checks whether the project’s quality factors are achieved and records a documentary evidence. Depending on the quality factors of the project, quality controls should include the following functions as shown in Table 6.1.
Table 6.1: Functions of Quality Controls
So far we have seen how quality can be measured, attained and ensured. For even better results, there may be a need to improve them. W. Edwards Deming, an influential management guru suggests that processes can be continuously improved. In his opinion, processes can be subjected to workers and the organization, which would look at them continuously for the sake of improving them.
Deming insists that improvements on processes can be done indefinitely so that they benefit:
1) Workers because they can control and take pride in their work.
2) Organizations because they can make increased profits
3) Customers because they get better quality.