C H A P T E R 4
The waterfall model is an established approach to software development and is widely used.
In the waterfall method, the steps are sequential. Each stage is completed until conclusion before the next stage begins. Additionally, each stage produces a product until it is input into the next stage. For example, all coding is completed before testing begins.
Fig. 4.1: Waterfall method
The principles of the waterfall method:
1) It is a series of steps.
2) Each step is well defined.
3) Each step creates a definite product.
4) Each product forms the basis for the next step.
5) Each step can be verified or validated.
The complete waterfall process method is like a series of small waterfalls because each stage produces a product before it can flow like water to the next stage. As water cannot flow up but can only flow down, once a step is executed, there is no going back.
The inputs and outputs for each step of waterfall process are shown in tabular form below:
Table 4.1: Waterfall Process Steps
An error uncovered at one stage cannot be solved by going backwards in the waterfall process model described above. So a variation of the process model is to provide feedback to the previous stage so that errors found in a stage can be corrected by going back to the immediate previous stage.
Fig. 4.2: Modified Waterfall Method
This model is based on the recognition that it involves enormous uncertainty at many stages and therefore, the involvement of risk assessments. These assessments are followed by deciding alternative actions, selecting the best action and re-planning.
The model spirals out from near the center of the process and passes through four phases. These are: 1) analyze risks and plan 2) analyze requirements 3) construct 4) analyze. These phases convey the increasing expenditure of time, effort and progress.
As we mentioned, the spiral model involves the idea of risk. There are uncertainties associated with software development and should be dealt with as carefully as possible. Some of the risks that could be experienced are:
1) The client changes some of his requirements.
2) During a time-consuming development, user’s requirements are ignored.
3) Someone quits the development team.
4) One of the component tasks of the development goes beyond its deadline.
5) The software performance is slow.
6) The software occupies too much main memory.
7) A target hardware configuration changes.
8) A bug is difficult to get rid of.
The spiral model makes provision for areas concerning the above uncertainties and therefore, minimizes the risk to the software project.
Each of the four phases of the spiral model can be further described as follows:
1) Meeting the objectives of the product of this stage – performance, functionality, ease of change.
2) Identifying the conditions that affect this stage – cost, deadlines, interfaces with other software components
3) Identifying risks
4) Identifying the alternative ways of implementing the stage – buying it, reusing something else, implementing it one way or the other.
5) Evaluating the alternative implementation schemes against the criteria set by the objectives and conditions.
6) Finding ways to overcome the risk.
7) Establishing the deadline for the next project phase.
This takes into account meeting the requirements for the next stage.
At this stage, the development of a product may be implemented. This stage may involve design, implementation, validation and verification. A product at this stage may be a specification, architectural design, prototype or software component.
Finally an evaluation is carried out to find out if the project is on track and if the entire development team and client(s) are happy. If yes, the project can head on to the next cycle.
Fig. 4.3 : The Spiral Model
A prototype is an initial design of the software system which is tried out to test concepts, design, find out more about the problem and figure out solutions. Rapid iterative development of the prototype is essential so that costs are minimum and stakeholders can experiment with the prototype in the early software process.
A software prototype can be used in a software development process to help monitor changes that may be needed.
1) In the requirements engineering process, a prototype can help with the elicitation and validation of system requirements.
2) In the system design process, a prototype can help with the discovery of software solutions and initiation of user interface design.
The system prototypes allow users to see how well the system supports their work. They may get new ideas for requirements and find strengths and weaknesses in the software. As the prototype is developed, it may reveal errors or omissions in the proposed requirements. This may lead to modification of the system specification.
While a system is designed, a system prototype may be used to test the feasibility of the proposed design. For instance, a database design may be prototyped to test if the system supports the volume of data required for the most common queries. As another instance, rapid prototyping with end user involvement is the best way to develop user interface design for software systems.
To reduce prototyping costs it may be wise to leave out some functionality of the prototype. For instance, you may decide to relax non-functional requirements such as response times and memory utilization.
The final stage of prototyping is prototype evaluation. This means to make provision for user training until they are comfortable with the new system and working on it routinely to discover requirements errors and omissions.
A process model for prototype development is shown below:
Fig. 4.4: The Prototyping Model
This process model is based on the development of an initial implementation, subjecting it to user comment and going through several versions until an adequate system is developed. Specification, development and validation are interleaved with rapid feedback from these activities.
Each increment or version of the system involves some of the functionality that is needed by the customer. The early increments include the most urgent or important functionality. This means that the customer can evaluate the system at an early stage in the development in order to find out if it delivers what is required. If not, only the current increment is to be modified and possibly new functionality for future increments.
The problems of incremental development become acute for large systems, with different teams working on different parts of the system. This has, therefore, to be planned in advance rather than developed incrementally.
Fig. 4.5: Incremental Development Model
Open source is a development approach in which the source code is completely free to access. The term “open source” applies to both the product and the development approach. Any individual is able to view it, modify it or duplicate it.
Some examples of larger open source products are Mozilla web browser, Apache web browser, GNU/Linux and GNU/HURD operating systems, MySQL database software, Perl programming language and My Office and Open Office suites.
Open source development is an extension of Hacker Community’s attitude to the development of software. Hackers are now regarded as a community of highly skilled programmers who relish writing code and enhance their reputation of writing code. Because knowledge and information are freely shared without restriction, it initiates group thinking and superior ideas on the overall.
Similarly, in open source development, the code is not confined to a small group of developers but shared by a bigger audience, which leads to more reliable code via greater thinking, ideas and rectification.
Because of the openness of the program code, the community has imposed its own license agreement for use on open source products. The GNU general public license (GPL) is a software license which protects its openness.
However, the openness and sharing of code does not mean the open source products are always free to buy. Companies often sell their software as a complete package including user manuals and additional help services.
There exists a split within open source development between ethics and philosophy. Nevertheless, the development strategies remain the same between the two.
The process is initiated by a single developer with requests for collaboration to the hacker community. The Internet often provides the bridge between the developers and distribution of source code via web, file transfer sites and email.
The head developer specifies most user requirements. Additional user requirements are worked out by developers themselves modifying the source code or through a communal process called code forking. Code forking occurs when the developer has conflicting ideas on a user requirement. The code is split giving arise to two copies from the base code. As the code is split this way, two different products are developed in parallel. The more significant or reliable one is retained and survives.
The design of the code is implemented by web-based tools. UML diagrams or cross reference displays using hyperlinks are used for deployment of the design. However, the design documentation lacks for open source products.
An explicit project manager or management group decides on the usefulness or appropriateness of contributions made by the wider development community. They usually add the patch to code and play the role of chief implementer of the code.
Once contributions have been made, beta versions of open source products are released. Releases are made frequently so that contributions for appropriateness can be tested quickly. Feedback on the latest contribution is given and the contributions cycle on the code continues until the community is satisfied with the eventual outcome.
Development communities and product websites are sources of support for users of open source products. The websites contain installation tutorials and user forum groups providing technical support.
As an alternative means of support, commercially supported versions of open source software are available for purchase along with supporting manuals and services.
Agile methods are incremental development methods in which the increments are small and new releases of the system are created and available every two-three weeks. They involve customers in the development process to get rapid feedback on changing requirements. They minimize documentation by using informal communication rather than formal meetings with documentation.
In the 1980s and early 1990’s, software engineering methods were heavy-weight, plan- driven approaches for developing large and long-lived software systems such as aerospace and government systems.
When these approaches are applied to small and medium-sized business systems, the overhead is so large that it dominates the software development process. More time is spent on how the system should be developed than on program testing and development. As the system requirements change, the specification and design has to change with the program.
Dissatisfaction with these heavyweight approaches led many developers in the 1990’s to propose new “agile methods”. These allowed the development team to focus more on the software rather than on its design and documentation. Agile methods universally rely on an incremental approach to software specification, development and delivery. They are best suited to application development where the system requirements change rapidly during the development process.
The philosophy behind agile methods is reflected in the agile manifesto that was agreed on by many of the leading developers of these methods. This manifesto puts emphasis on:
Table 4.2: Principles of Agile Development
Agile approaches to software development consider design and implementation to be the central activities in the software process. They incorporate other activities such as requirements analysis and testing into design and implementation. On the contrary, a plan-driven approach to software development includes separate stages in the software process with outputs associated with each stage. The outputs from one stage are used as a basis for planning for the following process activity.
In a plan-driven approach, iteration occurs within activities with formal activities for communicating between stages of the process. For example, requirements will arise and eventually a requirements specification will be produced. This then serves as an input to the design and implementation process. In an agile approach, iterations occur across activities and the requirements and the design are developed together rather than separately.
Fig. 4.6: Plan-driven and Agile Developments
Most software projects include practices from plan-driven and agile approaches. To decide on a balance between the plan-driven and the agile approaches, a range of questions need to be asked, some of which are given below:
1) Is it important to have a detailed specification and design before moving on to implementation? If yes, a plan-driven approach may be the solution.
2) Does an incremental delivery strategy, where you deliver the software to the customers and get rapid feedback from them seem feasible? If yes, go for an agile approach.
3) How large is the system that is being developed? Agile methods are best applied to small or medium-sized software systems where a small co-located team is assigned to work informal communications. Plan-driven methods are best applied to large system where large development teams are assigned.
4) What type of system is being developed? Systems requiring a lot of analysis before implementation would need a fairly detailed design to carry out this analysis. In that case, a plan-driven approach would be appropriate.
5) What is the expected system lifetime? Long-lifetime systems may require more design documentation to communicate the original intentions of the system developers to the support team. However, supporters of agile methods are right in arguing that for such systems, there is no need to keep documentation up-to-date nor will it be valid for long-term system maintenance.
6) What technologies are available to support system development? Agile methods rely on good tools for an evolving design. If the system uses an IDE that does not have good tools for analysis and visualization, then a more detailed design specification would be required and hence the need for a plan-driven approach.
7) How is the development team organized? If the development team is distributed or part of the team is being outsourced, then detailed design documents need to be distributed across the teams. These documents may need to be planned in advance and hence the need for plan-driven approach.
8) Are there cultural issues that may affect the system development? Traditional software engineering organization has the culture of plan-driven approach. This requires detailed design specification rather than informal knowledge in agile approaches.
9) How good are the designers and programmers in the development team? It is argued that agile approaches require higher skill levels than plan driven approaches where the design is simply translated into code. If the team consists of people with low level skills, among them the better ones should design while the others should code.
10) Is the system subjected to external regulations? Some software systems require an external regulator such as, Federal Aviation Authority (FAA) which requires to approve the software system of an aircraft. In those cases, detailed documentation would be required as part of the safety measures.
Extreme programming (XP) is probably the best known and widely used among agile methods. In XP, several new versions of a system may be developed, integrated and tested within a single day.
Fig. 4.7: Extreme Programming Process
In extreme programming, requirements are expressed as scenarios or user stories which are directly implemented as a series of tasks. Programmers work in pairs and develop tests for each task before writing the code. All code should run successfully and integrated into the system. There are short time gaps between releases of the system.
Extreme programming involves a number of practices, which are typical of agile approaches:
1) Incremental development is supported through small frequent releases of the system. Requirements are based on customer stories or scenarios, which form the basis for deciding what functionality should be included in an increment.
2) The customer is continuously engaged with the development team and the customer representative collaborates with the development team and is responsible for defining acceptance tests for the system.
3) People not process are involved in pair programming, get ownership of system code and do not have to work for long hours during the development process.
4) Change is embraced through regular system releases to customers, test-first development and refactoring (meaning improving the structure and organization of a program, which is also an important mechanism that supports change tolerance) to prevent code degeneration and integration of new functionality.
5) Maintaining simplicity is done through refactoring to improve code quality and by using simple designs that show no signs of future changes to the system.
In an XP process, customers are involved in specifying and prioritizing system requirements. The requirements are not listed as required system functionality. Rather the system customer is part of the development team and discusses scenarios with other team members. Together they develop a story card that encapsulates customer needs. The development team then includes a scenario in the next release of the software.
The story cards are the main inputs to the XP planning process. Once the story cards have been developed, developers break down these as tasks and figure out what effort and resources are required to implement each task. This usually involves the customer in discussion with the team to refine requirements. The customer than prioritizes these requirements and chooses those scenarios that will immediately deliver good business support in about two weeks when the next release of the software would be available.
As requirements change, the unimplemented stories change or may be discarded. If changes are required to a system that has already been delivered, new story cards are developed and again the customer decides whether these changes should have priority over new functionality.
Table 4.3: Extreme Programming Practice
During the planning game, there may arise questions, and solutions need to be explored. The team may carry out prototyping or trial development to deal with a problem and come to a solution. In XP terms, this is a spike, which is an increment where no programming is done. These spikes may also be used to design system architecture or develop system documentation.
Extreme programming takes the extreme approach to incremental development. New versions of the software may be built several times during the day and releases delivered to customers every two weeks. Release deadlines are never missed. However, if there are development problems, the customer is consulted and functionality removed from the planned release.
When a programmer builds the system to create a new version, he or she must run all existing tests of the system and also those for new functionalities. This only occurs if all tests run successfully. This then becomes the basis of the next iteration.
According to traditional software engineering, you should anticipate future changes and design for it so that these changes can be easily implemented. Extreme Programming has, however disregarded designing for change because it is often wasted effort. The changes anticipated often never materialize and often different change requests are made over time. Therefore, the XP approach accepts that changes will happen and reorganizes the software when these changes occur.
A problem with the incremental approach is that it degrades the software structure so that changes to the software becomes harder and harder to implement. Extreme programming tackles this problem by suggesting that the software be constantly refactored. This means that the programming team looks for improvements to the software and implements them immediately. When a team member sees code that can be improved, he makes those improvements even when there is no immediate need for them. Examples of refactoring include the reorganization of a class hierarchy to remove duplicate code, the tidying up and renaming of attributes and methods and the replacement of code with calls to a method embedded within a program library.
In theory then, software is always easy to understand and change. But in practice, this may not always be the case. Sometimes refactoring is delayed because time is devoted to the implementation of new functionality. In many cases, new features and changes cannot be implemented by code-level refactoring but require the system architecture to be modified.
In practice, all of the companies who have adopted the XP approach do not use all of the practices shown in Table 4.3. They pick and choose according to their convenience. For example, some companies find pair programming useful while others adopt individual programming and reviews. Some companies don’t do refactoring on parts they didn’t develop and use conventional requirements rather than user stories. However, most companies who have adopted an XP variant use small releases, test-first development and continuous integration.
Some approaches to incremental development have a very informal testing method compared to plan-driven approaches. Formal testing cannot be carried out in the incremental approach because there is no actual requirements specification to follow.
XP introduces an approach to testing that reduces the entry of undiscovered errors into the current version of the system.
The key features of testing in XP are as follows:
1) Test-first development
2) Incremental test development from scenarios.
3) User involvement in the test development and validation
4) Incremental automated testing frameworks
In test-first development, you write the test first instead of writing the code so that the test can be carried out while developing the code and errors can be discovered.
This approach helps to make clear the link between a system requirement and the code implementing that requirement. In XP this link is always obvious because the story cards representing the requirements are transformed into tasks and the tasks are the unit of implementation.
In test-first development, task implementers have to thoroughly understand the specification before they can write tests for the system. For this to happen omissions and ambiguities in the specification will have to be clarified before tests can be written. This also avoids the problem of test-lag. This happens when code developers are advancing at a faster pace than test implementers so that tests are skipped to maintain code development.
The development team expresses requirements as user stories or scenarios and the user prioritizes these for development. The development team assesses these scenarios and breaks them down into tasks. Each task generates one or more unit tests, which check the implementation within those task(s).
The role of the customer in the testing process is to help develop acceptance tests for the stories that are to be implemented in the next release of the system. Acceptance testing is the process where the system is tested using customer data to meet customer’s needs.
In XP, acceptance testing like development is incremental. The customer who is part of the development team writes tests as development proceeds. People who lead the customer role may have very limited available time and may not be willing to work full- time with the development team. The customer may think that the requirements were enough of a contribution and may be unwilling to get involved in the testing process.
Test automation is essential for test-first development. Tests are written as executable components before the task is implemented. An automated testing framework is a system that makes it easy to write executable tests and submit a set of tests