6.2 Good General Practices

Testing is a complete software engineering discipline in its own right. Volumes have been written about test scaffolds, regression testing, and the other techniques software engineers routinely employ to produce consistent results. This book isn't the place for that discussion—we list a few suggestions in Appendix A—but we do have a few general recommendations:

Perform automated testing

We advise you to automate your testing procedures as much as possible. As Brian W. Kernighan and Rob Pike explain in The Practice of Programming:

It's tedious and unreliable to do much testing by hand; proper testing involves lots of test, lots of inputs, and lots of comparisons of outputs. Testing should therefore be done by programs, which don't get tired or careless.

The best kind of automation, of course, would allow you to test your software against newly discovered vulnerabilities without modifying your test bed or procedures and without hand-coding new test cases. Luckily, some available testing tools achieve at least part of this dream. We'll point them out as we go along.

Test at every stage

Experience has taught us that it's important to test our software at every stage of the development cycle. This approach enhances not only the security of the software, but also the usability, reliability, and effectiveness of an application.

Make a test plan

An integral part of the software development process is the development of a test plan. This plan should be carefully thought out to ensure that the software is thoroughly tested for its business effectiveness. It may seem like a lot of additional work early on, but you will never regret it later.

Test all system components

Taking into account the notion of an application system as we described earlier, be sure to include all of the components of the system in your test plans. Beyond just the components, the test plan should consider issues such as the business impact of the application's failure, compromise, or unavailability. In addition, be sure to consider carefully the scope of the test plan. For example, will third-party software be tested for security, or will it be blindly accepted? Will the operations plan for the software be tested?

Follow a specific methodology

Testing is most effective when it is part of a carefully developed process.In this book, we have tried to remain largely neutral on your choice of process methodology. Nevertheless, we focus on the development steps generally, without regard to the actual steps (e.g., rapid prototype, waterfall lifecycle) that you take. Here are various standards we recommend that you consult. (Note that the ISO standard 17799, based on the British standard 7799, is a leader here.)

• ISO 17799 "Information Technology: Code of Practice for Information Security Management"

• ISO/IEC 15408 "Evaluation Criteria for IT Security" (the "Common Criteria")

• SSE-CMM "System Security Engineering Capability Maturity Model"

• ISO/IEC WD 15443 "Information Technology: Security Techniques" (for an overview)