6.4 Risk Assessment Methodologies

The testing and assessment tools and methodologies discussed in earlier sections are each applied at their respective stages of an application's development lifecycle. But in addition to these specific tools and methodologies, there are several approaches to reviewing the overall risk of an application system to a business that are, by and large, independent of where they are applied within the lifecycle. In this section we describe two advanced risk assessment methodologies: ACSM/SAR (Adaptive Countermeasure Selection Mechanism/Security Adequacy Review) and ASSET (Automated Security Self-Assessment Tool).

At least some of the components of ACSM/SAR and ASSET could also be performed at different points within the development lifecycle. For example, evaluating a risk level at design time using the ACSM/SAR process could save you considerable time and expense later.

6.4.1 ACSM/SAR

Some years ago, both of us were lucky enough to work directly on the Security Adequacy Review (SAR), a project initiated and managed at Sun Microsystems by Tim Townsend. The technical software and mathematical theory underpinning the SAR is known as the Adaptive Countermeasure Selection Mechanism (ACSM).

The goal of the ACSM/SAR project was to generate a set of software and processes that would produce a security "specification" for Sun's key applications—the applications Sun uses to run its own business. (Note that our discussion here has nothing to do with Sun's products.)

Please note that we aren't revealing confidential information here. We summarize only those aspects of the ACSM/SAR project that are public knowledge, having been previously described in a public white paper and a patent application.

The project team began with an analysis of general attacks and countermeasures, producing tables representing expert judgments as to how effective each countermeasure is in guarding against each kind of attack. The team developed, for each countermeasure, a set of five grades, or "strength levels," and then developed an estimate of the cost of each level of each countermeasure. With that data (and supporting processes and software) in place, Sun then instituted a program in which most key business applications were evaluated for security needs.

For each program a lengthy questionnaire must be completed, detailing the value of the assets manipulated by the application and the protective measures and design features already in place. Then, as described in the ACSM patent document:

A current strength level for a countermeasure is determined based on input data and rules corresponding to the application. The method and apparatus determine a recommended strength level for countermeasures based on the input data and security risk data. Based on the current strength level and the recommended strength level, the method determines and outputs a security model including a countermeasure and corresponding strength level.

In other words, ACSM produces a list of steps to follow that will take the application from the current security level to the level mandated by the value of the application and its assets to Sun. Of course, the same technique is routinely used to facilitate the secure design of applications from scratch.

While the ACSM/SAR software is not available (so far as we know) for use outside of Sun, the white paper is well worth your study. (Please see this book's companion web site for more information.)

6.4.2 ASSET

The second project was developed at the United States National Institute of Standards and Technology (NIST). The software that facilitates the recording and analysis of the answers supplied to the project's application questionnaires is called the Automated Security Self-Assessment Tool (ASSET). The document SP 800-30, which we quote back in Chapter 3, is one of the outcomes of the project. Because ASSET, unlike ACSM/SAR, was a publicly funded project, the software and documentation is freely available for download from the NIST Computer Security Resource Clearinghouse, at http://csrc.nist.gov/.

The similarity between ASSET and ACSM/SAR is notable. ASSET (which is somewhat broader in scope, because it deals with IT systems in general) also begins with a questionnaire (a "self-assessment" tool) and a detailed risk and threat assessment. Based on these factors and a set of effectiveness estimates calculated for each countermeasure, ASSET makes recommendations about which security countermeasures, at which of five levels, would be appropriate against the perceived threats.

The project documentation describes ASSET as follows:

The Automated Security Self-Evaluation Tool (ASSET) automates the process of completing a system self-assessment. ASSET will assist organizations in completing the self-assessment questionnaire contained in NIST Special Publication (SP) 800-26, Security Self-Assessment Guide for Information Technology Systems . . .

Self-assessments provide a method for agency officials to determine the current status of their information security programs and, where necessary, establish a target for improvement. This self-assessment guide utilizes an extensive questionnaire containing specific control objectives and techniques against which an unclassified system or group of interconnected systems can be tested and measured . . .

There are a total of 17 topics contained in the questionnaire; each topic contains critical elements and supporting security control objectives and techniques (questions) about the system.

In addition to the software that calculates security recommendations, we particularly like the mechanisms ASSET provides to handle and process questionnaires:

ASSET consists of two host-based applications: ASSET/System and ASSET/Manager.

ASSET/System: facilitates the gathering of individual system data. It provides a limited reporting capability and allows the user to determine the completeness of an individual system assessment in progress.

ASSET/Manager: aggregates individual system assessments created by ASSET/System. It assists managers in developing an organization-wide perspective on the state of IT system security.

The reporting features of ASSET are designed to provide users with a clear picture of the security status of their resources, as specified in NIST SP 800-26. The reports available from ASSET can be generated and interpreted by the users who use the application.

Should you undertake such a project, you will find (as we have) that collecting, collating, and cleansing the answers you get will be an extremely demanding task. In many cases, it will be harder than actually designing and implementing solutions to the security issues that are uncovered!

Nevertheless, we strongly recommend that you study the ASSET system, and consider adopting or adapting it for your needs.