SSDL Touchpoints: Security Testing (ST)

The overall goal of the Security Testing practice is quality control. Those performing security testing must ensure the detection and correction of security bugs. The SSG must enforce adherence to standards and the reuse of approved security features.

SSDL TOUCHPOINTS: SECURITY TESTING Use of black box security tools in QA, risk driven white box testing, application of the attack model, code coverage analysis.
	Objective	Activity	Level
ST1.1	execute adversarial tests beyond functional	ensure QA supports edge/boundary value condition testing	1
ST1.2	facilitate security mindset	share security results with QA	1
ST2.1	use encapsulated attacker perspective	integrate black box security tools into the QA process (including protocol fuzzing)	2
ST2.2	start security testing in familiar functional territory	allow declarative security/security features to drive tests
ST2.3	move beyond functional testing to attacker's perspective	begin to build/apply adversarial security tests (abuse cases)
ST3.1	include security testing in regression	include security tests in QA automation	3
ST3.2	teach tools about your code	perform fuzz testing customized to application APIs
ST3.3	probe risk claims directly	drive tests with risk analysis results
ST3.4	drive testing depth	leverage coverage analysis

one

ST Level 1: Enhance QA beyond functional perspective. The SSG must share its security knowledge and testing results with QA. QA must progress to include functional edge and boundary condition testing in its test suites.

ST1.1

Ensure QA supports edge/boundary value condition testing. The QA team goes beyond functional testing to perform basic adversarial tests. They probe simple edge cases and boundary conditions. No attacker skills required.

ST1.2

Share security results with QA. The SSG shares results from security reviews with the QA department. Over time, Quality Assurance Engineers learn the security mindset.

two

ST Level 2: Integrate the attacker perspective into test plans. QA must integrate black-box security testing tools into its process. QA must build test suites for functional security features and progress to building adversarial tests that simulate the attacker's perspective.

ST2.1

Integrate black box security tools into the QA process (including protocol fuzzing). The organization uses one or more black box security testing tools as part of the quality assurance process. The tools are valuable because they encapsulate an attacker's perspective, albeit in a generic fashion. Tools such as Rational AppScan or HP WebInspect are relevant for Web applications, and fuzzing frameworks such as PROTOS are applicable for most network protocols. In some situations, the other groups might collaborate with the SSG to apply the tools. For example, a testing team could run the tool but come to the SSG for help interpreting the results.

ST2.2

Allow declarative security/security features to drive tests. Testers target declarative security mechanisms and security features in general. For example, a tester could try to access administrative functionality as an unprivileged user or verify that a user account becomes locked after some number of failed authentication attempts.

ST2.3

Begin to build/apply adversarial security tests (abuse cases). Testing begins to incorporate test cases based on abuse cases provided by the SSG. Testers move beyond verifying functionality and take on the attacker's perspective. For example, testers might systematically attempt to replicate incidents from the organization's history.

three

ST Level 3: Deliver risk-based security testing. QA must include security testing in automated regression suites. The SSG must ensure this security testing and its depth is guided by knowledge about the codebase and its associated risks.

ST3.1

Include security tests in QA automation. Security tests run alongside functional tests as part of automated regression testing; the same automation framework houses both. Security testing is part of the routine.

ST3.2

Perform fuzz testing customized to application APIs. Test automation engineers customize a fuzzing framework to the organization's APIs. They could begin from scratch or use an existing fuzzing toolkit, but customization goes beyond creating custom protocol descriptions or file format templates. The fuzzing framework has a built-in understanding of the interfaces it calls into. Google's Lemon is such a framework. Lemon understands the conventions used by Google's application interfaces.

ST3.3

Drive tests with risk analysis results. Testers use architecture analysis results to direct their work. For example, if the architecture analysis concludes "the security of the system hinges on the transactions being atomic and not being interrupted partway through," then torn transactions will be become a primary target in adversarial testing.

ST3.4

Leverage coverage analysis. Testers measure the code coverage of their security tests in order to identify code that isn't being exercised. Code coverage drives increased security testing depth.