What's in a name

As we mentioned in the last post, we'd like to expand a bit on the presentation that Rory Alsop and Rory McCune made to OWASP Ireland.

While we've been working on 7 Elements, we've been putting some thought into how penetration testing is currently sold and delivered and how we can improve the process for customers and suppliers.

The first step for us was to understand some of the problems, as we see them, and the first of these is the name itself.

Penetration testing has come to mean a wide variety of things and it tends to get used interchangeably. It was originally understood to refer to a specific type of testing where the tester would emulate an attacker (generally in a black-box style of test) and try to get access to a specific set of services. A penetration test wasn't concerned necessarily with finding as many security issues as possible, but with proving whether an attacker could get unauthorised access to a system.

Now it seem to be used to refer to anything vaguely security testing related, from vulnerability scanning, through web application testing and code review, to actual penetration testing.

The major problem this causes is that it means that people are referring to "penetration testing" and having completely different ideas of what that testing will deliver.

This can cause problems in several areas, such as buying of testing services. How does a customer compare two companies selling penetration testing if one charges £400 a day and another charges £1200 a day?

Another problem comes when regulators or customers specify that an organisation must have a "penetration test", when what they really want to do is get some assurance that that organisation has addressed commonly occurring security issues across all parts of a given system.

So what's the answer to all this? Well we think that the best way forward is to move away from the "penetration test" terminology and begin to categorise types of security testing/assurance/review. We have been working with individuals across a range of organisations, including CREST, OWASP, buyers and vendors and have created a draft outline.

In our next post we plan to further develop this straw man into an industry ready draft.