Discussion re the Penetration Testing Industry

Chris over at Catch22 just posted up this excellent blog article.

A huge amount of commonality in thinking - Some extra thoughts on this:

Communication - over the last 12 or so years I have tried various training for testers along business lines etc., and there are very few who I would say are at the top of their game in both testing and reporting in business language. The few around are worth their weight in gold, but very rare, so my fallback solution was always to have a member of the team responsible for business QA and reporting. They'd still need to be at a high level of technical expertise, but the focus is different. (I do like Chris's idea of a tech reporting course though!)

Relevance - understanding the customer's needs is definitely key. As we've discussed, working with the customer so they understand what their options are, the value in different services etc., should be a part of every engagement.

Accountability - two thoughts on this. One is the name and shame as Chris mentions, but there are bound to be legal challenges, so the alternative is to use certifications (eg CREST, SANS etc) to be able to demonstrate to board level that you chose the right testers for the job, as the certification is effectively the entry qualification to the industry. In addition, you could go down the route of extensive logging (also would help for the repeatability section below) so you can prove every step.

Standards - absolutely! See our earlier posts on taxonomy and nomenclature to understand an element of where we see standards going, and we are planning to continue to work with a good range of experienced security individuals to define a set of industry standards.

Repeatability - I think where possible a number of organisations already do this. On a recent project, my customer wanted at least a minimum (including the parameters used and screenshots) to allow them to replicate the issue. That is only applicable for certain types of tests, but it goes a long way to help, and it is relatively light on resource so shouldn't price you out of the market.

The great thing is that more and more people are aiming the same direction. This has been a long time coming, but with passionate individuals, organisations and bodies, I think moving from the end of 2010 into 2011 will see a step change in the professionalisation of the industry.