Assessments .... Everyone should understand what the goal is

I know .... I have been quiet for a while .... ok a long while. That being said I have not disappeared and this topic has been one that has been near and dear to my heart over the last couple of years in particular.

Unlike many security professionals I do not dread assessments, accreditation exercises or any other sort of measurement criteria that is placed upon a system that I have either built, helped write policy around or have architected. I find the assessment exercise an educational one. Sometimes I find the educational aspect is that the assessors need to do a better job.

It is important when you look at assessments that they be viewed in light of the goal of the assessment. I have never been put in a position where the goal was to just fail the system being assessed. One of the reasons that happens is that I make sure I am involved upfront such that there is clear understanding of things on all sides. I want to know that the assessors understand the system and understand where some cookie cutter tools or criteria will not be demonstrable in the "standard" way. I have dealt with systems that have Linux OS built on a couple of hundred of packages versus the few thousand that is standard. I can assure you that  standard tools are not going to work in trying to assess that system. 

I have also been involved with systems where the assessment criteria is utilizing old or poorly written measurement goals. Believe it or not some have stated you must implement <> of <> rules. Pretty hard to successfully measure or defend against a finding there.

When it comes to an assessment, accreditation or audit I tend to like to follow a pattern to ensure that everyone is getting the most out of the exercise. The process for me does not change whether I am assessing or being assessed. The question/responses may change but the overall process is one I find gives the system/organization being assessed the best chance to ensure that the security of the system meets the business requirements of the organization. The flow I use includes:

• Ensure documentation is up to date and available in advance;
• Define the specific criteria for assessment and agree up front to what those criteria must entail;
• Ensure the assessor has been given an introduction to the system so they understand any challenges to assessing;
• Ensure that people with practical knowledge of the operations are available as well as people with the deep technical knowledge can be called upon quickly;
• Allow the system owner to review a draft report of the assessment;
• Work with the system owner to ensure clear understanding of the gaps in the implementation versus the defined criteria from above;
• Define a clear path to remediation with the System Owner.

The goal here is not to see how many flaws can be found but to ensure that whatever is found is clearly understood, that the system owner understands the possible risk and that a mitigation plan can be defined and delivered on.

I have been involved in assessments where the assessors try to get in and out, as quickly as possible, and assume the standard tools work in all environments. In the end the assessment comes out as a very faulty one. It has not truly assessed the system and in the end it looks bad on everyone. I have also been in assessments where a few things have been found that could improve things but the assessors walk away with a high degree of confidence that the system truly meets the criteria defined and the devil in the details is what is needed to clean up .... and clearing those details will certainly help the system owner in the long run.

Don't be afraid of assessments. If you have done your work and you work openly with your assessor it will be a success for all.