I know .... I have been quiet for a while .... ok a long while. That being said I have not disappeared and this topic has been one that has been near and dear to my heart over the last couple of years in particular.
Unlike many security professionals I do not dread assessments, accreditation exercises or any other sort of measurement criteria that is placed upon a system that I have either built, helped write policy around or have architected. I find the assessment exercise an educational one. Sometimes I find the educational aspect is that the assessors need to do a better job.
It is important when you look at assessments that they be viewed in light of the goal of the assessment. I have never been put in a position where the goal was to just fail the system being assessed. One of the reasons that happens is that I make sure I am involved upfront such that there is clear understanding of things on all sides. I want to know that the assessors understand the system and understand where some cookie cutter tools or criteria will not be demonstrable in the "standard" way. I have dealt with systems that have Linux OS built on a couple of hundred of packages versus the few thousand that is standard. I can assure you that standard tools are not going to work in trying to assess that system.
I have also been involved with systems where the assessment criteria is utilizing old or poorly written measurement goals. Believe it or not some have stated you must implement <> of <> rules. Pretty hard to successfully measure or defend against a finding there.
When it comes to an assessment, accreditation or audit I tend to like to follow a pattern to ensure that everyone is getting the most out of the exercise. The process for me does not change whether I am assessing or being assessed. The question/responses may change but the overall process is one I find gives the system/organization being assessed the best chance to ensure that the security of the system meets the business requirements of the organization. The flow I use includes:
- Ensure documentation is up to date and available in advance;
- Define the specific criteria for assessment and agree up front to what those criteria must entail;
- Ensure the assessor has been given an introduction to the system so they understand any challenges to assessing;
- Ensure that people with practical knowledge of the operations are available as well as people with the deep technical knowledge can be called upon quickly;
- Allow the system owner to review a draft report of the assessment;
- Work with the system owner to ensure clear understanding of the gaps in the implementation versus the defined criteria from above;
- Define a clear path to remediation with the System Owner.