Information Security Today Home

New Books

FISMA Principles and Best Practices: Beyond Compliance
Cyber Security Essentials
Mobile Device Security: A Comprehensive Guide to Securing Your Information in a Moving World
Security Strategy: From Requirements to Reality
Adaptive Security Management Architecture

Factors Impacting Information Security in Selection of a Compliance Methodology, Taxonomy, or Framework

Jeff Smith

Compliance in technology information security and assurance requires detailed investigation and selection of a credible practiced single or combination methodology, taxonomy, or framework (MTF) that produces measurable results while providing high accuracy in front and back end evaluation. The selection of a validated MTF that is appropriate for the applications and systems deployed contributes to an increased accuracy rate in the identification of emerging risks, threats and vulnerabilities.

An MTF must contain two highly productive data generating areas outside the routine process of conducting administrative interviews of people connected programs; i.e., training, human resources, and document review. First, compliance must be measured through the front end of system administration technical configuration and administration, such as active directory, servers, databases, firewalls, and switches. This includes compliance artifacts and measurement of system administrator operational knowledge while mapping results to mandated enterprise governance compliance polices.

The second assessment is at the back end, commonly referred to as "behind the curtain" or "unknowns to the organization," as compliance is measured through adoption of a vetted automated scripting engine in identification of vulnerabilities. Automated tool discovery results are measured against system operational knowledge in application of mandated enterprise governance compliance policies. In monitoring systems at the back end, a significant factor often overlooked is the impact of on-site vs. off-site automated engine scanning to ensure solid local topology coverage. Both local and remote scanning requires significant coordination over a period of testing as well as trial and error to match coverage with active hardware asset management. Correct asset management is accomplished through real time event reporting (RFID) and not human factor inventory as assets are dynamic to the business topology. All converged assets must be pre-tested to ensure scanning does not cause denial of service to critical systems that include sentinel events involving medical systems.

MTF selection must be approved from a top down approach, vetted and measured for consistency across the business organization. This aggressive strategy is far more reaching than the often misleading organizational view in which the MTF checklist is considered a one stop shopping tool for benchmarking compliance. Checklists should be viewed as an organizational entry point that requires modification with the understanding that it only provides initial base line coverage. A clear signal that the MTF was not vetted correctly or an initial risk assessment was not conducted on the system(s) under review is when compliance analysts are expected to negotiate modification or changes on-site to base coverage compliance controls from the MTF.

Outside of the initial checklist, base coverage is the starting point, where compliance analysts can provide advanced up-to-date emerging issues and real time recommendations while developing professional compliance analyst/site security officer discussion in keeping systems secure. This approach also requires the compliance analyst to provide valid, accepted, and peer reviewed substantiated information for any recommendation or remediation outside the baseline MTF checklist.

One area of caution for business contractual agreements in outsourcing information security is that specific clauses included in the vendor statement of work that clearly indicate any recommended remediation be developed fresh from the assessment MTF and not extracted from prior assessments, customers, or contract databases. This tactic is often used by outsourcing contractors and vendors to reduce the significant overhead required in producing a thorough and accurate assessment report, leaving susceptibility for significant gaps in compliance.

The selection of an MTF must be measurable and periodically statistically reviewed in identification of effectiveness and accuracy in application of compliance controls. Another overlooked factor is that enterprise security controls were adopted, yet not thoroughly understood at an accuracy level by both compliance analysts and on-site security officers. An insufficient understanding in validation of the interview, test, and observation assessment phase will result in the inability to produce validated measured findings or artifacts needed for development of the qualitative (narrative) and quantitative (measured) assessment report. This overlooked critical factor will lead to inconsistency turmoil across the entire enterprise by both compliance analysts and security officers and may cause more damage than good as the entire enterprise may be operating on a false sense of enterprise security culture thus increasing susceptibility to threats. Leading causes of, or key factors that may be present during inconsistent MTF application across a business enterprise are shown in Table 1.

Table 1. Issues Impacting Consistency in Deployment of Security Controls

Inaccurate Trending AnalysesOutliers in statistical analyses are not discussed, reviewed, or remediated or no use of a true statistical application; i.e., SPSS
Automated ToolsLack of implementation of automated scanning engine tool due to funding or inability to recognize available open source tools
Training GeneralLack of cradle to grave training on the MTF to include all applicable controls. Inability by compliance analysts and on-site security officers to demonstrate control knowledge. No use of job qualification boards (JQR) or personal qualification standards (PQS).
Training, Education and CertificationBasing auditor expertise on a few specific areas; i.e., education, certification, or experience without regard for past contribution to security field, writing, leadership, and organizational skills.
Improper MTF control applicationThe unauthorized removal or modification of base line checklist security controls assessment findings or injection of "not applicable" or "not tested" without proper authority.
Review ProcessNo real measurement of how long the MTF review process should take or rushed on-site review process impacting accuracy of report.
Artifact ReviewBasing compliance decisions on compliance artifacts during pre-reviews from database repository staging areas where documents have improper change control and revisions that mismatch what is actually practiced and deployed in the business.

Compliance cannot be accurately measured through a single methodology. Rather, a mixed approach of both quantitative and qualitative reporting is required in order to provide the agency a benchmark on how to improve continuous monitoring status through the use of statistical regression. Scores must be provided upon completion of the assessment in operational, administrative, and technical areas which can averaged or weighed. Specific guidelines on behalf of the agency assessment must present compliance results as either "noncompliant or compliant" without regard to any non-substantiated, biased, or unqualified opinion outside of the specific approved MTF.

Information assurance and security relies heavily on measurement, metrics, and visualization from management. Although often overlooked, one concept that has been in existence for over twenty years, is that "quality is measured and what gets measured gets done" and if the MTF cannot be measured, it "cannot be improved" (Peters, 1987 p. 90). Security metrics assist in "replacing fear, uncertainty, and doubt" and applied security visualization allows a business "to see it with their own eyes where a picture is worth a thousand log entries" (Jaquith, 2007; Marty, 2009).

The non-utilization of the many available security metrics, including the Center for the Internet Security (CIS) measurement formulas, in business functions of incident management, vulnerability management, patch management, application security, configuration management, and future metric recommendations leaves significant gaps in business assurance compliance. The Information Assurance Technology Analyses Center (IATAC) State of the Art Report in measuring cyber security and information assurance is a must read for all business and federal agencies Chief Information and Security Officers.

In order to obtain an accurate and consistent compliance strategy, security management professionals applying or measuring information security or assurance outcomes must move toward the ongoing and emerging analysis evaluation of security research. If there is an indication that any one security control is not fully understood by management, compliance analysts or on-site security officers for MTF implementation, the business or agency should consider rapid remedial intervention. Without correcting a skewed MTF compliance approach, it is realistically likely that the approach is inadequate in keeping business systems and data secure.

CIS Consensus Information Security Metrics (2009). Retrieved October 1st, 2009 from
Gallegos, F., Senft, S., Manson, D., and Gonzales C. (2008). Information Technology Control and Audit, 3rd ed. New York: Auerbach Publications.
Jaquith, A. (2007). Security Metrics: Replacing Fear, Uncertainty, and Doubt. MA: Pearson. Marty, R. (2009). Applied Security Visualization. MA: Pearson.
Peltier, T. (2004). Information Security Policies and Procedures: A Practitioner's Reference. New York: Auerbach Publications.
Peters, T. (1987). Thriving on Chaos, Handbook for a Management Revolution. IL: Video Publishing House.
Smith, J. (2009). Wanted: Engaged information security professionals for damage control and compliance. IA Newsletter, 12, (3), p. 16-18.
Whitman, M., and Mattord, J. (2009). Principles of Information Security, 3rd ed. Thomson: Canada.

About the Author
Jeffrey Smith has over 30 years of diverse experience in Information Technology and Security including analysis, implementation and testing of applications for both government and commercial organizations. Currently, he serves as a Federal Information Technology and Security Compliance Officer, has served in the position as a Federal Information Security Officer, and worked at two major defense contractors conducting project management and certification and accreditation.

Subscribe to Information Security Today

Powered by VerticalResponse

Share This Article

© Copyright 2011 Auerbach Publications