Critics of the 2002 Federal Information Security Management Act (FISMA) contend that it is a paper process that fails to bear the fruit of information assurance. The reason they give us is that FISMA requires an excessive amount of paperwork with little to no security benefit. They are correct in their assertion, but wrong in their reasoning.
FISMA requires Federal agencies to follow the guidance established by the National Institute of Standards and Technology (NIST). The guidance established by NIST requires agencies to create system security plans, contingency plans, risk assessments, security assessments, incident response plans, as well as every conceivable procedure on any aspect of security. Obviously, a substantial amount of paperwork is necessary according to the NIST guidance with its FISMA provenance. All of these documents, when combined form a systems accreditation package, which can easily exceed 200 pages.
Critics point out that all of this paper is irrelevant when a system is compromised. From their perspective, all of the effort put into the paper did little to protect the system. They claim it is wiser to expend the resources on fixing the real security problems as opposed to wasting it on paper. This seems like a logical argument, but it is unfortunately seriously flawed.
The complexity of modern information systems is well known. It is regrettable that a law such as FISMA is necessary to compel the owners of complex systems to implement modest system engineering practices. Much of the documentation required by FISMA supports proper development and engineering techniques.
Security documentation produced for a system is relevant from a systems engineering perspective. Sadly, it is less relevant when it is incomplete. It has been this author's experience that much of the system security documentation is woefully inadequate. Boundaries are often not explicitly identified and the enumeration of information flows are lacking. The depth and breadth of security control descriptions are incomplete. The following items are just some of the common weaknesses observed:
- Procedures: It is amazing to me that procedures are often lacking. People need to have explicit guidance on what to do and how it should be done. A lack of well-defined procedures allows inconsistency at minimum and inappropriate actions during a critical time at worst.
- Auditing: This is an essential control that should be documented in a system security plan. Indeed, auditing is an important and necessary component when investigating a breach or system misuse. What is often missing regarding this control is the extent of the auditing implemented. Auditing could be used in a plethora of opportunities for security purposes, such as routers, firewalls, workstations, servers, databases, and other applications. System security plans rarely identify audit generation, collection, and analysis from anything other than a server.
- Ports and protocols: Many organizations deploy an intrusion detection system (IDS), but fail to identify all of the ports and protocols in use on a system. The IDS analyst is burdened with validating what should be allowed to run and what should not. Knowing which ports and protocols are authorized will improve the ability of an IDS to detect internal and external attacks. This level of detail is essential in accreditation packages, yet is not always included.
- Processes: It is surprising that software running is inadequately identified. This is not to suggest that a software package is not identified, but the descriptive functionality is frequently missing. For instance, this author once identified Apache running on a Windows domain controller. None of the security staff responsible for documenting and monitoring the system could explain it. Further investigation revealed that Apache was running as an element of a system management package. The problem here suggests that not only did security management not understand the software implemented, but the Apache deployment was not properly configured and potentially opened a hole in the system. A lack of attention to detail is detrimental to information assurance.
- Superficial security assessments: Security tools provide an excellent way to quickly gather a substantial amount of vulnerability information. But, no security tool is a silver bullet. In fact, a collection of security tools does not approach perfection either. Excessive reliance on tools for security testing has supplanted manual methods, such as the flaw hypothesis, which might find critical weaknesses outside of the scope of the tools used.
Black Box Mentality
Why is it that people forego detailed analysis and documentation of these systems? One possibility is that people have become accustomed to a black box mentality of information technology (IT) products. A black box is something that people use with little understanding of it beyond its basic implementation. User friendliness is a common mantra in IT product development. With products becoming easier to operate, users have been required less and less to understand how IT works. Consider plug-and-play. Once upon a time, it was necessary to hand code settings in configuration files to get hardware devices to play nicely with the system. Today, thanks to plug-and-play people have little idea where the configurations are stored and how many there may be. This advancement in technology is but one example of black box mentality people have toward IT. People often do not understand how IT works, but they expect it to work properly. If the prevailing mentality is a lack of understanding, how can these same people hope to develop security documentation with sufficient detail? In effect, advancements in technology have lobotomized IT professionals.
The black box mentality not only perpetuates a lack of understanding, but creates a situation where documentation appears counter-intuitive. Consider a rental car company. In this case an automobile is nothing more than a black box. The automobiles are integrated into the workflow of individuals renting and the company providing the service. Few people really understand the complexity in modern vehicles. Yet, the vehicles are expected to work. There is no need for a rental car company to document all of the operational aspects of a car. It is accepted that it will work as advertised. It seems that people have the same perception about IT hardware and software. People simply expect it to properly function as advertised. Documenting all of the detailed aspects of a software package or hardware device seems trivial. However, it is evident that IT systems are very complex and necessitate a deeper level of understanding to ensure that the security services of a system are met. It is dangerous for IT folks to assume any less.
Suppose that the individuals tasked with developing security documentation did so with a real understanding of the system. Furthermore, assume that their resulting documentation precisely describes the depth and breadth of the security controls. Might this increase security? It should, with the caveat that the organization follows their detailed documentation. A sufficiently documented system security plan, with the controls properly implemented on the system, will result in a system with many countermeasures to attack. This does not mean a system will be impervious to attack. In contrast, sufficient management, operational, and technical controls should exist to assist in the prevention, detection, correction, recovery, deterrence, and compensation of weaknesses and exposures in the system. Well developed system security documentation increases its relevance by assisting human responses to security incidents and contingencies.
Sufficiently documented system security represents a maturity step for an organization. Organizations that devote time and resources to compile meaningful security documentation with sufficient detail enable an in-depth understanding of the system. An in depth understanding of architecture intricacies helps to conceptualize system risk. When vulnerabilities emerge, a concrete understanding of the system enables creative solutions to counter perceived and material threats. Quality documentation therefore facilities better risk management, which is the basis of information assurance.
Presently, the view advocated here is just as anecdotal as the FISMA detractors. Rather than simply argue a position, it is more beneficial to conduct an investigation to prove these points. Nevertheless, it is difficult to argue the positive implications to system security when the documentation exists with sufficient depth and breadth. It is more likely that maturity will accompany the security program and result in benefits that will confound the critics.
The problem with FISMA is not the amount of documentation required, but its quality. The lack of sufficient detail makes designing, implementing, and monitoring system security problematic at best. I speculate that many of the individuals developing the documentation lack the proper skills, as those of a systems engineer, to consider the importance of the depth and breadth of detail needed. This is not their fault, but rather a factor of our black box mentality toward commodity information technology products.
About the Author
Sean M. Price (CISA, CISSP) is an independent security researcher and consultant living in northern Virginia. He specializes in designing and evaluating organizational information assurance programs and system security architectures. Research interests include insider threat, information flows, and applications of artificial intelligence to information assurance problems. Prior publications include the
Information Security Management Handbook, Official (ISC)2 Guide to the CISSP CBK, IEEE Computer magazine, as well as other journals and conferences.