Introduction
This article is an extended version of the paper entitled “A Comprehensive Cybersecurity Audit Model to Improve Cybersecurity Assurance: The CyberSecurity Audit Model (CSAM)”, that was presented at the “2nd. International Conference on Information Systems and Computer Science - INCISCOS 2017” on November 24, 2017.
The initial paper introduced the CyberSecurity Audit Model (CSAM) and its design to the global scientific community. Furthermore, we now present the methodology of our case study research and the results from our Canadian post-secondary institution case study research.
Organizations are trying to protect cyber assets and implement cybersecurity measures and programs, but despite this continuing effort it is unavoidable to evade cybersecurity breaches and cyberattacks.
According to the Information Systems Audit and Control Association (ISACA), the origin of cybersecurity was published in a journal article in the early eighties, presenting the first proof of the concepts of self-replicating/self-propagating code linked to a computer worm. Pursuant to the fundamentals of the discipline defined by ISACA, cybersecurity is “The protection of information assets by addressing threats to information processed, stored and transported by internetworked information systems” - cybersecurity and information security are often mentioned interchangeably but cybersecurity is a component of information security. Proaño et al. (2017) highlight that IT auditors deal with subjectivity issues involved with emotions, technical skills or abilities in order to report audit findings and recommend the future implementation of knowledge-based systems for computer audits.
Our proposed CyberSecurity Audit Model (CSAM) has been designed to address the limitations and inexistence of cybersecurity controls to conduct comprehensive cybersecurity or domain-specific cybersecurity audits.
2. Methodology
In our previous paper, we reviewed relevant literature related to general cybersecurity, cybersecurity best practices, cybersecurity audits and cybersecurity frameworks from Protiviti, Deloitte, ISACA, ISO 27001, ISO 27001, NIST Cybersecurity Framework, Donaldson et al., Hollingsworth, Ross and Khan.
The CyberSecurity Audit Model (CSAM) has been tested, implemented and validated along with the Cybersecurity Awareness TRAining Model (CATRAM) in a Canadian higher education institution. The research project assessed the cybersecurity organizational strategy, implemented the CyberSecurity Audit Model (CSAM) and delivered cybersecurity awareness training to more than one hundred participants based on the Cybersecurity Awareness TRAining Model (CATRAM).
The case study research included several phases like plan, design, preparation, collection, analysis, sharing and dissemination. We intended to perform qualitative research by utilizing interpretive material practices such as online and paper surveys, interviews, classroom and online training and analysis of documentation, processes and procedures of the target institution. The organization provided their staff time to support the case study research, resources to conduct the cybersecurity audit, the provision of classroom space and time, computer use, Internet access for the delivery of the cybersecurity awareness training courses, the access to their computer systems to conduct the research and to design the online courses in their Learning Management System (Moodle).
3. The Cybersecurity Audit Model (CSAM)
The CyberSecurity Audit Model (CSAM) proposed in this article, is a new exhaustive model that encloses the optimal assurance assessment of cybersecurity in any organization and it can verify specific guidelines for Nation States that are planning to implement a National Cybersecurity Strategy (NCS) or want to evaluate the effectiveness of its National Cybersecurity Strategy or Policy already in place. The CSAM can be implemented to conduct internal or external cybersecurity audits, this model can be used to perform single cybersecurity audits or can be part of any corporate audit program to improve cybersecurity controls. Any audit team has either the options to perform a full audit for all cybersecurity domains or by selecting specific domains to audit certain areas that need control verification and hardening. The CSAM has 18 domains; domain 1 is specific for Nation States and domains 2-18 can be implemented at any organization. The organization can be any small, medium or large enterprise, the model is also applicable to any Non-Profit Organization (NPO).
The CyberSecurity Audit Model (CSAM) contains overview, resources, 18 domains, 26 sub-domains, 87 checklists, 169 controls, 429 sub-controls, 80 guideline assessments and an evaluation scorecard.
Overview
This section introduces the model organization, the working methodology and the possible options for implementation.
Resources
This component provides links to additional resources to help understanding some of the cybersecurity topics:
Cybersecurity: NIST Computer Security Resource Center, Financial Industry Regulatory Authority (FINRA) cybersecurity practices and Homeland Security cybersecurity.
National Cybersecurity Strategy (NCS): North Atlantic Treaty Organization (NATO) cybersecurity strategy, European Union Agency for Network and Information Security (ENISA) cybersecurity strategy and Organisation for Economic Co-operation and Development (OECD) comparative analysis of national cybersecurity strategies.
Governance: PricewaterhouseCoopers Board cybersecurity governance and MITRE cybersecurity governance.
Cyber Assets: NERC critical cyber assets.
Frameworks: Foresite common cybersecurity frameworks, United States Computer Emergency Readiness Team (US-CERT) framework and ISACA’s implementing the NIST cybersecurity framework.
Architecture: Trusted Computer Group (TCG) architect’s guide and US Department of Energy’s IT security architecture.
Vulnerability Management: SANS vulnerability assessment and Homeland Security vulnerability assessment and management.
Cyber Threat Intelligence: SANS - Who’s using cyberthreat intelligence and how?
Incident Response: Computer Security Incident Response Team (CSIRT) frequent asked questions.
Digital Forensics: SANS forensics whitepapers.
Awareness: National Cyber Security Alliance - Stay safe online and PCI DSS -Best practices for implementing security awareness program.
Cyber Defense: SANS- The sliding scale of cybersecurity.
Disaster Recovery: Financial Executives International (FEI) Canada - Cybersecurity and business continuity.
Personnel: Kaspersky - Top 10 tips for educating employees about cybersecurity.
Domains
The CSAM contains 18 domains. Domain 1 has been designed specifically for Nations States and domains 2-18 are applicable to any organization.
Sub-domains
All domains have at least one sub-domain but in certain cases there might be several sub-domains per domain.
The sub-domains are:
Cyberspace
Governance
Strategy
Legal and Compliance
Cyber Asset Management
Cyber Risks
Frameworks and Regulations
Architecture
Networks
Information
Systems
Applications
Vulnerability Management
Threat Intelligence
Incident Management
Digital Forensics
Awareness Education
Cyber Insurance
Active Cyber Defense
Evolving Technologies
Disaster Recovery
Onboarding
Hiring
Skills
Training
Offboarding
Controls
Each domain has sub-domains that are assigned a reference number. Controls are identified by clause numbers and an assigned checklist. In order to verify the control evaluation, the cybersecurity control is either in place or inexistent.
Checklists
Each checklist is linked to a specific domain and the subordinated sub-domain. The checklist verifies the validity of the cybersecurity sub-controls in alignment with a control clause. The cybersecurity auditors have the option to collect evidence to verify the sub-control compliance.
Sub-Controls
The Sub-Controls are evaluated using the checklists.
The assessment of each sub-control can be in compliance, with a minor nonconformity or with a major nonconformity:
-Compliant: The cybersecurity sub-control is active and aligned with the specific requirements.
-Minor Nonconformity: The cybersecurity sub-control has not been fulfilled and it represents a minor risk.
-Major Nonconformity: The cybersecurity sub-control does not exist or it is a complete failure and it represents an unacceptable risk.
Guideline Assessment
The guideline assessment only applies to the Nation States domain. The guidelines are evaluated for cybersecurity culture, National Cybersecurity Strategy (NCS), cyber operations, critical infrastructure, cyber intelligence, cyber warfare, cybercrime and cyber diplomacy.
Evaluation Scorecard
The control, guideline and sub-control evaluation is calculated after the audit has been completed. The evaluation consists in assigning scores and ratings for each control, guideline and sub-control.
We calculate the final cybersecurity maturity rating of the Nation States domain by using the following criteria. The score can be mapped to a specific maturity level:
Immature (I): 0-30
The Nation State does not have any plans to manage its cyberspace. A National Cybersecurity Strategy (NCS) or Policy is inexistent.
Developing (D): 31-70
The Nation State is starting to focus on national cybersecurity. If technologies are in place, the Nation State needs to focus on key areas to protect cyberspace.
Mature (M): 71-90
While the Nation State has a mature environment. Improvements are required to the key areas that have been identified with weaknesses.
Advanced (A): 91-100
Nation State has excelled in national cybersecurity and cyberspace practices. There is always room for improvement. Nation State could become an international leader and help other Nation States with cybersecurity and cyberspace matters.
And for domains 2-18, we calculate the final cybersecurity maturity rating of any organization by using the following criteria. The score can be mapped to a specific maturity level:
Immature (I): 0-30
The organization does not have any plans to manage its cybersecurity. Controls for critical cybersecurity areas are inexistent or very weak. The organization has not implemented a comprehensive cybersecurity program.
Developing (D): 31-70
The organization is starting to focus on cybersecurity matters. If technologies are in place, the organization needs to focus on key areas to protect cyber assets. Attention must be focused towards staff, processes, controls and regulations.
Mature (M): 71-90
While the organization has a mature environment. Improvements are required to the key areas that have been identified with weaknesses.
Advanced (A): 91-100
The organization has excelled in implementing cybersecurity best practices. There is always room for improvement. Keep documentation up-to-date and continually review cybersecurity processes through audits.
Cybersecurity Readiness Evaluation
We calculate the overall organizational cybersecurity readiness by using the following criteria. The score can be mapped to a specific cybersecurity readiness level:
Immature (I): 0-30
The organization does not have any plans to manage its cybersecurity. Controls for critical cybersecurity areas are inexistent or very weak. The organization has not implemented a comprehensive cybersecurity program. The Cybersecurity readiness is inexistent at this level.
Developing (D): 31-70
The organization is starting to focus on cybersecurity matters. If technologies are in place, the organization needs to focus on key areas to protect cyber assets. Attention must be focused towards staff, processes, controls and regulations. The Cybersecurity readiness is developing at this stage.
Mature (M): 71-90
While the organization has a mature environment. Improvements are required to the key areas that have been identified with weaknesses. The Cybersecurity readiness is at a mature level.
Advanced (A): 91-100
The organization has excelled in implementing cybersecurity best practices. There is always room for improvement. Keep documentation up-to-date and continually review cybersecurity processes through audits. The Cybersecurity readiness is at an advanced level, but the organization must continually update its cybersecurity strategy at all times.
4. Results
The research results were measured based on the implementation outcome of the CSAM and CATRAM models in our target institution. The organizational cybersecurity audit results are presented as an overall cybersecurity rating classified by the model’s domains.
The organization is starting to focus on cybersecurity matters. If technologies are in place, the organization needs to focus on key areas to protect cyber assets. Attention must be focused towards staff, processes, controls and regulations. The final cybersecurity maturity rating is positioned at the “Developing” level with a score of 51% (Table 1).
In addition, the radar chart (Figure 1) presents the domain evaluation results in order to provide the overall organizational cybersecurity readiness.
5. Discussion
This study presents the design of the CyberSecurity Audit Model (CSAM). The aim of this model is to introduce a cybersecurity audit model that includes all functional areas, in order to guarantee an effective cybersecurity assurance, maturity and cyber readiness in any organization or any Nation State that is auditing its National Cybersecurity Strategy (NCS). This model was envisioned as a seamless and integrated cybersecurity audit model to assess and measure the level of cybersecurity maturity and cyber readiness in any type of organization, no matter in what industry or sector the organization is positioned. Moreover, by adding guidelines assessment for the integration of a national cybersecurity policy, program or strategy at the country level.
Table 1
Final Cybersecurity maturity rating table
Figure 1
Final Cybersecurity Readiness Chart
Many cybersecurity frameworks are mostly oriented towards a specific industry like the “PCI DSS” for credit card security, the “NERC CIP Cyber Security” for the bulk power system or the “NIST Cybersecurity Framework” for protecting national critical infrastructure. But, all the existing frameworks do not provide a one-size fits all for planning and conducting cybersecurity audits. The necessity to mapping against specific cybersecurity frameworks is because of regulatory requirements, to satisfy the demands of industry regulators, to comply with internal or external audits, to satisfy business purposes and customer requirements or simply by improving the enterprise cybersecurity strategy.
Table 2
Comparison of some cybersecurity audit models
| Audit Model or Framework | Description |
|---|
| The Cybersecurity Framework (CSF) Version 1.1: NIST (2017) | The initial version was conceived in 2014 to improve cybersecurity of critical infrastructure. The version 1.1 manages cybersecurity risks for critical infrastructure. It is composed of the Framework Core, the Framework Implementation Tiers and the Framework profiles. The Framework Core includes five functions - Identify, Protect, Detect, Respond and Recover; then each of these functions have categories and subcategories. In addition, the Core contains Informative resources like cybersecurity standards, guidelines and best practices. The Tiers define cybersecurity context organized from partial to adaptive tier. The Profile presents the outcomes based on organizational needs. The current profile can later be compared with a target profile. |
| The Audit First Methodology: Donaldson et al. (2015) | This methodology considers other cybersecurity controls and leaves preventive control execution until the end. This audit includes five different phases: Threat analysis: This phase identifies Confidentiality, Integrity and Availability (CIA) threats that may impact IT and corporate data. Threat impact and indicators are defined. Audit controls: It includes the design of threat audit controls. Forensic controls: This phase helps to implement the required forensic controls for the enterprise cybersecurity functional areas: Systems administration Networks Applications Endpoints, servers and devices Identity, authentication and access Data protection and cryptography Monitoring, vulnerabilities and patch management Availability, disaster recovery and physical protection Incident management Supply chain and asset management Policy, audit, e-Discovery and training Detective controls: Detective controls are designed to alert, detect, stop and repel cyberattacks. Preventive controls:These controls block undesired activities and stop them from ocurring. |
| The CyberSecurity Audit Model (CSAM): Sabillon et al. (2017) | The CSAM comprises overview, resources, 18 domains, 26 sub-domains, 87 checklists, 169 controls, 429 sub-controls, 80 guideline assessments and an evaluation scorecard. Domain 1-Guideline assessment are specific for Nation States and domains 2-18 are applicable to any type of organization. Certain domains have specific sub-domains where controls are evaluated. Then the checklists verify compliance about specific sub-controls based on domain/sub-domain. The scorecard results determine the domains rating and score that will produce the overall cybersecurity maturity rating. |
We compared our model in Table 2 to highlight the main features against “The Cybersecurity Framework (CSF) Version 1.1: NIST (2017)” and “The Audit First Methodology:Donaldson et al. (2015)”. The CSAM is not for a specific industry, sector or organization - On the contrary, the model can be utilized to plan, conduct and verify cybersecurity audits everywhere. The CSAM has been designed to conduct partial or complete cybersecurity audits either by a specific domain, several domains or the comprehensive audit for all domains.
6. Conclusions
This study introduces the CyberSecurity Audit Model (CSAM) design and all its components, the aim of this model is to evaluate and measure the cybersecurity assurance, maturity and cyber readiness in any organization. In addition, the model can evaluate the effectiveness of cybersecurity guidelines for any Nation State linked to its national cybersecurity strategy or policy.
The CSAM was tested, implemented and validated along with the Cybersecurity Awareness TRAining Model (CATRAM) in a Canadian higher education institution. A research case study is being conducted to validate both models and the findings will be published accordingly.
Since there aren’t universal acceptance or standardization in terms of defining cybersecurity audit scopes, aims and domains, further research is required and encouraged in the cybersecurity areas of assurance and audits.
