“System Operations” is vague, we understand that. In a SOC 2 examination, systems operations refer to the following concepts and processes:
Use of detection and monitoring procedures (CC7.1)
Monitor for anomalies and analyze them to determine if they represent security events (CC7.2)
Evaluate security events to determine if they represent security incidents (CC7.3)
Respond to security incidents (CC7.4)
Recover from security incidents (CC7.5)
This blog post will provide a detailed overview of each criterion, key concepts assessed, typical evidence requests for each criterion, and the ByteChek difference.
Overview of CC7.0 (System Operations)
CC7.1 and CC7.2
CC7.1: To meet its objectives, the entity uses detection and monitoring procedures to identify (1) changes to configurations that result in the introduction of new vulnerabilities, and (2) susceptibilities to newly discovered vulnerabilities.
CC7.2: The entity monitors system components and the operation of those components for anomalies that are indicative of malicious acts, natural disasters, and errors affecting the entity’s ability to meet its objectives; anomalies are analyzed to determine whether they represent security events.
Key Concepts Assessed in CC7.1:
Typical evidence requests: System-generated reports from your vulnerability scan tool (number of vulnerability scans will be determined by the frequency of scans, quarterly is normal). Each report should show the scope of the vulnerability scan, identified vulnerabilities, and a description of the identified vulnerabilities. Along with the report, expect to provide evidence of a follow-up scan or remediation evidence for the critical and high vulnerabilities identified. Remediation should be completed within the specified timeframe in your information security policy.
With ByteChek, we will work closely with you to determine the tools and technologies your organization is utilizing to conduct vulnerability scans. We understand that a one-size-fits-all approach to evidence requests doesn’t work so we won’t ask you for evidence until we understand the tools and processes you’re following. If you’re using Amazon Inspector, our AWS integration will automatically collect the necessary evidence.
Bonus: If you’re using Amazon Inspector, we will provide you with recommendations to automate the vulnerability scan and remediation process. Our deep AWS expertise will turn this traditionally manual control into a set it once and forget about control, freeing up your team to focus on your business.
Security Incident and Event Management (SIEM) Solutions and Procedures:
Typical evidence requests: System-generated screenshots or observations of SIEM configurations showing the assets that are reporting to the SIEM solution, types of events being logged, and any alerting configured for each log event type. You should also be prepared for your auditors to sample a selection of instances or servers and request evidence that SIEM agents are installed or other system-generated evidence to prove that each instance is reporting to the SIEM solution.
With ByteChek, we eliminate the screenshots by integrating directly with your SIEM tool. Our integration with your SIEM solution and AWS will determine whether all in-scope assets are reporting to the SIEM solution.
Additionally, the ByteChek platform integrates with AWS to check for your use of AWS CloudTrail, a vital service on AWS to provide increased visibility into activity in your AWS account by recording information about AWS API calls made on the account.
S3 Bucket Security:
Typical evidence requests: Let’s be honest here, status quo auditors are not requesting evidence for S3 bucket security, despite the repeated S3 bucket breaches we read about in the news. Check out your latest SOC 2 report or one of your vendors that are hosted on AWS, do you see any controls addressing S3 bucket security?
With ByteChek, we are AWS experts and our SOC 2 examinations reflect that. Our integration into AWS checks the logging configuration of Amazon S3 buckets to ensure that detailed access logs are delivered hourly and include details about each request, such as the request type, the resources specified in the request, and the time and date the request was processed.
Additionally, with ByteChek, our integration checks buckets in Amazon S3 that have open access permissions. Bucket permissions that grant list access to everyone can result in higher than expected charges if objects in the bucket are listed by unintended users at a high frequency. Bucket permissions that grant Upload/Delete access to everyone create potential security vulnerabilities by allowing anyone to add, modify, or remove items in a bucket. This check examines explicit bucket permissions and associated bucket policies that might override the bucket permissions.
Typical evidence requests: System-generated screenshots or observations detailing the playbooks, stacks, cookbooks, or other configuration elements for your infrastructure. Be prepared to explain how tools such as Ansible, Terraform, CloudFormation, OpsWorks, or Chef work, these are not tools status quo auditors are familiar with. Evidence should demonstrate how changes are implemented along with the process to spin up new instances or servers.
With ByteChek, we understand that most modern architectures are built using an Infrastructure as Code (IaC) strategy. Our AWS integration allows us to plug directly into CloudFormation and AWS OpsWorks (which integrates with Chef and Puppet) to assess how this IaC strategy is configured and managed. Through the elimination of observations, screenshots, and explanations of technology, this control is completely automated with ByteChek.
Typical evidence requests: System-generated screenshots or observations that show the infrastructure monitoring tool you are using such as Datadog, Cloudwatch, Splunk, etc. Be prepared for your auditors to select a sample of instances and request evidence that each instance is being actively monitored by these infrastructure monitoring tools. This evidence should show which metrics are being monitored such as CPU utilization, memory utilization, disk I/O, etc. This often is captured during onsite observations as the screenshots will require an explanation for your auditors.
With ByteChek, we integrate with AWS which means we obtain the evidence directly from CloudWatch. We are AWS experts so we understand that every EC2 instance has basic monitoring enabled evaluating CPU load, disk I/O, and network I/O metrics at five-minute intervals. Our platform will determine any additional monitoring performed by Splunk or other tools your organization is using. We won’t require onsite observations to explain what is being monitored and why.
CC7.3: The entity evaluates security events to determine whether they could or have resulted in a failure of the entity to meet its objectives (security incidents) and, if so, takes actions to prevent or address such failures.
Key Concepts Assessed in CC7.3:
Security Incident Policies and Procedures
Typical evidence requests Your security incident policies and procedures that include approvals and version history. Your auditors may provide templates or websites where you can generate a policy that will sufficiently address this criterion. Be prepared to explain the process or tools used to communicate these policies and procedures to employees (i.e. acknowledged upon hire, stored on internal document repository, etc.)
With ByteChek, our platform is built with an intuitive information security policy generator which will help you quickly create a robust and detailed policy that includes security incident policies and procedures. If you already have a policy, you can upload it directly to our platform to address this control. We also take care of the communication to users, your employees can use the ByteChek platform to read and acknowledge their understanding of the information security policy (and other applicable policies and procedures).
Information security event response
Typical evidence requests: System-generated listing from your ticketing system, SIEM, or tracking tool showing all information security events that occurred during the reporting period. Your auditors will select a sample of events (i.e. tickets) and request evidence for each event to determine that each event was properly classified and responded to according to your security incident response policies. This concept or control area is generally complex and sensitive which requires detailed conversations with your auditors.
With ByteChek, we integrate directly with your SIEM solutions and ticketing system (e.g. JIRA) to determine whether or not security events were assessed, assigned a severity, and classified appropriately according to the information security policy created on the ByteChek platform. We understand that security events are nuanced and expect to discuss this concept with your team during our review but rest assured that the evidence collection for this control will be automated.
CC7.4 and CC7.5
CC7.4 The entity responds to identified security incidents by executing a defined incident response program to understand, contain, remediate, and communicate security incidents, as appropriate.
CC7.5 The entity identifies develops and implements activities to recover from identified security incidents.
Key Concepts Assessed in CC7.4 and CC7.5:
CC7.4 and CC7.5 both specifically discuss “identified security incidents.” At Bytechek, we evaluate these criteria using the AICPA definition for system incidents, which describes these as significant incidents that:
(a) where the result of controls that were not suitably designed or operating effectively to achieve one or more of the commitments and system requirements; or
(b) otherwise resulted in a significant failure in the achievement of one or more of those commitments and system requirements.
Additionally, in section 3 of your SOC 2 report, there is a requirement to disclose any significant incidents that meet the threshold above. The AICPA states that disclosures should not be made if a discussion of the incident would result in a higher security risk to your organization. A good rule of thumb here is if you had a press release or mass email to customers notifying of an incident, it would likely require disclosure.
The requirement in section 3 directly relates to CC7.4 and CC7.5, with this in mind we assess CC7.4 and CC7.5 through the lens of only “identified” significant incidents or incidents that have risen to the level of disclosure.
What does this mean for you and your report? If there was no identified security incident, the controls and tests here will not be tested. In simple terms, we will document that we did not test the security incident control because there were no security incidents to test.
This isn’t something we decided on our own, the AICPA recognizes that there will be an inability to test CC7.4 and CC7.5. In paragraph 4.88 of the SOC 2 guide (sidenote: our CFO, Jeff Cook, is listed as a contributor in this guide) the AICPA provides this example of CC7.4 and CC7.5 not being tested for a service organization:
“Example Service Organization’s description of its payroll system discusses its cybersecurity incident response and recovery plan (CIRP), which includes the controls implemented and operated to respond to and recover from security incidents. Example Service Organization’s CIRP includes procedures to help understand, contain, monitor, or eradicate a security incident; restore normal business operations in a timely manner with minimal, or no, business interruption or loss of data; and communicate with affected parties.
However, during the period [date] through [date], Example Service Organization did not experience a security incident that would warrant the operation of the response and recovery processes and controls within its CIRP. Because those controls did not operate during the period, we were unable to test and did not test, the operating effectiveness of those controls as evaluated using trust services criteria CC7.4, The entity responds to identified security incidents by executing a defined incident response program to understand, contain, remediate, and communicate security incidents, as appropriate, and CC7.5, The entity identifies, develops, and implements activities to recover from identified security incidents.”