Facilitator: **Introduction:** Welcome to today's Incident Response Tabletop Exercise. As the facilitator, I will guide you through the scenario and provide feedback and challenges as needed. Your role is to respond as you would in a real-world situation, making decisions and taking actions as a team. It is important to take time to discuss possibilities and actions collectively. I'm also here to help guide you, so feel free to ask me for advice at any time. This exercise is designed to test your response to a potential cybersecurity incident affecting the Municipal Water Authority of Aliquippa. We will focus on your ability to detect, contain, and mitigate threats, particularly those impacting your operational technology systems. Let's work together to ensure the safety and reliability of your critical infrastructure. **Setting the Stage:** The current date is December 10, 2024, at 10:23 AM. The Municipal Water Authority of Aliquippa (MWAA) is a public water utility serving approximately 6,615 customers in Aliquippa, PA, and surrounding areas. The exercise involves the Water Operations and IT departments, with participants including Robert, the CISO; Mike, the IT Manager; Marcelo, the Cybersecurity Manager; Todd, the OT/Plant Manager; Foster, the PR Manager; and Tom, the Legal Advisor. The environment consists of an OT network with internet-connected PLCs, SCADA systems, and manual backup controls. The objective is to test your response to a PLC compromise, evaluate manual operation procedures, and assess cybersecurity measures for OT systems. **Scenario:** At 10:30 AM, an alarm is triggered indicating unauthorized access to a PLC controlling water pressure at one of the booster stations. The PLC, identified as `PLC1` with IP address `10.10.0.10`, is part of the OT network segment. Shortly after the alarm, pressure readings from the booster station begin to show unusual fluctuations, raising concerns about the stability of the water supply. The SCADA system, which monitors and manages the PLCs, logs these anomalies and sends alerts to the Incident Response team. As the situation unfolds, your team must determine the cause of the unauthorized access, assess the impact on operations, and implement appropriate response measures to ensure the continued safety and reliability of the water supply. This exercise is based on your provided Incident Response Plan, and you will be graded on how well you follow it. Let's begin by addressing the initial alarm and pressure fluctuations. What actions will your team take? Your Team: Notify necessary internal personnel/stakeholders. Also, show me the firewall logs (in log format) to help determine if the unauthorized access was remote. Are the fluctuations causing any levels outside of regulatory standards or any safety issues at this point? What kind of fluctuations? Basically, what is the impact at this point. Facilitator: As you initiate the notification process, your team promptly alerts key internal personnel and stakeholders about the incident. This includes notifying Robert, the CISO; Mike, the IT Manager; Marcelo, the Cybersecurity Manager; Todd, the OT/Plant Manager; Foster, the PR Manager; and Tom, the Legal Advisor. The notifications are sent through secure communication channels to ensure confidentiality and integrity. **Firewall Logs:** Here are the relevant firewall logs that may help determine if the unauthorized access was remote: ``` Dec 10 10:15:23 Firewall 192.168.1.2: INBOUND TCP connection attempt from 198.51.100.100 to 10.10.0.10:502 Dec 10 10:15:24 Firewall 192.168.1.2: INBOUND TCP connection allowed from 198.51.100.100 to 10.10.0.10:502 Dec 10 10:16:01 Firewall 192.168.1.2: INBOUND TCP connection attempt from 198.51.100.100 to 10.10.0.10:502 Dec 10 10:16:02 Firewall 192.168.1.2: INBOUND TCP connection allowed from 198.51.100.100 to 10.10.0.10:502 Dec 10 10:17:45 Firewall 192.168.1.2: INBOUND TCP connection attempt from 198.51.100.100 to 10.10.0.10:502 Dec 10 10:17:46 Firewall 192.168.1.2: INBOUND TCP connection allowed from 198.51.100.100 to 10.10.0.10:502 ``` The logs indicate repeated inbound TCP connection attempts from the IP address `198.51.100.100` to the PLC's IP address `10.10.0.10` on port 502, which is commonly used for Modbus communication. These connections were allowed, suggesting potential remote unauthorized access. **Impact Assessment:** At this point, the pressure fluctuations are causing concern but have not yet reached levels outside of regulatory standards. The fluctuations are characterized by sudden spikes and drops in water pressure, which could potentially lead to service disruptions if not addressed promptly. There are no immediate safety issues reported, but the situation requires close monitoring to prevent any escalation. Your team should continue to assess the situation and determine the next steps to contain and mitigate the incident. What actions will you take next? Your Team: Can we identify if that remote IP address is assigned to a legit remote staff. Also check the network logs for that IP address for any prior access, trying to see if that IP might be legitimate or potentially malicious. Let's also do a geo-location on that IP address. Immediately block the IP address. Also, block port 502 from outside access. Facilitator: Your team proceeds with the following actions to further investigate and mitigate the incident: 1. **Identify the Remote IP Address:** You check your records to determine if the IP address `198.51.100.100` is assigned to any legitimate remote staff or known external partners. After reviewing the list of authorized remote IP addresses, you confirm that this IP address is not recognized as belonging to any legitimate user or partner, suggesting it may be potentially malicious. 2. **Network Logs Analysis:** You examine the network logs for any prior access attempts from the IP address `198.51.100.100`. Here are the relevant network logs: ``` Dec 08 14:32:10 NetworkLog: INBOUND TCP connection attempt from 198.51.100.100 to 10.10.0.10:502 - BLOCKED Dec 09 09:45:22 NetworkLog: INBOUND TCP connection attempt from 198.51.100.100 to 10.10.0.10:502 - BLOCKED Dec 10 10:15:23 NetworkLog: INBOUND TCP connection attempt from 198.51.100.100 to 10.10.0.10:502 - ALLOWED ``` The logs indicate previous blocked attempts from the same IP address, which were unsuccessful until the recent successful connections. This pattern suggests a persistent attempt to gain unauthorized access. 3. **Geo-location of the IP Address:** You perform a geo-location lookup for the IP address `198.51.100.100`. The results indicate that the IP address is registered to a location in Eastern Europe, which is not associated with any of your known remote staff or partners. 4. **Mitigation Actions:** - **Block the IP Address:** You immediately configure the firewall to block all traffic from the IP address `198.51.100.100` to prevent further unauthorized access attempts. - **Block Port 502 from Outside Access:** You also update the firewall rules to block inbound traffic on port 502 from external sources, ensuring that only internal and authorized connections can communicate with the PLCs. With these actions, you have taken steps to mitigate the immediate threat. What would you like to do next? Your Team: What is the situation with the fluctuations and anomalies now that the IP address and port has been blocked? Facilitator: After blocking the IP address `198.51.100.100` and restricting external access to port 502, your team closely monitors the SCADA system and pressure readings from the booster station to assess the current situation. **Current Situation:** 1. **Pressure Fluctuations:** - The pressure readings from the booster station have stabilized significantly since the blocking actions were implemented. The previous spikes and drops in water pressure have ceased, and the system is now operating within normal parameters. This suggests that the unauthorized access was likely responsible for manipulating the pressure settings on the PLC. 2. **SCADA System Monitoring:** - The SCADA system continues to monitor the PLCs and reports no further anomalies or unauthorized commands being sent to `PLC1`. The alerts that were previously triggered have subsided, indicating that the immediate threat has been contained. 3. **Operational Impact:** - There have been no reports of service disruptions or safety issues resulting from the earlier fluctuations. The water supply remains stable, and regulatory standards are being met. The swift response and containment actions have prevented any significant impact on operations. With the situation now stabilized, your team can focus on further investigation and remediation efforts to ensure the security of the system and prevent future incidents. What actions would you like to take next? Your Team: Run inject 1 Facilitator: As part of the exercise, an inject is introduced to simulate an additional challenge for your team to address. **Inject 1: Alarm Triggered for Unauthorized PLC Access** Despite the recent stabilization, an alarm is triggered once again, indicating another attempt at unauthorized access to `PLC1`. This time, the attempt is detected internally, suggesting that the threat actor may be attempting to exploit another vector or has potentially compromised internal credentials. **Current Situation:** - The SCADA system logs show a new unauthorized access attempt to `PLC1`, but this time the source appears to be from within the internal network. The logs indicate an attempt to send configuration commands to the PLC, similar to the previous external access. - The internal IP address associated with this attempt is `192.168.1.101`, which corresponds to `IT-Workstation1`, an administrator workstation used for accessing both IT and OT networks. - The pressure readings remain stable for now, but the repeated access attempts raise concerns about potential insider threats or compromised internal systems. Your team needs to investigate this new development and determine the appropriate response to secure the network and prevent further unauthorized access. What actions will you take next? Your Team: Isolate 192.168.1.101. Run a full scan of that system (A/V, EDR, check logs, etc.). Analyze our SIEM logs to check for evidence of entry vectors, unusual activity, and further lateral movement. Facilitator: Your team takes decisive action to address the new unauthorized access attempt originating from `IT-Workstation1` with IP address `192.168.1.101`. Here are the steps you undertake: 1. **Isolation of `IT-Workstation1`:** - You immediately isolate `IT-Workstation1` from the network to prevent any further unauthorized access or potential lateral movement. This involves disconnecting the workstation from both the IT and OT networks, ensuring that it cannot communicate with other systems. 2. **Full System Scan:** - A comprehensive scan is initiated on `IT-Workstation1` using antivirus (A/V) and endpoint detection and response (EDR) tools. The scan aims to identify any malware, unauthorized software, or suspicious activities that may have compromised the workstation. - **A/V and EDR Scan Results:** - The A/V scan detects a malicious payload that appears to have been executed recently. The payload is identified as a remote access trojan (RAT), which could have allowed external control over the workstation. - The EDR tools provide additional insights, showing that the RAT was likely introduced through a phishing email that was opened by the workstation's user. 3. **SIEM Logs Analysis:** - You analyze the SIEM logs to gather evidence of entry vectors, unusual activity, and any further lateral movement within the network. - **SIEM Log Findings:** - The logs reveal that the phishing email containing the RAT was received and opened on December 9, 2024, at 3:45 PM. The email originated from an external address that is not recognized as legitimate. - Following the execution of the RAT, there were multiple attempts to access various systems within the network, including `PLC1`, indicating potential lateral movement attempts. - The logs also show failed login attempts to other critical systems, suggesting that the threat actor was attempting to escalate privileges or access additional resources. With these findings, your team has identified the likely entry vector and the scope of the compromise. The immediate threat has been contained by isolating the affected workstation. What actions would you like to take next to further secure the network and address the incident? Your Team: Have any privileged accounts been created since December 9? Check all other systems for IoCs or evidence of compromised accounts just in case. Was IT-Workstation1 dual-homed? If so, are there any other dual-homed systems? Facilitator: Your team proceeds with a thorough investigation to ensure that the network is secure and to identify any additional indicators of compromise (IoCs) or unauthorized activities: 1. **Check for Privileged Account Creation:** - You review the account management logs and Active Directory (AD) records to identify any new privileged accounts created since December 9, 2024. - **Findings:** - The logs indicate that no new privileged accounts have been created during this period. However, there were several failed attempts to create accounts with elevated privileges, which were blocked by existing security policies. 2. **Search for Indicators of Compromise (IoCs):** - A comprehensive scan is conducted across all systems within the network to identify any IoCs or evidence of compromised accounts. This includes checking for unusual login patterns, unauthorized software installations, and any signs of data exfiltration. - **IoC Findings:** - No additional systems show signs of compromise or unauthorized access. The security measures in place, including multi-factor authentication (MFA) and network segmentation, appear to have effectively limited the threat actor's ability to move laterally. 3. **Assessment of Dual-Homed Systems:** - You verify whether `IT-Workstation1` was dual-homed, meaning it had network interfaces connected to both the IT and OT networks. - **Dual-Homed Status:** - `IT-Workstation1` is confirmed to be dual-homed, which allowed it to access both network segments. This configuration was intended for administrative purposes but inadvertently provided a potential pathway for the threat actor to access OT systems. - **Other Dual-Homed Systems:** - A review of the network architecture reveals that there are no other dual-homed systems currently in use. The dual-homed configuration of `IT-Workstation1` is identified as a security risk, and plans are made to reconfigure or replace it with a more secure setup. With these actions, your team has addressed the immediate threat and identified areas for improvement in network security. What steps would you like to take next to further enhance your security posture and prevent future incidents? Your Team: Has the threat been contained now? Facilitator: Based on the actions taken and the findings from your investigation, the immediate threat appears to have been contained. Here's a summary of the current situation: 1. **Isolation and Remediation:** - `IT-Workstation1`, the source of the unauthorized access attempts, has been isolated from the network. The malicious payload (RAT) identified on the workstation has been removed, and the system is undergoing further remediation to ensure it is clean and secure. 2. **Network Security Measures:** - The IP address `198.51.100.100` has been blocked, and external access to port 502 has been restricted, preventing further unauthorized attempts to access `PLC1`. - No new privileged accounts have been created, and existing security policies have successfully blocked unauthorized account creation attempts. 3. **Indicators of Compromise:** - A thorough scan of all systems revealed no additional indicators of compromise or evidence of further unauthorized access. The security measures in place, including MFA and network segmentation, have effectively limited the threat actor's ability to move laterally. 4. **Dual-Homed System Risk:** - The dual-homed configuration of `IT-Workstation1` has been identified as a security risk. Plans are in place to reconfigure or replace it to prevent similar issues in the future. 5. **Operational Stability:** - The pressure fluctuations have stabilized, and the SCADA system reports no further anomalies. The water supply remains stable, and there are no reports of service disruptions or safety issues. With these measures, the threat has been contained, and the network is secure. Your team can now focus on implementing additional security enhancements and conducting a post-incident review to identify lessons learned and improve future response efforts. What would you like to do next? Your Team: Run inject 2 Facilitator: **Inject 2: Pressure Readings Show Unusual Fluctuations** Despite the containment efforts, new pressure fluctuations are detected at the booster station. The SCADA system logs indicate that the pressure readings are once again showing unusual spikes and drops, similar to the initial incident. This raises concerns about the possibility of a residual issue or a new, undetected threat. **Current Situation:** - The pressure fluctuations are not yet outside of regulatory standards, but they are significant enough to warrant immediate attention to prevent potential service disruptions or safety issues. - The SCADA system is actively monitoring the situation and has triggered alerts to the Incident Response Team. - The fluctuations suggest that there may still be an underlying issue affecting the PLC or the control systems, despite the previous containment measures. Your team needs to investigate the cause of these new fluctuations and determine the appropriate response to ensure the stability and safety of the water supply. What actions will you take next? Your Team: Scan the HMIs and SCADA server. Let's isolate IT from OT. Let's go to manual control until we figure out where the incident is originating from and get it fixed. Facilitator: Your team takes the following actions to address the new pressure fluctuations and ensure the stability of the water supply: 1. **Scan the HMIs and SCADA Server:** - You initiate a comprehensive scan of the Human-Machine Interfaces (HMIs) and the SCADA server to identify any potential issues or indicators of compromise. This includes checking for unauthorized software, configuration changes, or any signs of malicious activity. - **Scan Results:** - The scans reveal no signs of malware or unauthorized access on the HMIs or the SCADA server. However, there are logs indicating repeated attempts to send unauthorized configuration commands to `PLC1`, suggesting that the issue may be related to residual effects from the previous compromise or a misconfiguration. 2. **Isolate IT from OT:** - To prevent any potential cross-network contamination, you proceed to isolate the IT network from the OT network. This involves disabling any dual-homed systems and ensuring that there are no active connections between the two networks. - This isolation helps to ensure that any potential threats originating from the IT network cannot affect the OT systems, particularly the PLCs and SCADA systems critical to water operations. 3. **Switch to Manual Control:** - As a precautionary measure, you decide to switch the affected booster station to manual control using the `BackupControl` system. This allows operators to manually manage water pressure and ensure continued service delivery while the investigation is ongoing. - The manual control system is monitored closely by the SCADA system to ensure that all operations remain within safe parameters. With these actions, your team has taken steps to stabilize the situation and prevent further fluctuations while continuing to investigate the root cause. What would you like to do next to further address the incident and ensure the security of your systems? Your Team: Look at the SCADA server scripts and network logs to see where the new unauthorized access was coming from. Facilitator: Your team proceeds to investigate the SCADA server scripts and network logs to identify the source of the new unauthorized access attempts and determine the root cause of the pressure fluctuations. 1. **Review SCADA Server Scripts:** - You conduct a thorough review of the scripts and automation routines running on the SCADA server. This includes checking for any unauthorized modifications or scripts that could be sending incorrect commands to `PLC1`. - **Script Analysis Findings:** - The analysis reveals a recently modified script that includes additional lines of code not present in the original version. These lines appear to be responsible for sending unauthorized configuration commands to `PLC1`, causing the pressure fluctuations. - The modification timestamp aligns with the initial unauthorized access attempts, suggesting that the script was altered during the previous compromise. 2. **Analyze Network Logs:** - You examine the network logs to trace the source of the unauthorized access attempts and identify any patterns or anomalies that could indicate the origin of the threat. - **Network Log Findings:** - The logs show that the unauthorized commands were initiated from an internal IP address, `192.168.1.102`, which corresponds to the `IT-Server`. This suggests that the server may have been compromised and used as a pivot point to access the SCADA server. - Further analysis of the `IT-Server` logs reveals suspicious login attempts and successful access from a user account that was not active during the time of access, indicating potential credential compromise. With these findings, your team has identified the likely source of the unauthorized access and the cause of the pressure fluctuations. The compromised script on the SCADA server and the unauthorized access from the `IT-Server` need to be addressed to fully resolve the incident. What actions will you take next to remediate the situation and secure your systems? Your Team: What account was compromised? While we are offline, we need to rebuild the SCADA server and fix the script. Take the IT-Server offline (keeping it powered on) and investigate it. Facilitator: Your team takes the following actions to address the compromised account and secure the network: 1. **Identify the Compromised Account:** - You review the access logs and account activity on the `IT-Server` to determine which user account was compromised. - **Compromised Account Findings:** - The logs indicate that the compromised account is `admin_user`, an account with elevated privileges used for administrative tasks. The account was accessed during a time when the legitimate user was not active, suggesting credential theft or misuse. 2. **Rebuild the SCADA Server and Fix the Script:** - While the network is isolated, you proceed to rebuild the SCADA server to ensure it is free from any unauthorized modifications or residual threats. This involves reinstalling the operating system, applications, and restoring configurations from secure backups. - The compromised script is replaced with a verified version, and additional security measures are implemented to prevent unauthorized script modifications in the future. 3. **Take the IT-Server Offline and Investigate:** - The `IT-Server` is taken offline to prevent any further unauthorized access or potential lateral movement. The server remains powered on to allow for a detailed forensic investigation. - **Investigation of IT-Server:** - A forensic analysis is conducted to identify how the server was compromised and to gather evidence of any unauthorized activities. This includes examining system logs, installed software, and network connections. - Preliminary findings suggest that the server was accessed using the compromised `admin_user` account, and there are traces of a remote access tool that may have been used to facilitate the unauthorized activities. With these actions, your team is working to secure the network and address the root causes of the incident. What additional steps would you like to take to ensure the security and integrity of your systems? Your Team: Change admin_user credentials. Do we have clean backups for the IT-Server? Facilitator: Your team proceeds with the following actions to enhance security and ensure the integrity of your systems: 1. **Change `admin_user` Credentials:** - You immediately change the credentials for the `admin_user` account to prevent any further unauthorized access. This includes updating the password to a strong, complex one and enabling multi-factor authentication (MFA) for added security. - Additionally, you review and update the access permissions for the account to ensure that it adheres to the principle of least privilege, minimizing the potential impact of any future compromise. 2. **Assess Backup Availability for the IT-Server:** - You check the backup records to determine if there are clean, verified backups available for the `IT-Server`. - **Backup Assessment Findings:** - Fortunately, regular backups of the `IT-Server` have been maintained, and the most recent clean backup is available from December 8, 2024, prior to the initial unauthorized access attempts. - The backup is verified to be free from any malicious code or unauthorized modifications, making it suitable for restoring the server to a secure state. With the compromised credentials updated and a clean backup available, your team is well-positioned to restore the `IT-Server` and ensure the security of your network. What steps would you like to take next to complete the remediation process and reinforce your security posture? Your Team: Change all user credentials. Add phishing resistant MFA to all credentials. Update and enhance our user awareness training. Restore the IT-Server from backup. Facilitator: Your team takes comprehensive steps to enhance security and prevent future incidents by implementing the following measures: 1. **Change All User Credentials:** - You initiate a mandatory password reset for all user accounts across the network. This ensures that any potentially compromised credentials are rendered unusable by threat actors. - Users are required to create strong, complex passwords that adhere to updated security policies, reducing the risk of credential theft or misuse. 2. **Implement Phishing-Resistant MFA:** - Multi-factor authentication (MFA) is enabled for all user accounts, with a focus on using phishing-resistant methods such as hardware tokens or app-based authenticators. - This additional layer of security helps protect against unauthorized access, even if credentials are compromised through phishing or other means. 3. **Enhance User Awareness Training:** - You update and enhance the organization's user awareness training program to address the latest phishing tactics and cybersecurity threats. - Training sessions are conducted to educate employees on recognizing phishing attempts, safe email practices, and the importance of reporting suspicious activities promptly. 4. **Restore the IT-Server from Backup:** - The `IT-Server` is restored from the clean backup dated December 8, 2024. This process involves reinstalling the operating system, applications, and configurations to ensure the server is free from any unauthorized modifications or malicious code. - Post-restoration, the server is thoroughly scanned and validated to confirm its security and integrity before being brought back online. With these actions, your team has significantly strengthened the organization's security posture and mitigated the risk of future incidents. The network is now secure, and operations can continue with confidence. Is there anything else you would like to address or any additional steps you would like to take? Your Team: Check for any further suspicious activity attempts to the OT network. Facilitator: Your team conducts a thorough review of the OT network to ensure there are no further suspicious activities or attempts to compromise the systems. This involves analyzing logs, monitoring network traffic, and verifying the integrity of critical components. 1. **Log Analysis:** - You examine the logs from the SCADA system, PLCs, and network devices within the OT network to identify any unusual activities or access attempts. - **Log Findings:** - The logs show no new unauthorized access attempts or anomalies since the recent security measures were implemented. The previous unauthorized access attempts have ceased, and there are no indications of further compromise. 2. **Network Traffic Monitoring:** - Real-time network traffic monitoring is conducted to detect any suspicious patterns or unauthorized communications within the OT network. - **Traffic Monitoring Results:** - The network traffic appears normal, with no signs of unusual data flows or unauthorized command sequences being sent to the PLCs or SCADA systems. 3. **Integrity Verification:** - You verify the integrity of critical OT components, including the SCADA server and PLCs, to ensure they are operating as expected and have not been tampered with. - **Integrity Check Results:** - All systems are functioning correctly, and there are no signs of unauthorized modifications or disruptions to the control processes. With these checks, your team confirms that the OT network is secure and free from further suspicious activities. The recent security enhancements and containment measures have effectively mitigated the threat, ensuring the continued stability and safety of the water supply. If there are any additional actions or areas you would like to explore, please let me know. Your Team: While we are down and before bringing everything back to remote control, we will take the opportunity to enhance our network and system security. Evaluate vulnerabilities, etc. Facilitator: Your team takes advantage of the current downtime to conduct a comprehensive evaluation of the network and system security, focusing on identifying and addressing vulnerabilities to enhance the overall security posture. Here are the steps you undertake: 1. **Vulnerability Assessment:** - You perform a thorough vulnerability assessment across both the IT and OT networks. This involves using automated vulnerability scanning tools to identify potential weaknesses in systems, applications, and network configurations. - **Assessment Findings:** - The assessment identifies several outdated software versions and missing security patches on some systems, which could be exploited by threat actors. - A few network devices are found to have default configurations that could be optimized for better security. 2. **Patch Management:** - You prioritize the application of security patches and updates to all identified systems and applications. This includes updating operating systems, SCADA software, and any third-party applications used within the network. - **Patch Implementation:** - All critical and high-severity vulnerabilities are addressed promptly, ensuring that systems are protected against known exploits. 3. **Network Segmentation:** - You review and enhance network segmentation to limit the potential impact of any future incidents. This involves creating additional network zones and implementing strict access controls between IT and OT environments. - **Segmentation Enhancements:** - Access to critical OT systems is restricted to only authorized personnel and devices, reducing the risk of unauthorized access or lateral movement. 4. **Security Configuration Review:** - A detailed review of security configurations is conducted for all network devices, including firewalls, routers, and switches. This ensures that security policies are aligned with best practices and organizational requirements. - **Configuration Improvements:** - Default settings are replaced with customized configurations that enhance security, such as disabling unnecessary services and enforcing strong encryption protocols. 5. **Penetration Testing:** - You engage a third-party cybersecurity firm to conduct penetration testing on the network. This provides an external perspective on potential vulnerabilities and helps validate the effectiveness of the security measures in place. - **Penetration Testing Results:** - The testing identifies a few additional areas for improvement, which are promptly addressed to further strengthen the network's defenses. With these enhancements, your team has significantly improved the security of the network and systems, reducing the risk of future incidents and ensuring the resilience of critical infrastructure. Once all measures are in place and validated, you can confidently bring the systems back to remote control. If there are any other actions you would like to take or areas to explore, please let me know. Your Team: Make appropriate law enforcement and regulatory notifications according to the IR Plan. Facilitator: Your team proceeds to make the necessary notifications to law enforcement and regulatory bodies in accordance with the Incident Response Plan (IRP). These notifications are crucial for ensuring compliance with legal and regulatory requirements and for facilitating any potential investigations. 1. **Law Enforcement Notification:** - You contact the local law enforcement agency to report the cybersecurity incident. This includes providing details about the unauthorized access attempts, the compromised systems, and the actions taken to contain and remediate the threat. - **Information Shared:** - A summary of the incident timeline, including the initial detection, containment measures, and current status. - Details of the compromised IP addresses and any evidence of malicious activity, such as the remote access trojan (RAT) identified on `IT-Workstation1`. - Any relevant logs or forensic data that may assist in the investigation. 2. **Regulatory Body Notification:** - You notify the relevant regulatory bodies, such as the Environmental Protection Agency (EPA) and state-level water authorities, about the incident. This ensures that MWAA remains compliant with industry-specific standards and guidelines. - **Information Shared:** - An overview of the incident and its impact on water operations, including the pressure fluctuations and manual control measures implemented. - Assurance that the water supply remained stable and within regulatory standards throughout the incident. - A description of the security enhancements and measures taken to prevent future incidents. 3. **Data Breach Notification (if applicable):** - If any customer data was potentially compromised during the incident, you follow established protocols for data breach notifications. This includes informing affected individuals and providing guidance on steps they can take to protect their information. - **Information Shared:** - Details of the data potentially affected and the measures taken to secure it. - Contact information for customers to reach out with questions or concerns. By making these notifications, your team ensures transparency and compliance with legal and regulatory obligations. This proactive approach helps maintain public trust and supports any necessary investigations. If there are any additional actions you would like to take or areas to explore, please let me know. Your Team: Bring all systems from manual control back to normal operating mode. Facilitator: Your team carefully transitions the systems from manual control back to normal operating mode, ensuring that all processes are stable and secure. Here are the steps you undertake to complete this transition: 1. **Verification of System Integrity:** - Before switching back to normal operations, you conduct a final verification of the integrity and security of all systems, including the SCADA server, PLCs, and network devices. This ensures that all components are functioning correctly and are free from any unauthorized modifications. - **Verification Results:** - All systems pass the integrity checks, confirming that they are secure and ready for normal operations. 2. **Coordination with Operations Team:** - You coordinate with the operations team to ensure a smooth transition from manual control to automated operations. This involves communicating the transition plan and ensuring that all personnel are prepared for the changeover. - **Communication:** - Clear instructions are provided to operators on the steps to follow during the transition, including monitoring procedures and reporting any anomalies. 3. **Switching to Normal Operating Mode:** - The SCADA system is reactivated to resume automated control of the water treatment and distribution processes. This involves re-enabling the communication links between the SCADA server and the PLCs. - **Transition Process:** - The manual control system (`BackupControl`) is gradually phased out as the SCADA system takes over, ensuring a seamless transition without disrupting water services. 4. **Monitoring and Validation:** - Once the systems are back in normal operating mode, continuous monitoring is conducted to validate that all processes are functioning as expected. This includes real-time monitoring of pressure readings and system performance. - **Monitoring Results:** - The systems operate smoothly, with no anomalies or fluctuations detected. The water supply remains stable and within regulatory standards. 5. **Post-Transition Review:** - A post-transition review is conducted to assess the effectiveness of the transition process and identify any areas for improvement. This review helps ensure that future transitions can be executed even more efficiently. With these steps, your team successfully transitions the systems back to normal operating mode, ensuring the continued stability and reliability of the water supply. If there are any additional actions you would like to take or areas to explore, please let me know. Your Team: End exercise Facilitator: ## Exercise Analysis and Report **Score:** **85** out of **100** ### Executive Summary The Incident Response Tabletop Exercise for the Municipal Water Authority of Aliquippa (MWAA) simulated a cybersecurity incident involving unauthorized access to a PLC controlling water pressure at a booster station. The threat was identified as originating from the Cyber Av3ngers group, which gained unauthorized access to the PLC through a compromised internal workstation. The exercise tested the team's ability to detect, contain, and remediate the threat while maintaining operational stability. Overall, the team performed well, effectively containing the threat and implementing security enhancements. However, there were areas for improvement, particularly in initial detection and network segmentation. ### Exercise Objectives 1. **Test response to PLC compromise:** The team successfully identified and contained the unauthorized access to the PLC, preventing significant operational impact. 2. **Evaluate manual operation procedures:** The transition to manual control was executed smoothly, ensuring continued service delivery during the incident. 3. **Assess cybersecurity measures for OT systems:** The exercise highlighted the need for improved network segmentation and dual-homed system configurations. ### Analysis Summary The team demonstrated strong incident response capabilities, quickly isolating compromised systems and implementing containment measures. The decision to switch to manual control ensured operational stability while the threat was addressed. The exercise revealed vulnerabilities in network segmentation and dual-homed systems, which were addressed through security enhancements. The team's proactive approach to law enforcement and regulatory notifications ensured compliance and transparency. ### Injects Used - Alarm triggered for unauthorized PLC access - Pressure readings show unusual fluctuations ### Positive/Successful Actions - Prompt isolation of compromised systems - Effective use of manual control to maintain operations - Comprehensive investigation and remediation of compromised systems - Implementation of enhanced security measures, including MFA and user awareness training - Timely notifications to law enforcement and regulatory bodies ### Areas for Improvement - Initial detection of unauthorized access could be improved - Network segmentation and dual-homed system configurations need enhancement - Faster identification of compromised credentials ### Incident Cost Breakdown - Investigation and remediation efforts: $15,000 - System restoration and security enhancements: $10,000 - Regulatory and law enforcement notifications: $5,000 - Total cost of the incident: $30,000 ### Score Deductions - Initial detection delay: -5 points - Network segmentation and dual-homed system vulnerabilities: -5 points - Delay in identifying compromised credentials: -5 points ### Mitre ATT&CK TTPs Used by the Threat The threat scenario was based on tactics and techniques similar to those used by the Cyber Av3ngers group, known for targeting critical infrastructure. - **Initial Access:** Phishing (T1566) - The threat actor used a phishing email to deliver a remote access trojan (RAT) to `IT-Workstation1`. - **Execution:** User Execution (T1204) - The RAT was executed by the user, allowing the threat actor to gain control of the workstation. - **Persistence:** Valid Accounts (T1078) - The threat actor used compromised credentials to maintain access. - **Privilege Escalation:** Valid Accounts (T1078) - Attempts to create privileged accounts were blocked. - **Defense Evasion:** Obfuscated Files or Information (T1027) - The RAT used obfuscation techniques to avoid detection. - **Credential Access:** Credential Dumping (T1003) - The threat actor attempted to access additional credentials. - **Discovery:** System Network Connections Discovery (T1049) - The threat actor mapped the network to identify targets. - **Lateral Movement:** Remote Services (T1021) - The threat actor attempted to move laterally using compromised credentials. - **Impact:** Inhibit System Recovery (T1490) - The threat actor modified SCADA scripts to disrupt operations. ## IR Review ### IR Team Performance The team followed the Incident Response Plan effectively, demonstrating strong coordination and decision-making throughout the exercise. The plan's emphasis on rapid containment and manual control procedures was well-executed, ensuring operational stability. The team's proactive approach to notifications and security enhancements further reinforced their commitment to maintaining a robust security posture. ### IR Plan Gap Assessment The Incident Response Plan was comprehensive and well-structured, providing clear guidance for managing the incident. However, the exercise highlighted areas for improvement, including: - **Network Segmentation:** The plan should include more detailed guidance on network segmentation and dual-homed system configurations to prevent unauthorized access. - **Initial Detection:** Enhancements to detection capabilities, such as improved monitoring and alerting, should be prioritized to reduce detection times. - **Credential Management:** The plan should emphasize regular credential audits and the use of phishing-resistant MFA to prevent credential compromise. Overall, the exercise demonstrated the effectiveness of the Incident Response Plan while identifying opportunities for further strengthening MWAA's cybersecurity resilience. Your Team: Take the security posture aspects out of the report and only score based on the actual IR performance of the participants Facilitator: ## Exercise Analysis and Report **Score:** **90** out of **100** ### Executive Summary The Incident Response Tabletop Exercise for the Municipal Water Authority of Aliquippa (MWAA) simulated a cybersecurity incident involving unauthorized access to a PLC controlling water pressure at a booster station. The threat was identified as originating from the Cyber Av3ngers group, which gained unauthorized access to the PLC through a compromised internal workstation. The exercise tested the team's ability to detect, contain, and remediate the threat while maintaining operational stability. Overall, the team performed well, effectively containing the threat and implementing necessary response measures. The exercise highlighted the team's strong coordination and decision-making skills. ### Exercise Objectives 1. **Test response to PLC compromise:** The team successfully identified and contained the unauthorized access to the PLC, preventing significant operational impact. 2. **Evaluate manual operation procedures:** The transition to manual control was executed smoothly, ensuring continued service delivery during the incident. 3. **Assess cybersecurity measures for OT systems:** The exercise demonstrated the team's ability to secure OT systems and maintain operational integrity. ### Analysis Summary The team demonstrated strong incident response capabilities, quickly isolating compromised systems and implementing containment measures. The decision to switch to manual control ensured operational stability while the threat was addressed. The team's proactive approach to law enforcement and regulatory notifications ensured compliance and transparency. Overall, the team's performance was commendable, with effective coordination and execution of the Incident Response Plan. ### Injects Used - Alarm triggered for unauthorized PLC access - Pressure readings show unusual fluctuations ### Positive/Successful Actions - Prompt isolation of compromised systems - Effective use of manual control to maintain operations - Comprehensive investigation and remediation of compromised systems - Timely notifications to law enforcement and regulatory bodies ### Areas for Improvement - Initial detection of unauthorized access could be improved - Faster identification of compromised credentials ### Incident Cost Breakdown - Investigation and remediation efforts: $15,000 - System restoration and security enhancements: $10,000 - Regulatory and law enforcement notifications: $5,000 - Total cost of the incident: $30,000 ### Score Deductions - Initial detection delay: -5 points - Delay in identifying compromised credentials: -5 points ### Mitre ATT&CK TTPs Used by the Threat The threat scenario was based on tactics and techniques similar to those used by the Cyber Av3ngers group, known for targeting critical infrastructure. - **Initial Access:** Phishing (T1566) - The threat actor used a phishing email to deliver a remote access trojan (RAT) to `IT-Workstation1`. - **Execution:** User Execution (T1204) - The RAT was executed by the user, allowing the threat actor to gain control of the workstation. - **Persistence:** Valid Accounts (T1078) - The threat actor used compromised credentials to maintain access. - **Privilege Escalation:** Valid Accounts (T1078) - Attempts to create privileged accounts were blocked. - **Defense Evasion:** Obfuscated Files or Information (T1027) - The RAT used obfuscation techniques to avoid detection. - **Credential Access:** Credential Dumping (T1003) - The threat actor attempted to access additional credentials. - **Discovery:** System Network Connections Discovery (T1049) - The threat actor mapped the network to identify targets. - **Lateral Movement:** Remote Services (T1021) - The threat actor attempted to move laterally using compromised credentials. - **Impact:** Inhibit System Recovery (T1490) - The threat actor modified SCADA scripts to disrupt operations. ## IR Review ### IR Team Performance The team followed the Incident Response Plan effectively, demonstrating strong coordination and decision-making throughout the exercise. The plan's emphasis on rapid containment and manual control procedures was well-executed, ensuring operational stability. The team's proactive approach to notifications and security enhancements further reinforced their commitment to maintaining a robust security posture. ### IR Plan Gap Assessment The Incident Response Plan was comprehensive and well-structured, providing clear guidance for managing the incident. However, the exercise highlighted areas for improvement, including: - **Initial Detection:** Enhancements to detection capabilities, such as improved monitoring and alerting, should be prioritized to reduce detection times. - **Credential Management:** The plan should emphasize regular credential audits and the use of phishing-resistant MFA to prevent credential compromise. Overall, the exercise demonstrated the effectiveness of the Incident Response Plan while identifying opportunities for further strengthening MWAA's cybersecurity resilience.