Final Containment and Ensuring Future Security
Introduction:
In our other two posts of this series, we explored both previous investigations at Joe’s Retail, each marked by significant oversights and containment failures. This final instalment will discuss our comprehensive investigation, how we successfully contained the breach that had persisted for over two years, and the steps Joe’s Retail is taking to secure their systems moving forward.
A Comprehensive Approach:
Following a third fraud notification from a card brand, Joe’s Retail were informed that they needed a full PFI investigation to be performed. When Foregenix was called in after two unsuccessful investigations, our goal was not only to identify all existing threats but also to ensure containment and implement measures to prevent future breaches.
Foregenix began with a comprehensive assessment of the network, seeking to understand the full extent of the infiltration and aimed to leave no stone unturned.
Four Key Findings and Successes:
-
Comprehensive Evidence Collection:
Foregenix began with an exhaustive evidence collection, including a full disk image, volatile data extracts, database dumps, and extended log data from all relevant sources. This comprehensive approach provided a complete view of the system and revealed malware and unauthorised activities which had remained undiscovered until this investigation.
-
Log Data Examination:
Analysing the log data from the past two years revealed patterns of unauthorised access and persistent activity. We discovered that the attacker(s) had accessed the administrative panel as early as July 2022, maintaining access to the environment through various backdoors. The extensive log review provided a detailed timeline of the attacker(s) activities, helping us to understand their methods and movement.
-
Persistent Malware Detection:
Despite previous malware removal efforts, our monitoring tools, ThreatView, alerted us to the attacker’s return and reintroduction of similar malware. This demonstrated the attacker's determination and sophistication. By monitoring real-time system activity, ThreatView detected anomalies indicative of the attacker’s presence.
-
Foothold:
Analysis of the available logs revealed over 30,000 persistent POST requests to the malware samples and arbitrary named files within /tmp/; a POST request is the user submitting data to the environment. These activities indicated ongoing attempts to exploit the backdoor. However, details of the POST requests were not stored within the logs, leaving gaps in understanding the attacker’s actions. The sheer volume of these requests highlighted the persistent nature of the attack.
Enhancing Security Posture:
-
Verified All Plugins Updated:
Ensured all plugins were up-to-date and verified their integrity.
-
Improved Multi Factor Authentication (MFA):
Changing the MFA method to SMS-based authentication provided an additional layer of security for user accounts. This method was chosen for its balance between security and user convenience, helping to protect against unauthorised access.
-
Manual Review of Web Application Firewall (WAF) Critical Alerts:
Recommended that critical WAF alerts are reviewed manually to catch any missed threats.
-
Implemented File Integrity Monitoring (FIM):
Deployed Foregenix’ ThreatView to detect any unauthorised changes to critical system files and the introduction of potential malware.
-
User Account Management:
We reviewed all user accounts, limiting sessions to 60 minutes and ensuring only one active session per user at a time. This helped reduce the risk of unauthorised access. In addition, we implemented strict access controls, ensuring that only necessary personnel had administrative privileges.
-
Continuous Monitoring:
Continuous monitoring tools like ThreatView are essential in detecting persistent threats. Real-time alerts and automated anomaly detection can provide early warnings of re-infection or new attacks, enabling swift response.
Conclusion and Recommendations
Our comprehensive investigation at Joe’s Retail uncovered the full extent of the breach and implemented robust measures to contain and eradicate the threats. By addressing the vulnerabilities and strengthening the overall security posture, we assisted Joe’s Retail to mitigate the risk of future breaches. The lessons learned and best practices outlined in this series can serve as a guide for other organisations to enhance their cybersecurity defences and achieve greater resilience.
Thank you for joining us on this journey through the investigation of Joe’s Retail. I hope these insights and best practices will help you strengthen your own organisation’s cybersecurity defences and achieve greater resilience in the face of potential threats.
Not all DFIR teams are the same.
You know what they say…. Pay cheap….. Pay twice - or in this case three times.
