The Rime of the Ancient Mariner, a poem by Samuel Taylor Coleridge about an old sailor who is compelled to tell strangers about the supernatural adventures that befell him at sea after he killed an albatross, a friendly sea bird. Water water everywhere, not any drop to drink. This metaphor was so apt in the middle of the sea; so it is in the middle of thousands of logging events. Most of the security articles we read, talk about the tools, products and how attacks can be prevented. A few mention of detection; and a handful on monitoring. So, here's my take and experience with such teams, processes and tools.
How important do you think they take the security monitoring? Let me know if you feel connected to one of the case studies
Case 1: Look under the hood, and know your artillery!
I was working with a client on the applications, and we performed some checks including,
- Server and Application Hardening
- Sanity Checks for the application
- Application Configuration
- Application Vulnerability Assessment and Penetration Testing
After the project was completed, we scheduled some meetings to have discussions with the client project teams, and talk about recommendations. A client which is highly regarded in the market, and is, therefore, expected to understand security is key surprised me and my team on many different levels.
During the meeting, one of the item(s) was inspecting the HTTP traffic. Let's say there were some issues which the application/ IT server team can't fix or remediate in due course; hence, it was proposed to inspect the traffic on IPS and drop if "something" malicious (which was easy to spot in this case) is detected. Now, they called their IT experts, and engineers and asked if they can procure an IPS to do this. We had a lot of meetings around WAF, IPS vendors etc. and we planned to go with IPS due to it's multi-usage beyond HTTP traffic.
Now, while the vendor discussion was going on, I asked which version of perimeter firewall are they using. And when they shared the details, it turned out their firewall supports IPS capabilities with as simple as enabling on a radio-button! I couldn't believe that they have the capability with the existing product, and yet a team of 7 senior IT personals are planning the budget, and sorting vendors for weeks to finalize the product. So, I got my first real case of #WTFmoment with experts' who have no or little idea of their deployed tools,
Know who's under the bed before you look outside the window.
Next few days we enabled the IPS configuration without spending a dime, and management was happy. Guess they saved their 1000s of dollars!
Case 2: Is someone sitting in the check-post of your castle?
I was working for a client, they hired me to perform security assessments on their network and applications. It was a long term project and I was working with them to streamline the security. Their understanding of security was to enable all the defense mechanism and think you are protected! But did this approach work? Let me tell you they had a suite of tools, well updated; up and running - Cisco Firewall, a good Antivirus, a good IPS/IDS, a good WAF and QRadar SIEM. On top of that assessment tools like Nessus, Qualys etc. So, one would think they are well protected? It wasn't the case!
After completing my tasks of assessment - network, application and even firewall rules; I had some time to kill as part of my dates. So, I asked if I can check the SIEM and understand the offense rules they have put in place to identify/ correlate the malicious traffic via logs.
It turned out they had the SIEM digesting 1000s of logs but no one put any offense rule. Then, I started checking the logs, events and realized one of the target IP is connected to 40% of the network. That raised the eyebrow; is this Google? or is this the bad one. On further checking, I found many such IP addresses connected with the traffic from many workstations and even servers. So, I searched online and turned out that someone was running a botnet and those were C&C IP Addresses. I immediately escalated the issue to the CISO, and his first message was - Why did Anti-virus didn't alert us? And I thought maybe it did and no one reported, or maybe your antivirus is not even updated. Anyways I got a new project to work on and write offense rules, notifications, and alerts templates with the correlation of data from different feeds. But, it was the #facepalm moment.
The best of the boats to sail the tides; but did someone notice the wet feet?
The network was heavily infected, the antivirus was updated and put on alert as it failed to detect this attack. The incident management process initiated, and rest is history. I guess they survived.
So, ideally there are many vendors who can provide the best of security tools, and work on the buzz-word products but it's important to understand the security and keep it simple. There have been good cases as well where the company did not invest heavily in the plethora of products but kept it really simple. No fancy glossy boxes or GUI but open source tools well customized to alert teams via emails and SMS. But that's because they not only understand security but the quick timeline(s) in the case of incidence. They have accepted the fact that it's not all about if you are hacked, but when you are hacked!
Next time you go to any firm for assessment, or within your firm, ask your boss - What is the matrix and workflow when we get hacked; or who are checking the alerts on the fancy products we have put in place. To me, a well designed SIEM plays a much important role than we think or assume. Someone should focus on the events, and understand when to raise the red flag! You can practically foresee the attack if you have controls on the events and anomalies.
Be safe & kind.