Incident Immunity: Real-Time Threat Response Without Downtime in Cloud Native Applications

The final session of our cloud native security series focuses on effective incident response for modern containerized and Kubernetes applications. This session will walk you through the process of identifying, responding to, and neutralizing active threats in real time, while keeping critical applications operational. We will cover strategies that minimize time to remediation, reduce the window of exposure, and prevent service disruptions.

You will learn how Aqua’s runtime protection capabilities automatically block or restrict unapproved activity, stop zero-day threats before they spread, and maintain operational integrity without downtime.

Gain the knowledge to create an incident response plan tailored to cloud native environments, ensuring rapid containment of threats and stronger preventive measures for the future. This practical guidance will help security teams respond faster, protect runtime workloads, and maintain the resilience of their cloud native applications against sophisticated attacks.
Duration 32:50
Presented By:
Gilad Elyashar
Chief Product Officer, Aqua
Transcript
Good day, everyone, and welcome. Thank you for joining the finale of our cloud native runtime security series. I'm Joe Murphy, and we look forward to guiding you through today's session on effective incident response within cloud native apps.

So throughout this series, we've laid the groundwork by diving into the uniqueness of run time security and the nuances of threat detection in cloud native. Today, as we culminate our journey by focusing on how to swiftly counteract effective threats, ensuring that our applications remain resilient and operational amidst security incidents.

Together, we'll explore comprehensive strategies to craft incident response plans that not only mitigate immediate risks, but will also bolster our defenses for long term security resilience.

And before we get started, quick agenda and some housekeeping.

So we're gonna start off today with a quick navigation of the complex cloud native threat landscape.

Will then make things real and dissect a really cool headcrab attack path, followed by why runtime security is crucial, and we're gonna wrap up with key functionality to consider in run time. As always, the session is being recorded. Copy over the recording and the slides will be shared via email. If you'd like any additional materials or would like to speak to a technical resource here at aqua, Feel free to let me know. I love playing the matchmaker and would be happy to to set that up for you.

So We've had, two wonderful groups in part one and part two. I see a lot of familiar faces here with us today, but curious to see where everyone else is joining us from. So let us know in the chat, and, let's get this thing kicked off. So joining me today, everyone, I'm very pleased to announce Gilad Elyashar, Aqua’s CPO, a true visionary that joins us from Microsoft. You may have heard of them.

He's very passionate about cloud native and runtime security among many other things and just an all around great great person.

So Galad, thank you so much for joining us today. And, that's enough for me. I'll pass it over to you.

Thank you very much, Joe. And, thank you everyone for joining. I'm very happy to be here today talking to you.

Very briefly about myself as, Joe, started off explaining.

I've been here in Agua for over six months now as the chief product officer before that, I spent over fifteen years in Microsoft that multiple product positions. The last eight of them were managing the product management of the, cloud security of sign up product in Microsoft. So kind of, basically, been experiencing, cloud native and cloud security pretty much since its inception, twenty fifteen, right, very, very even embryonic days of this market, and and believe it did grow and grow and grow, of course, with it, to the to to where to where this is today, which is, a very exciting and in exciting, innovative place with, a lot of, challenges, and and sophisticated attackers to deal with, which is partly what we are going talk about today.

So let's get started.

We're gonna talk today a little bit about incident immunity effective response, we're gonna give an example of one, a known incident that was, identified, in a published, in a, in multiple, questions, by the Aqua Notulus team, our research team. And, we're gonna share a bit details and kind of explain the insights of how things look and operate.

I'm not gonna repeat. I think this is kind of what Joe was talking about.

So let's start with a short recap of the previous sessions you had, with the with the math and more or so with the big as well. Where if you guys remember, we basically, kind of, mark the theoretical landscapes to basically things we call risks, which are kind of the openings that Eventually, we all end up living in our environments and in our systems, the vulnerabilities, misconfigurations, insecure coding, flows or in the eyes of the attackers, the opportunities.

Right?

And on the other side, of the house, these are the threats. The actions that the act that the bad actors actually do remote code execution and attempting a a kind of attacking from from from the outside, planting malware in multiple ways, working in memory driving suspicious activities and takeovers. So all of the different malicious activities that attackers can do.

And when these to meet, kind of this overlap in the middle where a risk actually means action and threat that together constitutes an attack.

We also talked about kind of broken down the model and went through this kind of tree of different of how attacks usually look, how do they look like, where do they start, and how do they go around and, propagate within your organization starting with a threat actor that is usually looking for some sort of opening risk or opportunity, as we said in their perspective, either by misconfiguration, vulnerability, something on the supply chain that allows them entry and presents a weakness.

And to and using that, Usually, they would like to download some sort of an exploit that can into your environment that will give them the ability to, basically, expand further, gain more, privileges and perform more actions, whether it's connecting to your Kubernetes launching their own images inside your environment, sending addresses, looking at data and probing for your environment.

They that's usually when they would be dropping their full payload that will then give them gives them access to the host.

And kind of break free and break out of, kind of the, of the encapsulate the environment that they're running in and enable them to kind of gain deeper access, either still keys and secrets, run their own malicious software or just kind of run Bitcoin minors. That's kind of the most common activity, that's used to kind of take your compute, basically, and and and leverage the leverage it to, to to mine a Bitcoin.

Create backdoors and eventually connect back to command and control to a report back to send data, etcetera.

And again, this is, basically a way to outline the world and what we are expecting to see.

Now I think one of the principles we like to define, in aqua when we're looking at cloud native attacks and, specifically, by our team, which is our research team that's comprised of, a risk of a state of the art world class security researchers were only focused on cloud native attacks identifying them, and, and, and, and, classifying them and then writing, of course, really kind of, to obnoxious signatures that identify them as part of the aqua products.

They deploy networks of honeypots across the web. They research, they appear in conventions, of course, and they are one of the most powerful engines that drive the aqua product eventually.

And I think one of the one of the most interesting things to understand is that when talking about threats, it's bill it's pretty much protection against the unknown There is, of course, the known threats. These are the malwares that were already that are exist and known and and known and can be identified by traditional tools there are the malicious URLs and IP addresses, and that are ex that exist in, in many threat intel streams known rootkits that are known to the public.

But then there is kind of the hidden part of this iceberg, which is which goes very deep with more and more sophisticated, threat actors. Some of them state sponsored, some of them just very sophisticated hacker groups, that actually go about, attacks that are unknown, leveraging vulnerabilities or other types of weaknesses or combinations of those and techniques that, are that are not fully known and identifiable.

And we're gonna talk about one today, the one on the bottom called headcrub.

Headcrub is a very advanced threat actor that utilizes a state of the art custom made malware.

It's important, and and there is a lot of, there is a lot of, kind of, anecdotes around it, how it was discovered, by aqua. These were shared by our Nautilus team in a convention. I think this was a year ago, and there is a lot of, and there is a lot of, there is a lot of interesting details there. But eventually, this is a threat actor, an unknown one, officially working alone, but but kind of, unofficially very, very, very sophisticated malware that was built, that can really evade, most, techniques which which can raise us questions.

It's unprecedented by it's undetectable by agentless and traditional antivirus solutions because of its really kind of the fact that it never actually drops files on your environment and works fully in memory.

And, we have identified, and, and, and, and it's marked that it's, has taken control of at least three thousand and three hundred servers, servers, creating, that kind of very, vast reach in its botnet.

Using that technique and malware.

So let's simplify a bit so that we can actually and get into the details a bit more and go over the actions that an attacker is usually trying to do. Starting with exploiting a risk, this is again the opportunity we're talking about and basically leveraging that to penetrate.

Execute the threat, which is basically getting into your system and your environment, expanding the foothold to gain a deeper, capabilities within the environment and allowing them to run their own code and eventually use that for malicious intent, whether it's disrupting the service, owning the workload, still information, and more. Let's see a little bit how it was done here in head ground.

Threat actors started with probing for misconfigurations and vulnerable, and vulnerable workloads and it leveraged a specific, application level misconfiguration, in a redis, workloads, right, applications from certain versions. And again, more details around that, that we published in our blog in the past that leverage that leverage super user capabilities.

These are things that, for example, showing you here the vulnerability scan, on the image level of, the redis, container.

This is something that would have been flagged, but it would have been flagged just when con scanning the entire container, and when scanning for this specific, misconfiguration.

It's not, it's not it will not have been it would have passed a regular vulnerability scanning test because it had no known vulnerabilities, but some misconfiguration.

And that's the basic entry point. This is, okay, this is, misconfiguration that was fixed in later versions, but it still runs on multiple on on many, many servers as we've seen.

Then the second step, the the attacker executes the threat. And this here is basically, a detection that would have shown how a database program spawned the shell. What attacker does here is basically this is running a radius database and program and basic, and that's that's the that's the application customer is running. And the alert here coming off is to basically indicate something that is not so common for a database program to spawn a UNIX shell within, within the environment, within the container. Which doesn't usually happen, and this is something that is alerted on. And that shell can then run code.

Which is exactly what we're seeing now.

Doctor Shell, this is another detection. Now, it's to go back you're seeing that the detection is coming off from these are detections coming from the notice team based on understanding this attack and similar ones. Right, because these are not specific only to this attack. They can identify similar attacks as well. This one is for medium severity. It's suspicious.

This one is already in high severity because it's called it continuous then. New executable, sorry, was dropped to memory during runtime. Basically, what we're seeing here is that, there is fileless loading technique that loads and executable directly into memory without any trace in, in the file system and on disk in a way, that in itself is very suspicious and that alerts, are our probes and identify that as an alert and alert the customer.

And of course, that is what is allowing the the attacker to run their malicious code in the in the workload.

Then once they have that and they are able to run the work that run their, code within, your environment, what they've done in this example that I'm running here, they've ran started running a crypto miner. To basically, leverage your compute and some of these databases usually run on very, very powerful machines that can there is a lot to extract from and to take out. And and and they're using that to run their crypto miner and basically leverage your infrastructure.

Now this, of course, could have continued in multiple other is when attackers could have used that, persistence that their their creator or their presence that they've created with an in memory tool to run other types of code, to break out from the container, to take over the host and to, continue with additional malicious activities.

Forty one fourteen hundred percent.

What would that be? How do we that's kind of that would be the increase that we're seeing, in in, in the number of attacks the past year.

Now how do we defend against this?

So the answer to everything that we've seen here And this is kind of a bit throwing the hints, in the slides, with the detections coming out, from the estimate from the aqua product, but basically the answer to all that is what we call runtime protection, which is immediate frictionless threat protection against known and unknown attacks, the target running workloads, the ability to identify and handle these attacks at real time.

That would include first of all, to secure all of your cloud workloads, containers, virtual machines, serverless, Kubernetes deployments, all of their complete assets that actually run code and can be then attacked through that.

Then continuously monitor them to detect cloud threats, like the ones we've shown here with different types of alerts on different, malicious or known threats, then the ability to actually stop these attacks across workloads in real time. When they happen.

And eventually, the ability to investigate incidents and incident response to gain insight and to get to the root cause of these attacks. You know, it's it's we're looking at it from from, from the from the security team's perspective. It's never, as you probably all know, it's never as clear as I was showing it right now. You see part of the chain, you're surprised by it. You need to kind of investigate back and understand exactly what it was, what else is impacted, and how to deal with it, and so that's why where this comes in.

So let's talk about the first key capability, detection.

This is, I think, the first capability that, that that is, very important for runtime protection, the ability to monitor, have sensors that monitor and instrument data at the deepest levels of the compute.

And when we're talking about, containers, Kubernetes and early Linux Snow, actually, to that, to that fact, this would be, going into kernel level visibility with EBPF technology that enables us to gain, instrumentations at the deepest level and the the richest level to really identify very granular problems.

Of any very granular actions and then, malicious actions.

On top of that, real time monitoring of suspicious activities, attack patterns, suspicious behaviors, real time monitoring of known threats such as malware detection. In files or in memory as we've just shown.

Buffers, and in registry and windows registry hives, of course.

Behavioral detection techniques that, based either an indication of compromise, that, that, that are known and, of course, on malicious behaviors that we model, as, as as malicious by, again, variety of inputs. Including our source, our network of honey pots, but, but not but not limited to that, including research, etcetera.

And in any way, anything that is abnormal that can be detected and flagged.

Then the completing and or kind of closing the loop capability to detection is, of course, the ability to prevent.

Now many, many, I would say security teams, start off, of course, with only detection. And, and that is almost the natural thing to do when you get started. Basically, first of all, identify the threats.

But the need to actually prevent or block them at a real time is a very important, for multiple reasons, now think the first one of them is just the sheer number of attacks. The huge increase in incoming attacks, the and the and the and the in the really, fast time that and kind of the the the the the short time where attackers can close such loops as I've shown here before. And so they need to be able to react automatically becomes more and more important.

And that capability, of course, is also, some indications to the maturity of a solution. Solutions that can automatically stop things, of course, need to have a very, very high, level of fidelity on what they detect. Otherwise, it would create other types of problems. So, of course, that kind of has to go end in hand, and it needs to have, of course, very granular a foothold to really only, block what's required and not kind of have any residual damage. And so this is, I think, a more sophisticated capability, but one that I believe is crucial when protecting against run time threats.

And it can be used to multiple, for for multiple different, means that I'm going to detail. First and foremost to enforce the immutability.

Make sure that, especially when working on cloud native with the images that you've built scanned and validated in your organizations, making sure that you are not seeing any code injected or are not right changes are happening to your workload in runtime. That nothing that you did not plan to be there is suddenly getting in there and running in your environment.

And this is one type of run time prevention that can be applied. Another one is prevent the executables and commands to run if there weren't in the original image, which is another, which is another type of, of of of control that can be run, and that really really limits attackers capabilities to, to gain foothold.

The block known and unknown malware on access or on download to block malicious activities that are in high fidelity that are there that are that are high fidelity attacks. Ones that are kind of identifies as attacks, in very high probability and actually block them at real time.

And also to apply virtual patching to temporary block access to vulnerable software component. This is another very sophisticated capability that, handles the case where you have a software that we know is vulnerable. But, is but it's but you you still need to run it, because, multiples of reason the custom the main one being that, there is no pack yet for this specific software, and there is no, back from the whether it's an open source vendor or a software, supplier or a software vendor that you bought this from, And so you're in a place where you have a vulnerability that you know of, and you wanna do something about it.

And virtual packaging allows you to temporarily block access to the specific software components that, the specific vulnerability leverages in order to penetrate again foothold or gain any type of attacker advantage and just kind of block those based on the runtime presence that the security tool would have and allow you to then still run your software, but be but but be secure, from that threat that you identified.

And then the third one should also be very important is what we call incident response. Whether you responded in real time or not, whether things were blocked in real time or detected and then moved to the saw to to kind of, handle manually.

There is usually the need, to understand what happened why did it happen? Where did it start? Where else did it affect? And so forensic level insights provide a detailed outline of that. Process that can really help you identify all that.

And this involves, as I said, forensics, it involves access reports, policies, ins and, and the ability to, manage your incidents in a multi cloud and complex environment.

It involves all kinds of, supporting tools, like, mapping them to timelines, decision, timelines, the process trees, mapping them to the Mitra attack framework that really kind of shows you how the attacker moved from gaining access to actually exploiting And eventually, it involves, of course, the more, straightforward, but very important integration into your, sock systems where you actually, investigate threats across other verticals as well, which is usually your same source systems.

So seeing how this tees up here with a with a with Asyn Acwa to see and stop the most sophisticated attacks using the Acwa EVF agent.

Starting off, as I said, by behavioral detection, stopping signature based attack by recognized known malware IDs files, etcetera, stopping threats by recognized patterns, basically attack patterns that we identify through our honey pots through our research and through multiple other sources that are not released during leverages, then on the second layer, preventing mutability and gain a higher level of protection and hardening of your environment, using isolation, using, runtime protection to isolate your environment and basically prevent attackers from gaining ground.

Then the next layer is protecting the application with controls over processes, resources, users, etcetera, with a very rich policy set that you can set up across the environment, across all of your workload or some of your workloads depending on the scopes and really apply them in a way that is actually prevented or executed at runtime.

And eventually, the ability to quickly understand attack patterns and determine organizational impact.

The incident response IR part.

To tee up what we just kind of went through went over here. I think What we were saying here, and this is very important, and we really, I think, surfaces from the head crop exam head crop example is that scanning and visibility are not enough They don't get you, all the way.

They're both because you need to monitor for run time. Activity, there is always a chance for attackers to come in even if you minimize the risk side, the attacker the opportunity for the attacker. There is always the second dimension of what the attacker is doing and the threats and the and the capabilities that they have and the ability to kind of create some level of attack. And so it's not enough. And as we've seen here, Sometimes they can get very sophisticated and not even show up for traditional tools because they're using memory techniques like, we've shown here in head scrub.

Second, security eventually means stopping attacks.

We believe that, and this is something that we see all the time.

Bironautilus team, by the way, because we actually monitor many customer environments, but also our own honey pots. And our own environments. And we're seeing that, the attackers are getting more sophisticated with usage with with use of AI, but with just use of basic automation as well. They're getting more sophisticated, both in terms of, identifying vulnerabilities, spin up a new VM online with a certain vulnerability, and it's prob it's it's around the hour, that, that it's kind of that's the time that we usually measure where, where where this, will be right, where attackers will try to exploit that vulnerability.

So there is continuous probing, their ability to close the loop is much, much faster using automation.

And that means that responders, that defenders needs to be much faster and much more scalable as well. And that requires automation. That requires the ability to close the loop automatically in efficient way, of course.

And I and and and this is kind of a second, very important principle.

I think a third one is what we call frustrating the attackers.

When you when you create, when when you kind of enforce immutability, when you enforce strict policies on what can run and and make sure that only things that went through your entire validation process that came in with the, with the emails, you can actually run-in production when you don't allow attackers to kind of cut through and get and run their workloads within your environment that, that in itself is, is basically that ability to prevent that drift of all types can really really make it harder for attackers. They need to get in much earlier. It really makes a lot of their techniques unusable and it's super important, and just kind of raising the bar, and effectively protecting your workloads everywhere.

And kind of to to tee this up for more information if you guys want to kind of dig in deeper into the headcrub, into the headcrub, attack story. And what we've discovered here in the continuous team, then, you're free to scan, this link and, really kind of, see, our blog and, and more, and more, information that we've published around it.

Thank you, Galad.

I also dropped a link to this webinar in the chat team.

This will be a great one. Researchers will be speaking, walk everyone through sort of the, the fascinating story of the attack, the tactics used, to actually communicate with the bad actor and their technical analysis of both the headcrap malware and the persistent tool.

After we wrap here everyone, you'll see a survey with some three quick questions, but, I just wanna thank everyone for your time. Galad, thank you so much for joining us today, and keep a lookout for future Aqua events, and we hope to see you at the next one. Thank you very much everyone. Take care.
Watch Next