For the past several years, the security and engineering community have repeatedly heard the adage that "identity is the new perimeter." While identity has long been a pillar to managing security risks in various environments, it's perhaps never been more relevant than it is today in the cloud. Earlier this month, Crowdstrike released "Nowhere to Hide," a 50+ page threat hunting report from observations their team has made from over 200 different adversary groups in the last year. The results demonstrate how attacks against cloud environments have changed and how threat actors are continuously honing their TTPs in order to adapt to the ever-changing cloud landscape.
In this blog article, we will unpack some components of Crowdstrike’s report, as well as couple that research with what we have observed in customers’ cloud environments and what we found in our recent Cloud Detection and Response survey. We will illustrate how threat actors are navigating across multiple layers of the cloud environment to orchestrate advanced attacks against environments. We will also provide some actionable steps your team can take in order to bolster your defenses against these attacks and better secure your cloud environment.
Over the course of a twelve month period between 2022-2023, Crowdstrike observed a 160% increase in attempts to gather secret keys and other credential materials via cloud instance metadata APIs. The APIs that drive much of the technology in the cloud have greatly improved the efficiency in the transfer of data and the speed of development and deployment lifecycles. The growth of API driven infrastructure like microservices, serverless technologies or hybrid cloud environments, in addition to CI/CD pipelines and data lakes has also brought with it a significant increase in the number of breaches caused by compromised API credentials.
These secrets are leaked across code repositories, CI systems, artifact repositories, cloud hosting environments and even team collaboration tools like Slack. Research conducted with a honeypot GitHub repo found that on average, It takes hackers 1 minute to find and abuse credentials exposed on GitHub. The findings the honeypot yielded are consistent with our own real world experience were we have observed accidentally disclosed keys being accessed by multiple threat actor groups within minutes of disclosure. In fact, in most situations, threat actors discover and leverage the keys to access, persist, and move laterally within victim environments within an hour or two. This is giving attackers access to sensitive data with ease - and often times if the credentials don’t yield sensitive information, toxic combinations in the permissions associated with they key or overly permissive configurations can give the attacker the opportunity to continue to move laterally, escalate their privileges, or access sensitive data elsewhere.
This problem is exacerbated by the growing number of credentials being sold on the dark web. In their report, Crowdstrike noted a 147% increase in access broker advertisements in criminal or underground communities. The growth in the supply of these services is a direct reflection of the increased demand from individuals or groups that don’t have the time, or ability to harvest these credentials themselves. Commodity actors often may not have the skills to harvest their own credentials, and thus leverage underground markets to purchase them and get to work.
Perhaps the biggest problem with provisioning API secrets is the lack of inherent visibility into who created the secrets, how they are being used or who is using them. This creates a significant blind spot for an organization’s ability to detect anomalies in the way those credentials are being used, if they’re being shared and other behavior that increases the security risk of these secrets.
Many organizations have tools to address the different layers of their cloud environment. One solution dedicated to securing their IDP if they are using Okta or Azure AD. Another solution for their IaaS/PaaS environments on AWS, Azure, or GCP, and sometimes yet another one for securing their containers. Its likely companies may also have a separate solution to secure their SaaS applications, and another tool for their CI/CD pipeline. While it’s a good thing that organizations are ensuring each layer of their cloud environment is secured, the silos between authentication boundaries of these systems and the corresponding solutions to help secure those applications and services becomes a blind spot, particularly to the advanced tactics of modern threat actors.
In 2019, the Capital One Breach shocked the security world, but even this attack, a mere four years later, is quite rudimentary compared to how cloud-based attacks are often orchestrated today.
Breaches where attackers exploit vulnerabilities from only a single cloud application or service are becoming less common. These days, attacks don’t always end with the threat actor getting access to an S3 bucket and stealing the data in it. Rather, AWS environments are often used as a jumping point to escalate privileges or harvest credentials that will give them access to another environment where they are able to access sensitive data, or find additional access keys. For example, a threat actor may get access to an artifact repository and steal a docker image or upload a nefarious one or breach a GitHub account and clone repositories.
In a far cry from what we witnessed with the malicious threat actor targeting a single service in Capital One, modern threat actors are adapting their TTPs to break across authentication boundaries with ease. The P0 labs team has observed multiple threat groups navigating across Okta > AWS > Azure > Salesforce > M365 > Jira > Terraform > Github — and the list goes on. They are poised to uncover methods to increase their privileges, gain access to more resources, and work their way up the chain of sensitive data they are able to access. This all happens within the matter of days, sometimes as little as hours. Attackers are objective oriented and don’t think in terms of boundaries - tools designed to detect their activities need to be able to traverse boundaries and show the entire picture of activity.
The example above illustrates how modern threat actors traverse multiple layers of cloud environments during the course of their campaign. Their goals as they move across these authentication boundaries is to learn more about the environment in order to harvest credentials, escalate their privileges and gain access to additional sensitive data. Modern cloud security solutions like CPSMs, ITDR, SSPMs and other tools simply aren’t built to detect elaborate attacks that span across multiple cloud environments. Sure, each solution may be able to fire alerts on atomic events that are occurring in each application or group of applications, but that doesn’t tell the whole story of exactly how the attack was orchestrated, where the attacker moved throughout the environment and more importantly, how they were able to move from one service or application to the next. You’re only seeing one small piece of a picture and in order to be able to piece this information together, you’re often scrambling over the course of days or weeks to dig through logs in an effort to replay the attack. Even then, you may only have one piece of what actually unfolded. You’re left with some fragmented data that doesn’t tell a comprehensive story of exactly how the attack was orchestrated.
Not only are these attacks orchestrated across environments where multiple applications and services are targeted and infiltrated, but many companies that are strategically targeted by threat actor groups often face a barrage of attacks against their network on a regular basis. Crowdstrike’s report observed one potential intrusion every 7 minutes and the average breakout time for interactive eCrime intrusion activity was 79 minutes. Advanced threat actor groups and commodity threat actors alike are constantly waging war against environments, either through strategic attacks or repeated broad-based, spray and pray methods.
A statistic previously cited by Crowdstrike, and mentioned yet again in their report is that 80% of breaches used compromised identities, of which 62% of interactive intrusions involving the abuse of valid accounts, with 34% of intrusions specifically involved with the use of domain accounts or default accounts. While many companies make an active effort to achieve least privilege for the identities in their environment, it really does only take one overprivileged identity to cause a serious data breach.
Identities entail human, machine, and vendor users. Much of the focus when it comes to managing the privileges falls with human users, and much less emphasis seems to fall on the latter two.
Over the course of observing customer environments last year, we found that, on average, over 90% of privileges assigned to identities are never even used. CSPM vendors, those vendors tasked with securing cloud environments are some of the biggest offenders of this. CSPM vendors will often ‘require’ a very high number of privileges, some of which are sensitive in nature, but only actually use a fraction of the privileges that have been granted to them. On average CSPM vendors monitored by Permiso are granted over 1500 privileges, and only use 11% (170) of those.
All of this has translated into a 95% increase in cloud exploitation in 2022 and a 3x increase in cases involving could-conscious threat actors according to Crowdstrike. Threat actors are leveraging compromised credentials with ease, either through scraping public repositories, finding them in environments they’ve gained access into, or purchasing them on the dark web. Once they’re able to gain access, they are able to move laterally with ease by escalating their privileges or leveraging other compromised identities. Rinse and repeat.
Looking at cloud security through the lens of posture and focusing on managing to eliminate misconfigured resources is no longer enough - it’s table stakes. Runtime visibility and monitoring will quickly become a must have in order to be able to keep up with the modern TTPs of threat actors and effectively stymie the dynamic attack patterns against your environment.
There are a few steps security and engineering teams can take in order to help better defend themselves against modern cloud attacks.
1.) Develop a comprehensive inventory of identities - this includes human, machine, vendors - anyone that is touching your environment. This should be a living, breathing document that maps the list of identities to what they have access to. How sensitive is the data they are able to access? For each of your identities, report and monitor on the ratio or percentage of privileges used vs. privileges granted.
2.) Keep your keys safe - compromised credentials will continue to be a driving factor behind a good number of attacks in the near future. Ensure that secrets are being rotated every 90 days - it’s not uncommon for us to observe environments where API secrets are 4+ years in age. Much like passwords, these static values present security risks as they age. In addition to using a secrets vault, companies should prioritize developing more visibility into how their secrets are being used. Who has provisioned them, who has access to them, and where are they being shared?
3.) Develop a baseline of normal user behavior for each identity - By developing a baseline of what normal behavior is for each user, you’re able to create a behavioral profile that can be used to identify anomalous behavior. In many ways, users tend to be creatures of habit and to an extent, their behavior is somewhat routine. By establishing the baseline, you can set up rules in your SIEM or logging tools to detect any deviation from the baseline. That may just be suspicious activity, but could also be malicious as well.