At Permiso Security, we’ve always believed that curiosity fuels innovation. What started as a routine investigation into the reliability of AWS CloudTrail logs for monitoring IAM policy changes quickly turned into something far more significant. Collaborating with Daniel Bohannon, and powered by our open-source framework, SkyScalpel, our team uncovered a subtle yet critical logging evasion vulnerability within AWS environments.

The issue lies in the differing size limitations of individual AWS CloudTrail logs versus the actual content being logged. By exploiting whitespace and other syntactic quirks, an attacker can create valid IAM policies that effectively bypass CloudTrail logging. These omissions leave security teams in the dark, unable to track unauthorized changes or detect malicious activity. While this blog post focuses on IAM policies as the desired resource to evade from being logged, the Permiso team identified dozens of additional resource types in numerous AWS services that exhibit this same evasive quality.

This discovery is not just about highlighting a theoretical flaw — it’s about identifying real-world risks and contributing to the continuous improvement of cloud security. In this blog, we will dive deep into how our research unfolded, the technical nuances of this logging evasion technique, and its implications for AWS users. Finally, we’ll discuss how AWS is working to address the issue and what organizations can do to protect themselves in the meantime.

Unpacking the Logging Evasion Vulnerability

AWS CloudTrail serves as a critical auditing and monitoring tool, logging activity across AWS environments and helping security teams detect unauthorized changes while maintaining compliance. However, our investigation uncovered a significant gap in how CloudTrail enforces requestParameters size limitations, which do not always align with the potential size of input values. A specific example will highlight these size differences that are not completely detailed in AWS documentation.

AWS documentation outlines specific size limits for IAM policies, varying based on the policy type — inline or managed. These limits range from 2,048 characters for smaller policies to 10,240 characters for larger ones, depending on the policy's purpose and type. However, the detail in AWS documentation that caught our attention is the statement: “IAM doesn’t count white space when calculating the size of a policy against this limit.”

During our prior research into IAM policy obfuscation, we examined how JSON handles Unicode encoding and found that characters like "a" (\\u0061) are represented as six characters instead of one, significantly inflating the size of the JSON structure and counting toward the IAM policy size limit.

Our next question was: “does whitespace really not count when calculating the policy size limit?” When creating a max-sized policy with no insignificant whitespace, we found the documentation to be correct (initially) - namely, adding whitespace did not prevent us creating or updating the IAM policy. However, we quickly noticed that the whitespace is still stored by AWS and not discarded as is the case during JSON document deserialization.

This led us to our next question: If a policy is evaluated as one size (excluding whitespace) for the policy size limit constraint during creation/modification, but AWS stores the complete, larger policy (including whitespace) on the backend, does the AWS backend have a separate size limitation for the policy including whitespace?

The following image illustrates the first undocumented size limitation, demonstrating what happens when a policy, inflated with whitespace, exceeds an unspecified size threshold during an iam:CreatePolicy operation. Despite the documentation's assertion, the validation process clearly counts the overall policy size (including whitespace), resulting in the following error:

The above error displays an undocumented total policy size limitation of 131,072 (2^17) characters, which includes all whitespace and Unicode syntax counted literally. But a policy of size ≤ 131,072 characters can successfully be created and is notably larger than the deserialized policy size limit of 2,048-10,240 characters detailed in AWS documentation. However, when policies approach this maximum size limit, CloudTrail introduces a separate issue revealing the second undocumented size limitation.

In the requestParameters field of CloudTrail logs, the policy that was created/updated is typically displayed, including details such as the policy name and the full policy document content. However, policies inflated with obscene amounts of insignificant whitespace (such as spaces, tabs, or line breaks) at a certain size begin to be omitted from CloudTrail logs and are instead rendered as seen below:

This CloudTrail logging evasion appears to occur within the range of 102,401 to 131,072 characters, including whitespace. Policies falling within this size range trigger CloudTrail’s logging size constraints, resulting in incomplete logging where critical details are omitted.

One noteworthy detail is that the policy that is too large to be logged by CloudTrail is still correctly stored by AWS and can be retrieved via the corresponding API call (e.g. iam:GetPolicyVersion for the managed policy example). However, in scenarios where a policy document is listed only in CloudTrail logs (e.g. sts:AssumeRole) or does not contain versions (e.g. inline policies) and can be created with oversized log evasion, used and then updated with a normal-sized benign policy, this logging evasion can spell total lack of visibility into the contents of the policy.

Steps to Reproduce the Vulnerability

The logging evasion occurs due to undocumented size limitations for the requestParameters field in AWS CloudTrail logs, specifically paired with the second undocumented IAM policy size evaluation pertaining to whitespace characters. While AWS does not count whitespace during deserialized policy size evaluation in the iam:CreatePolicy operation and does count whitespace in the undocumented 131,072-character serialized policy size limit, CloudTrail logs fail to capture policies that are “oversized” based on whitespace characters, leaving critical policy details omitted from logs.

Here is a step-by-step breakdown of how this vulnerability can be exploited:

1. Construct an oversized IAM policy: Start with a valid policy structure and inflate its size by adding whitespace. Ensure the policy’s total size falls within the size of 102,401-131,072 characters.

2. Create the policy: Use the AWS CLI or API to create or update the policy via the iam:CreatePolicy or iam:UpdatePolicy operation, respectively. The policy will be successfully created/updated in the environment, but its content will be omitted from CloudTrail logs.

3. Trigger logging evasion: CloudTrail logs for this operation will display requestParameters too large, effectively hiding the actual policy content from runtime visibility.

This technical behavior creates an opportunity for attackers to abuse policies without leaving a clear audit trail in CloudTrail. By exploiting this gap, threat actors can bypass monitoring performed through CloudTrail and can effectively obscure unauthorized changes.

Disclosure Timeline: AWS Response & Planned Mitigation

Below is the timeline of Permiso’s disclosure of this vulnerability to AWS and their timely response:

• 2024-12-23 - Permiso notifies AWS of CloudTrail logging evasion vulnerability and provides example oversized policy to reproduce issue.
• 2024-12-23 - AWS acknowledges they are looking into the issue.
• 2025-01-23 - AWS confirms that while IAM policy data evades CloudTrail logs in cases of excessive size, it is still logged in their backend systems and can be retrieved by AWS in rare cases where a customer requests their retrieval. AWS concludes it does not warrant an urgent security fix (their reasoning being: 1 - its low frequency of occurrence and 2 - its prerequisite that an unauthorized user must have policy mutation permissions in the first place) but confirms it is on the Q3 2025 roadmap to be resolved.

AWS highlighted their commitment to balancing audit completeness with system performance and recommended monitoring CloudTrail logs for unusual omissions and auditing policies for excessive whitespace until the fix is live.

Recommendations for Organizations

While AWS works on a fix, organizations should monitor CloudTrail logs for entries with the requestParameters property containing reason:”requestParameters too large” and omitted:true . Permiso identified numerous additional AWS services that encounter this same scenario, though none were deemed malicious as of the time of writing this blog. Permiso also recommends auditing IAM policies (via API and runtime CloudTrail logs) for excessive whitespace, particularly those nearing the 102,401-character threshold. If you need to check whether a policy contains excessive whitespace, you can use SkyScalpel to detect and flag such anomalies, but shifting focus to CloudTrail logs for additional suspicious policy obfuscation techniques is our strong recommendation for confirmed yet subtle scenarios not outlined in this blog post.

Permiso Coverage

For Permiso customers, the following alerts assist in detecting related threats:

Conclusion

Our findings reveal two significant gaps in how AWS handles policy size evaluation and large requests in CloudTrail logging. First, creating or updating an IAM policy exceeding 131,072 characters (including whitespace) fails validation, contradicting AWS documentation which states that whitespace does not count towards policy size limits. Second, CloudTrail logs fail to capture full request details for requests in the size range of 102,401–131,072 characters. Instead, the requestParameters field flags these requests as too large and omits their content in runtime logs entirely.

These findings expose a discrepancy between AWS’s documented behavior and the actual handling of large requests, resulting in operational challenges and blind spots for security teams monitoring CloudTrail logs. This reduced visibility into oversized or obfuscated requests could hinder organizations’ ability to detect unauthorized changes or activity.