CloudWatchLogs / Client / put_account_policy

put_account_policy

CloudWatchLogs.Client.put_account_policy(**kwargs)

Creates an account-level data protection policy, subscription filter policy, field index policy, transformer policy, metric extraction policy, or retention policy that applies to all log groups, a subset of log groups, or a data source name and type combination in the account. Account-level policies are Region-specific: a policy applies only to log groups in the Region where you create it. To apply a policy across multiple Regions, create the policy separately in each Region.

Warning

PutAccountPolicy is an account-wide administrative operation intended for CloudWatch Logs administrators. Because it affects all log groups (or a broad subset) in the account, you should grant logs:PutAccountPolicy permissions only to administrators who manage logging configuration across the account, not to application teams or individual log group owners.

Conflict resolution between account-level and log-group-level policies

When both an account-level policy and a log-group-level policy of the same type apply to a log group, the resolution depends on the policy type:

  • Data protection — The two policies are cumulative. Any sensitive term specified in either the account-level or the log-group-level policy is masked.

  • Subscription filters — Account-level and log-group-level subscription filters are additive. A log group can have up to 1 account-level and up to 2 log-group-level subscription filters.

  • Transformers — A log-group-level transformer overrides the account-level transformer. If a log group has its own transformer, it ignores the account-level transformer policy.

  • Field index policies — If a log group has its own field index policy (created with PutIndexPolicy), any account-level policy that uses LogGroupNamePrefix selection criteria or has no selection criteria is ignored for that log group. However, account-level policies that use DataSourceName and DataSourceType selection criteria still apply alongside the log-group-level policy.

  • Metric extraction policies — Metric extraction policies are account-level only and have no log-group-level equivalent, so no conflict resolution applies.

  • Retention policies — An account-level retention policy overrides the retention setting of the log group.

For field index policies, you can configure indexed fields as facets to enable interactive exploration of your logs. Facets provide value distributions and counts for indexed fields in the CloudWatch Logs Insights console without requiring query execution. For more information, see Use facets to group and explore logs.

To use this operation, you must be signed on with the correct permissions depending on the type of policy that you are creating.

  • To create a data protection policy, you must have the logs:PutDataProtectionPolicy and logs:PutAccountPolicy permissions.

  • To create a subscription filter policy, you must have the logs:PutSubscriptionFilter and logs:PutAccountPolicy permissions.

  • To create a transformer policy, you must have the logs:PutTransformer and logs:PutAccountPolicy permissions.

  • To create a field index policy, you must have the logs:PutIndexPolicy and logs:PutAccountPolicy permissions.

  • To configure facets for field index policies, you must have the logs:PutIndexPolicy and logs:PutAccountPolicy permissions.

  • To create a metric extraction policy, you must have the logs:PutMetricExtractionPolicy and logs:PutAccountPolicy permissions.

  • To create a retention policy, you must have the logs:PutRetentionPolicy and logs:PutAccountPolicy permissions. A dry run requires the same permissions as the operation that it previews, and no additional permissions.

Data protection policy

A data protection policy can help safeguard sensitive data that’s ingested by your log groups by auditing and masking the sensitive log data. Each account can have only one account-level data protection policy.

Warning

Sensitive data is detected and masked when it is ingested into a log group. When you set a data protection policy, log events ingested into the log groups before that time are not masked.

If you use PutAccountPolicy to create a data protection policy for your whole account, it applies to both existing log groups and all log groups that are created later in this account. The account-level policy is applied to existing log groups with eventual consistency. It might take up to 5 minutes before sensitive data in existing log groups begins to be masked.

By default, when a user views a log event that includes masked data, the sensitive data is replaced by asterisks. A user who has the logs:Unmask permission can use a GetLogEvents or FilterLogEvents operation with the unmask parameter set to true to view the unmasked log events. Users with the logs:Unmask can also view unmasked data in the CloudWatch Logs console by running a CloudWatch Logs Insights query with the unmask query command.

For more information, including a list of types of data that can be audited and masked, see Protect sensitive log data with masking.

To use the PutAccountPolicy operation for a data protection policy, you must be signed on with the logs:PutDataProtectionPolicy and logs:PutAccountPolicy permissions.

The PutAccountPolicy operation applies to all log groups in the account. You can use PutDataProtectionPolicy to create a data protection policy that applies to just one log group. If a log group has its own data protection policy and the account also has an account-level data protection policy, then the two policies are cumulative. Any sensitive term specified in either policy is masked.

Subscription filter policy

A subscription filter policy sets up a real-time feed of log events from CloudWatch Logs to other Amazon Web Services services. Account-level subscription filter policies apply to both existing log groups and log groups that are created later in this account. Supported destinations are Kinesis Data Streams, Firehose, and Lambda. When log events are sent to the receiving service, they are Base64 encoded and compressed with the GZIP format.

The following destinations are supported for subscription filters:

  • An Kinesis Data Streams data stream in the same account as the subscription policy, for same-account delivery.

  • An Firehose data stream in the same account as the subscription policy, for same-account delivery.

  • A Lambda function in the same account as the subscription policy, for same-account delivery.

  • A logical destination in a different account created with PutDestination, for cross-account delivery. Kinesis Data Streams and Firehose are supported as logical destinations.

Each account can have one account-level subscription filter policy per Region. If you are updating an existing filter, you must specify the correct name in PolicyName. To perform a PutAccountPolicy subscription filter operation for any destination except a Lambda function, you must also have the iam:PassRole permission.

Transformer policy

Creates or updates a log transformer policy for your account. You use log transformers to transform log events into a different format, making them easier for you to process and analyze. You can also transform logs from different sources into standardized formats that contain relevant, source-specific information. After you have created a transformer, CloudWatch Logs performs this transformation at the time of log ingestion. You can then refer to the transformed versions of the logs during operations such as querying with CloudWatch Logs Insights or creating metric filters or subscription filters.

You can also use a transformer to copy metadata from metadata keys into the log events themselves. This metadata can include log group name, log stream name, account ID and Region.

A transformer for a log group is a series of processors, where each processor applies one type of transformation to the log events ingested into this log group. For more information about the available processors to use in a transformer, see Processors that you can use.

Having log events in standardized format enables visibility across your applications for your log analysis, reporting, and alarming needs. CloudWatch Logs provides transformation for common log types with out-of-the-box transformation templates for major Amazon Web Services log sources such as VPC flow logs, Lambda, and Amazon RDS. You can use pre-built transformation templates or create custom transformation policies.

You can create transformers only for the log groups in the Standard log class.

You can have one account-level transformer policy that applies to all log groups in the account. Or you can create as many as 20 account-level transformer policies that are each scoped to a subset of log groups with the selectionCriteria parameter. If you have multiple account-level transformer policies with selection criteria, no two of them can use the same or overlapping log group name prefixes. For example, if you have one policy filtered to log groups that start with my-log, you can’t have another transformer policy filtered to my-logpprod or my-logging.

You can also set up a transformer at the log-group level. For more information, see PutTransformer. If there is both a log-group level transformer created with PutTransformer and an account-level transformer that could apply to the same log group, the log group uses only the log-group level transformer. It ignores the account-level transformer.

Field index policy

You can use field index policies to create indexes on fields found in log events for a log group or data source name and type combination. Creating field indexes can help lower the scan volume for CloudWatch Logs Insights queries that reference those fields, because these queries attempt to skip the processing of log events that are known to not match the indexed field. Good fields to index are fields that you often need to query for and fields or values that match only a small fraction of the total log events. Common examples of indexes include request ID, session ID, user IDs, or instance IDs. For more information, see Create field indexes to improve query performance and reduce costs

To find the fields that are in your log group events, use the GetLogGroupFields operation. To find the fields for a data source use the GetLogFields operation.

For example, suppose you have created a field index for requestId. Then, any CloudWatch Logs Insights query on that log group that includes requestId = value or requestId in [value, value, ...] will attempt to process only the log events where the indexed field matches the specified value.

Matches of log events to the names of indexed fields are case-sensitive. For example, an indexed field of RequestId won’t match a log event containing requestId.

You can have one account-level field index policy that applies to all log groups in the account. Or you can create as many as 20 account-level field index policies that are each scoped to a subset of log groups using LogGroupNamePrefix with the selectionCriteria parameter. You can have another 20 account-level field index policies using DataSourceName and DataSourceType for the selectionCriteria parameter. If you have multiple account-level index policies with LogGroupNamePrefix selection criteria, no two of them can use the same or overlapping log group name prefixes. For example, if you have one policy filtered to log groups that start with my-log, you can’t have another field index policy filtered to my-logpprod or my-logging. Similarly, if you have multiple account-level index policies with DataSourceName and DataSourceType selection criteria, no two of them can use the same data source name and type combination. For example, if you have one policy filtered to the data source name amazon_vpc and data source type flow you cannot create another policy with this combination.

If you create an account-level field index policy in a monitoring account in cross-account observability, the policy is applied only to the monitoring account and not to any source accounts.

CloudWatch Logs provides default field indexes for all log groups in the Standard log class. Default field indexes are automatically available for the following fields:

  • @logStream

  • @aws.region

  • @aws.account

  • @source.log

  • @data_source_name

  • @data_source_type

  • @data_format

  • traceId

  • severityText

  • attributes.session.id

CloudWatch Logs provides default field indexes for certain data source name and type combinations as well. Default field indexes are automatically available for the following data source name and type combinations as identified in the following list:

amazon_vpc.flow

  • action

  • logStatus

  • region

  • flowDirection

  • type

amazon_route53.resolver_query

  • transport

  • rcode

aws_waf.access

  • action

  • httpRequest.country

aws_cloudtrail.data, aws_cloudtrail.management

  • eventSource

  • eventName

  • awsRegion

  • userAgent

  • errorCode

  • eventType

  • managementEvent

  • readOnly

  • eventCategory

  • requestId

Default field indexes are in addition to any custom field indexes you define within your policy. Default field indexes are not counted towards your field index quota.

If you want to create a field index policy for a single log group, you can use PutIndexPolicy instead of PutAccountPolicy. If you do so, that log group will use that log-group level policy and any account-level policies that match at the data source level; any account-level policy that matches at the log group level (for example, no selection criteria or log group name prefix selection criteria) will be ignored.

Metric extraction policy

A metric extraction policy controls whether CloudWatch Metrics can be created through the Embedded Metrics Format (EMF) for log groups in your account. By default, EMF metric creation is enabled for all log groups. You can use metric extraction policies to disable EMF metric creation for your entire account or specific log groups.

When a policy disables EMF metric creation for a log group, log events in the EMF format are still ingested, but no CloudWatch Metrics are created from them.

Warning

Creating a policy disables metrics for Amazon Web Services features that use EMF to create metrics, such as CloudWatch Container Insights and CloudWatch Application Signals. To prevent turning off those features by accident, we recommend that you exclude the underlying log-groups through a selection-criteria such as LogGroupNamePrefix NOT IN ["/aws/containerinsights", "/aws/ecs/containerinsights", "/aws/application-signals/data"].

Each account can have either one account-level metric extraction policy that applies to all log groups, or up to 5 policies that are each scoped to a subset of log groups with the selectionCriteria parameter. The selection criteria supports filtering by LogGroupName and LogGroupNamePrefix using the operators IN and NOT IN. You can specify up to 50 values in each IN or NOT IN list.

The selection criteria can be specified in these formats:

LogGroupName IN ["log-group-1", "log-group-2"]

LogGroupNamePrefix NOT IN ["/aws/prefix1", "/aws/prefix2"]

If you have multiple account-level metric extraction policies with selection criteria, no two of them can have overlapping criteria. For example, if you have one policy with selection criteria LogGroupNamePrefix IN ["my-log"], you can’t have another metric extraction policy with selection criteria LogGroupNamePrefix IN ["/my-log-prod"] or LogGroupNamePrefix IN ["/my-logging"], as the set of log groups matching these prefixes would be a subset of the log groups matching the first policy’s prefix, creating an overlap.

When using NOT IN, only one policy with this operator is allowed per account.

When combining policies with IN and NOT IN operators, the overlap check ensures that policies don’t have conflicting effects. Two policies with IN and NOT IN operators do not overlap if and only if every value in the IN ``policy is completely contained within some value in the ``NOT IN policy. For example:

  • If you have a NOT IN policy for prefix "/aws/lambda", you can create an IN policy for the exact log group name "/aws/lambda/function1" because the set of log groups matching "/aws/lambda/function1" is a subset of the log groups matching "/aws/lambda".

  • If you have a NOT IN policy for prefix "/aws/lambda", you cannot create an IN policy for prefix "/aws" because the set of log groups matching "/aws" is not a subset of the log groups matching "/aws/lambda".

Retention policy

A retention policy sets the retention period for log groups in your account. You can use an account-level retention policy to set a default retention period for all log groups in the account, or for a subset of log groups that you specify with the selectionCriteria parameter. CloudWatch Logs automatically deletes log events that are older than the retention period.

Each account can have up to 20 account-level retention policies in each Region. To scope a retention policy to a subset of log groups, use LogGroupNamePrefix in selectionCriteria. Retention policies support log group name prefixes only. If you omit selectionCriteria, the policy applies to all log groups in the account.

Unlike other account-level policy types, retention policies can have overlapping selection criteria. When more than one retention policy applies to a log group, the policy with the longest retention period is used. A policy that specifies NEVER_EXPIRE is treated as the longest retention period. This lets you set a baseline retention period for the account, apply longer retention periods to specific log groups, and exempt specific log groups from expiring at all. If two policies that apply to the same log group specify the same retention period, the policy that was created first is the one reported in retentionPolicyName.

Note

Account-level retention policies require a minimum retention period of 30 days.

Note

A retention policy takes effect 24 hours after you create it. If you update a policy, the new retention period takes effect 24 hours after the update, and the previous retention period stays in effect until then. During this period, DescribeLogGroups and log event expiration use the retention period that is currently in effect, but a dry run reports the impact as if the policy were already in effect.

Before you create, update, or delete a retention policy, you can specify true for dryRun to preview how many log groups would have a longer, shorter, or unchanged retention period. For more information, see DryRunResult.

Warning

An account-level retention policy replaces the retention period set on a log group, including when the log group’s own retention period is longer. For example, if a log group is set to retain log events for 10 years and you create a retention policy that sets a 90-day retention period for that log group, CloudWatch Logs deletes the log events in that log group that are older than 90 days. Specify true for dryRun to see which log groups would have a shorter retention period before you create or update the policy.

See also: AWS API Documentation

Request Syntax

response = client.put_account_policy(
    policyName='string',
    policyDocument='string',
    policyType='DATA_PROTECTION_POLICY'|'SUBSCRIPTION_FILTER_POLICY'|'FIELD_INDEX_POLICY'|'TRANSFORMER_POLICY'|'METRIC_EXTRACTION_POLICY'|'RETENTION_POLICY',
    scope='ALL',
    selectionCriteria='string',
    dryRun=True|False
)
Parameters:

  • policyName (string) –

    [REQUIRED]

    A name for the policy. This must be unique within the account and cannot start with aws/.

  • policyDocument (string) –

    [REQUIRED]

    Specify the policy, in JSON.

    Data protection policy

    A data protection policy must include two JSON blocks:

    • The first block must include both a DataIdentifer array and an Operation property with an Audit action. The DataIdentifer array lists the types of sensitive data that you want to mask. For more information about the available options, see Types of data that you can mask. The Operation property with an Audit action is required to find the sensitive data terms. This Audit action must contain a FindingsDestination object. You can optionally use that FindingsDestination object to list one or more destinations to send audit findings to. If you specify destinations such as log groups, Firehose streams, and S3 buckets, they must already exist.

    • The second block must include both a DataIdentifer array and an Operation property with an Deidentify action. The DataIdentifer array must exactly match the DataIdentifer array in the first block of the policy. The Operation property with the Deidentify action is what actually masks the data, and it must contain the "MaskConfig": {} object. The "MaskConfig": {} object must be empty.

    For an example data protection policy, see the Examples section on this page.

    Warning

    The contents of the two DataIdentifer arrays must match exactly.

    In addition to the two JSON blocks, the policyDocument can also include Name, Description, and Version fields. The Name is different than the operation’s policyName parameter, and is used as a dimension when CloudWatch Logs reports audit findings metrics to CloudWatch.

    The JSON specified in policyDocument can be up to 30,720 characters long.

    Subscription filter policy

    A subscription filter policy can include the following attributes in a JSON block:

    • DestinationArn The ARN of the destination to deliver log events to. Supported destinations are:

      • An Kinesis Data Streams data stream in the same account as the subscription policy, for same-account delivery.

      • An Firehose data stream in the same account as the subscription policy, for same-account delivery.

      • A Lambda function in the same account as the subscription policy, for same-account delivery.

      • A logical destination in a different account created with PutDestination, for cross-account delivery. Kinesis Data Streams and Firehose are supported as logical destinations.

    • RoleArn The ARN of an IAM role that grants CloudWatch Logs permissions to deliver ingested log events to the destination stream. You don’t need to provide the ARN when you are working with a logical destination for cross-account delivery.

    • FilterPattern A filter pattern for subscribing to a filtered stream of log events.

    • Distribution The method used to distribute log data to the destination. By default, log data is grouped by log stream, but the grouping can be set to Random for a more even distribution. This property is only applicable when the destination is an Kinesis Data Streams data stream.

    Transformer policy

    A transformer policy must include one JSON block with the array of processors and their configurations. For more information about available processors, see Processors that you can use.

    Field index policy

    A field index filter policy can include the following attribute in a JSON block:

    • Fields The array of field indexes to create.

    • FieldsV2 The object of field indexes to create along with it’s type.

    It must contain at least one field index.

    The following is an example of an index policy document that creates indexes with different types.

    "policyDocument": "{ \"Fields\": [ \"TransactionId\" ], \"FieldsV2\": {\"RequestId\": {\"type\": \"FIELD_INDEX\"}, \"APIName\": {\"type\": \"FACET\"}, \"StatusCode\": {\"type\": \"FACET\"}}}"

    You can use FieldsV2 to specify the type for each field. Supported types are FIELD_INDEX and FACET. Field names within Fields and FieldsV2 must be mutually exclusive.

    Retention policy

    A retention policy must include the following attributes in a JSON block:

    retentionInDays

    The number of days to retain log events. Valid values are 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, and 3653. To retain log events indefinitely, specify the string NEVER_EXPIRE.

    enforcementMode

    Specifies which log groups the policy applies to. Valid values are NEW_ONLY and RETROACTIVE.

    RETROACTIVE applies the policy to every log group that matches the selection criteria, including log groups that existed before you created the policy.

    NEW_ONLY applies the policy only to log groups that are created after you create the policy. Log groups that already existed when you created the policy keep their own retention period.

    The following is an example of a retention policy document.

    "policyDocument": "{\"retentionInDays\": 90, \"enforcementMode\": \"RETROACTIVE\"}"

  • policyType (string) –

    [REQUIRED]

    The type of policy that you’re creating or updating.

  • scope (string) – Currently the only valid value for this parameter is ALL, which specifies that the data protection policy applies to all log groups in the account. If you omit this parameter, the default of ALL is used.

  • selectionCriteria (string) –

    Use this parameter to apply the new policy to a subset of log groups in the account or a data source name and type combination.

    Specifying selectionCriteria is valid only when you specify SUBSCRIPTION_FILTER_POLICY, FIELD_INDEX_POLICY, TRANSFORMER_POLICY, or RETENTION_POLICY for policyType.

    • If policyType is SUBSCRIPTION_FILTER_POLICY, the only supported selectionCriteria filter is LogGroupName NOT IN []

    • If policyType is TRANSFORMER_POLICY, the only supported selectionCriteria filter is LogGroupNamePrefix

    • If policyType is FIELD_INDEX_POLICY, the supported selectionCriteria filters are:

      • LogGroupNamePrefix

      • DataSourceName AND DataSourceType

    When you specify selectionCriteria for a field index policy you can use either LogGroupNamePrefix by itself or DataSourceName and DataSourceType together.

    • If policyType is RETENTION_POLICY, the only supported selectionCriteria filter is LogGroupNamePrefix. If you omit selectionCriteria, the retention policy applies to all log groups in the account.

    The selectionCriteria string can be up to 25KB in length. The length is determined by using its UTF-8 bytes.

    Using the selectionCriteria parameter with SUBSCRIPTION_FILTER_POLICY is useful to help prevent infinite loops. For more information, see Log recursion prevention.

  • dryRun (boolean) –

    Specifies whether to preview the impact of the policy without creating or updating it. If you specify true, CloudWatch Logs validates the request, does not save the policy, and returns the estimated impact in dryRunResult instead of accountPolicy. The default is false.

    Specifying dryRun is valid only when you specify RETENTION_POLICY for policyType. Only one dry run can be in progress for an account at a time.

Return type:

dict

Returns:

Response Syntax

{
    'accountPolicy': {
        'policyName': 'string',
        'policyDocument': 'string',
        'lastUpdatedTime': 123,
        'policyType': 'DATA_PROTECTION_POLICY'|'SUBSCRIPTION_FILTER_POLICY'|'FIELD_INDEX_POLICY'|'TRANSFORMER_POLICY'|'METRIC_EXTRACTION_POLICY'|'RETENTION_POLICY',
        'scope': 'ALL',
        'selectionCriteria': 'string',
        'accountId': 'string'
    },
    'dryRunResult': {
        'totalMatchedLogGroups': 123,
        'increasedRetentionLogGroupCount': 123,
        'decreasedRetentionLogGroupCount': 123,
        'unchangedRetentionLogGroupCount': 123,
        'failedAnalysisCount': 123,
        'topLogGroupsImpacted': [
            {
                'logGroupName': 'string',
                'currentRetention': 'string',
                'proposedRetention': 'string',
                'storedBytes': 123
            },
        ]
    }
}

Response Structure

  • (dict) –

    • accountPolicy (dict) –

      The account policy that you created.

      • policyName (string) –

        The name of the account policy.

      • policyDocument (string) –

        The policy document for this account policy.

        The JSON specified in policyDocument can be up to 30,720 characters.

      • lastUpdatedTime (integer) –

        The date and time that this policy was most recently updated.

      • policyType (string) –

        The type of policy for this account policy.

      • scope (string) –

        The scope of the account policy.

      • selectionCriteria (string) –

        The log group selection criteria that is used for this policy.

      • accountId (string) –

        The Amazon Web Services account ID that the policy applies to.

    • dryRunResult (dict) –

      The estimated impact of the policy on log groups in the account. This field is returned only when the request specifies dryRun as true.

      • totalMatchedLogGroups (integer) –

        The number of log groups in the scope of the policy that were analyzed. This value is the sum of increasedRetentionLogGroupCount, decreasedRetentionLogGroupCount, and unchangedRetentionLogGroupCount, and does not include log groups that could not be analyzed.

      • increasedRetentionLogGroupCount (integer) –

        The number of log groups whose effective retention period would become longer.

      • decreasedRetentionLogGroupCount (integer) –

        The number of log groups whose effective retention period would become shorter.

      • unchangedRetentionLogGroupCount (integer) –

        The number of log groups whose effective retention period would not change.

      • failedAnalysisCount (integer) –

        The number of log groups in the scope of the policy that could not be analyzed. These log groups are not included in totalMatchedLogGroups.

      • topLogGroupsImpacted (list) –

        Up to 20 log groups in the scope of the policy, sorted by stored bytes in descending order. This list can include log groups whose effective retention period would not change.

        • (dict) –

          Describes the estimated effect of an account-level retention policy change on one log group.

          • logGroupName (string) –

            The name of the log group.

          • currentRetention (string) –

            The current effective retention period of the log group, in days. A value of NEVER_EXPIRE indicates that log events in the log group never expire.

          • proposedRetention (string) –

            The effective retention period of the log group, in days, after the change. A value of NEVER_EXPIRE indicates that log events in the log group would never expire.

          • storedBytes (integer) –

            The number of bytes stored in the log group.

Exceptions