Amazon CloudWatch
Intro on CloudWatch & CloudWatch Logs from Cantrill’s SAA-C03
CloudWatch (CW) Fundamentals
CloudWatch Logs
CWLogs Insights
Ref: https://www.udemy.com/course/aws-certified-machine-learning-engineer-associate-mla-c01/learn/lecture/45357125
- 🔧 Allows to query/search and analyze log data stored in CWLogs
- Example: find a specific IP inside a log, count occurrences of “ERROR” in your logs…
- Provides a purpose-built query language
- Automatically discovers fields from AWS services and JSON log events
- Fetch desired event fields, filter based on conditions, calculate aggregate statistics, sort events, limit number of events…
- Can save queries and add them to CloudWatch Dashboards
- Can query multiple Log Groups in different AWS accounts
- ❗ It’s a query engine, NOT a real-time engine!
- Will only query your historical logs
- Diagram
CWLogs Subscriptions
Ref: https://www.udemy.com/course/aws-certified-machine-learning-engineer-associate-mla-c01/learn/lecture/45357125
- 🔧 Real-time delivery of log events from CWLogs (for processing & analysis)
- Supported destinations: Kinesis Data Streams, Data Firehose, Lambda
- Subscription Filter: filter which logs are delivered to which destination
- Diagram
- Cross-Account subscriptions (requires IAM permissions)
- Subscriptions can be used to for multi-region & multi-account log aggregation
- ‼️ Do NOT confuse with S3 Export!!
- S3 export dumps CWLogs into S3 in bulk… can take up to 12h → Not near-real-time even!