Ref: https://learn.cantrill.io/courses/1820301/lectures/41301375
CWLogs - Overview
- 🔧 Store, monitor & access logging data
- Log = timestamp + data (
YYYY-MM-DD::HH:MM:SS message )
- e.g.
2025-02-03::13:40:35 "User signed in"
- Public & regional service
- Accessible from AWS, on-premises, or even other cloud platforms
- Built-in integrations with many AWS services (EC2, VPC Flow Logs, Lambda, CloudTrail…)
- Security generally provided via IAM roles or service-linked roles
- Majority of AWS services send basic logging data to CWLogs by default
- CW Unified Agent: Integration SW that allows sending custom logs to CW, from anywhere
- AWS-external services must install it to send logs to CW
- Also needed to send custom application logs or OS-level logs from EC2 instances
- Alternative to CW Agent: AWS SDK (app code can use it to send logs too)
- 💡 CW Unified Agent is a modern and more powerful version of the CWLogs Agent, in case you see the latter mentioned
CWLogs - Architecture
- Logging sources: AWS services, mobile apps, DBs, external APIs, AWS-external compute…
- Inject Log Events (timestamp+message) into CWLogs
- Log stream = sequence of log events from same source
- Log group = container for log streams
- Stores same logs from different sources (e.g. different EC2 instances)
- Stores configuration settings for streams (e.g. retention setting & permission)
- Stores & defines metric filters → can trigger alarms
- 💡 Example: we have a group of EC2 instances. Each instance injects its
/var/log/messages (which stores Linux system diagnostics) into CWLogs.
- One log group contains all
/var/log/messages from all instances
- Every log stream inside the log group belongs to one specific instance
- Each log stream would contain the log events of its instance, in chronological order