• Contents:

Types of Data (according to structure)

Ref: https://www.udemy.com/course/aws-certified-machine-learning-engineer-associate-mla-c01/learn/lecture/45284653

• Rows and columns
• Consistent structure | - Relational DBs
• Spreadsheets or CSV files with consistent columns | | Unstructured | No schema nor predefined structure | - Requires heavy preprocessing to query
• May come in various formats | - Raw text (e.g. emails, word processing docs…)
• Raw media (videos, audio, images…) | | Semi-structured | Some level of structure (tags, hierarchies…) | - Elements might be tagged or categorized
• More flexible than structured
• Not as chaotic as unstructured | - XML & JSON files
• Email headers (structured: date, subject… + unstructured: body…)
• Log files with varied formats |

Properties of Data (3 Vs)

Ref: https://www.udemy.com/course/aws-certified-machine-learning-engineer-associate-mla-c01/learn/lecture/45284671

1. Volume
   • GBs? PBs?…
2. Velocity
   • High velocity → Real-Time or near-RT processing
3. Variety
   • Structured? Mixed? Multiple sources? Multiple formats?
4. 💡 …Veracity?
   • Some sources say there are 4 Vs ;)

Data Warehouses, Data Lakes & Data Lakehouses

Ref: https://www.udemy.com/course/aws-certified-machine-learning-engineer-associate-mla-c01/learn/lecture/45284683

• Data Warehouse (DWH) (e.g. Amazon Redshift)
  • Centralized repository optimized for analysis (read-heavy operations) where data from different sources is stored in a structured format
• Data Lake (e.g. Amazon S3 can be used as data lake)
  • Storage repository that holds vast amounts of raw data in its native format (predefined structure is not necessary)
  • Structured, semi-structured, & unstructured data

• Fast & complex queries requirements
• BI & analytics | - Big variety of data sources with different structure
• Scalable, flexible & cost-effective storage solution
• Advanced analytics, ML & data discovery |