• Contents:

Amazon OpenSearch Service

Ref: https://www.udemy.com/course/ultimate-aws-certified-generative-ai-developer-professional/learn/lecture/53684195

Elasticsearch

Ref: https://www.elastic.co/elasticsearch

• 🔧 Open-source, distributed search and analytics engine built for speed, scale, and AI applications
  • Built on top of Apache Lucene (Java-based indexing library)
• Amazon had forked Elasticsearch and provided a managed service with Amazon Elasticsearch Service
• In 2021 Elastic got into a dispute with Amazon → Amazon renamed their Elasticsearch fork into OpenSearch, and the service is now called Amazon OpenSearch Service

OpenSearch Service - Facts

• 🔧 At its core, it's a search engine that can ingest massive amounts of data (PBs)
  • But many data analysis tools have been added (visualization Dashboards)
• Can be part of a data pipeline, ingesting streams from e.g. Kinesis
• Two modes:
  1. Managed clusters
     • Horizontally scalable (uses shards)
  2. Serverless
• Use cases
  • Full-text search (core use case)
  • Log analytics
    • Application monitoring
    • Security analytics
    • Clickstream analytics
• Anti-patterns
  • OLTP → RDS or DynamoDB are better
  • Ad-hoc data querying → Athena is better
  • 💡 Remember OpenSearch is primarily for search & analytics at big scales

OpenSearch Dashboards

• 💡 Called “Kibana” in the OG Elasticsearch, “Dashboard” in Amazon's OpenSearch
• ❗ Great option for analytics if you don't want to e.g. use Google Analytics because you want to keep data away from Google, or if your data is too much for Google to ingest!

OpenSearch Shards & Clusters