GenAI apps in Amazon Bedrock

Amazon Bedrock Overview

🔧 Serverless marketplace for GenAI FMs within AWS
- Provides APIs to access FMs
- Higher level than SageMaker, but can integrate with SageMaker Canvas if you want to go low level
- Screenshot
Features
- Can invoke chat, text, or image models
  - Also “playground” environments provided
- Can use pre-built, custom fine-tuned or your own imported models
  - Can perform evaluations on any model
- 3rd-party models billed via AWS (consulting pricing!)
- Support for RAG
- Support for LLM agents
IAM permissions
- Must use with an IAM user (NOT root) → best practice to not use root anyway
- User must have relevant Bedrock permissions (e.g. AmazonBedrockFullAccess, AmazonBedrockReadOnly, admin policies have permissions already)

bedrock: Manage, train & deploy models
bedrock-runtime: Perform inference (execute prompts, generate embeddings) against these models
- Converse, ConverseStream, InvokeModel, InvokeModelWithResponseStream
- ❗ Converse = chat vs Invoke = single invocation, terminates
- 💡 Stream refers to the stream of tokens that an LLM generates (notice how ChatGPT usually answers with a stream of tokens that takes a while to get to the end? → that's what is meant with stream)
bedrock-agent: Manage, train & deploy LLM agents and knowledge bases
bedrock-agent-runtime: Perform inference against agents and knowledge bases
- InvokeAgent, Retrieve, RetrieveAndGenerate

‼️ Before using any FM in Bedrock, you must first request access!!
- 3rd-party models may require you to submit additional information
- You will be billed the 3rd party’s rates through AWS → check pricing
  - aws.amazon.com/bedrock/pricing
Amazon (Titan) models will approve immediately (because it's Amazon proprietary)
- But 3rd-party models only take a few minutes for approval
Screenshot