Amazon Bedrock, a generative artificial intelligence (AI) service, allows customers to build new and delightful user experiences using high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, and Meta. Users can use these models securely, privately, and responsibly through a single API, along with a broad set of capabilities for building generative AI applications. These capabilities help users monitor the health and performance of generative AI applications.
To help customers better understand and make use of Amazon Bedrock, Amazon has introduced an automatic dashboard in Amazon CloudWatch. This dashboard consolidates key metrics, like latency and invocation metrics, so users can quickly assess the health, performance, and usage of their Bedrock models. The dashboard also allows users to monitor specific models, track token usage, and detect invocation errors.
In addition to the automatic dashboard, users can use CloudWatch to build custom dashboards that combine metrics from multiple AWS services. Custom dashboards allow users to monitor their applications at a more granular level and can help with troubleshooting and debugging. One popular use of the custom dashboard is to implement Retrieval Augmented Generation (RAG) for a specific use case. RAG-based architectures involve multiple components, all of which need to be monitored.
Amazon Bedrock also allows for usage attribution. Using Amazon Bedrock invocation logs, users can monitor the invocation usage of different applications or users. These logs can help users understand who is using how many tokens or invocations. Users can then use this information to adjust their models or troubleshooting efforts accordingly.
The authors conclude by noting that Amazon Bedrock helps users overcome three common challenges in operationalizing generative AI applications: providing a consolidated view of model performance, integrating model monitoring with overall application monitoring, and attributing large language model (LLM) usage to specific users or applications. Thanks to Amazon Bedrock and CloudWatch, users can monitor their applications more effectively and make adjustments as necessary. The authors also provide a template for a custom dashboard on Github to help users get started.
