Amazon Bedrock, a managed service that offers a selection of foundation models from leading AI companies, empowers users to build new, delightful experiences for their customers using generative AI. As a response to end users’ curiosity for prescriptive ways to monitor generative AI applications’ health and performance in an operational environment, Amazon Bedrock has introduced a set of capabilities to provide quick and easy visibility into these workloads within the broader application context.
At the heart of this enhancement is the Amazon Bedrock automatic dashboard in CloudWatch. CloudWatch, which features automatic dashboards for monitoring AWS services’ health and performance, includes a new dashboard for Amazon Bedrock that delivers insights into key metrics of Bedrock models. Accessible via the AWS Management Console, this dashboard offers centralized visibility and insights into key performance metrics, like latency and invocation metrics.
Alongside the automatic dashboard, users can also develop custom dashboards using CloudWatch. These dashboards can combine metrics from multiple AWS services to monitor performance at the application level, useful for debugging and implementing custom logic.
Another significant feature is usage attribution, enabling users to monitor invocation usage from different applications. Users can utilize Amazon Bedrock’s invocation logs for enhanced visibility into the source and token consumption for each invocation. In addition to creating customized dashboards, the component allows users to break down usage by identity across their Amazon Bedrock invocations.
In addition to CloudWatch’s features, users can customize their dashboards using Retrieval Augmented Generation (RAG) to augment the model with domain-specific data. Monitoring these components is crucial for your overall monitoring strategy. Using the Amazon Bedrock GitHub repository’s example, users can learn how to create a custom dashboard that provides visibility and insights into the core components of a sample RAG based solution.
In conclusion, the Amazon Bedrock’s dashboard initiative addresses three common customer concerns: visibility into the performance of Amazon Bedrock models, integrating Amazon Bedrock monitoring with other application components, and attributing Large Language Model (LLM) usage to particular users or applications. This makes it easier for customers to monitor Amazon Bedrock metrics, create customized dashboards specific to their applications, and gather data for ongoing monitoring. The result is a significantly improved operational and monitoring environment for generative AI applications. Users can get started with Amazon Bedrock monitoring immediately using the example provided in the Amazon Bedrock GitHub repository and a custom dashboard template available in a separate GitHub repository.
