Meta Llama 3 inference is now available on Amazon Web Services (AWS) Trainium and AWS Inferentia-based instances in Amazon SageMaker JumpStart. Meta Llama 3 models are pre-trained generative text models that can be used for a range of applications, including chatbots and AI assistants. AWS Inferentia and Trainium, used with Amazon EC2 instances, provide a cost-effective solution for deploying Llama 3 models, offering up to a 50% reduction in deployment costs compared to other EC2 instances.
SageMaker JumpStart provides developers with access to publicly available and proprietary Machine Learning models. Users can access the Meta Llama 3 models through the SageMaker JumpStart on the Amazon SageMaker Studio console and the SageMaker Python SDK. Amazon EC2 instances, powered by AWS Trainium and AWS Inferentia, offer up to 50% lower costs than comparable instances, thereby reducing the time and expense involved in training and deploying large language models.
Developers can easily deploy Meta Llama 3 on AWS Trainium and AWS Inferentia-based instances in SageMaker JumpStart. Various Meta model variants can also be found by searching “neuron” in the search box. Developers can then view details about the model, such as the data used to train it, how to use it, and its license. Deployment can be done either through the provided interface or the example notebook, which offers end-to-end guidance on deploying the model and cleaning up resources.
The Meta Llama 3 models include Meta-Llama-3-8B, Meta-Llama-3-8B-Instruct, Meta-Llama-3-70B, and Meta-Llama-3-70B-Instruct. AWS Trainium and Inferentia provide lower deployment costs, and by using the SageMaker JumpStart console and Python SDK, deploying these models becomes flexible and easy. After deploying the models, developers can perform inference by invoking the endpoint.
There are also options to fine-tune parameters like sequence length, tensor parallel degree, and maximum rolling batch size. If developers no longer need to use existing resources, they can delete them. This announcement is an exciting step forward for developers wanting to build generative AI applications, as it provides a more cost-effective way for large-scale AI model deployment on AWS.
