Today AWS announced Trainium and Inferentia support for the Llama 3.1 models’ fine-tuning and inference. The Llama 3.1 is a collection of large language models (LLMs) available in three sizes: 8B, 70B, and 405B and supports a range of capabilities such as search, image generation, code execution, and mathematical reasoning. Notably, the Llama 3.1 405B is the world’s largest publicly available LLM and is ideal for enterprise applications, research, and development.
The Llama 3.1 models use an optimized transformer architecture and are fine-tuned through supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF). Users can further customize and optimize the models with appropriate safety measures, following Meta’s responsible use guide.
AWS users can access Llama 3.1 through Amazon Bedrock, powered by AWS Trainium, or fine-tune and deploy Llama 3.1 models with SageMaker for more control over the underlying resources. AWS Trainium and AWS Inferentia2 enable high performance and cost-effectiveness for Llama 3.1 models.
To start fine-tuning the Llama 3.1 8B or 70B, users can use the NeuronX Distributed library. Users can deploy their model using the Llama 3 8B Neuron sample code and update the model ID. They can also deploy models directly from SageMaker through the Hugging Face Model Hub or refer to the continuous batching guide to use vLLM for deployment.
AWS Trainium and Inferentia support high performance and low cost for the Llama 3.1 models, enabling users to build differentiated AI applications. For further help on getting started with AWS AI chips, users can refer to AWS Neuron Documentation’s Model Samples and Tutorials.
