AWS Machine Learning Blog

JUNE 11, 2024

release , you can now launch Neuron DLAMIs (AWS Deep Learning AMIs) and Neuron DLCs (AWS Deep Learning Containers) with the latest released Neuron packages on the same day as the Neuron SDK release. AWS DLCs provide a set of Docker images that are pre-installed with deep learning frameworks.

Deep Learning 128

Deep Learning ML Automation Auto-complete 128

AWS Machine Learning Blog

JUNE 13, 2023

In this post, we demonstrate how to deploy Falcon for applications like language understanding and automated writing assistance using large model inference deep learning containers on SageMaker. SageMaker large model inference (LMI) deep learning containers (DLCs) can help. code_falcon40b_deepspeed/model.py

Auto-complete 100

Auto-complete Deep Learning LLM Software Engineer 100

AWS Machine Learning Blog

OCTOBER 11, 2023

The decode phase includes the following: Completion – After the prefill phase, you have a partially generated text that may be incomplete or cut off at some point. The decode phase is responsible for completing the text to make it coherent and grammatically correct. The default is 32.

Auto-complete 126

Auto-complete LLM Deep Learning Machine Learning 126

AWS Machine Learning Blog

NOVEMBER 15, 2024

The integration of these multimodal capabilities has unlocked new possibilities for businesses and individuals, revolutionizing fields such as content creation, visual analytics, and software development. In the metadata.jsonl file, each example is a dictionary that contains three keys named file_name , prompt , and completion.

Auto-complete 112

Auto-complete ML Python AI Modeling 112

AWS Machine Learning Blog

MAY 9, 2023

In addition, you can now use Application Auto Scaling with provisioned concurrency to address inference traffic dynamically based on target metrics or a schedule. In this post, we discuss what provisioned concurrency and Application Auto Scaling are, how to use them, and some best practices and guidance for your inference workloads.

Auto-complete 91

Auto-complete Machine Learning ML Software Development 91

AWS Machine Learning Blog

MAY 31, 2023

In late 2022, AWS announced the general availability of Amazon EC2 Trn1 instances powered by AWS Trainium accelerators, which are purpose built for high-performance deep learning training. We use the Neuron SDK to run deep learning workloads on AWS Inferentia and Trainium-based instances. Choose Create repository.

Machine Learning 108

Machine Learning Auto-complete Deep Learning ML 108

AWS Machine Learning Blog

MAY 4, 2023

You can use ml.trn1 and ml.inf2 compatible AWS Deep Learning Containers (DLCs) for PyTorch, TensorFlow, Hugging Face, and large model inference (LMI) to easily get started. For the full list with versions, see Available Deep Learning Containers Images. These endpoints are fully managed and support auto scaling.

Generative AI 100

Generative AI Deep Learning Machine Learning Python 100

AWS Machine Learning Blog

APRIL 17, 2023

However, as the size and complexity of the deep learning models that power generative AI continue to grow, deployment can be a challenging task. Then, we highlight how Amazon SageMaker large model inference deep learning containers (LMI DLCs) can help with optimization and deployment.

Prompt Engineer 104

Prompt Engineer Prompt Engineering Deep Learning Large Language Models 104

The MLOps Blog

FEBRUARY 27, 2023

You would address it in a completely different way, depending on what’s the problem. 2 The more interesting ones are the ones that don’t have the data science teams, or sometimes they don’t even have software developers in the way that they are companies that live in the 21st century. It was data management.

Computer Vision 52

Computer Vision Auto-classification Auto-complete ML 52

Flipboard

DECEMBER 2, 2024

This enhancement builds upon the existing auto scaling capabilities in SageMaker, offering more granular control over resource allocation. To learn more, see Supercharge your auto scaling for generative AI inference – Introducing Container Caching in SageMaker Inference.

Auto-complete 135

Auto-complete Machine Learning ML Generative AI 135

AWS Machine Learning Blog

NOVEMBER 30, 2023

Because FM outputs could range from a single sentence to multiple paragraphs, the time it takes to complete the inference request varies significantly, leading to unpredictable spikes in latency if the requests are routed randomly between instances. In this post, we show you the new capabilities of IC-based SageMaker endpoints.

Auto-complete 121

Auto-complete Machine Learning ML Computer Vision 121

AWS Machine Learning Blog

JANUARY 17, 2024

Llama 2 is an auto-regressive generative text language model that uses an optimized transformer architecture. Trainium and AWS Inferentia, enabled by the AWS Neuron software development kit (SDK), offer a high-performance, and cost effective option for training and inference of Llama 2 models. train_and_test_dataset["train"].to_json("train.jsonl")

Auto-complete 133

Auto-complete Python Deep Learning Machine Learning 133

AWS Machine Learning Blog

FEBRUARY 19, 2024

As a result, an initial invocation to a model might see higher inference latency than the subsequent inferences, which are completed with low latency. To take advantage of automated model scaling in SageMaker, make sure you have instance auto scaling set up to provision additional instance capacity.

ML 129

ML Auto-complete Deep Learning Software Engineer 129

AWS Machine Learning Blog

JUNE 5, 2023

Today, as part of Amazon Web Services’ partnership with Hugging Face, we are excited to announce the release of a new Hugging Face Deep Learning Container (DLC) for inference with Large Language Models (LLMs). You can find our complete example notebook here. Qing Lan is a Software Development Engineer in AWS.

LLM 96

LLM Large Language Models Deep Learning Auto-complete 96

AWS Machine Learning Blog

DECEMBER 2, 2024

SageMaker Large Model Inference (LMI) is deep learning container to help customers quickly get started with LLM deployments on SageMaker Inference. For more details, see Amazon SageMaker inference launches faster auto scaling for generative AI models and Container Caching. The following table summarizes our setup.

Large Language Models 76

Large Language Models Auto-complete Machine Learning LLM 76

Flipboard

APRIL 13, 2023

Prime Air (our drones) and the computer vision technology in Amazon Go (our physical retail experience that lets consumers select items off a shelf and leave the store without having to formally check out) use deep learning. This is a giant leap forward in developer productivity, and we believe this is only the beginning.

Generative AI 182

Generative AI ML AI AI 182

AWS Machine Learning Blog

JULY 23, 2024

is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety. 405B-Instruct You can use Llama models for text completion for any piece of text.

Machine Learning 130

Machine Learning Computer Vision Python Auto-complete 130