AWS Machine Learning Blog

SEPTEMBER 4, 2024

Amazon SageMaker , a fully managed service to build, train, and deploy machine learning (ML) models, has seen increased adoption to customize and deploy FMs that power generative AI applications. To keep track of models, versions, and associated metadata, SageMaker Model Registry can be used as an inventory of models.

Generative AI 106

Generative AI Metadata ML AI 106

The MLOps Blog

JANUARY 23, 2023

Came to ML from software. Founded two successful software services companies. Founded neptune.ai , a modular MLOps component for ML metadata store , aka “experiment tracker + model registry”. Most of our customers are doing ML/MLOps at a reasonable scale, NOT at the hyperscale of big-tech FAANG companies.

DevOps 59

DevOps Metadata Software Engineer Data Scientist 59

The MLOps Blog

AUGUST 3, 2023

This article was originally an episode of the ML Platform Podcast , a show where Piotr Niedźwiedź and Aurimas Griciūnas, together with ML platform professionals, discuss design choices, best practices, example tool stacks, and real-world learnings from some of the best ML platform professionals. Stefan: Yeah.

ML 52

ML Data Scientist Software Engineer Python 52

TensorFlow

NOVEMBER 29, 2023

Posted by Marat Dukhan and Frank Barchard, Software Engineers CPUs deliver the widest reach for ML inference and remain the default target for TensorFlow Lite. The metadata can be added during model conversion using the _experimental_supported_accumulation_type attribute of the tf.lite.TargetSpec object: converter.

Metadata 97

Metadata Neural Network Software Engineer Computer Vision 97

AWS Machine Learning Blog

AUGUST 9, 2024

The embedding representations of text chunks along with related metadata are indexed in OpenSearch Service. Text embedding models are machine learning (ML) models that map words or phrases from text to dense vector representations. The application uses Amazon Textract to get the text and tables from the input documents.

Data Ingestion 127

Data Ingestion Metadata LLM Generative AI 127

The MLOps Blog

OCTOBER 3, 2023

This article was originally an episode of the ML Platform Podcast , a show where Piotr Niedźwiedź and Aurimas Griciūnas, together with ML platform professionals, discuss design choices, best practices, example tool stacks, and real-world learnings from some of the best ML platform professionals. It was two or three times more.

ML 52

ML Data Scientist Machine Learning Data Science 52

The MLOps Blog

JANUARY 23, 2023

Data version control in machine learning vs conventional software engineering Data version control in machine learning and conventional software engineering have some similarities, but there are also some key differences to consider. This is where data versioning comes in. DVC Git LFS neptune.ai neptune.ai

ML 52

ML Machine Learning Metadata Data Scientist 52

AWS Machine Learning Blog

JUNE 6, 2023

PyTorch is a machine learning (ML) framework that is widely used by AWS customers for a variety of applications, such as computer vision, natural language processing, content creation, and more. Instead, you can focus on the higher value-added effort of training jobs at scale in a shorter amount of time and iterating on your ML models faster.

ML 82

ML Deep Learning BERT Python 82

AWS Machine Learning Blog

APRIL 11, 2024

Traditional rule engines or ML-based classification can classify the documents, but often reach a limit on types of document formats and support for the dynamic addition of a new classes of document. An S3 prefix or S3 object metadata can be used to classify gallery images.

IDP 125

IDP Software Engineer Metadata Categorization 125

AWS Machine Learning Blog

SEPTEMBER 29, 2023

This guest post is co-written by Lydia Lihui Zhang, Business Development Specialist, and Mansi Shah, Software Engineer/Data Scientist, at Planet Labs. In this post, we illustrate how to use a segmentation machine learning (ML) model to identify crop and non-crop regions in an image.

Machine Learning 134

Machine Learning Data Scientist ML Python 134

AWS Machine Learning Blog

OCTOBER 31, 2024

Each response includes the annotator’s input and metadata such as acceptance time, submission time, and worker ID. Alan Ismaiel is a software engineer at AWS based in New York City. He focuses on building and maintaining scalable AI/ML products, like Amazon SageMaker Ground Truth and Amazon Bedrock Model Evaluation.

Software Engineer 116

Software Engineer Natural Language Processing Computer Vision Metadata 116

AWS Machine Learning Blog

DECEMBER 19, 2023

Artificial intelligence (AI) and machine learning (ML) are becoming an integral part of systems and processes, enabling decisions in real time, thereby driving top and bottom-line improvements across organizations. However, putting an ML model into production at scale is challenging and requires a set of best practices.

Machine Learning 129

Machine Learning ML Engineer DevOps ML 129

AWS Machine Learning Blog

DECEMBER 6, 2023

The search precision can also be improved with metadata filtering. To overcome these limitations, we propose a solution that combines RAG with metadata and entity extraction, SQL querying, and LLM agents, as described in the following sections. But how can we implement and integrate this approach to an LLM-based conversational AI?

Metadata 127

Metadata LLM NLP Conversational AI 127

AWS Machine Learning Blog

NOVEMBER 1, 2023

Organizations can search for PII using methods such as keyword searches, pattern matching, data loss prevention tools, machine learning (ML), metadata analysis, data classification software, optical character recognition (OCR), document fingerprinting, and encryption.

Software Engineer 115

Software Engineer Metadata Machine Learning NLP 115

The MLOps Blog

JUNE 27, 2023

Alignment to other tools in the organization’s tech stack Consider how well the MLOps tool integrates with your existing tools and workflows, such as data sources, data engineering platforms, code repositories, CI/CD pipelines, monitoring systems, etc. and Pandas or Apache Spark DataFrames.

Metadata 134

Metadata Machine Learning Data Scientist Data Quality 134

AWS Machine Learning Blog

NOVEMBER 14, 2024

We recently announced the general availability of cross-account sharing of Amazon SageMaker Model Registry using AWS Resource Access Manager (AWS RAM) , making it easier to securely share and discover machine learning (ML) models across your AWS accounts.

ML 92

ML Auto-complete Machine Learning Auto-classification 92

AWS Machine Learning Blog

APRIL 12, 2023

In the terminal with the AWS Command Line Interface (AWS CLI) or AWS CloudShell , run the following commands to upload the documents and metadata to the data source bucket: aws s3 cp s3://aws-ml-blog/artifacts/building-a-secure-search-application-with-access-controls-kendra/docs.zip. Expand Additional configuration.

Metadata 111

Metadata Software Engineer Algorithm ML 111

AWS Machine Learning Blog

FEBRUARY 20, 2024

In this post, we discuss deploying scalable machine learning (ML) models for diarizing media content using Amazon SageMaker , with a focus on the WhisperX model. With manual methods, a 30-minute episode can take between 1–3 hours to localize. Through automation, ZOO Digital aims to achieve localization in under 30 minutes.

Metadata 122

Metadata Auto-complete Machine Learning Deep Learning 122

AWS Machine Learning Blog

JANUARY 17, 2023

Amazon SageMaker Studio is a fully integrated development environment (IDE) for machine learning (ML) partly based on JupyterLab 3. Studio provides a web-based interface to interactively perform ML development tasks required to prepare data and build, train, and deploy ML models. cdk.json – Contains metadata, and feature flags.

Software Engineer 95

Software Engineer ML Engineer ML Machine Learning 95

AWS Machine Learning Blog

AUGUST 20, 2024

You can call the SageMaker ListWorkteams or DescribeWorkteam APIs to view workteams’ metadata, including the WorkerAccessConfiguration. About the Authors Sundar Raghavan is an AI/ML Specialist Solutions Architect at AWS, helping customers build scalable and cost-efficient AI/ML pipelines with Human in the Loop services.

Software Engineer 113

Software Engineer ML Machine Learning Metadata 113

AWS Machine Learning Blog

FEBRUARY 13, 2024

Amazon SageMaker Feature Store is a fully managed, purpose-built repository to store, share, and manage features for machine learning (ML) models. Features are inputs to ML models used during training and inference. SageMaker Feature Store now makes it effortless to share, discover, and access feature groups across AWS accounts.

ML 130

ML Machine Learning ML Engineer Data Scientist 130

ODSC - Open Data Science

MARCH 27, 2023

Learn about the flow, difficulties, and tools for performing ML clustering at scale Ori Nakar | Principal Engineer, Threat Research | Imperva Given that there are billions of daily botnet attacks from millions of different IPs, the most difficult challenge of botnet detection is choosing the most relevant data.

Machine Learning 98

Machine Learning Data Ingestion Explainability Deep Learning 98

Unite.AI

JUNE 24, 2024

MLOps , or Machine Learning Operations, is a multidisciplinary field that combines the principles of ML, software engineering, and DevOps practices to streamline the deployment, monitoring, and maintenance of ML models in production environments. What is MLOps?

Data Drift 124

Data Drift Machine Learning Data Scientist DevOps 124

AWS Machine Learning Blog

NOVEMBER 8, 2023

The MONAI AI models and applications can be hosted on Amazon SageMaker , which is a fully managed service to deploy machine learning (ML) models at scale. AHI provides API access to ImageSet metadata and ImageFrames. Metadata contains all DICOM attributes in a JSON document. In the customized inference.py

Metadata 122

Metadata AI AI Python 122

ODSC - Open Data Science

SEPTEMBER 17, 2024

Large Model Quality and Evaluation Anoop Sinha | Research Director, AI & Future Technologies | Google Large model development faces many challenges when it comes to ML quality and evaluation, including the coverage, scale, and wide use cases for what LLMs are used for. Check out a few of them below.

LLM 52

LLM Large Language Models Metadata Data Ingestion 52

Google Research AI blog

JUNE 14, 2023

Marcos Seefelder, Software Engineer, and Daniel Duckworth, Research Software Engineer, Google Research When choosing a venue, we often find ourselves with questions like the following: Does this restaurant have the right vibe for a date? Is there good outdoor seating? Are there enough screens to watch the game?

Software Engineer 129

Software Engineer Neural Network Computer Vision Metadata 129

AWS Machine Learning Blog

APRIL 4, 2024

In this post, we describe how Nielsen Sports modernized a system running thousands of different machine learning (ML) models in production by using Amazon SageMaker multi-model endpoints (MMEs) and reduced operational and financial cost by 75%. We can now research and deploy a new ML model in under 7 days, instead of over 1 month previously.

Machine Learning 96

Machine Learning Artificial Intelligence Artificial Intelligence ML 96

AWS Machine Learning Blog

SEPTEMBER 28, 2023

In the rapidly evolving world of AI and machine learning (ML), foundation models (FMs) have shown tremendous potential for driving innovation and unlocking new use cases. This table will hold the endpoint, metadata, and configuration parameters for the model.

Generative AI 123

Generative AI Metadata AI AI 123

AWS Machine Learning Blog

APRIL 23, 2024

Here are some features which we will cover: AWS CloudFormation support Private network policies for Amazon OpenSearch Serverless Multiple S3 buckets as data sources Service Quotas support Hybrid search, metadata filters, custom prompts for the RetreiveAndGenerate API, and maximum number of retrievals. Nitin Eusebius is a Sr.

Generative AI 129

Generative AI Metadata Automation Machine Learning 129

AWS Machine Learning Blog

JUNE 11, 2024

This allows machine learning (ML) practitioners to rapidly launch an Amazon Elastic Compute Cloud (Amazon EC2) instance with a ready-to-use deep learning environment, without having to spend time manually installing and configuring the required packages. You also need the ML job scripts ready with a command to invoke them.

Deep Learning 123

Deep Learning ML Automation Auto-complete 123

AWS Machine Learning Blog

NOVEMBER 16, 2023

Amazon SageMaker provides purpose-built tools for machine learning operations (MLOps) to help automate and standardize processes across the ML lifecycle. In this post, we describe how Philips partnered with AWS to develop AI ToolSuite—a scalable, secure, and compliant ML platform on SageMaker.

Data Scientist 114

Data Scientist ML Data Science Automation 114

AWS Machine Learning Blog

MARCH 2, 2023

When end-users ask natural language questions, Amazon Kendra uses machine learning (ML) algorithms to understand the context and return the most relevant answers. The Amazon Kendra’s S3 connector supports indexing documents and their associated metadata stored in an S3 bucket. Leave Metadata files prefix folder location blank.

Metadata 96

Metadata Machine Learning Software Engineer Algorithm 96

Viso.ai

MARCH 29, 2024

MLflow is an open-source platform designed to manage the entire machine learning lifecycle, making it easier for ML Engineers, Data Scientists, Software Developers, and everyone involved in the process. MLOps aims to automate and operationalize ML models, enabling smoother transitions to production and deployment.

Machine Learning 52

Machine Learning ML Automation Data Scientist 52

AWS Machine Learning Blog

APRIL 7, 2023

As one of the most prominent use cases to date, machine learning (ML) at the edge has allowed enterprises to deploy ML models closer to their end-customers to reduce latency and increase responsiveness of their applications. Even ground and aerial robotics can use ML to unlock safer, more autonomous operations. Instances[*].

BERT 96

BERT Metadata Natural Language Processing ML 96

AWS Machine Learning Blog

APRIL 26, 2024

In previous posts, we covered new capabilities like hybrid search support , metadata filtering to improve retrieval accuracy , and how Knowledge Bases for Amazon Bedrock manages the end-to-end RAG workflow. He frequently speaks at AI/ML conferences, events, and meetups around the world.

Data Ingestion 128

Data Ingestion Generative AI Python Software Engineer 128

AWS Machine Learning Blog

OCTOBER 25, 2023

Using SageMaker geospatial capabilities, you can efficiently transform or enrich large-scale geospatial datasets, accelerate model building with pre-trained machine learning (ML) models, and explore model predictions and geospatial data on an interactive map using 3D accelerated graphics and built-in visualization tools.

Metadata 114

Metadata Machine Learning ML Software Engineer 114

AWS Machine Learning Blog

JUNE 21, 2024

Amazon SageMaker JumpStart is a machine learning (ML) hub offering pre-trained models and pre-built solutions. Administrators can check the metadata of a specific model using the hub.describe_model(model_name= ) command. The private hub uses the SageMaker underlying infrastructure, allowing it to scale with enterprise-level ML demands.

Python 114

Python Machine Learning ML Software Engineer 114