AWS Machine Learning Blog

APRIL 21, 2023

If the model performs acceptably according to the evaluation criteria, the pipeline continues with a step to baseline the data using a built-in SageMaker Pipelines step. For the data drift Model Monitor type, the baselining step uses a SageMaker managed container image to generate statistics and constraints based on your training data.

Data Drift 74

Data Drift Metadata Data Quality ML 74

The MLOps Blog

JUNE 27, 2023

Databricks Databricks is a cloud-native platform for big data processing, machine learning, and analytics built using the Data Lakehouse architecture. Delta Lake Delta Lake is an open-source storage layer that provides reliability, ACID transactions, and data versioning for big data processing frameworks such as Apache Spark.

Machine Learning 134

Machine Learning Metadata Data Quality Data Scientist 134

AWS Machine Learning Blog

NOVEMBER 29, 2023

For instance, a notebook that monitors for model data drift should have a pre-step that allows extract, transform, and load (ETL) and processing of new data and a post-step of model refresh and training in case a significant drift is noticed. Run the notebooks The sample code for this solution is available on GitHub.

Data Drift 83

Data Drift BERT Data Scientist ML 83

Snorkel AI

MAY 12, 2023

Kishore will then double click into some of the opportunities we find here at Capital One, and Bayan will finish us off with a lean into one of our open-source solutions that really is an important contribution to our data-centric AI community. Compute, big data, large commoditized models—all important stages.

Machine Learning 52

Machine Learning Data Scientist ML Data Science 52

Snorkel AI

MAY 12, 2023

Kishore will then double click into some of the opportunities we find here at Capital One, and Bayan will finish us off with a lean into one of our open-source solutions that really is an important contribution to our data-centric AI community. Compute, big data, large commoditized models—all important stages.

Machine Learning 52

Machine Learning Data Scientist ML Data Science 52

AWS Machine Learning Blog

AUGUST 29, 2023

The batch inference pipeline includes steps for checking data quality against a baseline created by the training pipeline, as well as model quality (model performance) if ground truth labels are available. If the batch inference pipeline discovers data quality issues, it will notify the responsible data scientist via Amazon SNS.

Data Scientist 72

Data Scientist Data Quality Python ML 72

Snorkel AI

JULY 5, 2023

We thought we’d structure this more as a conversation where we walk you through some of our thinking around some of the most common themes in data centricity in applied AI. Is more data always better? One of them is that it is really hard to maintain high data quality with rigorous validation.

Large Language Models 52

Large Language Models Metadata AI AI 52