ODSC - Open Data Science

MARCH 27, 2023

ML Governance: A Lean Approach Ryan Dawson | Principal Data Engineer | Thoughtworks Meissane Chami | Senior ML Engineer | Thoughtworks During this session, you’ll discuss the day-to-day realities of ML Governance. Some of the questions you’ll explore include How much documentation is appropriate?

Machine Learning 98

Machine Learning Data Science Data Ingestion Deep Learning 98

AWS Machine Learning Blog

NOVEMBER 16, 2023

Amazon SageMaker provides purpose-built tools for machine learning operations (MLOps) to help automate and standardize processes across the ML lifecycle. In this post, we describe how Philips partnered with AWS to develop AI ToolSuite—a scalable, secure, and compliant ML platform on SageMaker.

Data Scientist 126

Data Scientist ML Data Science Automation 126

AWS Machine Learning Blog

FEBRUARY 13, 2024

Let’s demystify this using the following personas and a real-world analogy: Data and ML engineers (owners and producers) – They lay the groundwork by feeding data into the feature store Data scientists (consumers) – They extract and utilize this data to craft their models Data engineers serve as architects sketching the initial blueprint.

ML 128

ML Machine Learning ML Engineer Data Scientist 128

AWS Machine Learning Blog

FEBRUARY 19, 2025

In this post, we provide best practices to maximize the value of SageMaker HyperPod task governance and make the administration and data science experiences seamless. Access control When working with SageMaker HyperPod task governance, data scientists will assume their specific role.

Data Scientist 80

Data Scientist Data Science ML Engineer Generative AI 80

AWS Machine Learning Blog

JANUARY 10, 2024

Specialist Data Engineering at Merck, and Prabakaran Mathaiyan, Sr. ML Engineer at Tiger Analytics. The large machine learning (ML) model development lifecycle requires a scalable model release process similar to that of software development. This post is co-written with Jayadeep Pabbisetty, Sr.

ML 129

ML Machine Learning Data Scientist ETL 129

AWS Machine Learning Blog

OCTOBER 28, 2024

You can use this framework as a starting point to monitor your custom metrics or handle other unique requirements for model quality monitoring in your AI/ML applications. Data Scientist at AWS, bringing a breadth of data science, ML engineering, MLOps, and AI/ML architecting to help businesses create scalable solutions on AWS.

ML 115

ML Metadata Data Scientist DevOps 115

AWS Machine Learning Blog

AUGUST 31, 2023

Model cards are intended to be a single source of truth for business and technical metadata about the model that can reliably be used for auditing and documentation purposes. Depending on your governance requirements, Data Science & Dev accounts can be merged into a single AWS account.

ML 127

ML Data Scientist Machine Learning Data Science 127

The MLOps Blog

JANUARY 23, 2023

Came to ML from software. Founded neptune.ai , a modular MLOps component for ML metadata store , aka “experiment tracker + model registry”. Most of our customers are doing ML/MLOps at a reasonable scale, NOT at the hyperscale of big-tech FAANG companies. . – How about the ML engineer? Let me explain.

DevOps 59

DevOps Metadata Software Engineer Data Scientist 59

ODSC - Open Data Science

FEBRUARY 25, 2025

Topics Include: Agentic AI DesignPatterns LLMs & RAG forAgents Agent Architectures &Chaining Evaluating AI Agent Performance Building with LangChain and LlamaIndex Real-World Applications of Autonomous Agents Who Should Attend: Data Scientists, Developers, AI Architects, and ML Engineers seeking to build cutting-edge autonomous systems.

Data Scientist 52

Data Scientist Large Language Models Machine Learning ML Engineer 52

AWS Machine Learning Blog

MARCH 30, 2023

This allows for seamless communication of positional data and various outputs of Bundesliga Match Facts between containers in real time. The match-related data is collected and ingested using DFL’s DataHub. Both the Lambda function and the Fargate container publish the data for further consumption in the relevant MSK topics.

Machine Learning 100

Machine Learning Data Scientist Data Science Metadata 100

AWS Machine Learning Blog

OCTOBER 2, 2024

Additionally, you can enable model invocation logging to collect invocation logs, full request response data, and metadata for all Amazon Bedrock model API invocations in your AWS account. Leveraging her expertise in Computer Vision and Deep Learning, she empowers customers to harness the power of the ML in AWS cloud efficiently.

Generative AI 129

Generative AI Data Ingestion AI AI 129

AWS Machine Learning Blog

DECEMBER 13, 2023

ML operations, known as MLOps, focus on streamlining, automating, and monitoring ML models throughout their lifecycle. Data scientists, ML engineers, IT staff, and DevOps teams must work together to operationalize models from research to deployment and maintenance.

ML 123

ML Automation Metadata Software Development 123

AWS Machine Learning Blog

MARCH 1, 2023

Solution overview The ML solution for LTV forecasting is composed of four components: the training dataset ETL pipeline, MLOps pipeline, inference dataset ETL pipeline, and ML batch inference. ML engineers no longer need to manage this training metadata separately.

Automation 101

Automation ETL Data Drift ML 101

Viso.ai

MARCH 29, 2024

MLflow is an open-source platform designed to manage the entire machine learning lifecycle, making it easier for ML Engineers, Data Scientists, Software Developers, and everyone involved in the process. MLOps aims to automate and operationalize ML models, enabling smoother transitions to production and deployment.

Machine Learning 52

Machine Learning ML Automation Data Scientist 52

AWS Machine Learning Blog

AUGUST 31, 2023

Model cards are intended to be a single source of truth for business and technical metadata about the model that can reliably be used for auditing and documentation purposes. Depending on your governance requirements, Data Science & Dev accounts can be merged into a single AWS account.

ML 52

ML Data Scientist Machine Learning Data Science 52

The MLOps Blog

OCTOBER 3, 2023

So I was able to get from growth hacking to data analytics, then data analytics to data science, and then data science to MLOps. I switched from analytics to data science, then to machine learning, then to data engineering, then to MLOps. How do I get this model in production?

ML 52

ML Data Scientist Machine Learning Data Science 52

The MLOps Blog

AUGUST 3, 2023

This is Piotr Niedźwiedź and Aurimas Griciūnas from neptune.ai , and you’re listening to ML Platform Podcast. Stefan is a software engineer, data scientist, and has been doing work as an ML engineer. As you’ve been running the ML data platform team, how do you do that? Stefan: Yeah.

ML 52

ML Data Scientist Software Engineer Machine Learning 52

AWS Machine Learning Blog

SEPTEMBER 1, 2023

These data owners are focused on providing access to their data to multiple business units or teams. Data science team – Data scientists need to focus on creating the best model based on predefined key performance indicators (KPIs) working in notebooks.

Generative AI 126

Generative AI Prompt Engineering Prompt Engineer ML 126

The MLOps Blog

MAY 9, 2023

One of the most prevalent complaints we hear from ML engineers in the community is how costly and error-prone it is to manually go through the ML workflow of building and deploying models. Building end-to-end machine learning pipelines lets ML engineers build once, rerun, and reuse many times. Kale v0.7.0.

ML 98

ML Machine Learning Metadata Data Science 98

AWS Machine Learning Blog

NOVEMBER 14, 2024

However, model governance functions in an organization are centralized and to perform those functions, teams need access to metadata about model lifecycle activities across those accounts for validation, approval, auditing, and monitoring to manage risk and compliance. An experiment collects multiple runs with the same objective.

ML 105

ML Machine Learning Auto-complete Auto-classification 105

The MLOps Blog

MARCH 21, 2023

From gathering and processing data to building models through experiments, deploying the best ones, and managing them at scale for continuous value in production—it’s a lot. As the number of ML-powered apps and services grows, it gets overwhelming for data scientists and ML engineers to build and deploy models at scale.

Machine Learning 69

Machine Learning Data Scientist ML Metadata 69

AWS Machine Learning Blog

JANUARY 13, 2023

Enabling such a secure, compliant environment in the cloud within minutes relieves data scientists from the burden of handling cloud infrastructure, networking requirements, and security standards measures, to focus instead on the data science problem. The following diagram illustrates this architecture. Model deployment.

ML 98

ML Data Scientist Machine Learning Metadata 98

AWS Machine Learning Blog

JUNE 14, 2023

SageMaker Projects helps organizations set up and standardize environments for automating different steps involved in an ML lifecycle. Although notebooks are helpful for model building and experimentation, a team of data scientists and ML engineers sharing code need a more scalable way to maintain code consistency and strict version control.

ML 106

ML Data Scientist Automation DevOps 106

AWS Machine Learning Blog

APRIL 19, 2024

environment: HF_MODEL_ID: databricks/dolly-v2-7b HF_TASK: text-generation apiVersion: sagemaker.services.k8s.aws/v1alpha1 kind: Model metadata: name: flan-t5-xxl spec: modelName: flan-t5-xxl executionRoleARN: containers: - image: 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-tgi-inference:2.0.1-tgi0.9.3-gpu-py39-cu118-ubuntu20.04

Metadata 128

Metadata LLM Software Development Machine Learning 128

AWS Machine Learning Blog

NOVEMBER 13, 2024

You can now register machine learning (ML) models in Amazon SageMaker Model Registry with Amazon SageMaker Model Cards , making it straightforward to manage governance information for specific model versions directly in SageMaker Model Registry in just a few clicks.

Metadata 113

Metadata ML Software Engineer Machine Learning 113