Paul Yang

Increasingly, we are being asked by ML infrastructure teams what to do about Slurm, and more specifically, the emerging “Slurm on Kubernetes” trend. We work with dozens of such teams, and decided to weigh in here with a broad survey of the patterns and anti-patterns we see around Slurm (and Slurm-like) development in modern ML environments.

SLURM, SUNK, SLINKY, SLONK: Chasing Speed, Stability, and Scale for Bleeding-Edge ML

Kubernetes has emerged as the definitive compute substrate for machine learning by offering unmatched flexibility, scalability, and reproducibility across diverse workloads and environments. 

Kubernetes: The Winner for ML 

Platform engineering teams prioritize self-service and grade themselves on platform velocity. In ML, we should transition away from MLOps and manual enablement to a fully end-to-end system of self-service. 

MLOps is Dead, Long Live ML Platform Engineering

We will demonstrate the necessity of building learning loops for your AI models to continually refine the components of your production systems.

Why Should I Train and Tune My AI Models?

Donny Greenberg

Many ML teams are looking to adopt Ray. However, the adoption process requires a significant and usually painful infrastructural lift. We explore why this is and propose a better way to get your team productive with Ray. 

How to Adopt Ray into Your ML Platform (Gently)

Despite the hype around LLMs, we contend that even very large and sophisticated LLMs cannot replace ML training done over your first-party data. 

Will LLMs Replace ML Training (No, here's why)

Machine learning development is stuck in slow, fragmented cycles, far removed from the fast-paced, iterative nature of traditional software development. This blog will explore how we got here, define the essential requirements for a modern ML platform, and conclude by introducing Runhouse as a solution that was built to accelerate ML development.

AI Platforms: Past, Present, and Future

In this Lean Orchestration Manifesto, we advocate for a more streamlined approach to ML pipelines by narrowing the scope of orchestrators. Orchestrators should be used to schedule, log, and retry tasks, but should not be the primary runtime of your production ML pipelines.

The Lean Orchestration Manifesto: Narrowing Orchestrator Scope Creates Better ML Development 

Elijah ben Izzy

Plus: A nimble, composable example with Github Actions, Hamilton, and Runhouse

Lean Data Automation: A Principal Components Approach

What if we made building and deploying FastAPI apps even faster? 🤯

“LudicrouslyFastAPI:” Deploy Python Functions as APIs on your own Infrastructure

Lean, ergonomic, multi-cluster dispatch for high-velocity AI teams

Better GPU Cluster Scheduling with Runhouse

Caroline Chen

Matt Kandler

Josh Lewittes

You deserve better than pasting API keys into environment variables

Simple, Secure Secrets Management for AI

Deploy and Run Your Code on SageMaker in 10 Minutes with Runhouse

The Quickest AWS SageMaker Deployment in Existence

Buckle up, this is less a release blog post and more a wild ride through the Infra API Wonka Factory. We’ll talk through the innocent motivations which turned into upending half of Runhouse’s core execution model, and the gaggle of awesome new features which fell out.

Runhouse 0.0.12: rh.Module, Streaming, and too much more

Platform overhead occupies 50-75% of ML engineers’ and researchers’ time. Runhouse is a PyTorch-like unified interface to solve that.

A PyTorch Approach to ML Infrastructure

Part 1 of a series on data pipelines for the modern ML stack

Supercharging Apache Airflow with Runhouse

Machine Learning's hellish developer experience and research-to-production drag are largely due to widespread over-packaging - coupling systems rigidly together through misuse of containers and pipelines. There's finally an end in sight.

Runhouse Blog