Skip to content

getting-started.md

Latest commit

 

History

History
118 lines (82 loc) · 6.53 KB

getting-started.md

File metadata and controls

118 lines (82 loc) · 6.53 KB

Getting Started with CNCF ModelPack Specification

Welcome to the CNCF ModelPack Specification project! This guide will help you understand the specification and its role in the broader AI/ML ecosystem, as well as providing practical steps to get started.

What is the ModelPack Specification?

The CNCF ModelPack Specification is an open standard for packaging, distributing, and running AI models in cloud-native environments. It builds upon the proven OCI (Open Container Initiative) image specification to bring the same standardization and interoperability benefits that containers brought to application deployment to the world of AI models.

Why This Matters

We are entering the fourth age of infrastructure evolution:

  1. Machine-centric (GNU/Linux distributions)
  2. Virtual Machine-centric (Cloud computing, virtualization)
  3. Container-centric (Docker, Kubernetes, OCI standards)
  4. AI Model-centric (Current era - AI model development and deployment)

Just as OCI standards revolutionized how we package and distribute applications, the ModelPack specification aims to standardize AI model packaging and distribution, moving away from vendor-specific formats toward an open, interoperable standard.

The ModelPack Ecosystem

The ModelPack specification is designed to integrate seamlessly with existing cloud-native and AI/ML tools. Here's how the key components work together:

high level architecture

Core Infrastructure

This section lists the core infrastructure components that ModelPack is working with.

  • OCI Registries: Store model artifacts using the same proven infrastructure as container images. One example of such an implementation of an OCI registry is Harbor, which provides enterprise-grade OCI registry service that can host model artifacts with security scanning, policy management, and RBAC.
  • Model distribution service: Provides efficient model artifacts distribution. One example of such an implementation is Dragonfly, a P2P-based file distribution system at scale.

Model Management Tools

  • modctl: CLI tool for building, pushing, pulling, and managing OCI model artifacts
  • KitOps: ModelKit packaging and deployment platform that supports the ModelPack specification
  • AIKit: Package AI models as OCI artifacts from local, HTTP, or Hugging Face sources with extensible formats, including ModelPack specification

Kubernetes Integration

  • Model CSI Driver: Kubernetes CSI driver for mounting model artifacts as persistent volumes
  • OCI Volume Sources: Mount model artifacts directly as volumes in Kubernetes pods without init containers
  • CRI-O: Container runtime that supports OCI artifacts, enabling seamless model deployment in Kubernetes environments

Key Benefits

  • Standardization: The use of familiar OCI tooling and infrastructure for AI models
  • Interoperability: Models packaged once work across different platforms and tools
  • Security: Leverage existing OCI security features like image signing and vulnerability scanning
  • Efficiency: Native Kubernetes integration eliminates the need for manually downloading models
  • Versioning: The use of OCI tags and digests enables robust model version control
  • Ecosystem: Build on top of the mature container ecosystem, rather than creating new infrastructure and components

Getting Started

Prerequisites

  • Basic understanding of containers and OCI concepts
  • Access to an OCI-compatible registry (Docker Hub, Harbor, etc.)

Choose Your Tool

The ModelPack specification can be used with different tools depending on your needs:

  • modctl: CLI tool for building, pushing, pulling, and managing OCI model artifacts. Great for command-line workflows and CI/CD pipelines.
  • AIKit: Package AI models as OCI artifacts from local, HTTP, or Hugging Face sources with extensible formats.
  • KitOps: ModelKit packaging and deployment platform that supports the ModelPack specification.

Install Model CSI Driver

If you plan to use models in Kubernetes, install the Model CSI Driver by following the instructions in the Model CSI Driver repository.

Use Models in Kubernetes

Here's an example Kubernetes pod spec that mounts a model artifact using the model CSI driver. The model will be available under the /model directory inside the container.

apiVersion: v1
kind: Pod
metadata:
  name: model-inference-pod
spec:
  containers:
    - name: inference-server
      image: ubuntu:24.04
      command: ["sleep", "infinity"]
      volumeMounts:
        - name: model-volume
          mountPath: /model
          readOnly: true
  volumes:
    - name: model-volume
      csi:
        driver: model.csi.modelpack.org
        volumeAttributes:
          modelRef: "myregistry.com/mymodel:v1.0"

This example shows how to mount a model artifact directly into a Kubernetes pod using the model CSI driver. The contents of the model are available within the /model directory within the running pod.

Next Steps

  1. Get hands-on experience: Follow the step-by-step guides for modctl or AIKit
  2. Explore the full ModelPack specification for technical implementation details
  3. Join the community on CNCF Slack #modelpack
  4. Contribute to the ModelPack project - see our contributing guidelines

Additional Resources

The ModelPack specification represents the next evolution in infrastructure standardization, bringing the benefits of containerization to AI model management. Start with the basics, explore the ecosystem, and join our growing community of contributors and users building the future of cloud-native AI.