mxframe

⚡ Zero-copy bridge between PyArrow and MAX Engine — unlock GPU acceleration for your DataFrames.

🎯 Vision

mxframe bridges the gap between Python DataFrames and high-performance GPU compute. By leveraging PyArrow’s columnar memory format and MAX Engine’s accelerated kernels, we enable:

  • Zero-copy data sharing between PyArrow and MAX tensors
  • Seamless CPU/GPU execution with a unified API
  • Production-ready performance without leaving Python

🚀 What’s in v0.0.1

This initial release provides the core zero-copy bridge:

Feature Description
MXFrame class DataFrame wrapper backed by PyArrow
arrow_to_max_tensor() Zero-copy Arrow → MAX Tensor bridge
CPU Support True zero-copy (same memory)
GPU Support Automatic device transfer
Dtype Support int8-64, uint8-64, float32/64, date32

Benchmarked: 10M elements at 4.4 GB/s throughput — proves zero-copy.

⚡ Quick Start

from mxframe import MXFrame

# Create from dictionary
df = MXFrame({
    'price': [10.5, 20.3, 15.8, 42.0],
    'quantity': [100, 200, 150, 50],
})
print(df)

Zero-Copy to MAX Tensors

The key feature: convert columns to MAX tensors without copying data:

# Get a MAX tensor (zero-copy on CPU!)
price_tensor = df.to_max_tensor('price')
print(f"Tensor: {price_tensor.to_numpy()}")

# Verify same memory address
print(f"Same memory: {df.get_buffer_address('price') == df.to_numpy('price').ctypes.data}")

🎮 GPU Acceleration

Transfer to GPU when you need accelerated compute:

from max import driver

if driver.accelerator_count() > 0:
    gpu = driver.Accelerator()
    gpu_tensor = df.to_max_tensor('price', device=gpu)
    print(f"GPU tensor: {gpu_tensor.shape}")
else:
    print("No GPU available")

📦 Installation

Install from PyPI:

pip install mxframe

Or install latest from GitHub:

pip install git+https://github.com/abhisheksreesaila/mxframe.git

Requirements

  • Python 3.10+
  • PyArrow
  • MAX Engine SDK

🛠️ Developer Guide

# Clone the repository
git clone https://github.com/abhisheksreesaila/mxframe.git
cd mxframe

# Install in development mode
pip install -e .

# Make changes under nbs/ directory, then compile
nbdev_prepare

📚 Documentation

🗺️ Roadmap

📄 License

Apache 2.0