Performance Guide

This guide provides detailed performance benchmarks and recommendations for choosing between CPU (NumPy) and GPU (PyTorch) backends in nltools algorithms.

Why Performance Matters for Neuroimaging¶

Neuroimaging analyses often involve:

• High-dimensional data: fMRI datasets with 10k-200k voxels per brain

• Many samples: Hundreds to thousands of timepoints or subjects

• Iterative algorithms: Cross-validation, permutation testing, searchlight

• Large-scale studies: Analyzing dozens of subjects or conditions

A 10x speedup can transform workflows:

• Interactive analysis: Results in seconds instead of minutes

• Exploration: Try more models, parameters, and hypotheses

• Scalability: Analyze whole-brain data that was previously impractical

Backend Overview¶

nltools provides two computational backends:

NumPy Backend (CPU)¶

• Pros: No dependencies, works everywhere, excellent for small problems

• Cons: Single-threaded for most operations, slower for large problems

• Best for: Prototyping, small datasets, systems without GPU

PyTorch Backend (GPU)¶

• Pros: Massive parallelism, 10-30x speedup for large problems

• Cons: Requires PyTorch installation, GPU memory limits, transfer overhead

• Best for: Large datasets, cross-validation, production workflows

Auto Selection¶

• Recommendation: Use device='auto' for intelligent selection

• How it works: Evaluates problem size and chooses optimal backend

• Fallback: Automatically uses NumPy if PyTorch unavailable

Benchmark Results¶

Eshin-M3-Air¶

Host: Eshin-M3-Air
Platform: macOS-15.7.4-arm64-arm-64bit
Python: 3.11.14
NumPy: 2.4.4
PyTorch: 2.11.0
nltools: 0.5.1
GPU: MPS

GPU speedup

ridge — GPU speedup (mps)¶

inference — GPU speedup (mps)¶

Memory scaling

collection — peak RSS: lazy vs in-memory .mean()¶

Full results

ridge¶

predict¶

inference¶

collection¶

pikachu.ucsd.edu¶

Host: pikachu.ucsd.edu
Platform: Linux-6.17.0-1031-nvidia-aarch64-with-glibc2.39
Python: 3.11.15
NumPy: 2.4.6
PyTorch: 2.13.0+cu130
nltools: 0.5.1 @ 55e44f06
GPU: CUDA

GPU speedup

ridge — GPU speedup (cuda)¶

inference — GPU speedup (cuda)¶

Memory scaling

collection — peak RSS: lazy vs in-memory .mean()¶

Full results

ridge¶

predict¶

inference¶

collection¶

Recommendations¶

When to Use Each Backend¶

Set the compute device on the public API with device= ('cpu' / 'gpu' / 'auto') — e.g. Ridge(device='gpu'), brain.fit(model='ridge', device='auto'), brain.bootstrap(stat='weights', device='gpu').

Use CPU (device='cpu', NumPy) when:

• Problem size < 10 million elements (e.g., 100 samples × 100k features)

• Prototyping or exploratory analysis

• GPU not available

• Running on shared systems where GPU access is limited

Use GPU (device='gpu', PyTorch) when:

• Problem size > 30 million elements

• Running cross-validation (especially 5-fold or more)

• Fitting many models in a loop (e.g., searchlight)

• Production workflows where speed matters

Use Auto (device='auto') when:

• Unsure about problem size

• Want code to work optimally across systems

• Developing reusable analysis scripts

Problem Size Thresholds¶

Based on our benchmarks:

# Effective problem size = n_samples × n_features × cv_folds
problem_size = n_samples * n_features * cv_folds

if problem_size < 10_000_000:
    # Use CPU (GPU overhead not worth it)
    device = 'cpu'
elif problem_size > 30_000_000:
    # Use GPU if available (significant speedup)
    device = 'gpu'
else:
    # Medium range: auto-select based on GPU availability
    device = 'auto'

Memory Considerations¶

Both backends use float32 precision to reduce memory usage:

# Example: 1000 samples × 200k features
n_samples, n_features = 1000, 200000

# NumPy memory (approx)
memory_numpy = n_samples * n_features * 4 bytes / 1e9  # ~0.8 GB

# PyTorch memory (includes intermediate arrays)
memory_torch = memory_numpy * 2  # ~1.6 GB (rough estimate)

GPU memory limits:

• Typical GPU VRAM: 8-24 GB (consumer), 16-80 GB (professional)

• If problem doesn’t fit in GPU memory, use NumPy

• Monitor GPU memory with nvidia-smi (CUDA) or Activity Monitor (MPS)

Troubleshooting¶

Common Issues¶

GPU Not Detected¶

from nltools.algorithms.backends import check_gpu_available

available, info = check_gpu_available()
if not available:
    print(f"GPU not available: {info['device_name']}")
    print("Using CPU backend instead")

Solutions:

• CUDA: Install PyTorch with CUDA support: pip install torch --index-url https://download.pytorch.org/whl/cu118

• MPS (Apple Silicon): Ensure PyTorch ≥2.0: pip install torch>=2.0

• Fallback: Use device='cpu' explicitly

MPS (Apple Silicon) SVD Limitation¶

Symptom: Warning about linalg_svd falling back to CPU

UserWarning: The operator 'aten::linalg_svd' is not currently supported on the MPS backend
and will fall back to run on the CPU.

Explanation:

• PyTorch’s MPS backend doesn’t fully optimize SVD operations

• Falls back to CPU for SVD computation

• Performance may not exceed NumPy with Accelerate framework

Solutions:

• Use device='cpu' explicitly on Apple Silicon

• NumPy with Accelerate is often faster than MPS for ridge regression

• Wait for future PyTorch versions with improved MPS SVD support

Out of GPU Memory¶

Symptom: RuntimeError: CUDA out of memory or similar

Solutions:

1. Reduce problem size (subset voxels, downsample data)

2. Use the CPU device: device='cpu'

3. Process in batches if applicable

4. Use systems with more GPU memory

Slow First Run¶

Symptom: First GPU computation is slower than subsequent runs

Explanation:

• PyTorch compiles kernels on first use (JIT compilation)

• GPU initialization overhead

• Memory allocation and transfer

Solution: This is normal. Subsequent runs will be faster.

Best Practices¶

1. Profile Before Optimizing¶

import time
import numpy as np
from nltools.algorithms.ridge import ridge_svd
from nltools.algorithms.backends import Backend

X = np.random.randn(300, 100000)
y = np.random.randn(300)

# Time NumPy (algorithm-layer solvers use the internal `parallel=` name)
start = time.time()
coef_np = ridge_svd(X, y, parallel='cpu')
time_np = time.time() - start

# Time PyTorch
start = time.time()
coef_torch = ridge_svd(X, y, parallel='gpu')
time_torch = time.time() - start

print(f"NumPy: {time_np:.3f}s")
print(f"PyTorch: {time_torch:.3f}s")
print(f"Speedup: {time_np/time_torch:.1f}x")

2. Use Auto-Selection for Portability¶

# Good: Works optimally everywhere
result = ridge_cv(X, y, parallel='auto')

# Less portable: Assumes GPU available
result = ridge_cv(X, y, parallel='gpu')

3. Batch GPU Operations¶

# Good: Minimize host-device transfers
backend = Backend('torch')
X_device = backend.to_device(X)
results = [ridge_svd(X_device, y, parallel=backend) for y in y_list]

# Less efficient: Transfer on every call
results = [ridge_svd(X, y, parallel='gpu') for y in y_list]

4. Monitor Resource Usage¶

# GPU monitoring (NVIDIA)
watch -n 1 nvidia-smi

# CPU/Memory monitoring (all systems)
htop  # or top

Benchmark Methodology¶

Hardware & Software¶

The exact host, platform, Python, NumPy, and PyTorch versions for each run are recorded per section in Benchmark Results (auto-generated from each artifact’s .env.json). The runs currently committed are an Apple-Silicon MPS box and a GB10/aarch64 CUDA box (pikachu).

Benchmark domains¶

The harness (benchmarks/) sweeps four domains at realistic neuroimaging scale:

1. ridge — ridge_cv + BrainData.fit(model='ridge'), CPU vs GPU.

2. predict — BrainData.predict across whole_brain / roi / searchlight.

3. inference — one-/two-sample and correlation permutation tests, CPU vs GPU.

4. collection — BrainCollection over N subjects: lazy vs in-memory, n_jobs.

Metrics¶

• Execution time: median wall-clock across timed reps (after a warmup).

• Peak RSS: process high-water mark, sampled on a background thread.

• Peak GPU: CUDA max_memory_allocated (exact) or an MPS allocated-delta proxy.

• Speedup: ratio of CPU (NumPy) time to GPU (PyTorch) time.

Data Generation¶

# Synthetic data matching neuroimaging dimensions
X = np.random.randn(n_samples, n_features).astype(np.float32)
y = np.random.randn(n_samples).astype(np.float32)

# Ensures realistic memory/computation patterns
# Results generalizable to real fMRI data

See Also¶

• Backends - CPU/GPU backend documentation

• Ridge Regression - Algorithm details

• Algorithms - Complete algorithm reference

Note on MPS Performance: These benchmarks run on Apple Silicon with MPS backend. PyTorch’s SVD operation is not yet fully optimized for MPS and falls back to CPU, limiting speedup compared to CUDA GPUs. On NVIDIA GPUs with CUDA, expect significantly higher speedups (10-30x for large problems).

To regenerate benchmarks on your system: uv run python -m benchmarks.run then uv run python -m benchmarks.build_docs (see benchmarks/benchmarking-guide.md).