correlation¶
Correlation permutation test implementations.
This module provides CPU-parallel and GPU-batched implementations of correlation permutation tests for assessing statistical significance of correlations.
Methods:
| Name | Description |
|---|---|
correlation_permutation_test | Correlation permutation test. |
Classes¶
Methods¶
correlation_permutation_test¶
correlation_permutation_test(data1: np.ndarray, data2: np.ndarray, *, n_permute: int = 5000, metric: str = 'pearson', tail: int | str = 2, return_null: bool = False, device: str | None = 'cpu', n_jobs: int = -1, max_gpu_memory_gb: float | None = None, random_state: int | None = None, progress_bar: bool = False) -> dictCorrelation permutation test.
Tests whether the correlation between data1 and data2 is significantly different from zero by randomly permuting data1 and computing correlations.
Assumption: Observations are independent (i.i.d.). For autocorrelated time series, use timeseries_correlation_permutation_test with circle_shift or phase_randomize methods instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data1 | ndarray | Data to permute - shape (n_samples,) for single feature - shape (n_samples, n_features) for multi-feature | required |
data2 | ndarray | Data to correlate with - shape (n_samples,) for single feature - shape (n_samples, n_features) for multi-feature | required |
n_permute | int | Number of permutations (default: 5000) | 5000 |
metric | str | Correlation metric (default: ‘pearson’) - ‘pearson’: Pearson correlation (linear relationships) - ‘spearman’: Spearman rank correlation (monotonic relationships) - ‘kendall’: Kendall tau rank correlation (ordinal association, robust to ties) | ‘pearson’ |
tail | int | str | Test type — 2 | ‘two’ (two-tailed, default) or 1 |
return_null | bool | If True, return full null distribution (default: False) | False |
device | str | Parallelization method (default: ‘cpu’) - None: Single-threaded NumPy (for debugging/small problems) - ‘cpu’: CPU parallelization via joblib (default, 4-8× speedup) - ‘gpu’: GPU acceleration via PyTorch (fastest for large problems) | ‘cpu’ |
n_jobs | int | Number of CPU cores for parallelization (default: -1 = all cores) Only used when device=‘cpu’ | -1 |
max_gpu_memory_gb | float | Explicit GPU memory budget in GB. None (default) measures the device’s available memory. Controls automatic batching to prevent OOM errors. Only used with device=‘gpu’. Larger values allow more permutations per batch but risk OOM on smaller GPUs. | None |
random_state | int | Random seed for reproducibility | None |
progress_bar | bool | Show a progress bar over permutations (default: False) | False |
Returns:
| Name | Type | Description |
|---|---|---|
dict | dict | Dictionary with keys: - ‘correlation’ (float or np.ndarray): Observed correlation(s) - ‘p’ (float or np.ndarray): P-value(s) - ‘null_dist’ (np.ndarray): Null distribution (if return_null=True) - ‘device’ (str): Parallelization method used |
Examples:
>>> # Single feature (default CPU parallelization)
>>> x = np.random.randn(100)
>>> y = x + np.random.randn(100) * 0.5 # Correlated
>>> result = correlation_permutation_test(x, y, n_permute=5000)
>>> result['correlation']
0.85
>>> result['p']
0.001>>> # Multi-feature (2D arrays)
>>> data1 = np.random.randn(100, 10) # 100 samples, 10 features
>>> data2 = data1 + np.random.randn(100, 10) * 0.3 # Correlated
>>> result = correlation_permutation_test(data1, data2, n_permute=5000)
>>> result['correlation'].shape
(10,)
>>> result['p'].shape
(10,)>>> # GPU acceleration
>>> result = correlation_permutation_test(data1, data2, n_permute=5000, device='gpu')Notes
Default (device=‘cpu’): CPU parallelization with joblib (4-8× speedup)
GPU parallelization (‘gpu’): Fastest for large problems with automatic batching
Pearson: Fully vectorized across all features (5-20× speedup for multi-feature)
Spearman: GPU rank transform (average ties) + vectorized Pearson on ranks
Kendall: tie-corrected tau-b via pre-computed pairwise sign tensors; O(n²) memory per permutation, so batches are sized accordingly
Single-threaded (device=None): Use for small problems or debugging
For multi-feature data, each feature pair tested independently
Kendall is O(n^2) complexity, slower than Pearson/Spearman for large samples