nltools.algorithms¶
The statistical functions that operate on arrays and dataframes rather than on a data class: corrections, outlier and signal handling, similarity, permutation and bootstrap inference, intersubject measures, functional alignment, searchlight neighborhoods, and the GPU check.
nltools.algorithms — the functional core of nltools.
Every user-facing statistical function and algorithm is importable flat from
here (from nltools.algorithms import fdr, zscore, isc), organized into
focused submodules underneath:
- corrections: multiple-comparison corrections (FDR, Holm-Bonferroni, thresholding)
- outliers: outlier detection, winsorizing, z-scoring
- signal: temporal signal processing (resampling, filtering, basis functions)
- similarity: similarity metrics and Fisher transforms
- regression: standalone OLS regression on numpy arrays
- alignment: the
align/procrustesentry points and the shared-response estimators behind them - inference: permutation tests, bootstrap resampling, and intersubject statistics (ISC/ISFC/ISPS), parallelized across joblib workers
- backends: device selection and memory budgeting for the ridge paths;
check_gpu_availableis how you ask before requestingdevice='gpu'
Ridge regression lives in nltools.models.Ridge, which delegates its numerics
to the Himalaya library and is reached through BrainData.fit(model='ridge').
Functions:
| Name | Description |
|---|---|
align |
Align subject data into a common response model. |
align_states |
Align state weight maps by minimizing pairwise distance between group states. |
calc_bpm |
Calculate instantaneous BPM from beat to beat interval. |
check_gpu_available |
Check whether GPU acceleration is available. |
circle_shift |
Circular shift for time-series data. |
compute_searchlight_neighborhoods |
Compute sphere neighborhoods for all voxels in a brain mask. |
compute_similarity |
Compute row-wise similarity between two data arrays. |
correlation_permutation_test |
Permutation test for whether the correlation between two arrays differs from zero. |
distance_correlation |
Compute the distance correlation between two arrays to test for multivariate dependence. |
downsample |
Downsample a Polars DataFrame/Series to a new target frequency or number of samples using averaging. |
fdr |
Determine an FDR threshold for an array of p-values. |
find_spikes |
Identify spikes (motion artifacts, intensity outliers) in 4D fMRI data. |
fisher_r_to_z |
Convert correlation coefficients to Fisher z values. |
fisher_z_to_r |
Convert Fisher z back to a correlation coefficient. |
holm_bonf |
Determine a Holm-Bonferroni (step-down) threshold for an array of p-values. |
isc |
Compute intersubject correlation across the subject axis of an aligned array. |
isc_group |
Test the difference in pairwise intersubject correlation between two groups. |
isfc |
Compute intersubject functional connectivity (ISFC) from per-subject matrices. |
isps |
Compute dynamic intersubject phase synchrony (ISPS) from an observations-by-subjects array. |
make_cosine_basis |
Create basis functions for a discrete cosine transform. |
matrix_permutation_test |
Matrix permutation test (Mantel test) for correlating two square matrices. |
multi_threshold |
Threshold a statistic image at several p-values and count the passes per voxel. |
one_sample_permutation_test |
One-sample permutation test using sign flipping. |
phase_randomize |
FFT-based phase randomization for time-series data. |
procrustes_distance |
Test matrix similarity using Procrustes superposition. |
regress |
Fit an OLS regression of |
threshold |
Threshold a statistic image by the p-values in a separate image. |
transform_pairwise |
Transform data into pairwise differences with balanced labels for ranking. |
trim |
Trim a Polars DataFrame/Series by replacing outlier values with NaNs. |
two_sample_permutation_test |
Two-sample permutation test using group-label shuffling. |
upsample |
Upsample a Polars DataFrame/Series to a new target frequency or number of samples using interpolation. |
winsorize |
Winsorize a Polars DataFrame/Series with the largest/lowest value not considered outlier. |
zscore |
Z-score every column of a Polars or pandas DataFrame/Series. |
Functions:¶
align ¶
Align subject data into a common response model.
Aligns a group of subjects either by Procrustes-based hyperalignment
(Haxby et al., 2011) or by the Shared Response Model (Chen et al., 2015),
the latter through _SRM/_DetSRM.
The common model is the shared response (SRM) or the centered group template
(Procrustes). Transformed data can be projected back into each subject's
original space with its transformation matrix. To align a single BrainData
to another, use BrainData.align instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
list[BrainData] | list[ndarray]
|
Subjects to align; all elements must be the same type. Arrays are observations x features. |
required |
method
|
str
|
One of |
'deterministic_srm'
|
n_features
|
int | None
|
Number of features in the common space (SRM only).
None uses the smallest subject's size along the aligned axis (voxels
for |
None
|
axis
|
int
|
Axis to align on: 0 aligns timepoints (ISC computed per voxel), 1 aligns voxels (ISC computed per timepoint). Defaults to 0. |
0
|
n_iter
|
int
|
Number of |
10
|
random_state
|
int
|
Seed forwarded to the constructed |
0
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Examples:
# Hyperalign using procrustes transform
out = align(data, method='procrustes')
# Align using shared response model
out = align(data, method='probabilistic_srm', n_features=None)
# Project aligned data back into original data space
original_data = [
np.dot(t.data, tm.T)
for t, tm in zip(out['transformed'], out['transformation_matrix'])
]
align_states ¶
align_states(
reference,
target,
*,
metric="correlation",
return_index=False,
replace_zero_variance=False,
)
Align state weight maps by minimizing pairwise distance between group states.
This function uses the Hungarian algorithm for state alignment, which is different from aligning multiple subjects' data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
reference
|
ndarray
|
Reference pattern x state matrix. |
required |
target
|
ndarray
|
Target pattern x state matrix to align to |
required |
metric
|
str
|
Distance metric passed to |
'correlation'
|
return_index
|
bool
|
If True return the remapping index instead of the reordered data. Defaults to False. |
False
|
replace_zero_variance
|
bool
|
Replace zero-variance columns with uniform random numbers before computing distances; avoids NaNs with the correlation metric. Integer inputs are converted to float so the replacement noise survives. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
ndarray
|
If |
calc_bpm ¶
Calculate instantaneous BPM from beat to beat interval.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
beat_interval
|
int
|
Number of samples between beats (typically the R-R interval). |
required |
sampling_freq
|
float
|
Sampling frequency in Hz. |
required |
Returns:
| Type | Description |
|---|---|
float
|
Beats per minute for the time interval. |
check_gpu_available ¶
Check whether GPU acceleration is available.
Returns:
| Type | Description |
|---|---|
tuple[bool, dict[str, Any]]
|
|
circle_shift ¶
circle_shift(
data: ndarray,
shift_amount: int | ndarray | None = None,
random_state: int | RandomState | None = None,
) -> ndarray
Circular shift for time-series data.
Performs a circular shift that preserves autocorrelation structure. Useful for permutation tests on autocorrelated time series (e.g., fMRI). For 1D data, shifts by a single amount. For 2D data, shifts each feature (column) independently.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
ndarray
|
Time series, shape (n_samples,) or (n_samples, n_features). |
required |
shift_amount
|
int | ndarray | None
|
Shift amount: an int for 1D data, or an array of length n_features (one shift per column) for 2D data. Shifts wrap, so a shift at or beyond the series length is the same as its remainder. None draws random shift(s). Defaults to None. |
None
|
random_state
|
int | RandomState | None
|
Random seed used when
|
None
|
Returns:
| Type | Description |
|---|---|
ndarray
|
Circularly shifted data with the same shape as the input. |
Examples:
compute_searchlight_neighborhoods ¶
compute_searchlight_neighborhoods(
mask_img: Nifti1Image, radius: float = 10.0
) -> _SphereNeighborhoods
Compute sphere neighborhoods for all voxels in a brain mask.
For each voxel in the mask, this function identifies all other voxels within the specified radius (in millimeters).
The algorithm uses sklearn's BallTree for efficient radius queries in world coordinates (mm), ensuring accurate neighborhoods regardless of voxel resolution.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mask_img
|
Nifti1Image
|
NIfTI mask image defining the brain region |
required |
radius
|
float
|
Radius of spheres in millimeters (default: 10.0) |
10.0
|
Returns:
| Type | Description |
|---|---|
_SphereNeighborhoods
|
_SphereNeighborhoods with precomputed adjacency matrix |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the mask is not 3-D, or has no non-zero voxels |
Examples:
compute_similarity ¶
Compute row-wise similarity between two data arrays.
The array engine behind BrainData.similarity.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data1
|
ndarray
|
First data array, shape (n_samples1, n_features). |
required |
data2
|
ndarray
|
Second data array, shape (n_samples2, n_features). |
required |
metric
|
str
|
'correlation' (or 'pearson'), 'spearman' (or 'rank_correlation'), 'dot_product', or 'cosine'. Defaults to 'correlation'. |
'correlation'
|
Returns:
| Type | Description |
|---|---|
ndarray
|
Similarities of shape (n_samples1, n_samples2), squeezed: a 1D array when either input has a single row, a scalar when both do. |
Examples:
correlation_permutation_test ¶
correlation_permutation_test(
data1: ndarray,
data2: ndarray,
*,
n_permute: int = 5000,
metric: str = "pearson",
tail: int | str = 2,
return_null: bool = False,
n_jobs: int = -1,
random_state: int | None = None,
progress_bar: bool = False,
) -> dict
Permutation test for whether the correlation between two arrays differs from zero.
Builds the null distribution by randomly permuting the observations of data1
and re-correlating with data2. Assumes observations are independent (i.i.d.);
for autocorrelated time series use _timeseries_correlation_permutation_test,
whose 'circle_shift' and 'phase_randomize' methods preserve temporal
structure. With 2D inputs each column of data1 is tested against the
matching column of data2, independently.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data1
|
ndarray
|
Data to permute, shape (n_samples,) for a single feature or (n_samples, n_features) for several. |
required |
data2
|
ndarray
|
Data to correlate with, same shape as |
required |
n_permute
|
int
|
Number of permutations. Defaults to 5000. |
5000
|
metric
|
str
|
'pearson' (linear), 'spearman' (rank-based, monotonic), or 'kendall' (tau-b, ordinal association, tie-corrected). Defaults to 'pearson'. |
'pearson'
|
tail
|
int | str
|
|
2
|
return_null
|
bool
|
Also return the full null distribution. Defaults to False. |
False
|
n_jobs
|
int
|
Number of joblib workers, -1 = all cores. Defaults to -1. Results are identical at every worker count. |
-1
|
random_state
|
int | None
|
Random seed for reproducibility. |
None
|
progress_bar
|
bool
|
Show a progress bar over permutations. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys 'correlation' (float, or np.ndarray of shape (n_features,) for
2D inputs: the observed correlation), 'p' (float or np.ndarray, the
matching p-values), and 'null_dist' (np.ndarray of shape
(n_permute,) or (n_permute, n_features)) when |
Examples:
import numpy as np
from nltools.algorithms import correlation_permutation_test
# Single feature
x = np.random.randn(100)
y = x + np.random.randn(100) * 0.5
result = correlation_permutation_test(x, y, n_permute=5000)
result["correlation"] # → 0.85 (approximately)
result["p"] # → 0.0002
# Multi-feature: each column pair tested independently
data1 = np.random.randn(100, 10)
data2 = data1 + np.random.randn(100, 10) * 0.3
result = correlation_permutation_test(data1, data2, n_permute=5000)
result["correlation"].shape # → (10,)
result["p"].shape # → (10,)
Note
Kendall's tau is O(n²) in the number of samples, so it is markedly slower than Pearson or Spearman for large samples.
distance_correlation ¶
distance_correlation(
x: ndarray,
y: ndarray,
bias_corrected: bool = True,
ttest: bool = False,
) -> dict
Compute the distance correlation between two arrays to test for multivariate dependence.
Distance correlation detects linear and non-linear dependence. The arrays must match on their first dimension. Prefer the bias-corrected version (the default), which can also perform a t-test; that test operates on a statistic that is approximately the squared distance correlation, which is also returned.
Distance correlation is the normalized covariance of two centered Euclidean distance matrices. Each distance matrix holds the distances between rows (if x or y is 2d) or scalars (if 1d). Each matrix is centered before the covariance is computed, either by double-centering or by U-centering, which corrects the bias that grows with the number of dimensions. U-centering is almost always preferable and also permits a one-tailed directional t-test on the normalized covariance (Szekely & Rizzo, 2013). Distance correlation is normally bounded between 0 and 1, but U-centering can produce negative estimates, which are never significant.
Validated against dcor and dcor.ttest in the R package energy and
dcor.distance_correlation, dcor.u_distance_correlation_sqr, and
dcor.independence.distance_correlation_t_test in the Python package dcor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
ndarray
|
1d or 2d array of observations by features. |
required |
y
|
ndarray
|
1d or 2d array of observations by features. |
required |
bias_corrected
|
bool
|
If True, U-center the distance matrices; if False,
double-center them, which gives a biased estimate that converges to 1
as the number of dimensions grows. Must be True when |
True
|
ttest
|
bool
|
Perform a t-test on the bias-corrected distance correlation. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Key 'dcorr' (float, distance correlation); with |
Raises:
| Type | Description |
|---|---|
ValueError
|
If arrays are not 1d or 2d, or if |
Examples:
downsample ¶
Downsample a Polars DataFrame/Series to a new target frequency or number of samples using averaging.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
DataFrame | Series
|
Data to downsample. |
required |
sampling_freq
|
float
|
Sampling frequency of the data in Hz. |
None
|
target
|
float
|
Downsampling target. |
None
|
target_type
|
str
|
Unit of |
'samples'
|
method
|
str
|
Aggregation within each bin, 'mean' or 'median'. Defaults to 'mean'. |
'mean'
|
Returns:
| Type | Description |
|---|---|
DataFrame | Series
|
Downsampled data (same type as input). |
Note
Rows are binned by floor(row / n_samples), the same rule
DesignMatrix.downsample uses, so a non-integer ratio spreads its
leftover rows across the bins rather than into an extra final one.
fdr ¶
Determine an FDR threshold for an array of p-values.
Benjamini-Hochberg procedure at false discovery rate q (valid under
independence or positive dependence). Written by Tal Yarkoni.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
p
|
ndarray
|
Vector of p-values. |
required |
q
|
float
|
False discovery rate level. Defaults to 0.05. |
0.05
|
Returns:
| Type | Description |
|---|---|
float
|
The p-value threshold; |
find_spikes ¶
find_spikes(
data,
global_spike_cutoff=3,
diff_spike_cutoff=3,
*,
TR: float | None = None,
sampling_freq: float | None = None,
)
Identify spikes (motion artifacts, intensity outliers) in 4D fMRI data.
Each voxel's linear trend is removed before either detector runs, so slow scanner drift does not inflate the standard deviation the cutoffs are scaled by. Already-detrended data gives the same result.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
BrainData | Nifti1Image
|
4D functional data. |
required |
global_spike_cutoff
|
float | None
|
Cutoff in standard deviations for spikes in the per-TR global mean signal; None skips this detector. Defaults to 3. |
3
|
diff_spike_cutoff
|
float | None
|
Cutoff in standard deviations for spikes in the per-TR mean absolute frame-to-frame difference; None skips this detector. The difference is attributed to the later of the two volumes, as framewise displacement is. Defaults to 3. |
3
|
TR
|
float | None
|
Repetition time in seconds; sets the returned
DesignMatrix's |
None
|
sampling_freq
|
float | None
|
Sampling frequency in Hz (1 / TR). See |
None
|
Returns:
| Type | Description |
|---|---|
DesignMatrix
|
One indicator column per detected spike TR, named
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If both cutoffs are None, if |
fisher_r_to_z ¶
fisher_z_to_r ¶
holm_bonf ¶
Determine a Holm-Bonferroni (step-down) threshold for an array of p-values.
The step-down procedure applies progressively less correction to larger p-values. It is more conservative than FDR but much more powerful than plain Bonferroni correction.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
p
|
ndarray
|
Vector of p-values. |
required |
alpha
|
float
|
Family-wise alpha level. Defaults to 0.05. |
0.05
|
Returns:
| Type | Description |
|---|---|
float
|
The p-value threshold; |
isc ¶
isc(
data,
*,
n_samples=5000,
summary="median",
summary_statistic="pairwise",
method="bootstrap",
ci_percentile=95,
exclude_self_corr=True,
tail=2,
metric="correlation",
return_null=False,
n_jobs=-1,
random_state=None,
progress_bar=False,
)
Compute intersubject correlation across the subject axis of an aligned array.
ISC is summarized with the median, as Chen et al. (2016) recommend;
summary='mean' instead averages after the Fisher r-to-z transform and
converts back, which avoids inflating the estimate.
Three null distributions are available. The default subject-wise bootstrap
(Chen et al., 2016) resamples subjects with replacement and recomputes the
chosen summary statistic — for 'pairwise', the similarity matrix, where a
subject drawn twice correlates perfectly with itself, so those entries are
set to NaN when exclude_self_corr=True. P-values use the percentile
method, as in Brainiak. The classic surrogate methods instead circle-shift
or phase-randomize each time series (Lancaster et al., 2018), preserving
its temporal autocorrelation, and recompute ISC.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
ndarray | DataFrame | DataFrame
|
Observations by
subjects, shape |
required |
n_samples
|
int
|
Number of bootstrap draws or surrogate permutations. Defaults to 5000. |
5000
|
summary
|
str
|
|
'median'
|
summary_statistic
|
str
|
Which cross-subject comparison to summarize.
|
'pairwise'
|
method
|
str
|
|
'bootstrap'
|
ci_percentile
|
int
|
Confidence-interval width in percent. Defaults to 95. |
95
|
exclude_self_corr
|
bool
|
Set self-correlations (the same subject bootstrapped twice) to NaN. Applies to the pairwise statistic only. Defaults to True. |
True
|
tail
|
int | str
|
|
2
|
metric
|
str
|
Pairwise similarity metric; any metric accepted by
sklearn's |
'correlation'
|
return_null
|
bool
|
Include the null distribution in the result. Defaults to False. |
False
|
n_jobs
|
int
|
CPU workers for the resamples; -1 (default) picks the count from available memory. |
-1
|
random_state
|
int | RandomState | None
|
Seed or generator for the resampling. |
None
|
progress_bar
|
bool
|
Display a progress bar. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
Note
exclude_self_corr=True (the default) sets a subject's correlation
with itself to NaN when the bootstrap draws that subject twice;
turning it off inflates ISC. Resamples are counted with n_samples
here, for the surrogate methods as well as the bootstrap —
n_permute belongs to the permutation tests in nltools.algorithms
and is not accepted by this function.
References
Chen, G., Shin, Y. W., Taylor, P. A., Glen, D. R., Reynolds, R. C., Israel, R. B., & Cox, R. W. (2016). Untangling the relatedness among correlations, part I: nonparametric approaches to inter-subject correlation analysis at the group level. NeuroImage, 142, 248-259.
Hall, P., & Wilson, S. R. (1991). Two guidelines for bootstrap hypothesis testing. Biometrics, 757-762.
Lancaster, G., Iatsenko, D., Pidde, A., Ticcinelli, V., & Stefanovska, A. (2018). Surrogate data for hypothesis testing of physical systems. Physics Reports, 748, 1-60.
isc_group ¶
isc_group(
group1,
group2,
*,
n_samples=5000,
summary="median",
method="permute",
ci_percentile=95,
exclude_self_corr=True,
return_null=False,
tail=2,
metric="correlation",
n_jobs=-1,
random_state=None,
progress_bar=False,
)
Test the difference in pairwise intersubject correlation between two groups.
ISC within each group is summarized with the median, as Chen et al. (2016)
recommend (summary='mean' averages after the Fisher r-to-z transform), and
the observed statistic is group1 - group2.
Two null distributions are available. The default subject-wise permutation
(Chen et al., 2016) pools the subjects, computes pairwise similarity within
and between groups, then reshuffles the group labels and recomputes the
difference. The subject-wise bootstrap instead resamples subjects with
replacement within each group; a subject drawn twice correlates perfectly
with itself, so those entries are set to NaN when exclude_self_corr=True.
P-values use the percentile method (Hall & Wilson, 1991).
Runs on plain arrays; _isc_group_permutation_test exposes the same engine
with leave-one-out ISC.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
group1
|
ndarray | DataFrame | DataFrame
|
Observations by subjects for the first group. |
required |
group2
|
ndarray | DataFrame | DataFrame
|
Observations by subjects for the second group (same number of observations). |
required |
n_samples
|
int
|
Number of permutations or bootstrap draws. Defaults to 5000. |
5000
|
summary
|
str
|
|
'median'
|
method
|
str
|
|
'permute'
|
ci_percentile
|
float
|
Confidence-interval width in percent. Defaults to 95. |
95
|
exclude_self_corr
|
bool
|
In the bootstrap, set self-correlations to NaN. Defaults to True. |
True
|
return_null
|
bool
|
Include the null distribution in the result. Defaults to False. |
False
|
tail
|
int | str
|
|
2
|
metric
|
str
|
Pairwise similarity metric; any metric accepted by
sklearn's |
'correlation'
|
n_jobs
|
int
|
CPU workers for the resamples; -1 (default) picks the count from available memory. |
-1
|
random_state
|
int | RandomState | None
|
Random seed for reproducibility. |
None
|
progress_bar
|
bool
|
Display a progress bar. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
References
Chen, G., Shin, Y. W., Taylor, P. A., Glen, D. R., Reynolds, R. C., Israel, R. B., & Cox, R. W. (2016). Untangling the relatedness among correlations, part I: nonparametric approaches to inter-subject correlation analysis at the group level. NeuroImage, 142, 248-259.
Hall, P., & Wilson, S. R. (1991). Two guidelines for bootstrap hypothesis testing. Biometrics, 757-762.
isfc ¶
Compute intersubject functional connectivity (ISFC) from per-subject matrices.
Uses the leave-one-out approach of Simony et al. (2016): for each subject,
average the other subjects' data and correlate every voxel/ROI time series
of the target subject with every voxel/ROI time series of that average.
Subjects are independent, so they are processed in parallel with joblib
unless n_jobs=1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
list[ndarray]
|
One matrix per subject, each
|
required |
method
|
str
|
Only |
'average'
|
n_jobs
|
int
|
Parallel workers; -1 (default) uses all cores, 1 runs serially. |
-1
|
random_state
|
int | RandomState | None
|
Unused. ISFC's
leave-one-out computation is deterministic and draws no random
samples; the parameter exists for signature parity with the rest
of the ISC family ( |
None
|
progress_bar
|
bool
|
Display a progress bar over subjects. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
list[ndarray]
|
One |
References
Simony, E., Honey, C. J., Chen, J., Lositsky, O., Yeshurun, Y., Wiesel, A., & Hasson, U. (2016). Dynamic reconfiguration of the default mode network during narrative comprehension. Nature Communications, 7, 12141.
isps ¶
Compute dynamic intersubject phase synchrony (ISPS) from an observations-by-subjects array.
Instantaneous phase synchrony across subjects for a single voxel/ROI time
series, after Glerean et al. (2012): the data are narrow-band filtered
(Butterworth) and Hilbert-transformed to get each subject's instantaneous
phase angle at every time point. Across subjects, the result gives the
mean phase angle, the mean resultant vector length, and a parametric
p-value from the Rayleigh test for circular uniformity (Fisher, 1995).
With pairwise=True these are computed on pairwise phase-angle differences
(inter-site phase coupling in the EEG literature) rather than on the raw
angles (inter-trial phase coupling).
The default band, 0.04-0.07 Hz, follows Glerean et al. (2012). It is close to the "slow-4" band (0.025-0.067 Hz; Zuo et al., 2010; Penttonen & Buzsáki, 2003) but excludes ~0.03 Hz, which carries aliased respiration (Birn et al., 2006).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
ndarray | DataFrame | DataFrame
|
Observations by subjects. |
required |
sampling_freq
|
float
|
Sampling frequency in Hz. Defaults to 0.5. |
0.5
|
low_cut
|
float
|
Lower band-pass cutoff in Hz. Defaults to 0.04. |
0.04
|
high_cut
|
float
|
Upper band-pass cutoff in Hz. Defaults to 0.07. |
0.07
|
order
|
int
|
Butterworth filter order. Defaults to 5. |
5
|
pairwise
|
bool
|
Compute on pairwise phase-angle differences instead of the raw phase angles. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
References
Birn, R. M., Smith, M. A., Bandettini, P. A., & Diamond, J. B. (2006). Separating respiratory-variation-related fluctuations from neuronal-activity-related fluctuations in fMRI. NeuroImage, 31, 1536-1548.
Buzsáki, G., & Draguhn, A. (2004). Neuronal oscillations in cortical networks. Science, 304(5679), 1926-1929.
Fisher, N. I. (1995). Statistical analysis of circular data. Cambridge University Press.
Glerean, E., Salmi, J., Lahnakoski, J. M., Jääskeläinen, I. P., & Sams, M. (2012). Functional magnetic resonance imaging phase synchronization as a measure of dynamic functional connectivity. Brain Connectivity, 2(2), 91-101.
make_cosine_basis ¶
Create basis functions for a discrete cosine transform.
Based on the implementation in spm_filter and spm_dctmtx because
scipy DCT can only apply transforms but not return the basis functions. Like
SPM, this does not add a constant (i.e. intercept), but does retain the first
basis (i.e. sigmoidal/linear drift).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
nsamples
|
int
|
Number of observations (e.g. TRs). |
required |
sampling_interval
|
float
|
Seconds between observations (the TR), SPM's
|
required |
filter_length
|
int
|
Filter length in seconds. |
required |
unit_scale
|
bool
|
Scale the basis functions to the range [-1, 1]. Defaults to True. |
True
|
drop
|
int
|
Number of leading (slowest) bases to drop after the constant is
removed; |
0
|
Returns:
| Type | Description |
|---|---|
ndarray
|
Basis matrix of shape (nsamples, n_bases). |
Note
The basis count follows spm_dctmtx's k = fix(2*(n*RT)/HParam + 1) less
the constant that spm_filter drops, so 100 TRs of 2 s under a 128 s
filter give three bases, the same three SPM uses.
matrix_permutation_test ¶
matrix_permutation_test(
data1: ndarray,
data2: ndarray,
*,
n_permute: int = 5000,
metric: str = "pearson",
how: str = "upper",
include_diag: bool = False,
tail: int | str = 2,
return_null: bool = False,
n_jobs: int = -1,
random_state: int | None = None,
progress_bar: bool = False,
) -> dict
Matrix permutation test (Mantel test) for correlating two square matrices.
Tests whether the correlation between the elements of two matrices is
significant by permuting the rows and columns of one matrix together
(data1[perm][:, perm]) while keeping the other fixed. Each permutation
preserves the matrix's structure (including symmetry) but destroys its
relationship to data2; the p-value is the fraction of permuted correlations
at least as extreme as the observed one. Assumes both matrices are square and
the same size, and that row/column ordering is exchangeable under the null.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data1
|
ndarray
|
First square matrix (n×n). |
required |
data2
|
ndarray
|
Second square matrix (n×n). |
required |
n_permute
|
int
|
Number of permutations. Defaults to 5000. |
5000
|
metric
|
str
|
Correlation metric, one of 'pearson', 'spearman', or 'kendall'. Defaults to 'pearson'. |
'pearson'
|
how
|
str
|
Which elements to compare: 'upper' (upper triangle; assumes
symmetric matrices), 'lower' (lower triangle), or 'full' (all elements,
see |
'upper'
|
include_diag
|
bool
|
Include diagonal elements (only when |
False
|
tail
|
int | str
|
|
2
|
return_null
|
bool
|
Also return the null distribution. Defaults to False. |
False
|
n_jobs
|
int
|
Number of joblib workers, -1 = all cores. Defaults to -1. Results are identical at every worker count. |
-1
|
random_state
|
int | None
|
Random seed for reproducibility. |
None
|
progress_bar
|
bool
|
Show a progress bar over permutations. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys 'correlation' (float, observed correlation), 'p' (float,
Phipson-Smyth corrected p-value), and 'null_dist' (np.ndarray) when
|
References
Chen, G. et al. (2016). Untangling the relatedness among correlations, part I: nonparametric approaches to inter-subject correlation analysis at the group level. NeuroImage, 142, 248-259.
Mantel, N. (1967). The detection of disease clustering and a generalized regression approach. Cancer Research, 27(2), 209-220.
Examples:
import numpy as np
from nltools.algorithms.inference import matrix_permutation_test
# Two 20×20 similarity matrices sharing a common pattern
rng = np.random.default_rng(42)
pattern = rng.standard_normal((20, 10))
data1 = np.corrcoef(pattern + rng.standard_normal((20, 10)) * 0.5)
data2 = np.corrcoef(pattern + rng.standard_normal((20, 10)) * 0.5)
result = matrix_permutation_test(data1, data2, n_permute=1000)
print(f"Correlation: {result['correlation']:.3f}, p = {result['p']:.4f}")
multi_threshold ¶
Threshold a statistic image at several p-values and count the passes per voxel.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t_map
|
BrainData
|
Statistic image (e.g. t-values or betas). |
required |
p_map
|
BrainData
|
P-value image with the same voxels as |
required |
thresh
|
list[float]
|
P-value thresholds to apply. |
required |
Returns:
| Type | Description |
|---|---|
BrainData
|
Cumulative map. Positive values count how many thresholds a positive statistic passed; negative values count the same for negative statistics. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If either image is not a |
Note
Calling threshold once per level gives separate images; this returns a
single map of the threshold hierarchy, which nilearn.image.threshold_img
cannot produce.
one_sample_permutation_test ¶
one_sample_permutation_test(
data: ndarray,
*,
n_permute: int = 5000,
tail: int | str = 2,
return_null: bool = False,
n_jobs: int = -1,
random_state: int | None = None,
progress_bar: bool = False,
) -> dict
One-sample permutation test using sign flipping.
Tests whether the mean of data differs from zero by randomly flipping the
sign of each observation — the permutation analogue of a one-sample t-test.
Multi-feature (voxel-wise) data tests each column independently against
the same permutations.
Assumes errors are distributed symmetrically around zero. For strongly
skewed data, prefer bootstrap resampling. NaN observations are dropped
from the observed and every permuted mean (np.nanmean), feature by
feature.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
ndarray
|
Data to test, shape |
required |
n_permute
|
int
|
Number of permutations. Defaults to 5000. |
5000
|
tail
|
int | str
|
|
2
|
return_null
|
bool
|
If True, include the full null distribution in the result. Defaults to False. |
False
|
n_jobs
|
int
|
Number of joblib workers. Defaults to -1 (all cores). Results are identical at every worker count. |
-1
|
random_state
|
int | None
|
Random seed for reproducibility. |
None
|
progress_bar
|
bool
|
Whether to display a progress bar. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
Examples:
# Single feature
data = np.random.randn(30)
result = one_sample_permutation_test(data, n_permute=5000)
result["p"] # → 0.23
# Voxel-wise test
data = np.random.randn(30, 10000) # 30 subjects, 10K voxels
result = one_sample_permutation_test(data, n_permute=5000)
result["mean"].shape # → (10000,)
result["p"].shape # → (10000,)
phase_randomize ¶
phase_randomize(
data: ndarray,
*,
random_state: int | RandomState | None = None,
) -> ndarray
FFT-based phase randomization for time-series data.
Preserves the power spectrum (and therefore the autocorrelation) exactly, up to numerical precision, while destroying nonlinear temporal structure. Used to test whether data was generated by a linear Gaussian process or contains nonlinear dynamics.
The signal is transformed with an FFT, each positive frequency is multiplied
by exp(iφ) with φ drawn uniformly from [0, 2π], the matching negative
frequency by the conjugate exp(-iφ) so the inverse FFT is real, and the
result is transformed back.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
ndarray
|
Time series, shape (n_samples,) or (n_samples, n_features). |
required |
random_state
|
int | RandomState | None
|
Random seed for reproducibility. |
None
|
Returns:
| Type | Description |
|---|---|
ndarray
|
Phase-randomized data with the same shape as the input. |
Examples:
procrustes_distance ¶
Test matrix similarity using Procrustes superposition.
Matrices need to match in size on their first dimension only, as the smaller matrix on the second dimension will be padded with zeros. After aligning two matrices using the Procrustes transformation, use the computed disparity between them (sum of squared error of elements) as a similarity metric. Shuffle the rows of one of the matrices and recompute the disparity to perform inference (Peres-Neto & Jackson, 2001).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mat1
|
ndarray
|
1d or 2d array; must have the same number of rows as
|
required |
mat2
|
ndarray
|
1d or 2d array; must have the same number of rows as
|
required |
n_permute
|
int
|
Number of permutation iterations. Defaults to 5000. |
5000
|
tail
|
int | str
|
|
2
|
n_jobs
|
int
|
Number of CPUs for the permutations; -1 (default) uses all. |
-1
|
random_state
|
int | RandomState | None
|
Seed or generator for the row shuffling. Defaults to None. |
None
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
regress ¶
Fit an OLS regression of Y on X.
Does not add an intercept; include one in X explicitly. If Y is 2D, a
separate regression is fit to each column.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
ndarray
|
Design matrix, shape (n_samples, n_regressors). |
required |
Y
|
ndarray
|
Response, shape (n_samples,) or (n_samples, n_targets). |
required |
stats
|
str
|
'full' returns the 6-tuple below, 'betas' returns just |
'full'
|
tail
|
int | str
|
2 or 'two' for two-tailed p-values (default); 1 or 'one' for a one-tailed test of beta > 0 (negate a regressor for the other direction). |
2
|
Returns:
| Type | Description |
|---|---|
tuple
|
|
Note
Residual degrees of freedom are n - rank(X), so a rank-deficient design
is scored against the subspace it actually fits. A target whose fit is
undefined (a NaN in Y, say) scores t and p as NaN; a perfect fit,
whose standard error is finite but near zero, scores t = 0, p = 1.
threshold ¶
Threshold a statistic image by the p-values in a separate image.
Voxels whose p-value is at or above thr are set to zero in a copy of stat.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stat
|
BrainData
|
Statistic image (e.g. betas or t-values). |
required |
p
|
BrainData
|
P-value image with the same voxels as |
required |
thr
|
float
|
P-value threshold; voxels with |
0.05
|
return_mask
|
bool
|
Also return the binary thresholding mask. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
BrainData | tuple[BrainData, BrainData]
|
The thresholded image, or the
tuple |
Raises:
| Type | Description |
|---|---|
ValueError
|
If either argument is not a |
Note
BrainData.threshold and nilearn.image.threshold_img threshold an image
by its own values; this function is the only one that thresholds one
image by the p-values of another.
transform_pairwise ¶
Transform data into pairwise differences with balanced labels for ranking.
Turns an n-class ranking problem into a two-class classification problem: every pair of samples with different target values becomes one difference row, and signs are flipped so that the -1 and +1 classes are balanced.
Reference: Herbrich, R., Graepel, T., & Obermayer, K. "Large Margin Rank Boundaries for Ordinal Regression".
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
X
|
ndarray
|
Data, shape (n_samples, n_features). |
required |
y
|
ndarray
|
Target labels, shape (n_samples,) or (n_samples, 2). A second column groups the samples; pairs from different groups are skipped. |
required |
Returns:
| Type | Description |
|---|---|
tuple[ndarray, ndarray]
|
|
trim ¶
Trim a Polars DataFrame/Series by replacing outlier values with NaNs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
DataFrame | Series
|
Data to trim. |
required |
cutoff
|
dict
|
A dictionary with keys |
None
|
Returns:
| Type | Description |
|---|---|
DataFrame | Series
|
Trimmed data (outliers replaced with NaN), the same type as the input. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
two_sample_permutation_test ¶
two_sample_permutation_test(
data1: ndarray,
data2: ndarray,
*,
n_permute: int = 5000,
tail: int | str = 2,
return_null: bool = False,
n_jobs: int = -1,
random_state: int | None = None,
progress_bar: bool = False,
) -> dict
Two-sample permutation test using group-label shuffling.
Tests whether two independent groups have different means by randomly reassigning observations to groups — the permutation analogue of an independent-samples t-test. Group sizes may differ. Multi-feature (voxel-wise) data tests each column independently against the same permutations.
Assumes exchangeability under the null (group assignment is arbitrary):
independent samples from similarly shaped distributions. NaN observations
are dropped from the observed and every permuted mean (np.nanmean),
feature by feature.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data1
|
ndarray
|
Group 1 data, shape |
required |
data2
|
ndarray
|
Group 2 data, shape |
required |
n_permute
|
int
|
Number of permutations. Defaults to 5000. |
5000
|
tail
|
int | str
|
|
2
|
return_null
|
bool
|
If True, include the full null distribution in the result. Defaults to False. |
False
|
n_jobs
|
int
|
Number of joblib workers. Defaults to -1 (all cores). Results are identical at every worker count. |
-1
|
random_state
|
int | None
|
Random seed for reproducibility. |
None
|
progress_bar
|
bool
|
Whether to display a progress bar. Defaults to False. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
Keys |
Examples:
# Single feature
data1 = np.random.randn(20) # Group 1: 20 subjects
data2 = np.random.randn(25) # Group 2: 25 subjects
result = two_sample_permutation_test(data1, data2, n_permute=5000)
result["p"] # → 0.45
# Voxel-wise test
data1 = np.random.randn(20, 10000) # 20 subjects, 10K voxels
data2 = np.random.randn(25, 10000) # 25 subjects, 10K voxels
result = two_sample_permutation_test(data1, data2, n_permute=5000)
result["mean_diff"].shape # → (10000,)
result["p"].shape # → (10000,)
upsample ¶
Upsample a Polars DataFrame/Series to a new target frequency or number of samples using interpolation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
DataFrame | Series
|
Data to upsample. Non-numeric columns are dropped from a DataFrame; Boolean columns count as numeric and are interpolated as 0/1. |
required |
sampling_freq
|
float
|
Sampling frequency of the data in Hz. |
None
|
target
|
float
|
Upsampling target. |
None
|
target_type
|
str
|
Unit of |
'samples'
|
method
|
str
|
Interpolation method, one of 'linear', 'nearest', 'zero', 'slinear', 'quadratic', or 'cubic'; 'zero', 'slinear', 'quadratic' and 'cubic' refer to spline interpolation of zeroth, first, second or third order (default: 'linear'). |
'linear'
|
Returns:
| Type | Description |
|---|---|
DataFrame | Series
|
Upsampled data, the same type as the input. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Note
Dropping non-numeric columns emits a UserWarning naming them.
winsorize ¶
Winsorize a Polars DataFrame/Series with the largest/lowest value not considered outlier.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
DataFrame | Series
|
Data to winsorize. |
required |
cutoff
|
dict
|
A dictionary with keys |
None
|
replace_with_cutoff
|
bool
|
If True, replace outliers with the cutoff value; if False, replace them with the closest existing values (default: True). |
True
|
Returns:
| Type | Description |
|---|---|
DataFrame | Series
|
Winsorized data, the same type as the input. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Note
Missing values are skipped when the cutoffs are computed and left in
place in the result, so winsorize(trim(x)) is meaningful.
zscore ¶
Z-score every column of a Polars or pandas DataFrame/Series.
Pandas inputs are converted to Polars; the result is always Polars (a DataFrame for DataFrame input, a Series for Series input).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
DataFrame | Series | DataFrame | Series
|
Data to z-score. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame | Series
|
Same shape as the input, each column z-scored with the sample standard deviation (ddof=1). |