Roc ¶

Roc(
    *,
    input_values=None,
    binary_outcome=None,
    method="optimal_overall",
    forced_choice=None,
)

Compute receiver operating characteristic curves for single-interval or forced-choice classification.

The Roc class is based on Tor Wager's Matlab roc_plot.m function and allows a user to easily run different types of receiver operator characteristic curves. For example, one might be interested in single interval or forced choice.

Parameters:

Name	Type	Description	Default
`input_values`	`array - like`	1-D continuous decision values, one per observation.	`None`
`binary_outcome`	`array - like`	Boolean class label per observation.	`None`
`method`	`str`	Threshold-selection variant, naming what the chosen threshold maximizes or minimizes: `'optimal_overall'` maximizes the number of correct classifications, so the larger class dominates; `'optimal_balanced'` maximizes balanced accuracy, the mean of sensitivity and specificity, weighting the two classes equally; `'minimum_sdt_bias'` minimizes the signal-detection response bias `c`, which places the threshold midway between the two classes' estimated distributions. With equal class sizes the first two often agree.	`'optimal_overall'`
`forced_choice`	`array - like`	Subject id per observation for forced-choice classification (each subject contributes one positive and one negative observation).	`None`

Attributes:

Name	Type	Description
`input_values`	`ndarray`	Decision values.
`binary_outcome`	`ndarray`	Boolean labels.
`method`	`str`	Configured threshold-selection variant. Set at construction; `calculate`'s `method=` argument reads this as its default and never writes back to it, so an explicit override passed to `calculate` only affects that call.
`forced_choice`	`ndarray \| None`	Subject ids for forced-choice classification.
`criterion_values`	`ndarray`	Thresholds at which `tpr`/`fpr` were evaluated; set by `calculate`.
`tpr`	`ndarray`	True positive rate per criterion value; set by `calculate`.
`fpr`	`ndarray`	False positive rate per criterion value; set by `calculate`.
`auc`	`float`	Area under the ROC curve; set by `calculate`.
`class_thr`	`float`	Selected classification threshold; set by `calculate`.
`sensitivity`	`float`	Sensitivity at `class_thr`; set by `calculate`.
`specificity`	`float`	Specificity at `class_thr`; set by `calculate`.
`ppv`	`float`	Positive predictive value at `class_thr`; set by `calculate`.
`accuracy`	`float`	Classification accuracy; set by `calculate`.
`accuracy_se`	`float`	Standard error of the accuracy; set by `calculate`.
`accuracy_p`	`BinomTestResult`	`scipy.stats.binomtest` result comparing accuracy against chance (read `.pvalue`); set by `calculate`.
`tpr_smooth`	`ndarray`	Gaussian-model true positive rate curve; set by `plot(method='gaussian')`. Never read by `calculate`.
`fpr_smooth`	`ndarray`	Gaussian-model false positive rate curve; set by `plot(method='gaussian')`. Never read by `calculate`.
`aucn`	`float`	Area under the Gaussian-model curve (`tpr_smooth`/`fpr_smooth`); set by `plot(method='gaussian')`. Never read by `calculate`.
`gaussian_sensitivity`	`float`	Gaussian-model sensitivity estimate for forced-choice data; set by `plot(method='gaussian')`. Never read by `calculate`.
`gaussian_specificity`	`float`	Gaussian-model specificity estimate for forced-choice data; set by `plot(method='gaussian')`. Never read by `calculate`.
`gaussian_ppv`	`float`	Gaussian-model positive predictive value for forced-choice data; set by `plot(method='gaussian')`. Never read by `calculate`.
`gaussian_auc`	`float`	Gaussian-model area under the curve for forced-choice data; set by `plot(method='gaussian')`. Never read by `calculate`.

Methods:

Name	Description
`calculate`	Calculate ROC metrics and store them on the instance.
`plot`	Create a ROC plot.
`summary`	Display a formatted summary of ROC analysis.

Methods:¶

calculate ¶

calculate(
    *,
    input_values=None,
    binary_outcome=None,
    criterion_values=None,
    method=None,
    forced_choice=None,
    balanced_acc=False,
    tail=2,
)

Calculate ROC metrics and store them on the instance.

Parameters:

Name	Type	Description	Default
`input_values`	`array - like`	1-D continuous decision values, one per observation. Defaults to the values given at construction.	`None`
`binary_outcome`	`array - like`	Boolean class label per observation. Defaults to the labels given at construction.	`None`
`criterion_values`	`array - like`	Thresholds at which to evaluate `fpr` and `tpr`. Defaults to a dense grid over the range of `input_values`.	`None`
`method`	`str`	Threshold-selection variant, one of `'optimal_overall'` (maximize correct classifications), `'optimal_balanced'` (maximize balanced accuracy, the mean of sensitivity and specificity), or `'minimum_sdt_bias'` (minimize signal-detection response bias). Defaults to `None`, which uses the instance's configured `method` (set at construction, or by assigning `self.method` directly). An explicit value overrides the configured `method` for this call only and does not change `self.method`.	`None`
`forced_choice`	`array - like`	Subject id per observation for forced-choice classification.	`None`
`balanced_acc`	`bool`	Report balanced accuracy (mean of sensitivity and specificity) instead of overall accuracy. Only affects the accuracy estimate, not the p-value or the threshold used for sensitivity/specificity.	`False`
`tail`	`int \| str`	`2`/`'two'` for two-tailed (default); `1`/`'one'` for one-tailed (accuracy > chance) in the binomial test for `accuracy_p`.	`2`

plot ¶

plot(*, method='gaussian', balanced_acc=False)

Create a ROC plot.

Runs calculate first, then plots either a Gaussian-smoothed ROC curve fit to the decision values or the observed empirical curve. The underlying calculate call re-runs with the instance's configured method (it never overrides the threshold rule), and the Gaussian-model curve estimates are stored on their own attributes rather than overwriting calculate's sensitivity, specificity, ppv, and auc.

Parameters:

Name	Type	Description	Default
`method`	`str`	Type of plot, `'gaussian'` or `'observed'`.	`'gaussian'`
`balanced_acc`	`bool`	Passed to `calculate`; report balanced accuracy.	`False`

Returns:

Type	Description
`Figure`	The ROC figure.

Note

For method='gaussian' on forced-choice data, this also sets gaussian_sensitivity, gaussian_specificity, gaussian_ppv, and gaussian_auc from the fitted Gaussian model. For method='gaussian' on either kind of data, it also sets tpr_smooth, fpr_smooth, and aucn (the smoothed curve and its AUC). None of these attributes are read by calculate.

summary ¶

summary()

Display a formatted summary of ROC analysis.