attainable-information

Finite-sample ceilings on how much of one modality any model can recover from another. The package implements the estimators from Image-Identifiable Genomic Subspaces: A Linear-Gaussian Model of Radiogenomic Recoverability, where the two modalities are genomics and imaging, but nothing in the code depends on that interpretation.

Given paired matrices G (n x p) and X (n x d), two quantities are reported.

Channel recoverability is the infinite-data limit. Regularized canonical correlation analysis between G and X gives the spectrum R_i = rho_i^2, the identifiable directions of each modality, and the Gaussian mutual information

I(G; X) = -1/2 * sum_i log(1 - rho_i^2)

Attainable information is the finite-sample ceiling. For a direction with channel recoverability rho^2 and a d*-dimensional representation of X, no learning algorithm trained on n samples attains out-of-sample R^2 above

R_n = rho^2 * n / (n + nu),        nu = d* * (1 - rho^2) / rho^2
I_n = -1/2 * sum_i log(1 - R_n,i)

nu is the learning cost of the direction: the cohort size at which half of the channel value is attainable, equal to the feature count divided by the channel signal-to-noise ratio, and equal to the Bayes-optimal ridge penalty in whitened coordinates. Because R_n scales as rho^4 for weak channels, marginal directions are doubly penalized at fixed n.

Installation

pip install git+https://github.com/pachterlab/attainable_information.git

Dependencies are numpy, scipy, and scikit-learn. For a development install with tests and the example figure:

git clone https://github.com/pachterlab/attainable_information.git
cd attainable-information
pip install -e ".[dev,examples]"
pytest

Usage

import numpy as np
import attainable_information as ai

G, X, truth = ai.make_synthetic_radiogenomics(n=300, p=60, d=20, k=3, random_state=0)

fit = ai.fit_recoverability(G, X, n_components=5)       # channel spectrum
fit.recoverability                                       # rho_i^2, descending
fit.mutual_information()                                 # I(G;X) in nats

cv = ai.cross_validated_recoverability(G, X, n_components=5).mean(0)   # held-out R^2
obs, null, p = ai.cv_permutation_test(G, X, n_components=5, n_perm=200)  # effective rank

n, d_star = G.shape[0], X.shape[1]
ai.attainable_recoverability(fit.recoverability, n, d_star)   # per-direction ceiling R_n
ai.attainable_information(fit.recoverability, n, d_star)      # I_n in bits
ai.learning_cost(fit.recoverability, d_star)                  # nu per direction
ai.sample_size_for_fraction(fit.recoverability, d_star, 0.9)  # n for 90% of channel

The command-line entry point prints the same table. With no arguments it uses synthetic data with known ground truth.

attainable-information
attainable-information --G genomics.csv --X imaging.csv --n-components 5

Reproducing the ceiling figure

python examples/synthetic_ceiling.py

This plants a channel with known canonical correlations, redraws the signal direction in every replicate, trains ridge and ordinary least squares on n samples to predict the leading canonical score, and scores them on a large test set. Held-out R^2 stays below the ceiling and ridge tracks it, while in-sample R^2 sits above the channel value at small n.

Recommended workflow on real data

1. Align rows across modalities and standardize columns. For heavy-tailed features apply gaussian_rank_transform to each modality so the fit keys on dependence rather than marginal shape.
2. Fit fit_recoverability with Ledoit-Wolf shrinkage (the default) when p or d is comparable to n.
3. Report cross_validated_recoverability rather than the in-sample spectrum, and use cv_permutation_test to determine the effective rank. The in-sample permutation test builds its null on an inflated scale.
4. Compare distance_correlation_test against the linear spectrum to check for nonlinear dependence the model would miss.
5. Compute the ceiling at the cohort size and working dimension actually used. optimal_working_dimension selects d* when spectra are available at several representation sizes.

API

Function	Purpose
fit_recoverability	Regularized CCA; returns a RecoverabilityFit with spectrum, directions, covariances
posterior	Bayes-optimal map E[g given x] and posterior covariance
direction_recoverability	R(v) for a named direction v
cross_validated_recoverability	Held-out R_i per canonical direction
permutation_test, cv_permutation_test	Null distributions for the leading directions
distance_correlation, distance_correlation_test	Model-free dependence check
gaussian_rank_transform	Column-wise copula marginal transform
attainable_recoverability, attainable_information	Finite-n ceiling per direction and in total
learning_cost, sample_size_for_fraction	nu and the design formula
weak_direction_bound	Quartic small-signal bound
optimal_working_dimension	Best d* at fixed n
max_total_information, max_information_given_fourth_moment	Spectrum-free ceilings from scalar budgets
anchor_decomposition	Chain-rule split of I(G;X) into anchor and residual terms
auc_ceiling	Bayes AUC bound for binarized targets
make_synthetic_radiogenomics, true_recoverability	Ground-truth generator and closed-form spectrum

Citation

See CITATION.cff.

License

MIT.