Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Nearest-neighbor MIE

The nearest-neighbor estimator of Kozachenko and Leonenko estimates the entropy of a set of coordinates from the distance from each frame to its nearest neighbor among the other frames. The mutual information expansion (MIE) combines such estimates for single coordinates, pairs, and optionally triples and quadruples into an estimate for the whole molecule: order 1 is the sum of the single-coordinate entropies, order 2 also subtracts the mutual information of every pair, and so on. Select the order with --mie-order (default 2), mie_order= in Python, or the mie_order argument in Rust.

Periodic coordinates use the distance around the circle. --no-periodic treats every coordinate as linear, which is faster but loses some correctness for torsions.

Options

In Rust these are the fields of EntropyOptions. The CLI and Python have matching settings: --exact-constant or exact_constant=True, and --theiler-window or theiler_window=. For file-based calculations the metrics come from the coordinates; for a raw array in Python, pass periods=.

  • metrics: per-coordinate CoordinateMetric::Linear or CoordinateMetric::Periodic { period }. The default, None, treats every coordinate as linear.
  • constant: FiniteSampleConstant::Asymptotic (the default) or FiniteSampleConstant::Exact.
  • theiler_window: frames j with |i - j| <= theiler_window are never frame i's nearest neighbor. The default, 0, excludes only frame i itself.

Joint estimate

The MIE builds the entropy from estimates in one to four dimensions. A single nearest-neighbor estimate over a whole set of coordinates is also available, with no expansion: estimate_joint_entropy in Rust, for up to 8 coordinates (MAX_JOINT_DIMENSION), and the split estimator's --soft-estimator joint. It has no truncation error, but its bias grows quickly with the number of coordinates, so it suits small sets such as the soft torsions of a ligand. On a periodic box the nearest-neighbor search skips the periodic images that cannot hold a closer neighbor, which keeps it fast in higher dimensions.

The mutual information spanning tree (MIST; King, Killian and Tidor, J. Phys. Chem. B 116, 2891, 2012) uses the same one- and two-dimensional terms as MIE-2, but keeps only the pairwise mutual information along the maximum spanning tree of the coordinates: the sum of the marginal entropies minus one mutual information per coordinate after the first. With exact terms it is an upper bound on the joint entropy. MIE-2 subtracts every pair, so when many coordinates are coupled, as in a folded chain, it can count the same dependence several times and fall far below the joint entropy. MIST cannot. It is estimate_mist_entropy in Rust and the split estimator's --soft-estimator mist.

A tree of pairs still misses dependence among three or more coordinates, such as the backbone and side-chain torsions of a structured peptide moving together. The Bayesian network estimator extends MIST by the chain rule: it takes the coordinates in the order the spanning tree adds them and conditions each on up to p earlier ones (its parents), H = H(X1) + Σ H(Xi | parents of Xi). Each coordinate's first parent is its tree link, so with p = 1 it is MIST. Further parents are added greedily from the 10 earlier coordinates of highest pairwise mutual information with it, each time the one that lowers the conditional entropy most, while any does. The conditional entropies are differences of joint nearest-neighbor entropies of up to p + 1 coordinates, so p is at most 7. With exact terms it is still an upper bound on the joint entropy, and never above MIST. It is estimate_network_entropy(data, frames_end, parents, &options) in Rust and the split estimator's --soft-estimator network (--network-parents, default 5). On peptides it roughly halves MIST's error; see Validation.

Notes

  • The default second-order mutual-information expansion is a truncation and can omit higher-order correlations.
  • Following the historical Python implementation, the default nearest-neighbor constant uses the large-sample approximation ln(N) + EulerGamma in place of the exact finite-sample term psi(N) - psi(1) = H_(N-1). Their per-entropy difference is approximately 1/(2N) and is negligible for the intended 50,000-frame calculations, but it can accumulate across MIE terms or become important for very small samples. The convention is a property of the estimator, not of the total alone: pairwise mutual information shifts by exactly H_(N-1) - (ln(N) + EulerGamma) per pair when it changes, so per-coordinate results must be computed under the same convention as the total they are compared against. The Python implementation truncated EulerGamma to 0.57721566490153; this crate uses the full double-precision value, so results can differ from it by roughly 1e-14.
  • Exact duplicate samples use the nearest distinct coordinate point. Many ties usually indicate inadequate coordinate precision or sampling and should be investigated.

For correlated frames, see Correlated frames.