AutoHestia

AutoHestia automatically sweeps all combinations of partitioning algorithms and similarity metrics on your dataset, evaluates each combination with a k-nearest neighbours probe, and applies guardrails to select the best-performing splits.

Automatic benchmarking and selection of dataset partitioning strategies.

Sweeps all combinations of built-in (or custom) partitioning algorithms and pre-computed similarity metrics, evaluates each combination using a k-nearest neighbours probe across similarity thresholds from 0.1 to 1.0, and ranks them by monotonicity and mean performance. A guardrail filter then retains only experiments that meet minimum test-size and dynamic-range requirements, returning the top-ranked partitions ready for model training.

Initialise AutoHestia.

Parameters:

Name	Type	Description	Default
df	DataFrame	DataFrame containing the dataset entities.	required
field_name	str	Column name in df that identifies each entity (e.g. sequence, SMILES, or PDB path).	required
x	ndarray	Pre-computed feature matrix of shape (n_samples, n_features) used to train and evaluate the KNN probe.	required
y	ndarray	Label array of shape (n_samples,). Arrays with fewer than 10 unique values are treated as classification; otherwise regression.	required
sim_dfs	Dict[str, DataFrame]	Mapping of metric name → pre-computed pairwise similarity Polars DataFrame (as returned by the hestia.similarity functions).	required
verbose_level	str	Logging verbosity. One of 'debug', 'info', 'warning', or 'error'. Defaults to 'debug'.	'debug'

best_guardrailed_splits(part_algs=None, custom_algs=None, top_k_parts=3, top_l_sims=3, min_test_size=0.185, min_dynamic_range=0.4, overwrite=False, save_dir='tmp')

Run the full guardrailed partitioning sweep and return the best splits.

For every (partitioning algorithm, similarity metric) pair, partitions are generated at thresholds 0.1–1.0. Each partition is evaluated with a KNN probe (MCC for classification, Spearman CC for regression). Results are then filtered and ranked:

1. Dynamic-range filter – combinations whose threshold range spans less than min_dynamic_range are discarded.
2. Top-k algorithms – the top_k_parts algorithms with the highest worst-case mean performance are retained.
3. Top-l similarities – for each retained algorithm, the top_l_sims similarity metrics with the best mean performance are kept.

Intermediate results are written to <save_dir>/ as CSV files and partition pickle files.

Parameters:

Name	Type	Description	Default
part_algs	Optional[List[str]]	List of algorithm names to include. Must be keys of AVAILABLE_ALGORITHMS ('ccpart', 'cdhit', 'sim-umap', 'perimeter_split', 'maximum_dissimilarity', 'butina'). Defaults to all available algorithms.	None
custom_algs	Optional[Dict[str, Callable]]	Optional mapping of name → callable for user-supplied partitioning functions. The callable must accept the same keyword arguments as the built-in partition functions (df, sim_df, field_name, threshold).	None
top_k_parts	int	Number of top-performing partitioning algorithms to retain after the guardrail filter. Defaults to 3.	3
top_l_sims	int	Number of similarity metrics to retain per algorithm. Defaults to 3.	3
min_test_size	float	Minimum fraction of the dataset that must fall in the test split for a threshold to be included. Defaults to 0.185.	0.185
min_dynamic_range	float	Minimum required span of valid thresholds for an algorithm–metric combination to pass the guardrail. Defaults to 0.4.	0.4
overwrite	bool	If True, existing output directories are reused. Defaults to False.	False
save_dir	str	Root output directory. Defaults to 'tmp'.	'tmp'

Returns:

Type

Description

dict

Dictionary with the following keys: 'raw-experiments' – :class:pandas.DataFrame of per-threshold KNN results for every combination. 'main-stats' – :class:pandas.DataFrame of aggregated statistics (monotonicity, dynamic range, mean performance) after the guardrail filter. 'after-guardrail' – subset of 'main-stats' containing only the top-k × top-l combinations. 'top-combination' – (part_alg, sim_metric) tuple identifying the single best combination. 'best-parts' – dict mapping similarity threshold → {'train': np.ndarray, 'test': np.ndarray} for the top combination.

Raises:

Type	Description
ValueError	If part_algs contains an unrecognised algorithm name, or if no combinations survive the dynamic-range guardrail.

plot_good_curves(save_dir='tmp', overwrite=False)

Plot GOOD curves for all evaluated algorithm–metric combinations.

Saves a good.png figure under <save_dir>/figures/ and returns the Matplotlib figure object.

Parameters:

Name	Type	Description	Default
save_dir	str	Root directory where the figures/ sub-folder will be created. Defaults to 'tmp'.	'tmp'
overwrite	bool	If True, an existing figures/ directory is reused without raising an error. Defaults to False.	False

Returns:

Type	Description
matplotlib.figure.Figure	Matplotlib Figure with one line per algorithm–metric combination.

Raises:

Type	Description
RuntimeError	If called before :meth:best_guardrailed_splits.