dynadojo.challenges.FixedComplexity#

class dynadojo.challenges.FixedComplexity#

Bases: Challenge

Challenge where complexity is fixed, training set size is varied, and error is measured.

Methods

__init__(l, t, N, system_cls, reps, ...[, ...])

Initialize the class.

evaluate(algo_cls[, algo_kwargs, ...])

Evaluates an algorithm class (NOT an instance) on a dynamical system over a set of experimental parameters.

plot(data[, latent_dim, embedding_dim, ...])

returns:

A plot of the data.

system_run(rep_id, latent_dim, embed_dim, ...)

For a given system latent dimension and embedding dimension, instantiates system and for a specific N, evaluates the algorithm on the system.
__init__(l, t, N, system_cls, reps, test_examples, test_timesteps, e=None, max_control_cost_per_dim=1, control_horizons=0, system_kwargs=None, verbose=True)#

Initialize the class.

Parameters:

  • l (int) – Latent dimension.

  • t (int) – Number of timesteps.

  • N (list[int]) – List of training set sizes.

  • system_cls (type[AbstractSystem]) – System class.

  • reps (int) – Number of repetitions.

  • test_examples (int) – Number of test examples.

  • test_timesteps (int) – Number of test timesteps.

  • e (int, optional) – Embedding dimension. Defaults to None.

  • max_control_cost_per_dim (int, optional) – Maximum control cost per dimension. Defaults to 1.

  • control_horizons (int, optional) – Number of control horizons. Defaults to 0.

  • system_kwargs (dict, optional) – System kwargs. Defaults to None.

  • verbose (bool) –

evaluate(algo_cls, algo_kwargs=None, fit_kwargs=None, act_kwargs=None, ood=False, noisy=False, id=None, num_parallel_cpu=-1, seed=None, reps_filter=None, L_filter=None, rep_l_filter=None)#

Evaluates an algorithm class (NOT an instance) on a dynamical system over a set of experimental parameters.

Parameters:

  • algo_cls (type[AbstractAlgorithm]) – Algorithm class to be evaluated.

  • algo_kwargs (dict or None, optional) – Keyword arguments to be passed to algo_cls.

  • fit_kwargs (dict or None, optional) – Keyword arguments to be passed to algo_cls.fit.

  • act_kwargs (dict or None, optional) – Keyword arguments to be passed to algo_cls.act.

  • ood (bool, optional) – If True, also test on out-distribution initial conditions for the test set. (For FixedError, search is performed on ood_error if ood=True.) Defaults to False. If False, generate out-of-distribution initial conditions for the test set.

  • noisy (bool, optional) – If True, add noise to the train set. Defaults to False. If False, no noise is added.

  • id (int or None, optional) – Algorithm ID associated with evaluation results in the returned DataFrame.

  • num_parallel_cpu (int, optional) – Number of CPUs to use in parallel. Defaults to -1, which uses all available CPUs.

  • seed (int or None, optional) – Seed to seed the random number generator for seeding systems and algorithms. Defaults to None. Is overridden by seeds in system_kwargs or algo_kwargs.

  • reps_filter (list of int or None, optional) – If provided, will only evaluate system_runs with the given rep_ids. Defaults to None, which evaluates all repetitions.

  • L_filter (list of int or None, optional) – If provided, will only evaluate system_runs with the given latent dimensions. Defaults to None, which evaluates all latent dimensions.

  • rep_l_filter (list of tuple(int, int) or None, optional) – If provided, will only evaluate system_runs with the given (rep_id, latent_dim) pairs. Defaults to None, which evaluates all (rep_id, latent_dim) pairs.

Returns:

A DataFrame where each row is the result of an algorithm trained and evaluated on a single system.

Return type:

pandas.DataFrame

static plot(data, latent_dim=None, embedding_dim=None, show=True, show_stats=False)#
Returns:

A plot of the data.

Return type:

matplotlib.axes.Axes

Parameters:

  • latent_dim (int | None) –

  • embedding_dim (int | None) –

  • show (bool) –

  • show_stats (bool) –

system_run(rep_id, latent_dim, embed_dim, algo_cls, algo_kwargs=None, fit_kwargs=None, act_kwargs=None, noisy=False, test_ood=False, system_seed=None, algo_seed=None, **kwargs)#

For a given system latent dimension and embedding dimension, instantiates system and for a specific N, evaluates the algorithm on the system. Across runs, the algorithm is re-initialized with the same seed.

Note

The seed in algo_kwargs and system_seed in system_kwargs takes precedence over the seed passed to this function.

Parameters:

  • algo_cls (type[AbstractAlgorithm]) –

  • algo_kwargs (dict | None) –

  • fit_kwargs (dict | None) –

  • act_kwargs (dict | None) –

  • noisy (bool) –

  • test_ood (bool) –