dynadojo.challenges.FixedTrainSize#

class dynadojo.challenges.FixedTrainSize#

Bases: Challenge

Challenge where the size of the training set is fixed, the complexity of the system is varied, and the error is measured.

Methods

__init__(n, L, E, t, system_cls, reps, ...)

Initialize the class.

evaluate(algo_cls[, algo_kwargs, ...])

Evaluates an algorithm class (NOT an instance) on a dynamical system over a set of experimental parameters.

plot(data[, n, show, show_stats, plot_ood, ax])

returns:

A plot of the data.

system_run(rep_id, latent_dim, embed_dim, ...)

For a given system latent dimension and embedding dimension, instantiates system and for a specific N, evaluates the algorithm on the system.
__init__(n, L, E, t, system_cls, reps, test_examples, test_timesteps, max_control_cost_per_dim=0, control_horizons=0, system_kwargs=None, verbose=True)#

Initialize the class.

Parameters:

  • n (int) – The size of the training set.

  • L (int) – The complexities of the system.

  • E (int) – The embedding dimensions of the system.

  • t (int) – The number of timesteps to simulate.

  • max_control_cost_per_dim (int) – The maximum control cost per dimension.

  • control_horizons (int) – The number of control horizons to consider.

  • system_cls (type) – The system class to use.

  • reps (int) – The number of repetitions to run.

  • test_examples (int) – The number of test examples to use.

  • test_timesteps (int) – The number of timesteps to simulate for the test examples.

  • system_kwargs (dict) – The keyword arguments to pass to the system class.

  • verbose (bool) –

evaluate(algo_cls, algo_kwargs=None, fit_kwargs=None, act_kwargs=None, ood=False, noisy=False, id=None, num_parallel_cpu=-1, seed=None, reps_filter=None, L_filter=None, rep_l_filter=None)#

Evaluates an algorithm class (NOT an instance) on a dynamical system over a set of experimental parameters.

Parameters:

  • algo_cls (type[AbstractAlgorithm]) – Algorithm class to be evaluated.

  • algo_kwargs (dict or None, optional) – Keyword arguments to be passed to algo_cls.

  • fit_kwargs (dict or None, optional) – Keyword arguments to be passed to algo_cls.fit.

  • act_kwargs (dict or None, optional) – Keyword arguments to be passed to algo_cls.act.

  • ood (bool, optional) – If True, also test on out-distribution initial conditions for the test set. (For FixedError, search is performed on ood_error if ood=True.) Defaults to False. If False, generate out-of-distribution initial conditions for the test set.

  • noisy (bool, optional) – If True, add noise to the train set. Defaults to False. If False, no noise is added.

  • id (int or None, optional) – Algorithm ID associated with evaluation results in the returned DataFrame.

  • num_parallel_cpu (int, optional) – Number of CPUs to use in parallel. Defaults to -1, which uses all available CPUs.

  • seed (int or None, optional) – Seed to seed the random number generator for seeding systems and algorithms. Defaults to None. Is overridden by seeds in system_kwargs or algo_kwargs.

  • reps_filter (list of int or None, optional) – If provided, will only evaluate system_runs with the given rep_ids. Defaults to None, which evaluates all repetitions.

  • L_filter (list of int or None, optional) – If provided, will only evaluate system_runs with the given latent dimensions. Defaults to None, which evaluates all latent dimensions.

  • rep_l_filter (list of tuple(int, int) or None, optional) – If provided, will only evaluate system_runs with the given (rep_id, latent_dim) pairs. Defaults to None, which evaluates all (rep_id, latent_dim) pairs.

Returns:

A DataFrame where each row is the result of an algorithm trained and evaluated on a single system.

Return type:

pandas.DataFrame

static plot(data, n=None, show=True, show_stats=False, plot_ood=True, ax=None)#
Returns:

A plot of the data.

Return type:

matplotlib.axes.Axes

Parameters:

  • data (DataFrame) –

  • n (int | None) –

  • show (bool) –

  • show_stats (bool) –

system_run(rep_id, latent_dim, embed_dim, algo_cls, algo_kwargs=None, fit_kwargs=None, act_kwargs=None, noisy=False, test_ood=False, system_seed=None, algo_seed=None, **kwargs)#

For a given system latent dimension and embedding dimension, instantiates system and for a specific N, evaluates the algorithm on the system. Across runs, the algorithm is re-initialized with the same seed.

Note

The seed in algo_kwargs and system_seed in system_kwargs takes precedence over the seed passed to this function.

Parameters:

  • algo_cls (type[AbstractAlgorithm]) –

  • algo_kwargs (dict | None) –

  • fit_kwargs (dict | None) –

  • act_kwargs (dict | None) –

  • noisy (bool) –

  • test_ood (bool) –