Natural Evolution Strategies (NES)

class pypop7.optimizers.nes.nes.NES(problem, options)[source]

Natural Evolution Strategies (NES).

This is the abstract class for all NES classes. Please use any of its instantiated subclasses to optimize the black-box problem at hand.

Note

NES is a family of well-principled population-based randomized search methods with a relatively clean derivation from first principles, which maximize the expected fitness along with (estimated) natural gradients. In this library, we have converted it to the minimization problem, in accordance with other modules.

For some interesting applications of NES, please refer to [Xu et al., 2024, ICLR], [Liu et al., 2024, TC (Columbia University, NVIDIA Research, Nokia Bell Labs, etc.)], [Xuan Zhang et al., 2024, IEEE-LRA], [Conti et al., 2018, NeurIPS], to name a few.

Parameters:

problem (dict) –
problem arguments with the following common settings (keys):
- ’fitness_function’ - objective function to be minimized (func),
- ’ndim_problem’ - number of dimensionality (int),
- ’upper_boundary’ - upper boundary of search range (array_like),
- ’lower_boundary’ - lower boundary of search range (array_like).
options (dict) –
optimizer options with the following common settings (keys):
- ’max_function_evaluations’ - maximum of function evaluations (int, default: np.inf),
- ’max_runtime’ - maximal runtime to be allowed (float, default: np.inf),
- ’seed_rng’ - seed for random number generation needed to be explicitly set (int);
and with the following particular settings (keys):
- ’n_individuals’ - number of offspring/descendants, aka offspring population size (int),
- ’n_parents’ - number of parents/ancestors, aka parental population size (int),
- ’mean’ - initial (starting) point (array_like),
  - If not given, it will draw a random sample from the uniform distribution whose search range is bounded by problem[‘lower_boundary’] and problem[‘upper_boundary’].
- ’sigma’ - initial global step-size, aka mutation strength (float).

mean

initial (starting) point, aka mean of Gaussian search/sampling/mutation distribution. If not given, it will draw a random sample from the uniform distribution whose search range is bounded by problem[‘lower_boundary’] and problem[‘upper_boundary’], by default.

Type:: array_like

n_individuals

number of offspring/descendants, aka offspring population size (should > 0).

Type:: int

n_parents

number of parents/ancestors, aka parental population size (should > 0).

Type:: int

sigma

final global step-size, aka mutation strength or overall std of Gaussian search distribution (should > 0.0).

Type:: float

References

Hüttenrauch, M. and Neumann, G., 2024. Robust black-box optimization for stochastic search and episodic reinforcement learning. Journal of Machine Learning Research, 25(153), pp.1-44.

Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J. and Schmidhuber, J., 2014. Natural evolution strategies. Journal of Machine Learning Research, 15(1), pp.949-980.

Schaul, T., 2011. Studies in continuous black-box optimization. Doctoral Dissertation, Technische Universität München.

Yi, S., Wierstra, D., Schaul, T. and Schmidhuber, J., 2009, June. Stochastic search using the natural gradient. In Proceedings of International Conference on Machine Learning (pp. 1161-1168).

Wierstra, D., Schaul, T., Peters, J. and Schmidhuber, J., 2008, June. Natural evolution strategies. In IEEE Congress on Evolutionary Computation (pp. 3381-3387). IEEE.

Please refer to the official Python source code from PyBrain (now not actively maintained): https://github.com/pybrain/pybrain

NES: