# Design Philosophy¶

Given a large number of (black-box) optimizers which keep increasing almost every day, we need some (possibly) widely acceptable criteria to select from them, as presented below in details:

Respect for

**Beauty (Elegance)**From the

*problem-solving*perspective, we empirically prefer to choose the best optimizer for the black-box optimization problem at hand. For the new problem, however, the best optimizer is often*unknown*in advance (when without*a prior*knowledge). As a rule of thumb, we need to compare a (often small) set of available/well-known optimizers and finally choose the best one according to some predefined performance criteria. From the*academic research*perspective, however, we prefer so-called**beautiful**optimizers, though always keeping the No Free Lunch Theorems in mind. Typically, the beauty of one optimizer comes from the following attractive features:**model novelty**,**competitive performance on at least one class of problems**,**theoretical insights (e.g., convergence)**,**clarity/simplicity for understanding and implementation**, and**repeatability**.If you find any to meet the above standard, welcome to launch issues or pulls. We will consider it to be included in the

*pypop7*library as soon as possible, if possible. Note that any superficial imitation to well-established optimizers (i.e. Old Wine in a New Bottle) will be**NOT**considered here. Sometimes, several**very complex**optimizers could obtain the top rank on some competitions consisting of only*artificially-constructed*benchmark functions. However, these optimizers may become**over-skilled**on these artifacts. In our opinions, a good optimizer should be elegant (at least understandable) and generalizable. If there is no persuasive application reported for it, we will not consider any**very complex**optimizer in this library, in order to aovid the possible repeatability and overfitting issues.

Note

*“If there is a single dominant theme in this …, it is that practical methods of numerical computation can be
simultaneously efficient, clever, and –important– clear.”*—Press, W.H., Teukolsky, S.A., Vetterling, W.T. and
Flannery, B.P., 2007. Numerical recipes: The art of scientific computing. Cambridge University Press.

Respect for

**Diversity**Given the universality of

**black-box optimization (BBO)**in science and engineering, different research communities have designed different optimizers/methods. The type and number of optimizers are continuing to increase as the more powerful optimizers are always desirable for new and more challenging applications. On the one hand, some of these methods may share*more or less*similarities. On the other hand, they may also show*significant*differences (w.r.t. motivations / objectives / implementations / communities / practitioners). Therefore, we hope to cover such a diversity from different research communities such as artificial intelligence (particularly machine learning (evolutionary computation and zeroth-order optimization)), mathematical optimization/programming (particularly global optimization), operations research / management science, automatic control, electronic engineering, open-source software, physics, chemistry, and others.Respect for

**Originality**For each optimizer included in

*PyPop7*, we expect to give its original/representative reference (sometimes also including its good implementations/improvements). If you find some important references missed, please do NOT hesitate to contact us (and we will be happy to add it if necessary).

Note

*“It is both enjoyable and educational to hear the ideas directly from the creators”.*—Hennessy, J.L. and Patterson,
D.A., 2019. Computer architecture: A quantitative approach (Sixth Edition). Elsevier.

Respect for

**Repeatability**For randomized search, properly controlling randomness is very crucial to repeat numerical experiments. Here we follow the Random Sampling suggestions from NumPy. In other worlds, you must

**explicitly**set the random seed for each optimizer. For more discussions about**repeatability**from machine learning, evolutionary computation, and metaheuristics communities, refer to the following papers, to name a few:Swan, J., Adriaensen, S., Brownlee, A.E., Hammond, K., Johnson, C.G., Kheiri, A., Krawiec, F., Merelo, J.J., Minku, L.L., Özcan, E., Pappa, G.L., et al., 2022. Metaheuristics “in the large”. European Journal of Operational Research, 297(2), pp.393-406.

López-Ibáñez, M., Branke, J. and Paquete, L., 2021. Reproducibility in evolutionary computation. ACM Transactions on Evolutionary Learning and Optimization, 1(4), pp.1-21.

Sonnenburg, S., Braun, M.L., Ong, C.S., Bengio, S., Bottou, L., Holmes, G., LeCunn, Y., Muller, K.R., Pereira, F., Rasmussen, C.E., Ratsch, G., et al., 2007. The need for open source software in machine learning. Journal of Machine Learning Research, 8, pp.2443-2466.

We expect to see more discussions about the **Beauty of Black-Box Optimizers (BBO)**!