Hooke-Jeeves (HJ)

class pypop7.optimizers.ds.hj.HJ(problem, options)

Hooke-Jeeves direct (pattern) search method (HJ).


HJ is one of the most-popular and most-cited DS methods, originally published in one top-tier Computer Science journal (i.e., JACM) in 1961. Although sometimes it is still used to optimize low-dimensional black-box problems, it is highly recommended to attempt other more advanced methods for large-scale black-box optimization.

  • problem (dict) –

    problem arguments with the following common settings (keys):
    • ’fitness_function’ - objective function to be minimized (func),

    • ’ndim_problem’ - number of dimensionality (int),

    • ’upper_boundary’ - upper boundary of search range (array_like),

    • ’lower_boundary’ - lower boundary of search range (array_like).

  • options (dict) –

    optimizer options with the following common settings (keys):
    • ’max_function_evaluations’ - maximum of function evaluations (int, default: np.Inf),

    • ’max_runtime’ - maximal runtime to be allowed (float, default: np.Inf),

    • ’seed_rng’ - seed for random number generation needed to be explicitly set (int);

    and with the following particular settings (keys):
    • ’sigma’ - initial global step-size (float, default: 1.0),

    • ’x’ - initial (starting) point (array_like),

      • if not given, it will draw a random sample from the uniform distribution whose search range is bounded by problem[‘lower_boundary’] and problem[‘upper_boundary’].

    • ’gamma’ - decreasing factor of global step-size (float, default: 0.5).


Use the optimizer to minimize the well-known test function Rosenbrock:

 1>>> import numpy
 2>>> from pypop7.benchmarks.base_functions import rosenbrock  # function to be minimized
 3>>> from pypop7.optimizers.ds.hj import HJ
 4>>> problem = {'fitness_function': rosenbrock,  # define problem arguments
 5...            'ndim_problem': 2,
 6...            'lower_boundary': -5*numpy.ones((2,)),
 7...            'upper_boundary': 5*numpy.ones((2,))}
 8>>> options = {'max_function_evaluations': 5000,  # set optimizer options
 9...            'seed_rng': 2022,
10...            'x': 3*numpy.ones((2,)),
11...            'sigma': 0.1,  # the global step-size may need to be tuned for better performance
12...            'verbose_frequency': 500}
13>>> hj = HJ(problem, options)  # initialize the optimizer class
14>>> results = hj.optimize()  # run the optimization process
15>>> # return the number of function evaluations and best-so-far fitness
16>>> print(f"HJ: {results['n_function_evaluations']}, {results['best_so_far_y']}")
17HJ: 5000, 0.22119484961034389

For its correctness checking of coding, refer to this code-based repeatability report for more details.


decreasing factor of global step-size.




final global step-size (changed during optimization).




initial (starting) point.




Kochenderfer, M.J. and Wheeler, T.A., 2019. Algorithms for optimization. MIT Press. https://algorithmsbook.com/optimization/files/chapter-7.pdf (See Algorithm 7.5 (Page 104) for details.)


Kaupe Jr, A.F., 1963. Algorithm 178: Direct search. Communications of the ACM, 6(6), pp.313-314. https://dl.acm.org/doi/pdf/10.1145/366604.366632

Hooke, R. and Jeeves, T.A., 1961. “Direct search” solution of numerical and statistical problems. Journal of the ACM, 8(2), pp.212-229. https://dl.acm.org/doi/10.1145/321062.321069