Many pressing real world problems can be stated as problems of global optimization, where target function is a black box. Such problems are best approached with a library of optimization methods to help study the nature of the problem. We show how to use Scipy.optimize and Scikit-learn modules to create global optimization methods with desired properties.
Many pressing real world problems can be stated as problems of global optimization, where target function is given by values only (a black box function). They range from robot parameter tuning and optimization of chemical processes to biological systems analysis. Typical examples from financial engineering include model calibration, pricing, hedging, VaR/CVaR computation, credit rating assignment, forecasting, strategy optimization in electronic trading systems. The choice of the algorithm for decision making should depend on the type of the objective function and availability of a priori information. Such problems are best approached with a library of optimization methods to help study the nature of the problem. Several such libraries are readily available, such as Matlab Global Optimization Toolbox, that contains a minimal set of modules allowing to create a universe of optimization methods by making hybrids. We suggest to use use Python scipy and scikit learn. By varying parameters of the standard optimization procedures and making hybrids it is possible to obtain the whole universe of methods. For example, Nelder Mead algorithm (represented by scipy.optimize.fmin() function) needs a frequent restart and can be efficiently combined with random search for solving problems of global optimization. Many well known standard tools such as principal component analysis can be rewritten in a incremental (or online) form using machine learning techniques, where new data is incorporated at each iteration, making such algorithms suitable for big data handling.