 |
||||
| Decision and Reinforcement Learning |
Racing algorithms and Schemata Search
We have developed "racing" algorithms that take a kind of bounded rationality approach to evaluating many models in parallel (in randomized order) and use blocked comparisons to, in the early stages, prune away unpromising models that, with very high confidence, will not eventually evaluate as the best. We extended, with an algorithm called Schemata Search to the problem of searching over binary sets of features for best feature sets.
Blackbox Noisy Optimization: Autonomous Response Surface Methods
The apparently humble task of parameter tweaking for noisy systems is of great importance whether the parameters being tweaked are for an algorithm, a robot, a real manufacturing process, a simulation, or a scientific experiment.
We're very excited about this. Our reasons for excitement are two-fold. First, we want to be loud proponents of the importance of machine learning as an as-yet under-exploited tool for efficient blackbox optimization of noisy continuous functions. It is a domain in which the previous state of the art (stochastic approximation, response surface methods, numerical optimization, and evolutionary computation) all fail to meet at least one of the criteria of
The second reason for our enthusiasm has been the completion of our first release of a pragmatic blackbox noisy optimizer christened Q2. We believe that it meets all the above criteria. We can show that Q2 retains the desirable property of second order convergence of Newton’s method in benign noiseless cases, yet avoids instability. It makes statistical inferences about where to gather data for noisy and non-quadratic functions. We extensively compared Q2 with its closest rivals in Monte Carlo tests, demonstrating empirical improvements on every highly noisy task we examined, and on all but one low-noise task. The algorithms resulting from this research have been embedded in a commercial software system, and are currently being evaluated in collaborations with partners. In proposed future work plan to also actively learn protein identification for HPLCs, and the "science" of optimizing robot parameters.
In ongoing work, we are investigating how machine learning and data mining methods can help in the global optimization of functions of very many real variables.
|
|
||||
| Decision and Reinforcement Learning |