FOREWORD by Tom Mitchell ix

PREFACE xi

1 INTRODUCTION 1

1.1 Motivation 1
1.2 Lifelong Learning 3
1.3 A Simple Complexity Consideration 8
1.4 The EBNN Approach to Lifelong Learning 13
1.5 Overview 16

2 EXPLANATION-BASED NEURAL NETWORK LEARNING 19

2.1 Inductive Neural Network Learning 20
2.2 Analytical Learning 27
2.3 Why Integrate Induction and Analysis? 31
2.4 The EBNN Learning Algorithm 33
2.5 A Simple Example 39
2.6 The Relation of Neural and Symbolic Explanation-Based Learning 43
2.7 Other Approaches that Combine Induction and Analysis 45
2.8 EBNN and Lifelong Learning 47

3 THE INVARIANCE APPROACH 49

3.1 Introduction 49
3.2 Lifelong Supervised Learning 50
3.3 The Invariance Approach 55
3.4 Example: Learning to Recognize Objects 59
3.5 Alternative Methods 74
3.6 Remarks 90

4 REINFORCEMENT LEARNING 93

4.1 Learning Control 94
4.2 Lifelong Control Learning 98
4.3 Q-Learning 102
4.4 Generalizing Function Approximators and Q-Learning 111
4.5 Remarks 125

5 EMPIRICAL RESULTS 131

5.1 Learning Robot Control 132
5.3 Simulation 141
5.4 Approaching and Grasping a Cup 146
5.5 NeuroChess 152
5.6 Remarks 175

6 DISCUSSION 177

6.1 Summary 177
6.2 Open Problems 181
6.3 Related Work 185
6.4 Concluding Remarks 192

A AN ALGORITHM FOR APPROXIMATING VALUES AND SLOPES WITH ARTIFICIAL NEURAL NETWORKS 195

A.1 Definitions 196
A.2 Network Forward Propagation 196
A.3 Forward Propagation of Auxiliary Gradients 197
A.4 Error Functions 198
A.5 Minimizing the Value Error 199
A.6 Minimizing the Slope Error 199
A.7 The Squashing Function and its Derivatives 201
A.8 Updating the Network Weights and Biases 202

B PROOFS OF THE THEOREMS 203

C EXAMPLE CHESS GAMES 207

C.1 Game 1 207
C.2 Game 2 219

REFERENCES 227

LIST OF SYMBOLS 253

INDEX 259