This paper introduces a new representation for learning complex programs. This new representation, Neural Programming (NP) has been developed with the goal of incorporating positive aspects of both artificial neural networks and genetic programming. Neural Programming is a connectionist programming language that has been designed to make internal reinforcement, hither-to unaccomplished in genetic programming, possible.
Both GP [6] and ANNs [9] continue to be well investigated fields. In ANNs, the focus on improving the power of the technique has not been on changing what is inside an ``artificial neuron.'' Works like [4, 10] have, however, investigated the possible additional benefit of complicating and un-homogenizing artificial neurons. To the best of our knowledge, in the context of ANNs and principled update policies like backpropagation, these investigations have not yet extended to arbitrary, potentially non-differentiable functions like those typically used by human programmers and by evolving GP programs.
In GP, the focus of investigation for increased power of the technique has not been on changing the GP representation or on finding principled (non-random) update policies. Some work, has, however, been done in these areas. [8] describes a process for trying to find sub-functions in an evolving GP function that are more likely than randomly selected ones to contribute positively to fitness when crossed-over into other programs. [3] and [5] describe possible approaches for allowing the mechanism of evolution to provide self-adaptation all the way down to the single node level. For example, in a finite state machine or a GP program, each node can express an evolved preference for interaction with the update strategies like mutation and crossover.
In brief, ANN is a successful representative of the machine learning practice of explicit credit-assignment. GP is a successful representative of empirical credit-assignment [2]. Empirical credit-assignment is the machine learning practice of allowing the dynamics of the system to implicitly determine credit and blame. Evolution does just this [1]. The goal of this work is to begin to bridge this credit-assignment gap by finding ways in which explicit and empirical credit-assignment can find mutual benefit in a single machine learning technique.
In general, it is not possible to give an ANN an input for every possible parameterization of each user defined primitive function a GP program can be given. And it is far from obvious how to work complex functions into the middle of an otherwise homogeneous network of simple non-linear functions. Yet, gradient-descent learning procedures, like backpropagation in ANNs, are an extremely powerful idea. Backpropagation is not only a kind of performance guarantee, it is a kind of performance explanation. It is to achieve this kind of dual benefit that the research this paper reports was undertaken.
We show in this paper how we can accumulate explicit credit-assignment information in Neural Programming. These values will be collectively referred to as the Credit-Blame Map. A representation that would facilitate a credit-blame map would accomplish two things. First, by looking over a program's credit-blame map, humans could, with considerably less work, sort out what a particular successful program was doing and thereby increase the interpretability of GP programs. Second, by organizing the GP programs in a network of nodes but by replacing program flow of control with flow of data, we can use the credit-blame map to propagate punishment and reward through the program, getting the benefits from both the GP and ANN worlds. In short, it would be desirable to be able, in GP, to have reinforcement of the programs be more specific (directed towards particular parts or aspects of a program) and more appropriate (telling the system how to change those specific parts). ANNs, on the other hand, might benefit greatly from the use of the kind of program primitives that are natural to GP and often found to be advantageous to the learning process.
This paper will do three things. First, we will contribute the Neural Programming representation as a new, connectionist, representation in which to evolve programs. Second, this paper will describe how this representation can be used to deliver explicit, useful, internal reinforcement to the evolving programs. And third, an experiment on a illustrative signal classification problem will be used to demonstrate the promise of both the representation and its associated internal reinforcement strategy.