EDU.gatech.cc.is.learning
Class i_ReinforcementLearner_id

java.lang.Object
  |
  +--EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
Direct Known Subclasses:
i_AverageLearner_id, i_PriQLearner_id, i_QLearner_id, i_SLearner_id

public abstract class i_ReinforcementLearner_id
extends java.lang.Object
implements java.lang.Cloneable, java.io.Serializable

Reinforcement learning class with discrete inputs and outputs. The module will learn to select a discrete output based on state and a continuous reinforcement input. The "i"s in front of and behind the name imply that this class takes integers as input and output. The "d" indicates a double for the reinforcement input (i.e. a continuous value). No committment is made to model-free or model-based learning, the class may be extended to either.

Copyright (c)2000 Tucker Balch

See Also:
Serialized Form

Field Summary
protected  boolean logging
          Indicates whether logging is turned on or not.
protected  int numactions
          The number of actions to select from.
protected  int numstates
          The number of situations or states the object may be in.
protected  java.lang.String policyfilename
          The name of the file to save or read the policy in/from.
 
Constructor Summary
i_ReinforcementLearner_id(int numstatesin, int numactionsin)
          Instantiates a reinforcement learner using default parameters.
 
Method Summary
abstract  void endTrial(double V, double reward)
          Called when the current trial ends.
 double getAvgReward()
          return statistical info about the learner.
 int getPolicyChanges()
          return statistical info about the learner.
 int getQueries()
          return statistical info about the learner.
abstract  int initTrial(int statein)
          Called to initialize for a new trial and get the first action.
 void log(java.lang.String out)
          Print something to the logfile.
 void loggingOff()
          Turn logging off.
 void loggingOn()
          Turn logging on, but without opening a file.
 void loggingOn(java.lang.String filename)
          Turn on logging for data gathering.
abstract  int query(int statein, double rewardin)
          Select an output based on the state and reward.
abstract  void readPolicy()
          Read the policy from a file.
abstract  void savePolicy()
          Write the policy to a file.
 void setPolicyFileName(java.lang.String filename)
          Set the filename for policy reading and writing.
 java.lang.String toString()
          Generate a String that describes the learner.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

logging

protected boolean logging
Indicates whether logging is turned on or not.

numstates

protected int numstates
The number of situations or states the object may be in.

numactions

protected int numactions
The number of actions to select from.

policyfilename

protected java.lang.String policyfilename
The name of the file to save or read the policy in/from.
Constructor Detail

i_ReinforcementLearner_id

public i_ReinforcementLearner_id(int numstatesin,
                                 int numactionsin)
Instantiates a reinforcement learner using default parameters. Individual implementations may allow additional parameters to be adjusted using accessor methods.
Parameters:
numstates - int, the number of states the system could be in.
numactions - int, the number of actions or outputs to select from.
Method Detail

toString

public java.lang.String toString()
Generate a String that describes the learner.
Overrides:
toString in class java.lang.Object
Returns:
a String describing the learner.

query

public abstract int query(int statein,
                          double rewardin)
Select an output based on the state and reward.
Parameters:
statein - int, the current state.
rewardin - double, reward for the last output, positive numbers are "good."

endTrial

public abstract void endTrial(double V,
                              double reward)
Called when the current trial ends. This is used for tasks that have "absorbing states."
Parameters:
V - double, the value of the absorbing state. Usually a large positive number if the final state is desirable, or negative otherwise.
reward - double, the single-step reward for the last state/action.

initTrial

public abstract int initTrial(int statein)
Called to initialize for a new trial and get the first action.
Parameters:
statein - int, the current state.

getAvgReward

public double getAvgReward()
return statistical info about the learner. Logging must be on for this to work.
Returns:
the average reward per query for the trial.

getPolicyChanges

public int getPolicyChanges()
return statistical info about the learner. Logging must be on for this to work.
Returns:
the number of policy changes for the trial.

getQueries

public int getQueries()
return statistical info about the learner. Logging must be on for this to work.
Returns:
the number of queries for the trial.

setPolicyFileName

public void setPolicyFileName(java.lang.String filename)
Set the filename for policy reading and writing.
Parameters:
filename - String, the directory and filename to use.

loggingOn

public void loggingOn(java.lang.String filename)
Turn on logging for data gathering. It is up to the extended classes to actually do the writing to the file.
Parameters:
filename - String, the directory and filename to use.

log

public void log(java.lang.String out)
Print something to the logfile. Called by the classes that extend this one.
Parameters:
out - String, the string to print.

loggingOff

public void loggingOff()
Turn logging off.

loggingOn

public void loggingOn()
Turn logging on, but without opening a file. This indicates we should keep track of some items, but not save them to a file.

readPolicy

public abstract void readPolicy()
                         throws java.io.IOException
Read the policy from a file. Use setPolicyFileName() to determine where the policy is saved.
Parameters:
filename - String, the name of the file to read from.

savePolicy

public abstract void savePolicy()
                         throws java.io.IOException
Write the policy to a file. Use setPolicyFileName() to determine where the policy is read from.
Parameters:
filename - String, the name of the file to write to.