EDU.gatech.cc.is.learning
Class i_AverageLearner_id

java.lang.Object
  |
  +--EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
        |
        +--EDU.gatech.cc.is.learning.i_AverageLearner_id

public class i_AverageLearner_id
extends i_ReinforcementLearner_id
implements java.lang.Cloneable, java.io.Serializable

An object that learns to select from several actions based on a reward. Uses the Q-learning method but assuming average rewards.

The module will learn to select a discrete output based on state and a continuous reinforcement input. The "i"s in front of and behind the name imply that this class takes integers as input and output. The "d" indicates a double for the reinforcement input (i.e. a continuous value).

Copyright (c)2000 Tucker Balch

See Also:
Serialized Form

Fields inherited from class EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
logging, numactions, numstates, policyfilename
 
Constructor Summary
i_AverageLearner_id(int numstatesin, int numactionsin)
          Instantiate a Q learner using default parameters.
 
Method Summary
 void endTrial(double Vn, double rn)
          Called when the current trial ends.
 int initTrial(int s)
          Called to initialize for a new trial.
 int query(int yn, double rn)
          Select an output based on the state and reward.
 void readPolicy()
          Read the policy from a file.
 void savePolicy()
          Write the policy to a file.
 void setRandomRate(double r)
          Set the random rate for the Average-learner.
 void setRandomRateDecay(double r)
          Set the random decay for the Average-learner.
 java.lang.String toString()
          Generate a String that describes the current state of the learner.
 
Methods inherited from class EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
getAvgReward, getPolicyChanges, getQueries, log, loggingOff, loggingOn, loggingOn, setPolicyFileName
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

i_AverageLearner_id

public i_AverageLearner_id(int numstatesin,
                           int numactionsin)
Instantiate a Q learner using default parameters. Parameters may be adjusted using accessor methods.
Parameters:
numstates - int, the number of states the system could be in.
numactions - int, the number of actions or outputs to select from.
Method Detail

setRandomRate

public void setRandomRate(double r)
Set the random rate for the Average-learner. This reflects how frequently it picks a random action. Should be between 0 and 1.
Parameters:
r - double, the new value for random rate (0 < r < 1).

setRandomRateDecay

public void setRandomRateDecay(double r)
Set the random decay for the Average-learner. This reflects how quickly the rate of chosing random actions decays. 1 would never decay, 0 would cause it to immediately quit chosing random values. Should be between 0 and 1.
Parameters:
r - double, the new value for randomdecay (0 < r < 1).

toString

public java.lang.String toString()
Generate a String that describes the current state of the learner.
Overrides:
toString in class i_ReinforcementLearner_id
Returns:
a String describing the learner.

query

public int query(int yn,
                 double rn)
Select an output based on the state and reward.
Overrides:
query in class i_ReinforcementLearner_id
Parameters:
statein - int, the current state.
rewardin - double, reward for the last output, positive numbers are "good."

endTrial

public void endTrial(double Vn,
                     double rn)
Called when the current trial ends.
Overrides:
endTrial in class i_ReinforcementLearner_id
Parameters:
Vn - double, the value of the absorbing state.
reward - double, the reward for the last output.

initTrial

public int initTrial(int s)
Called to initialize for a new trial.
Overrides:
initTrial in class i_ReinforcementLearner_id
Tags copied from class: i_ReinforcementLearner_id
Parameters:
statein - int, the current state.

readPolicy

public void readPolicy()
                throws java.io.IOException
Read the policy from a file.
Overrides:
readPolicy in class i_ReinforcementLearner_id
Parameters:
filename - String, the name of the file to read from.

savePolicy

public void savePolicy()
                throws java.io.IOException
Write the policy to a file.
Overrides:
savePolicy in class i_ReinforcementLearner_id
Parameters:
filename - String, the name of the file to write to.