EDU.gatech.cc.is.learning
Class i_SLearner_id

java.lang.Object
  |
  +--EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
        |
        +--EDU.gatech.cc.is.learning.i_SLearner_id

public class i_SLearner_id
extends i_ReinforcementLearner_id
implements java.lang.Cloneable, java.io.Serializable

An object that learns to select from several actions based on a reward. Uses the S-learning method as defined by Mataric. "S" is for "single-step."

The module will learn to select a discrete output based on state and a continuous reinforcement input. The "i"s in front of and behind the name imply that this class takes integers as input and output. The "d" indicates a double for the reinforcement input (i.e. a continuous value).

Copyright (c)2000 Tucker Balch

See Also:
Serialized Form

Fields inherited from class EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
logging, numactions, numstates, policyfilename
 
Constructor Summary
i_SLearner_id(int numstatesin, int numactionsin)
          Instantiate an S learner using default parameters.
i_SLearner_id(int numstatesin, int numactionsin, int criteriain)
          Instantiate an S learner.
 
Method Summary
 void endTrial(double Vn, double rn)
          Called when the current trial ends.
 double getAvgReward()
          Report the average reward per step in the trial.
 int getPolicyChanges()
          Report the number of policy changes in the trial.
 int getQueries()
          Report the number of queries in the trial.
 int initTrial(int s)
          Called to initialize for a new trial.
 int query(int yn, double rn)
          Select an output based on the state and reward.
 void readPolicy()
          Read the policy from a file.
 void savePolicy()
          Write the policy to a file.
 void saveProfile(java.lang.String profile_filename)
          Write the policy profile to a file.
 
Methods inherited from class EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
log, loggingOff, loggingOn, loggingOn, setPolicyFileName, toString
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

i_SLearner_id

public i_SLearner_id(int numstatesin,
                     int numactionsin,
                     int criteriain)
Instantiate an S learner.
Parameters:
numstates - int, the number of states the system could be in.
numactions - int, the number of actions or outputs to select from.
criteria - int, IGNORED, but retained for compatibility.

i_SLearner_id

public i_SLearner_id(int numstatesin,
                     int numactionsin)
Instantiate an S learner using default parameters.
Parameters:
numstates - int, the number of states the system could be in.
numactions - int, the number of actions or outputs to select from.
Method Detail

query

public int query(int yn,
                 double rn)
Select an output based on the state and reward.
Overrides:
query in class i_ReinforcementLearner_id
Parameters:
statein - int, the current state.
rewardin - double, reward for the last output, positive numbers are "good."

endTrial

public void endTrial(double Vn,
                     double rn)
Called when the current trial ends.
Overrides:
endTrial in class i_ReinforcementLearner_id
Parameters:
Vn - double, the value of the absorbing state.
reward - double, the reward for the last output.

initTrial

public int initTrial(int s)
Called to initialize for a new trial.
Overrides:
initTrial in class i_ReinforcementLearner_id
Tags copied from class: i_ReinforcementLearner_id
Parameters:
statein - int, the current state.

getAvgReward

public double getAvgReward()
Report the average reward per step in the trial.
Overrides:
getAvgReward in class i_ReinforcementLearner_id
Returns:
the average.

getQueries

public int getQueries()
Report the number of queries in the trial.
Overrides:
getQueries in class i_ReinforcementLearner_id
Returns:
the total.

getPolicyChanges

public int getPolicyChanges()
Report the number of policy changes in the trial.
Overrides:
getPolicyChanges in class i_ReinforcementLearner_id
Returns:
the total.

saveProfile

public void saveProfile(java.lang.String profile_filename)
                 throws java.io.IOException
Write the policy profile to a file.
Parameters:
filename - String, the name of the file to write to.

readPolicy

public void readPolicy()
Description copied from class: i_ReinforcementLearner_id
Read the policy from a file. Use setPolicyFileName() to determine where the policy is saved.
Overrides:
readPolicy in class i_ReinforcementLearner_id
Tags copied from class: i_ReinforcementLearner_id
Parameters:
filename - String, the name of the file to read from.

savePolicy

public void savePolicy()
Description copied from class: i_ReinforcementLearner_id
Write the policy to a file. Use setPolicyFileName() to determine where the policy is read from.
Overrides:
savePolicy in class i_ReinforcementLearner_id
Tags copied from class: i_ReinforcementLearner_id
Parameters:
filename - String, the name of the file to write to.