EDU.gatech.cc.is.learning
Class i_SLearner_id

java.lang.Object
  |
  +--EDU.gatech.cc.is.learning.i_ReinforcementLearner_id
        |
        +--EDU.gatech.cc.is.learning.i_SLearner_id

public class i_SLearner_id

extends i_ReinforcementLearner_id

implements java.lang.Cloneable, java.io.Serializable

An object that learns to select from several actions based on a reward. Uses the S-learning method as defined by Mataric. "S" is for "single-step."

The module will learn to select a discrete output based on state and a continuous reinforcement input. The "i"s in front of and behind the name imply that this class takes integers as input and output. The "d" indicates a double for the reinforcement input (i.e. a continuous value).

Copyright (c)2000 Tucker Balch

See Also:

Serialized Form

Fields inherited from class EDU.gatech.cc.is.learning.i_ReinforcementLearner_id

logging, numactions, numstates, policyfilename

Constructor Summary

i_SLearner_id(int numstatesin, int numactionsin)
Instantiate an S learner using default parameters.

i_SLearner_id(int numstatesin, int numactionsin, int criteriain)
Instantiate an S learner.

Method Summary

void	endTrial(double Vn, double rn) Called when the current trial ends.
double	getAvgReward() Report the average reward per step in the trial.
int	getPolicyChanges() Report the number of policy changes in the trial.
int	getQueries() Report the number of queries in the trial.
int	initTrial(int s) Called to initialize for a new trial.
int	query(int yn, double rn) Select an output based on the state and reward.
void	readPolicy() Read the policy from a file.
void	savePolicy() Write the policy to a file.
void	saveProfile(java.lang.String profile_filename) Write the policy profile to a file.

Methods inherited from class EDU.gatech.cc.is.learning.i_ReinforcementLearner_id

log, loggingOff, loggingOn, loggingOn, setPolicyFileName, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

i_SLearner_id

public i_SLearner_id(int numstatesin,
                     int numactionsin,
                     int criteriain)

Instantiate an S learner.

Parameters:

numstates - int, the number of states the system could be in.

numactions - int, the number of actions or outputs to select from.

criteria - int, IGNORED, but retained for compatibility.

i_SLearner_id

public i_SLearner_id(int numstatesin,
                     int numactionsin)

Instantiate an S learner using default parameters.

Parameters:

numstates - int, the number of states the system could be in.

numactions - int, the number of actions or outputs to select from.

Method Detail

query

public int query(int yn,
                 double rn)

Select an output based on the state and reward.

Overrides:

query in class i_ReinforcementLearner_id

Parameters:

statein - int, the current state.

rewardin - double, reward for the last output, positive numbers are "good."

endTrial

public void endTrial(double Vn,
                     double rn)

Called when the current trial ends.

Overrides:

endTrial in class i_ReinforcementLearner_id

Parameters:

Vn - double, the value of the absorbing state.

reward - double, the reward for the last output.

initTrial

public int initTrial(int s)

Called to initialize for a new trial.

Overrides:

initTrial in class i_ReinforcementLearner_id

Tags copied from class: i_ReinforcementLearner_id

Parameters:

statein - int, the current state.

getAvgReward

public double getAvgReward()

Report the average reward per step in the trial.

Overrides:

getAvgReward in class i_ReinforcementLearner_id

Returns:

the average.

getQueries

public int getQueries()

Report the number of queries in the trial.

Overrides:

getQueries in class i_ReinforcementLearner_id

Returns:

the total.

getPolicyChanges

public int getPolicyChanges()

Report the number of policy changes in the trial.

Overrides:

getPolicyChanges in class i_ReinforcementLearner_id

Returns:

the total.

saveProfile

public void saveProfile(java.lang.String profile_filename)
                 throws java.io.IOException

Write the policy profile to a file.

Parameters:

filename - String, the name of the file to write to.

readPolicy

public void readPolicy()

Description copied from class: i_ReinforcementLearner_id

Read the policy from a file. Use setPolicyFileName() to determine where the policy is saved.

Overrides:

readPolicy in class i_ReinforcementLearner_id

Tags copied from class: i_ReinforcementLearner_id

Parameters:

filename - String, the name of the file to read from.

savePolicy

public void savePolicy()

Description copied from class: i_ReinforcementLearner_id

Write the policy to a file. Use setPolicyFileName() to determine where the policy is read from.

Overrides:

savePolicy in class i_ReinforcementLearner_id

Tags copied from class: i_ReinforcementLearner_id

Parameters:

filename - String, the name of the file to write to.