Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!udel!gatech!concert!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: Q: Same Inputs Different Outputs ?
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <Cz3AGE.CEo@unx.sas.com>
Date: Fri, 11 Nov 1994 06:09:50 GMT
References:  <39u70s$p6i@jeeves.niehs.nih.gov>
Nntp-Posting-Host: hotellng.unx.sas.com
Organization: SAS Institute Inc.
Lines: 32


In article <39u70s$p6i@jeeves.niehs.nih.gov>, davids@jester.niehs.nih.gov (Dr. David V. Schreibman) writes:
|> What do you do when you have several training pairs with identical input,
|> but different output?

This is not a problem at all. If you think of the network as a
regression model, it should be apparent that it is fine to have
replicated cases, and this can even be useful for estimating the
error/noise variance, which can in turn be used in various model
selection criteria such as Cp and PSE.

If you are doing least-squares estimation, you can speed up training by
combining each set of replicates into a single case where the target
value is the mean of the target values in the replicates and the case
is given a weight equal to the number of replicates combined.

|> To accommodate a training set where multiple subsets of identical input patterns
|> that have different outputs, what I would like to do is incorporate the
|> variance of the output patterns into the weight delta equation in
|> backpropagation.

This sort of thing would only be useful if you wanted to fit a
heteroscedastic model, i.e. one in which the error/noise variance is not
assumed to be constant. However, this is a very tricky undertaking and
should not be attempted without the assistance of a statistician
well-versed in such methodology.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
