Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!oitnews.harvard.edu!news.dfci.harvard.edu!narnia.ccs.neu.edu!chaos.dac.neu.edu!usenet.eel.ufl.edu!news.mathworks.com!udel!gatech!howland.reston.ans.net!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: Multiplication with NN
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <D4vpK4.9qx@unx.sas.com>
Date: Fri, 3 Mar 1995 19:16:52 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <3j4o4j$ilb@charm.magnus.acs.ohio-state.edu>
Organization: SAS Institute Inc.
Lines: 31


In article <3j4o4j$ilb@charm.magnus.acs.ohio-state.edu>, jhines@magnus.acs.ohio-state.edu (J W Hines) writes:
|> ...
|> I'm really trying to get to the problem of estimating or modeling logical
|> AND and OR gates with continuously values inputs.  Not the binary input
|> gates that a MLP can model well, but a probability type of gate:
|>
|> P(c)=p(a*P(b)  or   p(c)=p(a)+p(b)-p(a)p(b)
|>
|> both of these have multiplicationsa in them, and ANNs sm to have difficulties
|> learning or at least generalizing multiplication over a large range such as
|> 1e-6 to 1-3 to 1.   These are the ranges that may be useful in a fault tree.

The first thing you need to consider is the training/error/estimation
function. Presumably, the importance of an error is relative to the size
of the correct answer; e.g., an error of .1 would be very serious if you
were multiplying 1e-6 * 1e-6 but would be minor if you were multiplying
1e6 * 1e6. Hence you do not want to use the sum of these squared errors
as the training criterion. You can either use weighted least squares
(divide the error by the correct answer before you square it) or use a
logarithmic transformation on the target values. Multiplying by zero
requires special treatment and is easier to handle by weighted least
squares--divide the error by max(abs(correct answer), epsilon) where
epsilon is some small number.


-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
