This is sort of a hail mary pass.. I've been trying to fix this for 2 days now with no success and I'm about to start all over again if i cant find an answer..

Basically im implementing a feed-forward neural network and using back propagation to train it. As far as i can tell, the feed-forward part of things is working fine, but the training algorithm is not. I've uploaded the code.. its setup to try and train for OR at the moment (0,0 = 0, 0,1 / 1,0 / 1,1 = 1) and if you run it you can see its not working properly... the outputs after training are completely whacky. I havent implemented an RMS calculator yet - so training simply works on iterations.. RMS will probably be the next and last thing i try before deleting and starting again.. i'm running out of time

code:

http://users.bigpond.net.au/Strik3r/nn/network.h // header file w/ classes etc.

http://users.bigpond.net.au/Strik3r/nn/network.cpp // implementation of class fuctions etc.

http://users.bigpond.net.au/Strik3r/nn/main.cpp // main function.

if anyone can fix this i will love you for ever :o

Hi, I do C++ all day at work, and did a bit of neural net stuff at uni but am very rusty. I had a look at your code but couldn't quite understand your function for calculating the error, I think this may be where the problem lies.

I adjusted the calculate error function like so:

void Network::calculateError(int layer, double * desiredOut)

{

	register int x,y;

	double enet;

	if(layer == numLayers) // Output Units

	{

		for(x=0;x<<layers[numLayers].numNeurons;x++)

		{

			layers[layer].neurons[x].error = (desiredOut[x] - layers[layer].neurons[x].output);						

		}

	}

	else

	{

		for(x=0;x<layers[layer].numNeurons;x++) // Hidden Units

		{

			enet=0;

			for(y=0;y<layers[layer+1].numNeurons;y++)

			{

				enet+=(layers[layer+1].neurons[y].error * links[layer][x][y].weight);

			}

			

			layers[layer].neurons[x].error = enet;

		}

	}

}

(hope that works)

I also adjusted the parameters for your neural net. I increased the number of nodes in the hidden layer to 500, and ran the training with the following parameters:

nn.trainBP(t,2000,0.1,0.5);

This seems to yield fairly good results, such as:

Output for 0,0: 8.49413e-014
Output for 1,0: 0.886431
Output for 0,1: 0.883769
Output for 1,1: 0.92118

With a bit more tweaking you could probably improve the results a bit more. Hope this helps...


last edited by habib at 00:45:04 05/May/06

last edited by habib at 00:45:52 05/May/06

last edited by habib at 00:46:16 05/May/06

Also, you've got a memory leak in the main() function. You 'new'ed the input and output arrays when it wasn't really necessary, and then forgot to clean them up with a 'delete' after. Also in future you may want to consider using classes like std::vector<> instead of managing dynamically allocated arrays yourself as it makes memory management easier as well as having a few nifty other features

If I could understand half the stuff you just said I'd help ya.

Neural nets are for suckers. C4.5 learning algorithm and AI for ever!

skynet will f*** your s*** up, obes.

hey habib..

firstly thanks a tonne for looking through it.. unfortunately, i dont think that you've quite found the problem.. i inserted your code and tried to train for XOR with no success.. so it seems that it was just lucky that the way the code works it happens to spit out semi correct values for OR.

firstly, to calculate error, the formula is (derivative of acivation function) * error... Im using a sigmoidal activation function f(x) = 1-1+exp(-x) and the derivative of that is f(x) * (1-f(x))..

soo basically, the code i was using to calculater the error was correct ->

layers[layer].neurons[x].output * // this is the output of neuron (ie: f(x)
(1-layers[layer].neurons[x].output) * // 1-output
(desiredOut[x] - layers[layer].neurons[x].output); // calculated error


thats last line is slightly different for hidden nodes as the 'calculated error' is the error of all the nodes on the following layer multiplied by the link weight. leaving my error code intact and modifying the number of hidden units and training parameters i was able to produce the same results as you for OR but it doesnt seem to work for anything else.

Ive started from scratch... hopefully will do better this time around :P

thanks again