Data Mining

Download Free Data Mining Source Code In C/C++, C#, Visual Basic, Visual Basic.NET, Java,
and other programming languages
Welcome to Data Mining Sign in | Join | Help
in Search
Home Blogs Forums Photos

Data Mining Source Code Newsletter

Subscribe By Email | Subscribe By RSS Feed
Data Mining » Data Mining » Data Mining Source Code Downlo... » C# ARTIFICIAL NEURAL NETWORK MULTILAYER FEEDFORWARD BACKPROPAGATION ALGORITHM VERSION 1.1 - OPEN SOURCE CODE RELEASE

Announcing The Data Mining Source Code Newsletter!

Subscribe By Email | Subscribe By RSS Feed

C# ARTIFICIAL NEURAL NETWORK MULTILAYER FEEDFORWARD BACKPROPAGATION ALGORITHM VERSION 1.1 - OPEN SOURCE CODE RELEASE

Last post 10-25-2004, 19:48 by Kingsley Tagbo. 0 replies.
Sort Posts: Previous Next

  •  10-25-2004, 19:48 3069

    C# ARTIFICIAL NEURAL NETWORK MULTILAYER FEEDFORWARD BACKPROPAGATION ALGORITHM VERSION 1.1 - OPEN SOURCE CODE RELEASE

    Attachment: NeuralNetworksCSVB101.zip
    Locked

    Monday, October 25, 2004 07:10 AM
    PLEASE DOWNLOAD THE ATTACHED C# ARTIFICIAL NEURAL NETWORK MULTILAYER FEEDFORWARD BACKPROPAGATION ALGORITHM VERSION 1.1 - OPEN SOURCE CODE RELEASE

    DOCUMENTATION: http://www.kdkeys.net/ShowForum.aspx?ForumID=1022

    FEEDBACK: PLEASE POST YOUR FEEDBACK IN THE FORUMS:

    C# NEURAL NETWORK MULTILAYER FEEDFORWARD BACKPROPAGATION ALGORITHM VERSION 1.1 - OPEN SOURCE CODE RELEASE

    A Neural Network is a Network modeled after neurons in brain tissues and is a set of neurons in an input layer connected to one or more hidden layers of neurons which are in turn connected to an output layer of neurons.

    The connections between the cells in the input layer and the hidden layer and the hidden layer and output layer are associated with weights.

    For simplicity sake, each Neuron could be said to consist of an INPUT, TRAINING OUTPUT, TARGET OUTPUT, BIAS, ERROR, CONNECTING WEIGHT.

    INPUT LAYER

    Typically one or more neurons or cells exist in an input layer. The input layer could receive data normalized between 0 and 1 or -1 and 1. The input layer converts the data it receives to an OUTPUT using a Sigmoid Function or Squashing Function that generally maps any input to a value between 0 and 1 as an output.

    Each cell in the INPUT LAYER is connected to all the cells in the HIDDEN LAYER by weighted connections. For modeling or simplicity purposes, the bias and weights in the INPUT LAYER are set to zero.

    HIDDEN LAYER

    There can be one or more HIDDEN LAYERS in a FeedForward Neural Network, but for most practical applications, the number of hidden layers is one (1).

    Each cell in the HIDDEN LAYER is fully connected to each cell in the INPUT LAYER, next HIDDEN LAYER or OUTPUT LAYER and each of these connections are weighted.

    The number of cells in a HIDDEN LAYER can be adjusted during the training of a Neural Network to avoid overfitting. The weights and bias in the hidden layer are initialized to random values between -1.0 and 1.0 or -0.5 and 0.5. These initial values are used to compute the OUTPUT from each cell in the HIDDEN LAYER which will be fed to the next HIDDEN LAYER or OUTPUT LAYER.

    Assume that a cell in the hidden layer (C3) is connected to two cells in the input layer (C1 and C2)

    If the input to cell C1 is 0.4 and the input to cell C2 is 0.8, then the outputs from cell C1 and C2 respectively are 0.4 and 0.8

    Let's the connection between the cell in the hidden layer (C3) and the cell C1 in the input layer be represented as W31 and the connection between the cell C3 and C2 to be represented by W32

    Since W31, W32 and the hidden cell bias B3 are initialized randomly, assume that W31 = -0.25, W32 = 0.46 and B3 = -0.25

    The input to cell C3 = (OUTPUT from input cell C1 * W31) + (OUTPUT form cell C2 * W32) + (BIAS of cell C3)

    = (0.4 * - 0.25) + (0.8 * 0.46) - 0.25

    = 0.13

    Each cell both in the HIDDEN LAYER and OUTPUT LAYER calculates it's OUTPUT using a mathematical Sigmoid / Activation / Squashing / Logistic Function that is represented by OUTPUT O = (1.0 / (1.0 + System.Math.Exp(-val))) in Visual Basic .NET (VB.NET) or C#.

    Applying the Sigmoid Function or Logistic Function to 0.13 gives a value of 0.53245430638731872 as the OUTPUT of cell C3.

     

    OUTPUT LAYER

    Each cell in the OUTPUT LAYER is fully connected to each cell in the hidden layer and has a bias and weighted connections which are also initialized to random values.

    Each cell in the OUTPUT LAYER receive inputs from every cell in the last hidden layer. The INPUT and OUTPUT of cells in the OUTER LAYER are calculated like that of cells in the HIDDEN LAYER.

     

    FEED-FORWARD

    The Neural Network Algorithm that I have been describing is a Multi-Layer Feed-Forward Back-Propagation Neural Network Algorithm.

    After the weights and bias are initialized for all the weights in the Neural Network, the INPUTS and OUTPUTS are calculated for all the cells in the HIDDEN LAYER and OUTPUT LAYER in a process described as a Feed Forward. A network topology that includes 1 HIDDEN LAYER and 1 OUTPUT LAYER is a 2 LAYER Network. If it has N LAYERS in the HIDDEN LAYER then it is described as an (N + 1) LAYER Network where N representing the number of hidden layers in the Network.

    After this process, the Neural Network compares the OUTPUT from each cell in the OUTPUT LAYER to the expected or training or target value to calculate the ERROR.

    The ERROR in a cell in the OUTPUT LAYER = OUTPUT * (1 - OUTPUT) * (TARGET - OUTPUT) where OUTPUT is the actual OUTPUT from a cell and TARGET is the expected or training or target value.

    The ERROR is used to update the weights and bias in the OUTPUT LAYER and the HIDDEN LAYER, where,

    NEW BIAS VALUE = OLD BIAS VALUE + (OLD BIAS VALUE * ERROR * LEARNING RATE )



    C# NEURAL NETWORK MULTILAYER FEEDFORWARD BACKPROPAGATION ALGORITHM VERSION 1.0 - OPEN SOURCE CODE RELEASE


    // NeuralNetwork.cs

    //

    // Copyright (C) 2004 Kingsley Tagbo

    //

    // LICENSE:

    // You have my permission to use this software, redistribute it and/or

    // modify it under the terms of the GNU General Public License

    // (SEE HTTP://WWW.FSF.ORG OR THE COPY OF THE GNU PUBLIC LICENSE BELOW)

    // as published by the Free Software Foundation; either version 2

    // of the License, or (at your option) any later version

    // AS LONG AS YOU INCLUDE A REFERENCE TO KINGSLEY TAGBO -

    // HTTP://WWW.VISUAL-BASIC-DATA-MINING.NET AS THE ORIGINAL AUTHOR

    // OF THIS PROGRAM AND YOU DO NOT REMOVE THE COPYRIGHT AND LICENSE

    // NOTICES IN THIS FREE SOFTWARE

    //

    // This program is distributed in the hope that it will be useful,

    // but WITHOUT ANY WARRANTY; without even the implied warranty of

    // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the

    // GNU General Public License for more details.

    //

    // You should have received a copy of the GNU General Public License

    // along with this program; if not, write to the Free Software

    // Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

    //

    //

    //C# NEURAL NETWORK MULTILAYER FEEDFORWARD BACKPROPAGATION ALGORITHM VERSION 1.0

    //DATE : OCTOBER 25, 2004 4:38 AM

    //AUTHOR : KINGSLEY TAGBO

    //LICENSE : DISTRIBUTED UNDER GNU GENERAL PUBLIC LICENSE VERSION 2, JUNE 1991 (SEE WWW.FSF.ORG)

    // : YOU CAN ALSO READ THE LICENSE INCLUDED BELOW AFTER THE SOURCE CODE

    //VERSION : 1.0

    //COPYRIGHT : KINGSLEY TAGBO - HTTP://WWW.KDKEYS.NET

    //RELEASE: : GOLD


          SOURCE CODE DOCUMENTATION

    NeuralNetwork

    Artificial MiultiLayer FeedForward Neural Network With BackPropagation Algorithm

    rand field

    Static variable for random numbers

    input_count field

    Static variable for number of neurons or cells in INPUT LAYER

    hidden_count field

    Static variable for number of neurons or cells in HIDDEN LAYER

    output_count field

    Static variable for number of neurons or cells in OUTPUT LAYER

    iterations field

    Static variable for the number of processing cycles of the Neural Network

    learning_rate field

    Represents the LEARNING RATE used in gradient descent to prevent weights from converging at sub-optimal solutions.

    input_data field

    A two dimensional array of input data from a training sample An INPUT represented as double [,] inputDataArray = new Double [,] {{0.20, 0.80}, {0.80, 0.4}} would represent 2 training instances for an INPUT LAYER with 2 NEURONS or CELLS

    output_data field

    A two dimensional array of output data from a training sample An OUTPUT represented as double [,] outputDataArray = new Double [,] {{0}, {1}} would represent 2 training instances for an OUTPUT LAYER with 1 NEURON or CELL

    input field

    A collection of neurons representing the input layer

    hidden field

    A collection of neurons representing the hidden layer

    output field

    A collection of neurons representing the output layer

    Initialize(Double[0:,0:],Double[0:,0:],Int32) method

    Initializes the LAYERS in the NEURAL NETWORK

    Parameters

    inputData
    INPUTS data for the NEURONS in the INPUT LAYER
    outputData
    OUTPUT data for the NEURONS in the OUTPUT LAYER
    hidden_layer_count
    The number of neurons in the HIDDEN LAYER

    FeedForward(Int32) method

    Initializes the NEURAL NETWORK with training data input or a real world data input for classification. FeedForward feeds the INPUT to the INPUT LAYER neurons FeedForward feeds the OUTPUT from the INPUT LAYER to the HIDDEN LAYER FeedForward feeds the OUTPUT from the HIDDEN LAYER to the OUTPUT LAYER FeedForward uses the Sigmoid Function or Logistic Function to calculate the OUTPUT from the INPUT in the HIDDEN and OUTPUT LAYERS

    Parameters

    sampleNumber
    A numeric ordered value representing the training or classification instance. If the dataset contains 10 instances or rows, the first row has a sampleNumber = 0 and the last row or instance has a sample number = 9

    BackPropagate() method

    Recalculates the BIAS and ERROR in the HIDDEN LAYER and OUTPUT LAYER. Adjustes the WEIGHTS between the OUTPUT LAYER and HIDDEN LAYER and between the HIDDEN LAYER and the INPUT LAYER using the derivative of the Sigmoid Function or Logistic Function

    Main() method

    Executes a Neural Network MultiLayer FeedForward BackPropagation Algorithm for Learning or Classification

    Random property

    Generates random double values between -1.0 and +1.0

    NeuralNetwork.Neuron

    A Neuron is the basic building block of a Neural Network

    Neuron(Int32,Int32) constructor

    Public constructor for initializing a neuron

    Parameters

    layer
    Input Layer = 0, Hidden Layer = 1 and Output Layer = 2
    index
    A number assigned to a cell for ordering the cell or identifying it

    player field

    Internal storage for Neuron.Layer public property

    pindex field

    Internal storage for Neuron.Index public property

    pinput field

    Internal storage for Neuron.Input public property

    poutput field

    Internal storage for Neuron.Output public property

    poutputTraining field

    Internal storage for Neutron.OutputTraining public property

    pweight field

    Internal storage for Network.Weight public property

    pbias field

    Internal storage for Network.Bias public property

    perror field

    Internal storage for Network.Error public property

    LogisticFunction(Double) method

    Sigmoid Function or Logistic Function or Activation Function is applied to the INPUT to a NETWORK LAYER to get an OUTPUT

    Parameters

    val
    The value to calculate a Sigmoid or Logistic Function for

    Returns

    System.Double datatype of the Logistic or Sigmoid Function result

    LogisticFunctionDerivative(Double) method

    Derivative of the Sigmoid Function or Logistic Function or Activation Function

    Parameters

    val
    The value to calculate the derivative of the Sigmoid Function for

    Returns

    System.Double datatype of the numeric result

    Layer property

    INPUT LAYER = 0, HIDDEN LAYER = 1, OUTPUT LAYER = 2

    Index property

    Identifies a Neuron or the position of a Neuron in a LAYER

    Input property

    Input data fed to the Neural Network

    Output property

    Calculated Ouput from the INPUT, HIDDEN or OUTPUT LAYER

    OutputTraining property

    Expected or Target or Learning output for a neutron in the OUPUT LAYER

    Weight property

    Storage for array of weights from OUTPUT to HIDDEN LAYER and from HIDDENLAYER TO INPUT LAYER. Each Neuron in the OUTPUT and HIDDEN LAYER is connected to an array of Neurons

    Bias property

    Varies the OUTPUT of a Neuronin a HIDDEN or OUTPUT LAYER

    Error property

    Error = (ACTUALOUTPUT * (1 - ACTUALOUTPUT) * (EXPECTEDOUTPUT - ACTUALOUTPUT))

    NeuralNetwork.Neurons

    A Neural Network Layer or collection of cells or neurons

    Add(Neuron) method

    Adds a neuron to a Neural Network Layer or collection of cells

    Parameters

    newNeuron
    The neuron to add to a Neural Network Layer

    Insert(Int32,Neuron) method

    Adds a neuron to a Neural Network Layer at a specific position

    Parameters

    index
    The position where a neuron will be inserted
    newNeuron
    The neuron to add to a Neural Network Layer

    this[Int32] indexer

    ReadOnly indexer - retrieves the neuron at a specific position orlocation in the Neural Network Layer or Neural Network Collection


    Sign-up For Data Mining Source Code Newsletter
Sign-Up For Free Data Mining Source Code Newsletter |  View as RSS news feed in XML

Announcing The Data Mining Source Code Newsletter!

Subscribe By Email | Subscribe By RSS Feed