Using Func(Of TResult) Delegate

Dominick Marciano

I'm building a neural network library pretty much to see if I can do it and advance my skills in programming. I'm currently building the Neuron class, and as you may or may not know, a neuron in a neural network has what's called an activation function; however my question below does not have to do with neural networks directly, strictly with the use of Func(Of TResult) delegate So within my neural network class I have several built in activation functions:

Private Function Identity(value As Double) As Double
Return value
End Function

Private Function \[Step\](value As Double) As Double
    If value <= 0 Then
        Return 0
    Else
        Return 1
    End If
End Function

Private Function LogarithmicSigmoid(value As Double, alpha As Double) As Double
    If value < -45.0 Then
        Return 0
    ElseIf value > 45.0 Then
        Return 1
    Else
        Return (1.0 / (1.0 + Exp(-alpha \* value)))
    End If
End Function

Private Function SymmetricSigmoid(value As Double) As Double
    Return HyperbolicTangent(value)
End Function

Private Function HyperbolicTangent(value As Double) As Double
    If value < -45 Then
        Return -1.0
    ElseIf value > 45.0 Then
        Return 1.0
    Else
        Return Tanh(value)
    End If
End Function

I also have an enumeration for the user to select which activation function they want to use for the neuron:

Public Enum ActivationFunctions
IDENTITY
[STEP]
LOGARITHMIC_SIGMOID
SYMMETRIC_SIGMOID
HYPERBOLIC_TANGENT
End Enum

The first problem I'm having is with the following methods:

Private activationFunction As Func(Of Double)

Public Sub SetActivationFunction(activationFunction As ActivationFunctions, value As Double, Optional alpha As Double = 1)
Select Case activationFunction
Case ActivationFunctions.IDENTITY
Me.activationFunction = Function(value As Double) '<---PROBLEM LINE
End Select
End Sub

Public Sub SetActivationFunction(ByVal activationFunction As Func(Of Double))
'DON'T KNOW WHAT TO DO HERE
End Sub

I can't figure out how to set the instance variable activationFunction to a specific method using the AddressOf operator while passing it a parameter. Specifically

TnTinMn

For as function with return type Double and a parameter of type Double, use the Func(T, TResult) delegate[^]. However, in this case, a delegate solution is not feasible as you have two different function signatures: Func(Of Double, Double) Func(Of Double, Double, Double) An alternative would be:

Private Function GetActivationFunctionValue(ByVal functiontype As ActivationFunctions, _
ByVal value As Double, _
Optional ByVal alpha As Double = 1) _
As Double
Select Case functiontype
Case ActivationFunctions.IDENTITY
Return Identity(value)
Case ActivationFunctions.LOGARITHMIC_SIGMOID
Return LogarithmicSigmoid(value, alpha)
' other enum cases
End Select
End Function

Dominick Marciano

That works fine for built-in functions but how can I generalize it so that a user can supply their own activation function to use instead of just the ones I define?

TnTinMn

Quote:

how can I generalize it so that a user can supply their own activation function to use instead of just the ones I define?

You should probably ask this question as a new question to get input from others. The best advice that I can offer on that is to define the allowed function structure and develop an expression builder so that the user must follow a structure you define. The expression could then be processed to emit a DynamicMethod[^] that can called if selected. This will take a fair amount of work.