create_dl_layer_loss_cross_entropyT_create_dl_layer_loss_cross_entropyCreateDlLayerLossCrossEntropyCreateDlLayerLossCrossEntropycreate_dl_layer_loss_cross_entropy (Operator)

Name

create_dl_layer_loss_cross_entropyT_create_dl_layer_loss_cross_entropyCreateDlLayerLossCrossEntropyCreateDlLayerLossCrossEntropycreate_dl_layer_loss_cross_entropy — Create a cross entropy loss layer.

Signature

Description

The operator create_dl_layer_loss_cross_entropycreate_dl_layer_loss_cross_entropyCreateDlLayerLossCrossEntropyCreateDlLayerLossCrossEntropyCreateDlLayerLossCrossEntropycreate_dl_layer_loss_cross_entropy creates a cross entropy loss layer whose handle is returned in DLLayerLossCrossEntropyDLLayerLossCrossEntropyDLLayerLossCrossEntropyDLLayerLossCrossEntropyDLLayerLossCrossEntropydllayer_loss_cross_entropy. This layer computes the two dimensional cross entropy loss on the input (provided by DLLayerInputDLLayerInputDLLayerInputDLLayerInputDLLayerInputdllayer_input) given the corresponding target (provided by DLLayerTargetDLLayerTargetDLLayerTargetDLLayerTargetDLLayerTargetdllayer_target) and weight (provided by DLLayerWeightsDLLayerWeightsDLLayerWeightsDLLayerWeightsDLLayerWeightsdllayer_weights).

Cross entropy is commonly used to measure the similarity between two vectors.

Example:

Illustrative example, where we have a pixel-level classification problem with three classes.

The input vector for a single pixel is (e.g., the output of a softmax layer) which means that the predicted value (e.g., probability) is 0.7 for the class at index 0, 0.1 for the class at index 1 and 0.2 for the class at index 2.

The target vector is with a probability of 1.0 for the actual class and 0.0 else. Entropy is calculated by the dot product of these two vectors. Since the target vector has only one non-zero entry, it can be given by the index of the actual class instead of a vector, in this case .

The cross entropy is then simply the value of the input vector at the target class index, hence . Using this simplification, the cross entropy loss function over an input image can be defined by where the input consists of one prediction vector for each pixel, the target and weight consist of one value and for each input pixel, is the number of pixels and is the sum over all weights.

Hence, this layer expects multiple incoming layers:

• DLLayerInputDLLayerInputDLLayerInputDLLayerInputDLLayerInputdllayer_input: Specifies the prediction (e.g., a softmax layer, commonly with logarithmized results).

• DLLayerTargetDLLayerTargetDLLayerTargetDLLayerTargetDLLayerTargetdllayer_target: Specifies the target sequences (originating from the ground truth information).

• DLLayerWeightsDLLayerWeightsDLLayerWeightsDLLayerWeightsDLLayerWeightsdllayer_weights: Specifies the weight sequences.

The parameter LayerNameLayerNameLayerNameLayerNamelayerNamelayer_name sets an individual layer name. Note that if creating a model using create_dl_modelcreate_dl_modelCreateDlModelCreateDlModelCreateDlModelcreate_dl_model each layer of the created network must have a unique name.

The parameter LossWeightLossWeightLossWeightLossWeightlossWeightloss_weight determines the scalar weight factor with which the loss, calculated in this layer, is multiplied. This parameter can be used to specify the contribution of the cross entropy loss to the overall network loss in case multiple loss layers are used.

The following generic parameters GenParamNameGenParamNameGenParamNameGenParamNamegenParamNamegen_param_name and the corresponding values GenParamValueGenParamValueGenParamValueGenParamValuegenParamValuegen_param_value are supported:

'is_inference_output'"is_inference_output""is_inference_output""is_inference_output""is_inference_output""is_inference_output":

Determines whether apply_dl_modelapply_dl_modelApplyDlModelApplyDlModelApplyDlModelapply_dl_model will include the output of this layer in the dictionary DLResultBatchDLResultBatchDLResultBatchDLResultBatchDLResultBatchdlresult_batch even without specifying this layer in OutputsOutputsOutputsOutputsoutputsoutputs ('true'"true""true""true""true""true") or not ('false'"false""false""false""false""false").

Default: 'false'"false""false""false""false""false"

'num_trainable_params'"num_trainable_params""num_trainable_params""num_trainable_params""num_trainable_params""num_trainable_params":

Number of trainable parameters (weights and biases) of the layer.

Certain parameters of layers created using this operator create_dl_layer_loss_cross_entropycreate_dl_layer_loss_cross_entropyCreateDlLayerLossCrossEntropyCreateDlLayerLossCrossEntropyCreateDlLayerLossCrossEntropycreate_dl_layer_loss_cross_entropy can be set and retrieved using further operators. The following tables give an overview, which parameters can be set using set_dl_model_layer_paramset_dl_model_layer_paramSetDlModelLayerParamSetDlModelLayerParamSetDlModelLayerParamset_dl_model_layer_param and which ones can be retrieved using get_dl_model_layer_paramget_dl_model_layer_paramGetDlModelLayerParamGetDlModelLayerParamGetDlModelLayerParamget_dl_model_layer_param or get_dl_layer_paramget_dl_layer_paramGetDlLayerParamGetDlLayerParamGetDlLayerParamget_dl_layer_param. Note, the operators set_dl_model_layer_paramset_dl_model_layer_paramSetDlModelLayerParamSetDlModelLayerParamSetDlModelLayerParamset_dl_model_layer_param and get_dl_model_layer_paramget_dl_model_layer_paramGetDlModelLayerParamGetDlModelLayerParamGetDlModelLayerParamget_dl_model_layer_param require a model created by create_dl_modelcreate_dl_modelCreateDlModelCreateDlModelCreateDlModelcreate_dl_model.

Execution Information

• Multithreading type: reentrant (runs in parallel with non-exclusive operators).
• Multithreading scope: global (may be called from any thread).
• Processed without parallelization.

Parameters

Module

Deep Learning Training