A model returns three logits: 2, 1, and 0. Those numbers are scores, not confidence percentages, and their absolute scale is not directly interpretable. To see what cross-entropy is teaching, follow the scores through softmax and calculate the loss for a known class.
From Scores To Probabilities
Softmax turns logits z into probabilities by exponentiating each score and dividing by the sum of all exponentials. For logits [2, 1, 0], the resulting probabilities are approximately [0.665, 0.245, 0.090]. The largest score gets the largest probability, but the other classes still matter because the denominator includes them.
Calculate The Example Loss
Suppose the correct class is the first one. Its probability is about 0.665, so the cross-entropy loss for this example is -ln(0.665), or approximately 0.408. If the model assigned the correct class a smaller probability, the loss would be larger; assigning a confident probability to the wrong class is penalized especially strongly. For a single labeled example, the loss depends on the probability assigned to its target class.
Read The Gradient Carefully
For softmax followed by cross-entropy, the gradient with respect to each logit is its predicted probability minus its target value, where the target is 1 for the correct class and 0 for the others. In this example, the first logit gets a gradient near -0.335, while the other two get positive gradients near 0.245 and 0.090. Gradient descent therefore pushes the correct score up relative to the alternatives; it does not simply tell the model to make every score larger.
Keep Computation Numerically Stable
Exponentiating large logits directly can overflow, so implementations commonly subtract the largest logit before computing the softmax or loss. This leaves the probabilities unchanged because the same factor cancels from numerator and denominator. When checking a training pipeline, verify that labels match the model’s class ordering and that the loss receives logits or probabilities in the format it expects.