logits is (B, T, V) and targets is (B, T), where padded positions hold ignore_index. Return the mean cross-entropy over the real positions only.

Flatten the batch and time dimensions together first -- cross-entropy wants (N, V) against (N,) -- and let it skip the ignored positions rather than averaging them in as if the model had got them wrong.

Input

logits =
tensor([[[2.0000, 1.0000],
         [0.5000, 2.5000]],

        [[1.0000, 1.0000],
         [3.0000, 0.0000]]])
targets =
tensor([[   0,    1],
        [   1, -100]])
ignore_index = -100

Output

tensor(0.3778)