Debugging an LLM Training Run
LLM training has more moving parts than classic machine learning, but the debugging method is similar. Engineers check the training pipeline, data, objective, optimization, and evaluation in that order.
The fastest first test is to overfit a tiny dataset.
Take 100 examples and train on them many times. The model should almost memorize them. If it cannot, the problem is probably not data coverage. The problem may be the loss function, masking, tokenizer, optimizer, gradient flow, or training code.
After that, look at training and validation loss.
If training loss does not go down, the model is not learning. If training loss is good but validation loss is bad, the model may be overfitting. If both losses look good but the real output is still bad, the training objective may not match the behavior you want.
This last case is common in LLM training.
A low next-token prediction loss does not mean the model can follow instructions, call tools, return valid JSON, or make good decisions. The model may be learning exactly what the loss asks for while still failing the real task.
Data is another major source of problems. Engineers check bad samples, duplicate data, wrong labels, data mixture, missing cases, and train-test distribution.
Aggregate accuracy is often not enough. A model with 90% accuracy may still fail on one important class. A confusion matrix can show where the failures are. Engineers then inspect those examples and group them into error types.
Training signals also help find lower-level problems. Engineers watch learning rate, gradient norm, clipping, NaN values, token distribution, sequence length, and batch statistics. A sudden gradient spike may point to a bad batch.
Next is ablation. Change one thing at a time. Remove one dataset. Change one data mixture. Remove synthetic data. Change the learning rate. Compare the result against a stable baseline.
Here is a practical debugging process:
- Can it memorize a tiny dataset?
- Is training loss healthy?
- Is validation loss healthy?
- Which classes fail?
- Which examples fail?
- Is the label or objective wrong?
- Change one thing and test again.