Hyperparameter
Hiperparameter
Settings you choose before training, which the training process does not learn.
Parameters are learned from data; hyperparameters are chosen by you or by a search process. Learning rate, batch size, number of layers, context length at training time: these are decisions made before the run, and they often matter more than any single model choice. Good defaults exist, which is why a small team can match a large one on a narrow task, and why bad defaults waste weeks of compute.
Related terms
Training
The expensive process that sets a model's parameters by exposing it to data.
Parameters
The learned numbers inside a model. Their count is the model size.
Evaluation
Measuring whether the system does the job, on a fixed set of cases, repeatedly.
Dataset
A collection of examples used to train, evaluate, or ground a system.
Want this working inside your team?
It starts with one conversation about your workflow.