Parameters

Parameters are the internal variables within a machine learning model that are learned from the training data. In the context of neural networks, these primarily consist of weights and biases. These values are adjusted by optimization algorithms, such as gradient descent, to minimize the difference between the model's predicted output and the actual target output.

These variables are stored within the model's architecture and determine how input data is transformed as it passes through the network layers. While hyperparameters are set by the developer before training begins to control the learning process, parameters are the resulting values that define the model's final state and its ability to perform specific tasks.

The total number of parameters is often used as a metric for model scale. For example, in a Large Language Model (LLM), the parameter count refers to the sum of all weights across its transformer layers. A model with 7 billion parameters has 7 billion individual numerical values that have been tuned to recognize patterns in language.