What is a p-value?

P-value is the probability of obtaining observed data (or more extreme) if the null hypothesis is true. In statistics and machine learning, the p-value helps assess the significance of test or model results.

If the p-value is small (usually less than 0.05), it indicates that the observed data are unlikely under the null hypothesis and should be rejected. If the p-value is large, there is not enough evidence to reject the null hypothesis.

Example: when testing whether a new algorithm affects model accuracy, the null hypothesis is "the algorithm has no effect." If the p-value = 0.01, it means there is a 1% chance of obtaining such data if there is no effect, and it can be concluded that the algorithm does indeed affect accuracy.

p-value — is the probability of obtaining observed data (or more extreme) if the null hypothesis is true. In simpler terms, it measures how well the data agree with the assumption that there is no effect or difference.

Purpose of p-value:

• Helps decide whether to reject or not reject the null hypothesis in a statistical test.
• If the p-value is less than a pre-selected significance level (e.g., 0.05), it is considered that the observed data are unlikely under the null hypothesis, and it is rejected.

For example, in testing the effectiveness of a drug, the null hypothesis might be that the drug has no effect on recovery. If the p-value is very small, it indicates that the data are unlikely if there is no effect, and it can be assumed that the drug indeed works.