Image source: bayesianbiologist.com
P-value is used to determine the significance of the results in hypothesis testing. It indicates the probability of occurrence of a given event and also indicates how data is incompatible with a specified statistical model.
Hypothesis tests show the validity of the claim concluded on a given population and the p-value is used to weigh the strength of the evidence. A larger p-value shows that the statistical data used in the model is unusual but cannot indicate the validity of the assumption.
Look at the following pros and cons to easily understand p values.
Pros:
1. Support decision making: P-values are important for evidence-based decision-making through hypothesis testing on various claims or experimental data. It seeks to access the evidence relevant to the given scenario.
2. Hypothesis testing: P-value uses a hypothesis test to determine the validity of the claim and determine its significance. It doesn’t rely on the probability that the null hypothesis is true because the hypothesis does not contain classical statistics.
3. Detect significance: P values give you more powers to detect the significance of numbers although this will increase type 1 errors.
4. Show data incompatibility: It shows how data is incompatible with a particular statistical model. It summarizes all the incompatibilities between sets of data.
5. Measure consistency: The p-values can measure the consistency between the results obtained through trial and pure chance as well as give a detailed explanation of the results.
6. Show clear distinction: P-values show a clear distinction between the experimental evidence and the inferential conclusions. It also provides a framework for combining the prior evidence with the current evidence.
7. Transparency of reporting: To get proper inferences, the researchers need to have a full reporting on the number of hypotheses explored, data collection decisions, and show the statistical analysis conducted.
8. Widely used: The P-value is widely used in statistical analysis of various fields of research specifically in null hypothesis significance testing. These fields include economics, finance, psychology, sociology, and criminology among others.
Cons:
1. The accuracy of data: P-values only take into account the random error but don’t consider systematic error affecting the accuracy of the estimated data.
2. Relative risk: The magnitude of association and sample size determines the p-values. A large sample size can offset the magnitude of association leading to statistically significant results.
3. Limited information: A P value cannot provide a good measure of evidence on a model or hypothesis by itself due to the limited information provided. It can’t show the reason for testing the hypothesis and there is no information on the magnitude of the evidence supporting the tests.
4. Objective method: Using P-value to determine statistical significance is an objective technique to determine whether a null hypothesis should be rejected.
5. Quantifies values: P-value shows that you have eliminated randomness in any particular quantification and data analysis shouldn’t end with the calculation of the p-value
6. Doesn’t indicate the importance of results: P-value whether small or big does not have any importance of the result or measure the size of an effect. A small effect can produce a small value if the sample size is high and vice versa.
7. Doesn’t support reasoning: P-value doesn’t have reasoning capability about the probabilities of the hypothesis. It is only used as a tool for deciding whether to reject the null hypothesis.
8. Doesn’t indicate the size: P-value doesn’t indicate the size or the importance of an observed effect. Therefore, a small p-value of less than 0.001 doesn’t measure the strength of the relationship between two variables.
