Social Research Glossary A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Home
Citation reference: Harvey, L., 201220, Social Research Glossary, Quality Research International, http://www.qualityresearchinternational.com/socialresearch/ This is a dynamic glossary and the author would welcome any email suggestions for additions or amendments.

A fastpaced novel of conjecture and surprises 
_________________________________________________________________
Significance test
A significance test is a statistical procedure that is applied to random samples to take account of sampling error.
Introduction
Essentially, a statistical significance test indicates whether any observed difference between samples or relationships between variables in a sample is due to random chance or is indicative of a 'real' difference or a 'real' relationship.
There are a large number of tests of significance dealing with a large variety of situations. Tests of significance are usually divided into parametric tests and nonparametric tests.
Parametric tests are those that test sample parameters such as means, proportions, standard deviations, correlation coefficients, etc.
Nonparametric tests compare entire distrubutions rather than parameters. Nonparametric statistical tests (also known as distributionfree tests) do not involve the estimation of a population parameter and generally require no assumptions about how the scores in question are distributed. Nonparametric tests are used when parametric tests are inappropriate, for example, the data is not interval scale. However, nonparamertic tests lack some of the power of parametric tests.
The following determine a suitable test for a given situation :
a. whether samples are related or independent
b. the level of measurement of the sample data
c. the number of samples being compared
d. which parameter, if any, is being compared
The level of confidence (or confidence level) is the degree of reliance that may be placed on a particular interval estimate for a population parameter. Measured as the number of times out of 100 that the confidence interval can be expected to include the 'true' parameter value, if the research were repeated a large number of times.
The level of significance (or significance level) is the level (or percentage) at which a statistically significant result may be incorrect. It is the probability of falsely rejecting the null hypothesis. The significance level is equivalent to 100% minus the confidence level.
Critical values are borderline values, which may be specified for any given statistical test, dividing the region of acceptance from the region of rejection of the null hypothesis.
There are very many miinterpretations of significance tets and these are explored in Greenland, S. et al., 2016.
Note that signifcant difference is not the same as substantive difference see: Note on substantive difference.
Colorado State University (1993–2013) defines
Matched TTest: A statistical test used to compare two sets of scores for the same subject. A matched pairs Ttest can be used to determine if the scores of the same participants in a study differ under different conditions. For instance, this sort of ttest could be used to determine if people write better essays after taking a writing class than they did before taking the writing class.
TTest: A statistical test. A ttest is used to determine if the scores of two groups differ on a single variable. For instance, to determine whether writing ability differs among students in two classrooms, a ttest could be used.
If the results of a test have statistical significance, it means that they are not likely to have occurred by chance alone. In such cases, we can be more confident that we are observing a 'true' result.
Power of significance tests
How good a statistical test is at identifying differences not accounted for by sampling error is referred to as the power of the test.
Technically, the power of a statistical test is the probability of the test not accepting a null hypthesis when it is false.
Accepting a null hypthesis when it is false is known as a type 2 error (sometimes denoted by β).
A type 1 error is rejecting a null hypothesis when it is true and this is the level of significance (see above) (sometimes denoted by α).
Different tests are more or less powerful depending on how they operate.
For any given test, the factors that affect the power of the test include the sample size (the larger the sample the more power the test has, which is logical as there is less chance of 'rogue' values affecting the result) and the significance level, the power of the test is reduced the significance level is reduced (there is more chance of not rejecting the null hypothesis).
See also
Researching the Real World Section 8.3.14
Greenland, S. et al., 2016, 'Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations', European Journal of Epidemiology, 31, pp. 337–50 available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4877414/, accessed 27 May 2019.
NHS, undated,
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Home