RESEARCHING THE REAL WORLD



MAIN MENU

Basics

Orientation Observation In-depth interviews Document analysis and semiology Conversation and discourse analysis Secondary Data Surveys Experiments Ethics Research outcomes
Conclusion

References

Activities

Social Research Glossary

About Researching the Real World

Search

Contact

© Lee Harvey 2012–2019

Page updated 25 January, 2019

Citation reference: Harvey, L., 2012–2019, Researching the Real World, available at qualityresearchinternational.com/methodology
All rights belong to author.


 

A Guide to Methodology

CASE STUDY: Generalisation in qualitative research (Payne and Williams, 2005)

Geoff Payne and Malcolm Williams explored the issue of generalisationwhen undertaking qualitative rsearch and provided some examples one of which, referring to focus groups, is reproduced here.

The Structure of Argument in Qualitative Sociology

The second example of generalizing practice (Wray, 2003) is a persuasive and stimulating study of gender, age and health which also addresses national policy. It is based on 'in-depth semi-structured interviews and focus groups' with 170 women in several minority ethnic groups. Its strengths, however, have nothing to do with the larger number of interviewees, but rather with the comparatively greater caution in the conclusions it draws. The final section in the article combines summary statements about what respondents have said, with more general evaluative sentences about older women as a whole. The author uses the format of examples from fieldwork—such as 'most women noted how despite changes to the agility of their bodies....Being in good health was linked to...' (Wray, 2003: 525)—as a basis for making immediately juxtaposed general claims:

...the needs of older women should be recognized and prioritized by local councils and other funding agencies. If older women are to have a good quality of life they need to feel in control of their present and future. This meant making their own decisions, being included in the wider social and political arena and continuing to have a role in society. (p. 525, emphasis added)

This latter quotation shows first how the specificities of the study are reinterpreted as policy imperatives of general applicability, and then in the last italicized sentence, pulled back by use of the past tense to how the author understands the empirical statements that have been elicited from informants. The juxtaposition makes it clear from which part of the evidence the generalization is coming. On the other hand, the quotation also demonstrates how closely empirical specificities are linked to generalities, and how easy it is to move from one to the other without explicit discussion of whether generalization is justified on the basis of the data actually collected.

To be more precise, the author has collected data by interviews and focus groups, from nine different self-defined ethnic groups (p. 513). If we assume (because we are not told) that all of the nine ethnic groups were included in some numbers, it seems a reasonable moderatum position to take that the experiences of aging that the author discusses are not confined to only a few ethnic groups, but are widely encountered. To the extent that the experiences of physical activity and sense of self highlighted by the article were systematically reported by very many of the women (and this again is not reported), it is a reasonable moderatum position to suggest that many other (un-interviewed) older women (elsewhere in Britain) might also share the experience. The kinds of empowering physical activities reported may plausibly offer the benefits suggested, but the article would be stronger if the grounds for claiming plausibility were spelled out.

However, to claim that these particular activities have the same benefits for all older women in every part of Britain would be a riskier generalization. We might also ask whether the activities might benefit younger women as well, or indeed, older men. How much counter-evidence (older women exercising but not reporting benefits) would it take to impugn the researcher's conclusions? This second example is by no means atypical in not addressing these points. Sociological conventions in research papers do not currently require such justifications. If the intention is to generalize from the situation of a small number of people (in this case 170) to local authority policy throughout Britain, then we would suggest that it is incumbent on researchers to confront the need to establish the grounds on which generalizing claims are being made, and the limits to the generalization. Equally, even when the research goal is less concerned with policy and simply addresses enhanced understanding, if researchers are to make extensive generalizing statements like 'A code of ethics exists among collectors' or 'an instrumental society...tends to empty life of its richness' on the basis of eight interviews (Kjølsrød, 2003: 469, 474), it is not unreasonable to ask on what grounds such statements can be justified.

We can summarize the generalization issues exemplified by this article as follows:

1 Generalization is more likely to be plausible if it is approached with caution, moderating the range of the generalizing conclusions. Too ambitiousa set of conclusions undermines the credibility of otherwise competent research.

2 While the distinction between evidence and conclusion needs to be clearly demarcated, so that the reader is enabled to distinguish between the two, generalizations are more credible if the exposition connects the generalization to the specifics of data that provide its foundation. This can be partially achieved by simple juxtaposition, but explicit linkages with the relevant sub-set of data are an even better way of sustaining the generalization. This is not simply an aspect of final reportage. Good research design helps to identify the data necessary for the kinds of generalizing conclusions that may be anticipated at the outset.

3 Where several elements are involved in the study, these need to be fully reported. Thus if there are several sub-groups among the informants, if there are a variety of activities that operationalize the focal concepts, or if outcomes may differ for wider population groups, the reader needs to have access to published information about similarities and difference in those sub-groups, activities and wider populations. This enables readers to evaluate the status of the researcher's generalizations.

4 Similarly, researchers should indicate their own assumptions about similarities and differences between the research site and the other ('receiving') sites, and between the research's informants and other actors. This process would not only assist the reader, but clarify in the researchers' own minds what it is that they are claiming.

5 Even when the quality of the data is good, and the evidence is consistent with the conclusions drawn, research design may not have eliminated alternative explanations or constraints on generalization. The constraints on generalization need to become a standard part of the analytical discussion.

Extract from Payne and Williams (1999, pp. 303–5)

Top

Return to Focus groups (Section 4.4.3.6)

Return to Generalisability (Section 1.10.1)