Analytic Quality Glossary

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Citation reference: Harvey, L., 2004–8, Analytic Quality Glossary, Quality Research International, http://www.qualityresearchinternational.com/glossary/

This is a dynamic glossary and the author would welcome any e-mail suggestions for amendments or additions.

____________________________________________________________________

Audit

core definition

Audit, in the context of quality in higher education, is a process for checking that procedures are in place to assure quality, integrity or standards of provision and outcomes.

explanatory context

Audit and institutional audit tend to be used interchangeably. However, there is a distinction between an external and internal audit (the former by an external agency) and between an institutional audit and a sub-institutional audit. There are then, four possibilities, external institutional audit, internal institutional audit, external sub-institutional audit and internal sub-institutional audit.

Audit tends also to be shorthand for quality audit, although it might also include, or specifically be about, auditing standards. In a sense, external examining is a standards auditing procedure.

Audits might establish the existence of procedures or may attempt to audit their effectiveness.

Audit may be to ensure conformance with regulations or procedures specified by an external body, such as regulations relating to examination of students.

Audit may aim to ensure compliance to policy objectives or system-wide practices. For the majority of policy objectives, this is probably most effectively done at the institutional level. However, if policy is focused on specific issues such as encouraging problem-based learning, the development of employability skills, developing inclusive curricula, encouraging equal opportunities, then programme-level assessments are useful tools to see to what extent such initiatives have been taken on board.

Normally, audits check whether procedures are in place, which usually requires an institution to specify its internal quality-monitoring procedures, including identification of responsibilities and intra-institutional communication and co-ordination of practices. Such processes will usually be circumscribed by established national or international protocols, codes of practice or tradition (Harvey, 2002, pp. 13–14).

In some cases, audit is specifically targeted at improvement plans and processes, such as the quality audits of Finnish polytechnics undertaken by the Finnish Higher Education Evaluation Council (HEEC, 1997). In Sweden, the approach to audit undertaken by the National Agency was to focus on the stated improvement agendas of institutions and explore the efficacy of improvement projects and the approach appears to have aided the development of quality awareness and quality work in the institutions (Askling, 1997; 1998).

Often an audit team (usually of peers) will receive documentation and visit the institution to establish the effectiveness of the stated processes via panel discussion augmented by informal conversations and observation. Audits do not usually attempt to evaluate the institution as such, just to ensure that the institution has clearly-defined internal quality monitoring procedures linked to effective action.

Although higher education quality audit is an approach adopted, in various forms, by several countries, including New Zealand, Australia, Sweden, Finland Norway and the USA (Spanghel, 2001)[1] it was initiated in the UK and is a term designed to focus on procedures rather than quality per se. Webb sets out the historical genesis and intent:

The term ‘audit’ was suggested by Professor Stewart Sutherland, Vice-Chancellor of the University of London and chairman of the CVCP’s Academic Standards Group and of the Management Board of the Academic Audit Unit between 1990 and 1992, partly in order to distinguish academic audit from other kinds of peer reviews exercises, and partly to hint at the intended methodology that would characterize this form of external academic review. Academic audit would seek to borrow and adapt some techniques from financial audit, such as the scrutiny of an institution’s documented procedures and practice, the extermination of these in operation through detailed inquiries and discussions with staff (and students), and the sampling of a wide range of activity and evidence, to establish and test the robustness and effectiveness of academic quality assurance. The analogy with financial audit was known to be far from perfect; it was not meant to imply that a university’s teaching and learning activities are comparable with a company’s balance sheet, or that they are susceptible to similar forms of analysis. Indeed, the starting point for an academic audit was, and remains, an institution’s stated aims and objectives and its stated means of assuring the quality, in particular, of its activities associated whit teaching and learning. This approach has enjoined audit teams to bring an open, non-prescriptive perspective to each institutional audit and to review arrangements for quality assurance agnostically, rather than from a conviction that a given set of procedures or an approach must be right, irrespective of the institutional context or an institution’s declared aims and objectives. (Webb, 1994, p. 47)

analytical review

For INQAAHE (2001, p. 2) audit is about evaluating processes rather than evaluating quality. The definition also identifies external sub-institutional audit:

Audit is the evaluation or review of procedures, processes and mechanism in an HEI or a part of an HEI (e.g., school of Medicine, school of Business & Management). Not quality itself is subject of review, but procedures and mechanism to assure the quality and the processes to achieve the mission and goals… A quality audit evaluates especially the procedures and processes for the assurance of the quality. The main assumption is that if QA-procedures are in place, one may expect that HEI will deliver quality. … As soon as the quality of the core activities (educational activities and research outcomes) are judged too, it is called institutional assessment.

Green, (1994, p.12) also implicitly differentiates assurance and management audits but argues that quality audit:

focuses on the robustness of quality assurance and quality management systems, and is therefore legitimately the responsibility of the institutions themselves.

Fraser (1994, p.106) disassociates audit from any form of evaluation of mission or outcomes:

A scrutiny by a group external to the university checking that the quality assurance and quality control processes are appropriate and working properly has been described as a ‘quality audit’…. Quality Audit is neither concerned with a university’s mission (objectives, inputs) nor with how successfully these objectives have been attained (outputs), but solely with the processes by which the university checks on the relations between its inputs and outputs.

The European Training Foundation (1998, p. 12) defines audit as:

Quality audit: Evaluation of an institution’s processes for quality management.

The Australian Universities Quality Agency (2004) defines audit as:

‘Quality audit’ is defined as “a systematic and independent examination to determine whether activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives” (Australian/New Zealand Standard, 1994). This definition forms the basis upon which AUQA has developed its approach to audit.

University of Canberra (2002), has a slightly different take, however:

The major aim of the audit is to consider and review the processes an organisation has in place to monitor and achieve its objectives and the outcomes being achieved.

Southern Cross University (2003) states:

Academic audits of self-accrediting institutions will be whole-of-institution audits based on a self-assessment and a site visit. The audit will assess the extent to which the university is “fit for its purpose” and achieving its missions and objectives

Woodhouse (1999) reminds us that ISO (Standards New Zealand, 1994) defines quality audit as a three-part process, checking: 1) the suitability of the planned quality procedures in relation to the stated objectives; 2) the conformity of the actual quality activities with the plans; and 3) the effectiveness of the activities in achieving the stated objectives (

The British Standards Institution has a generic definition that focuses on the compliance of processes with planned intent:

Quality audit is a systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives. (BSI, 1989)

Tempus (2001) defines quality system audit as both an internal and external process:

Quality System Audits – systematic and independent investigation with the purpose of determining whether the actions and results referring to quality are in accordance with the set regulations, whether these regulations are suitable for achieving the set goals and whether they are really being implemented. It can be internal and external.

The University of Aberdeen (2002) equates audit with external processes:

Quality Audit: A systematic and independent examination to determine whether quality control and quality assurance are satisfactory, implemented effectively, and are suitable to achieve objectives.

Woodhouse (1999, pp. 30–31) defines audit as:

a check on an organisation’s explicit or implicit claims about itself. When an institution states objectives, it is implicitly claiming that this is what it will do, and a quality audit checks the extent to which the institution is achieving its own objectives. When the claims are explicit (as in financial reporting or if the institution has done a self-quality audit), audit becomes a validation (or otherwise) of those claims. Audit asks “are your processes effective?” (i.e. in achieving your objectives). The output of an audit is a description of the extent to which the claims are correct. An audit is sometimes called a review.

Haakstad (2001, p. 77) sees audits as having an improvement function:

The audits were to evaluate how the institutions accomplish their responsibility for quality assurance and quality work, however in a way that addresses the internal capacity of the institution and motivates efforts at self-improvement and institutional learning.

The UNESCO glossary defines quality audit:

Quality Audit: The process of quality assessment by which an external body ensures that (i) the institution of programme quality assurance procedures or (ii) that the overall (internal and external) quality assurance procedures of the system are adequate and are actually being carried out. Quality audit looks to the system for achieving good quality and not at the quality itself. A quality audit can be realized only by persons (i.e., quality auditors) who are not directly involved in the areas being audited. Quality audits can be undertaken to meet internal goals (internal audit) or external goals (external audit). The results of the audit must be documented (audit report). See, also, audit). (Vlãsceanu et al., 2004, p. 50)

The UNESCO definition also suggests, under audit that:

In the United Kingdom, when an audit is an institutional process carried out internally, the process is described (since 2002) as an “institutional review” process. (Vlãsceanu et al., 2004, pp. 22–23)

There is a conflict in the last sentence with the UNESCO definition of institutional audit elsewhere: the quote above implies review is an internal process but Vlãsceanu et al., (2004, p. 23) state that it is a peer review process, implying an external process.

CHEA (2001) defines quality audit as:

Quality Audit: A test of an institution's quality assurance and control system through a self-evaluation and external review of its programs, staff, and infrastructure. Designed to provide an assessment of an institution's system of accountability, internal review mechanisms, and effectiveness with an external body confirming that the institution's quality assurance process complies with accepted standards.

CHEA (2001) has a separate definition of audit see external sub-institutional audit.

Kristoffersen (2003) does not talk of audit but of evaluations of quality assurance mechanisms. However, this appears to be similar to audit but involves an assessment of the impact of quality processes.

Evaluations of quality assurance mechanisms aim to assess the quality assurance mechanisms within, typically, a specific system, institution or programme, and how these affect the quality of the educational activities.

A totally different meaning of audit comes from the United States (which has not been accommodated in the core definition):

Audit: Enrolling in a class on an audit basis means the class would not count for credit or grade point average. In some cases the audit fee is less than the tuition rate. Registration for audit may require the permission of the instructor. (NTNC, 2002)

associated issues

related terms

See also

external institutional audit

internal institutional audit

external sub-institutional audit

internal sub-institutional audit

management audit

audit report

sources

Askling, B., 1997, ‘Quality monitoring as an institutional enterprise’, Quality in Higher Education, 3(1), pp. 17–26.

Askling, B., 1998, Quality Work in Swedish Universities in a Period of Transformation (Report No. 1998:04). (Göteborg; Dept. of Education and Educational Research, Göteborg University).

Australian Universities Quality Agency (AUQA), 2003, Audit Manual v1 (My 2002), http://www.auqa.edu.au/qualityaudit/auditmanual_v1/chapter02/index.shtml, last updated 09/12/2003.

British Standards Institution (BSI), 1989, Quality Systems Auditing, BS7229 (London, BSI).

Campbell, C. & Rozsnyai, C., 2002, Quality Assurance and the Development of Course Programmes. Papers on Higher Education Regional University Network on Governance and Management of Higher Education in South East Europe Bucharest, UNESCO.

Council For Higher Education Accreditation (CHEA) 2001, Glossary of Key Terms in Quality Assurance and Accreditation, http://www.chea.org/international/inter_glossary01.html, updated 8 May 2001.

Council on Higher Education, Higher Education Quality Committee (HEQC), 2004, Criteria for Institutional Audits, April (Pretoria, Council on Higher Education). Avalable at http://quality.up.ac.za/docs/index.html

Duff, T., Hegarty, J. and Hussey, M., 2000, Academic Quality Assurance in Irish Higher Education: Elements of a handbook, (Dublin, Blackhall).

European Training Foundation (ETF), 1998, Quality Assurance in Higher Education: Manual of Quality Assurance in Higher Education: Procedures and Practices (Turin: European Training Foundation).

Fraser, M., 1994, ‘Quality in higher education: an international perspective’ in Green, D. (Ed.), 1994, What is Quality in Higher Education? pp. 101–111 (Buckingham, Open University press and Society for Research into Higher Education)

Green, D., 1994, ‘What is quality in higher education? Concepts, policy and practice’, in Green, D. (Ed.), 1994, What is Quality in Higher Education? pp. 3–20 (Buckingham, Open University press and Society for Research into Higher Education).

Harvey, L., 2002, ‘Quality assurance in higher education: some international trends’ Higher Education Conference, Oslo, 22-23 January.

Higher Education Evaluation Council (Finland) (HEEC), 1997, Action Plan for 1998–1999. Publications of HEEC, 5:1997. Edita, Helsinki.

International Network of Quality Assurance Agencies in Higher Education (INQAAHE), 2001, Annex: Clarification and Glossary, to a questionnaire conducted in December, 2001. www.inqaahe.nl/public/docs/definities.doc [this is no longer the current site of INQAAHE although this document was still accessible, 23 October 2004]

Kristoffersen, D., 2003, ‘Denmark’ in Educational Evaluation around the World An International Anthology (Copenhagen, The Danish Evaluation Institute) ISBN 87-7958-132-3. Available as a pdf at http://www.eva.dk/

Northeast Texas Network Consortium (NTNC), 2002, Distance Learning College Glossary. http://www.netnet.org/students/student%20glossary.htm

Southern Cross University (2003) Academic Quality and Office http://www.scu.edu.au/admin/acqua/quality/quality_audit.html, June

Spanghel, S.D., 2001, ‘Quality improvement alive and vital in US accreditation’ paper at the Sixth Quality in Higher Education International Seminar, The End of Quality?, Birmingham, UK, 25–26 May, 2001.

Standards New Zealand, 1994, Quality Management and Quality Assurance – Vocabulary, Australian/New Zealand Standard AS/NZS ISO 8402.

Tempus, 2001, Glossary of the terms related to quality assurance Development of Quality Assurance System in Higher Education (QUASYS) Tempus Joint European Project, UM JEP-16015-2001 http://www.unizg.hr/tempusprojects/glossary.htm

University of Abeerdeen, 2002, Academic Quality Guide: Section 2: Quality Assurance in Higher Education: an Overview http://www.abdn.ac.uk/registry/quality/section2.hti#2_1, last modified, 15 May 2002.

University of Canberra, 2002, Report of an Audit of The University of Canberra http://www.canberra.edu.au/corporate/quality/audit/UCAUQAReport.pdf

Vlãsceanu, L., Grünberg, L., and Pârlea, D., 2004, Quality Assurance and Accreditation: A Glossary of Basic Terms and Definitions (Bucharest, UNESCO-CEPES) Papers on Higher Education, ISBN 92-9069-178-6. http://www.cepes.ro/publications/Default.htm

Webb, C., 1994, ‘Quality audit in the universities’ in Green, D. (Ed.), 1994, What is Quality in Higher Education? pp. 46–59 (Buckingham, Open University press and Society for Research into Higher Education)

Woodhouse, D., 1999, ‘Quality and Quality Assurance’ in Organisation for Economic Co-Operation and Development (OECD),, 1999, Quality and Internationalisation in Higher Education, pp. 29–44, Programme on Institutional Management in Higher Education (IMHE), Paris, OECD.

---

[1] The largest US accreditor, the North Central Association of Colleges and Schools (NCA), proposes to implement a form of improvement audit. Fundamentally different from traditional quality assurance, which uses sporadic inspections to judge institutions against fixed standards, the new approach (AQIP) uses institutions’ ongoing documentation of quality improvement to provide public quality assurance. It aims to help an institution incorporate quality improvement principles and strive toward self-chosen improvement targets to strengthen its competitive position (Spanghel, 2001).