Analytic Quality Glossary
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
Citation reference: Harvey, L., 2004–8, Analytic Quality Glossary, Quality Research International, http://www.qualityresearchinternational.com/glossary/
This is a dynamic glossary and the author would welcome any e-mail suggestions for amendments or additions.
____________________________________________________________________
Assessment
core definition
A general term that embraces all methods used to judge the performance of an individual, group or organisation.
explanatory context
Assessment includes assessment of quality, through an external process of some kind as well as assessment of student learning and assessment of teaching and learning including teacher performance. These are dealt with elsewhere.
Also, note the slightly confusing use of assessment in the US context where it relates to the Assessment Movement, discussed below.
Note, also, that assessment of teacher performance is not the same as Teaching Quality Assessment, as used in the UK, which is a quality assessment process explored below.
Quality assessments set out to ÔmeasureÕ the level of quality of inputs, processes and, sometimes, outputs. This may be a judgment of the overall quality of an institution or programme or of specified component elements. In France, for example, the ComitŽ National dÕƒvaluation (CNE) evaluates each institution holistically (Staropoli, 1991; Ribier, 1995). [i]
Assessment has also focused on research, most notably on research outputs. Finland undertook such evaluations in the early 1980s (Luukkonen and StŒhle, 1990) and more recently Lithuania has evaluated research performance (Mockiene and Vengris, 1995). Probably the most extensive approach is the periodic Research Assessment Exercise undertaken in the United Kingdom, which not only provides a public rating of research for all higher education institutions across more than 60 subject areas but also allocates significant government funding to institutions on the basis of the rating Ñ the higher the rating the more money is allocated (HEFCE/SHEFC/HEFCW, 1993).
analytical review
The International Network of Quality Assurance Agencies in Higher Education (INQAAHE, 2001, p. 2) refer to assessment as follows:
Assessment is the evaluation of the quality itself. Assessment tries to collect data, information and evidence of the quality of the HEI as a whole (institutional assessment) or its core activities (education, research and community service) separately (program assessment). It goes beyond quality procedures (although it will be included) and tries to judge the quality of input, process and output.
A diagnostic form of quality review and evaluation of teaching, learning, and programs based on a detailed examination of curricula, structure, and effectiveness of the institution, its internal review, and quality control mechanisms. (CHEA, 2001)
Woodhouse (1999, p. 32) focuses on the grading aspect of assessment:
Assessment is an evaluation that results in a grade, whether numeric (e.g. a percentage or a shorter scale of say 1 to 4), literal (e.g. A to F) or descriptive (excellent, good, satisfactory, unsatisfactory). There may or may not be a pass/fail boundary somewhere along the grade spectrum (or it may simply be a two-point scale). Assessment asks Òhow good are your outputs?Ó The output of an assessment is the grade. An assessment is sometimes called an evaluation.
The UNESCO definition suggests two meanings pertinent to higher education quality. The first sees assessment as a judgemental evaluation. The second, is a specific evaluation process focusing on learning and teaching and reflects the issues discussed under assessment of learning and teaching.
1. The process of the systematic gathering, quantifying, and using of information in view of judging the instructional effectiveness and the curricular adequacy of a higher education institution as a whole (institutional assessment) or of its educational programmes (programme assessment). It implies the evaluation of the core activities of the higher education institution (quantitative and qualitative evidence of educational activities and research outcomes).
2. A technically designed process for evaluating student learning outcomes and for improving student learning and development as well as teaching effectiveness. (Vl‹sceanu et al., 2004)
Vl‹sceanu et al., (2004) state that institutional effectiveness can be assessed via:
Outcomes Assessment: The process of evaluation and improvement of specific results of a higher education institution in order to demonstrate its institutional effectiveness. Assessment may concern the performance of teaching staff, the effectiveness of institutional practices, and/or the functioning of departments or programmes (e.g., programme reviews, budget reviews, etc.). It is a formative procedure used for institutional self-study, financial retrenchment, programme evaluation, and better understanding of the current needs of students.
And effectiveness of student learning outcomes via:
Student Outcome Assessment: The act of assembling, analyzing, and using both quantitative and qualitative evidence of teaching and learning outcomes, in order to examine their congruence with stated purposes and educational objectives and to provide meaningful feedback that will stimulate improvement. (Vl‹sceanu et al., 2004, p. 43)
The UNESCO glossary also has an entry for quality assessment, which it equates with review:
Quality Assessment/Quality Review: Indicates the actual process of external evaluation (reviewing, measuring, judging) of the quality of higher education institutions and programmes. It consists of those techniques, mechanisms, and activities that are carried out by an external body in order to evaluate the quality of the higher education processes, practices, programmes, and services. Some aspects are important when defining and operating with the concept of quality assessment: (i) the context (national, institutional); (ii) the methods (self-assessment, assessment by peer review, site visits); (iii) the levels (system, institution, department, individual); (iv) the mechanisms (rewards, policies, structures, cultures); (v) certain quality values attached to quality assessment such as academic values, traditional values (focusing upon the subject field), managerial values (focusing on procedures and practices); pedagogical values (focusing on staff and their teaching skills and classroom practice); employment values (emphasizing graduate output characteristics and learning outcomes). (Vl‹sceanu et al., 2004, p. 48)
For Kisuniene (2004):
Quality assessment means a process of external assessment: studentsÕ achievements and performance, school documentation, statistical data, observations, interviews.
Another view relates assessment to subject review, emphasising direct exploration of provision:
The area of quality assurance where standards of teaching and learning in institutions are directly evaluated. This may be by self-assessment, visits by external teams or, usually, some combination of the two. (AEC, 2004)
Quality Assessment : External assessment by peers of actual provision in particular subjects, by scrutiny of institutional documentation and student work, direct observation, interview, and by reference to performance indicators. (University of Aberdeen)
The latter draws on the UK system of quality assessment (now concluded) that:
examines the wide range of influences that shape the learning experiences and achievements of students. It covers the full breadth of teaching and learning activities, including: direct observation of classroom/seminar/workshop/ laboratory situations, the methods of assessing studentsÕ work, studentsÕ work and achievements, the curriculum, staff and staff development, the application of resources (library, information technology, equipment), and student support and guidance. This range of activities is captured within a core set of six aspects of provision, each of which is graded on a four-point assessment scale (1 to 4), in ascending order of merit. The aspects of provision are: curriculum design, content and organisation; teaching, learning and assessment; student progression and achievement; student support and guidance; learning resources; quality management and enhancement. (QAA, 2004)
One view sees quality assessment as the same as review, measurement or evaluation:
Quality assessment, quality measurement, and review of quality are all taken here to be synonymous with evaluation, especially when there is an external element to the procedure. (Campbell & Rozsnyai, 2002, p. 32).
Quality assessment can also be used in conjunction with quality control, essentially as the vehicle for testing the quality control system:
Quality assessment: a system of actions which guarantees that the quality control will be efficient and includes assessment of products and production, that is, of the measuring system. That is a mechanism used to establish whether the quality system works within the set boundaries of tolerance defined on the basis of the end userÕs judgement. (Tempus, 2001)
In the US, the development of the Assessment Movement resulted in a slightly different use of the term assessment, viz. assessment of improvement processes:
Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education (Thomas A. Angelo, AAHE Bulletin, November 1995, p.7).
associated issues
Both the INQAAHE and UNESCO definitions state that while assessment does not lead to accreditation, accreditation decisions must be based on accreditation evidence.
Assessment does not necessarily lead to a formal accreditation decision. However, formal accreditation needs to be based on assessment. (INQAAHE, 2001, p. 2)
Assessment is necessary in order to validate a formal accreditation decision, but it does not necessarily lead to an accreditation outcome. (Vl‹sceanu et al., 2004)
Not everyone agrees that accreditation requires judgement on the level of quality as is implied by assessment, other than perhaps a basic threshold judgment. The concern is that some accreditation processes do not actually measure/assess quality (even at a threshold level) but establish that requirements have been complied with, which is taken as a proxy for a quality judgement.
Assessment is primarily an accountability process. However, it can also be used to encourage improvement. In the UK, the QAA (2004) asserts that the purposes of quality assessment are:
á to ensure that the public funding provided is supporting education of an acceptable quality,
á to provide public information on that education through the publication of reports,
á and to provide information and insights to encourage improvements in education.
In the United States, it has been claimed that:
No state has yet developed an adequate approach to assessing student learning. And yet surely this is the most important category of all (Bennett, 2001).
The key to assessing quality, it is argued, is the value added by an institution or programme.
Virtually everyone who has thought carefully about the question of assessing quality in higher education agrees that "value added" is the only valid approach (Bennett, 2001).
Assessment measures
Harvey (2002, p. 14Ð15), noted:
Measurement may be against externally-set criteria (both implicit and explicit), against internally-specified objectives or missions, or a mutually agreed set of criteria. Many assessments are supposedly of fitness for purpose and thus institutions or programmes are assessed against mission-based criteria. In practice, there are a set of overarching expectations and the mission-based variability operates within narrow tolerances. Assessment might include a complex grading system or might be based on a simple satisfactory/non-satisfactory dichotomyÉ.
Assessment thus may focus on inputs (such as teaching staff, learning resources) or process (such as teaching, learning, support services) or outcomes, (such as students academic standards of achievement or professional competence, employment rates, student perception of their learning). Assessment evidence includes statistical indicators, direct observation, direct evaluation of research outputs, student and graduate views, employer views, student performance, self-assessment and other documentation, discussion and interviews with teachers, students and managers, and perceptions of other agencies, such as professional bodies.
Assessment and benchmarking
Assessment may also ÔbenchmarkÕ[ii] against other institutions, national norms or against oneself over time.
related terms
also see
assessment of student learning
assessment of teaching and learning
sources
Angelo, T.A., 1995, AAHE Bulletin, November, p.7
Association europeenne des conservatoires [Academies de musique et musikhochschulen] (AEC), 2004, Glossary of terms used in relation to the Bologna Declaration http://www.aecinfo.org/glossary%20and%20faq%20english.pdf undated, accessed September 2004
Bennett, D.C., 2001, Assessing quality in higher education Ð Perspectives, Liberal Education, Spring. http://www.findarticles.com/p/articles/mi_m0NKR/is_2_87/ai_88581415
Campbell, C. & Rozsnyai, C., 2002, Quality Assurance and the Development of Course Programmes. Papers on Higher Education Regional University Network on Governance and Management of Higher Education in South East Europe Bucharest, UNESCO.
Council For Higher Education Accreditation (CHEA) 2001, Glossary of Key Terms in Quality Assurance and Accreditation, http://www.chea.org/international/inter_glossary01.html October 23, 2002
Harvey, L., 2002, ÔQuality assurance in higher education: some international trendsÕ Higher Education Conference, Oslo, 22-23 January 2002.
HEFCE/SHEFC/HEFCW, 1993
International Network of Quality Assurance Agencies in Higher Education (INQAAHE), 2001, Annex: Clarification and Glossary, to a questionnaire conducted in December, 2001. www.inqaahe.nl/public/docs/definities.doc [this is no longer the current site of INQAAHE although this document was still accessible, 23 October 2004]
Kisuniene, G., 2004, Quality Assurance: Priority of the Education Reform. http://www.phare.lt/previous/97/EN/en04a.htm
Quality Assurance Agency for Higher Education (QAA), 2004, Assessing the Quality of Education http://www.qaa.ac.uk/revreps/subjrev/assessingquality.htm undated retrieved September 2004
Ribier, 1995
Staropoli, 1991;
Tempus, 2001, Glossary of the terms related to quality assurance Development of Quality Assurance System in Higher Education (QUASYS) Tempus Joint European Project, UM JEP-16015-2001 http://www.unizg.hr/tempusprojects/glossary.htm
University of Abeerdeen, 2002, Academic Quality Guide: Section 2: Quality Assurance in Higher Education: an Overview http://www.abdn.ac.uk/registry/quality/section2.hti#2_1, last modified, 15 May 2002.
Vl‹sceanu, L., GrŸnberg, L., and P‰rlea, D., 2004, Quality Assurance and Accreditation: A Glossary of Basic Terms and Definitions (Bucharest, UNESCO-CEPES) Papers on Higher Education, ISBN 92-9069-178-6. http://www.cepes.ro/publications/Default.htm