Assessing Quality - 2nd Athens International Conference on University Assessment HELLENIC AMERICAN UNIVERSITY

Abstracts

(Abstracts are arranged in alphabetical order, according to speaker’s name.)

Barbara Brittingham

"American Accreditation: Standards and Approaches at Home and Abroad“

The U.S. has, depending on how one measures such developments, 50-100 years of experience with its systems of accreditation. A large, diverse, and decentralized system of higher education has informed and shaped accreditation, as have the social, demographic, economic, and political conditions in our country. And while the experience of exporting the American model of higher education approaches 150 years, the past 15 years have seen an acceleration and complication of the issues raised for U.S. accreditors in considering the overseas operations of U.S. institutions abroad and the question of accrediting free-standing institutions in other countries.

This presentation will summarize the history, influences, and current conditions in which U.S. accreditation operates and illustrate their impact on how the New England Association of Schools and Colleges recently revised its Standards for Accreditation.

Bruno Curvale

“Transforming External Quality Assurance” 

The development of competition in higher education sets new questions to the management of public higher education systems and has consequences on the work and objectives of national quality assurance agencies like CNE in France. Growing institutions’ autonomy, increasing competition and strategic national policy question the action principles of quality assurance operators. This talk will present some of the core questioning behind the recent transformation of CNE approach of quality in higher education.

Nachum Finger

“From Accreditation to Assessment and Evaluation: The Current Israeli Experience.”

Since its creation in 1958, the Israeli Council for Higher Education dealt mainly with the accreditation of academic programs and the authorization to establish academic-degree granting institutions. Israel is undergoing a transition towards mass-higher education and with it a proliferation of new institutions and programs. This and an increasing demand by stakeholders for "Quality Assurance" and "Accountability" lead to the initiation of a top-down effort for overall "Quality Assessment and Evaluation". Some issues concerning this move are presented and discussed.

Hans-Uwe Erichsen

“Quality Assurance in the European Higher Education Area”

By signing the Bologna Declaration in June 1999, 29 European ministers have committed their governments and their countries to create a European Higher Education Area. They have confirmed this commitment in the Prague Communiqué of May 2001 and the Berlin Declaration of September 2003 and they will again deal with this subject at the Bergen Conference in May 2005.

The Berlin Communiqué says: “the quality of higher education has proven to be the heart of setting up a European higher education area”. To improve and to guarantee quality in most of the participating countries of the Bologna Process, systems of quality assurance have been established. Among the tools of quality assurance are accreditation and evaluation, which in the frame of the Bologna Process are seen as equal components.

Evaluation is aiming at the development of quality, it is addressing those being responsible for the quality of an higher education institution or a study program and provides them with information and ideas how to develop and enhance quality.

Accreditation means a formal decision concerning the quality of a program or an institution expressed by a recognized body taking the decision following predetermined standards.

Accreditation and evaluation are of great importance for promoting mobility within the European higher education area. Essential for the further development of mobility is the mutual recognition of credits and degrees. To achieve this, the ministers of the signatories of the Bologna Process have declared it necessary “to develop an agreed set of standards, procedures and guidelines on quality assurance, to explore ways of insuring an adequate peer review system for quality assurance and / or accreditation agencies or bodies”. This set of standards, procedures and guidelines has to take care of the cultural richness and linguistic diversity based on different traditions of the systems of higher education in Europe. A question is, what can in addition in the field of quality assurance be done to promote the cross border recognition of degrees and credits.

Malcolm Frazer

“Is the Four-stage Model of Quality Evaluation always Appropriate?”  

In 1990 there were five countries which had well defined systems of quality evaluation of higher education. Based on a survey of these systems van Vught identified the four stage model, viz.: an agency providing independence from government and institutions, a self-evaluation report, an external review and site visit, a report. Fifteen years later there are about seventy countries with quality evaluation systems. With only minor variations, they have all adopted and use the four stage model.

It will be asserted that this model is not always appropriate for the diverse purposes (e.g. accreditation, assuring stakeholders, grading, mutual recognition of qualifications and credit, improvement), and diverse focuses (e.g. institution, programme, subject, research, strategic planning, student services, e-learning) of quality evaluation. The time has come to develop more sophisticated approaches.

Bruce G. Hammond

"The American Consumer Mindset and How Overseas Institutions Can Recruit U.S. Students"

If an American student were asked why he or she chose a particular university, the student would likely cite academic programs as the most important factor. Most students would be sincere in this belief, but they would be deceiving themselves.
There are, it seems to me, two central determinants of college choice among U.S. students of traditional age. First, students want to go to an institution that is prominent or perceived as such. Admission to college is the primary marker of distinction in a young person's life, and quality is often determined less by the experience once there than by the difficulty in gaining entrance.
Second, the college and university experience in America is not only about getting a degree, but also about acquiring a lifelong affiliation. The runaway popularity of ranking publications such as U.S. News & World Report has less to do with students seeking an academic experience than with alumni wondering how their alma mater will stack up this year. The rankings are not unlike another annual frenzy, the collegiate athletic contests in American football and basketball. These games are about pride in one's alma mater, and for institutions not blessed with Ivy-League pedigree, winning them is the best way to become more selective and thereby more elite.
There is, however, a window of opportunity for less-known institutions from overseas which would like to recruit Americans. The events of September 11, 2001, have helped to spur an increased interest in overseas study among U.S. students. More are considering degrees overseas. U.S. colleges are also expanding study-abroad opportunities, and institutions throughout the English-speaking world have stepped up their recruitment of Americans.
To be successful, overseas institutions must understand American college choice. A ranking in U.S. News may not be possible, but by penetrating the communities that shape awareness of higher education, these institutions can begin to create a reputation for themselves. Institutions need to speak the language of selectivity that is expected in the U.S., and tailor an admission process that gives the expected cues.
Finally, institutions should consider the qualify-of-life expectations of U.S. students, and how these relate to the American concept of university as a rite of passage and affiliation for life. Often, it is non-academic factors that shape college choice and then hold student loyalty during the college years and throughout later life.

Hiroshi Hokama

Transformation of Quality Assurance System of Higher Education in Japan

The Japanese system of quality assurance of higher education has changed profoundly in recent years. Before 2004, the major framework of the system was as follows: the Ministry of Education of the National Government grants approval of the establishment of university; university is required by law to conduct self-study of its operations and make the results public; Japan University Accreditation Association (JUAA), through its periodical evaluation following its own standards, criteria and procedures, assures independently of the Government the quality of its member-universities; non-member universities except national universities have remained un-assessed by any external evaluation agency; national universities are requested by the Ministry of Education to go through the evaluation of its own agency, the National Institute for Academic Degrees and University Evaluation (NIAD-UE).

The amendments of education laws, which took effect on April 1, 2004, have brought about a radical change in the system. The amended laws put all universities under the legal obligation to go through evaluations every seven years by external certified agency; in addition to its power to grant approval of the establishment of university, the Ministry of Education is given by the new law the right to certify evaluation agency; JUAA, which is a voluntary association of universities and used to be entirely independent of the Government, has come under the control of the Ministry of Education through its certification process; the law anticipates that multiple bodies will be certified as evaluation agencies, which will entail competition among those agencies; three bodies including JUAA and NIAD-UE have been certified thus far; JUAA was the first to initiate the evaluation activities under the new law and has recently published the results of its evaluation of 34 universities.

Martin Ince

 “Are Rankings Relevant?”  

In this paper, Martin Ince will set out the increasing demand for university and other rankings, and the reasons why world rankings are of especial interest.

He will discuss the criteria used to determine the Times Higher Education Supplement’s World University Rankings and ways in which these and other rankings can be enhanced.

He will conclude that indeed, rankings are relevant!

Volli Kalm

"Higher education reforms and the Birth of Quality Assessment System - Lessons Learned in Central and Eastern Europe."

The aim of the presentation is to examine reforms in CEE higher education (HE) sector in 1989-2004, and to identify the main trends together with the convergences and divergences between CEE and the developments in European Union (EU), lately with the Bologna process.

Since the late 1980s, three to four different “styles” of HE and qualifications have been in force in most of CEE countries. When comparing CEE HE systems with those of the West, one is confronted with Western HE systems that are quite heterogeneous, themselves also going through change, and obviously the “copy-paste” method in reforming of HE does not work.

In the early 1990s, public universities entered into the market economy conditions as the private HE institutions emerged. Massification of HE in CEE resulted an 2-4-fold increase in student numbers in last ten years. This has resulted in a considerable increase in lecturer’s load and relevant financial load to the universities, which is, with no doubt, a serious threat to the quality of tuition.

The quality matters in HE became important in the early 1990s with massification of HE and emergence of new HE institutions. Two major trends quality assessment were evident in CEE: 1) towards the definition of national standards; and 2) towards competence-based (as distinct from knowledge-based) assessment.

Similarly to many countries from the EU the Bologna process in EE countries can not be seen as the beginning of HE reforms. However, permanent “revolutionary situation” in HE has created a considerable amount of reluctance and hesitation in academic community to all changes, regardless the aim of the change. In many cases at all levels of administration the big picture and fundamental problems were lost from focus when dealing with details.

Unfortunately, central financing of the reforms has mostly been limited and majority of the direct costs (e.g. parallel running of two sets of curricula; site-visits to learn international experience, analysis and consultancy) and all indirect costs (e.g. labor cost for additional workload of people involved in reforms) were covered by the universities involved.

Generally, young CEE HE accreditation systems faced a capacity problem in assessing a great number of new programmes, since in many countries only accredited programmes lead to state-recognized diplomas and degrees.

As a rule, the interest groups in the HE quality assessment process – the universities, the ministries, employer’s representatives and the HE quality assessment councils – have realized, that the role of institutional evaluation must increase in expense of a single programme’s accreditations.

Fair recognition and comparison of the earlier offered degrees and qualifications has become very important in providing equal oportunities in further career or learning possibilities. Another development is related to recognition of informal learning and its relevance to the formal study programme.

Peter Knight

“Demonstrating Quality in Higher Education: What's being done about complex outcomes?”

Universities around the world measure student learning outcomes in order to demonstrate the quality of their work. The universities face problems, though, when their clients want evidence that universities have successfully promoted complex outcomes.

This presentation elaborates the point by considering the national work of the Enhancing Student Employability Team (ESECT) in England. International research shows that employers value graduates who are rich in complex achievements; this is illustrated with data from the UK, mainland Europe and Japan.

The great challenge here for universities is to show that they are making a difference. Reliable and valid data are elusive – for example, employment rates are an invalid indicator and complex achievements, by their very nature, resist measurement.

ESECT’s response to the problem is described and complemented by a view of the degree of support this response has in English universities.

Jimmy Leach

“Measuring Quality in Higher Education - What business is it of newspapers?”

 Universities are reluctantly - make that “very reluctantly” - coming round to the idea that university league tables are here to stay. The, not unreasonable, argument that such tables are an arbitrary set of statistics given random weighting to provide a ranking, which is then set in stone, has great sway in academic circles. However, increasingly, there is an acknowledgement that if UK universities want to charge students up to £3,000 a year, then those who are paying that figure have some right to know what they are getting for their money.

In an age of government-backed performance targets, one might expect such things to be centralised. Yet, the closest we have had to a universal measure of quality in higher education, the teaching quality assessment from the Quality Assurance Agency (QAA), was abandoned in 2001, with universities citing the burden of paperwork, and its replacement lacked the same comprehensive breadth. Into such a vacuum, newspapers will swoop...

Details

Jacob David Leibenluft

 "Life at the Top: The Impact of Rankings and Competitiveness on Students and Applicants at America's Top Universities."  

Jacob David Leibenluft will address three questions in relation to increased competitiveness at elite American universities:

  1. What is the source of this competitiveness, and how is it manifested in college rankings?
  2. How well do these rankings reflect the quality of education at the top-ranked universities?
  3. What is the impact of this competitiveness on student experiences both in secondary schools and at universities?

Tryphon Pneumaticos

“The Cyprus Experience in Quality Higher Education through Accreditation”

Higher Education (HE) in Cyprus, essentially started in the 80”s with the first private colleges offering a limited number of programmes, mainly in Business and Hotel Management fields.

Prior to that, there were of course a number of state institutions of HE, but the establishment of the Department of HE within the Ministry of Education in 1984, gave the momentum needed to the following:

  • Registration of a number of private institutions of HE in 1991;
  • Establishment of the University of Cyprus in 1992;
  • Functioning of the Council for Recognition of Higher Education Qualifications (KySATS) in 2000;
  • Accreditation of the first programmes of study offered by the registered institutions in 2000;
  • Reaccredidation of the accredited programmes in 2004.

More than 150 programmes of study have been successfully examined and accredited during the process, by the Council of Accreditation and by 2004 most of these programmes gained reaccredidation, after detailed scrutiny.

During the decade, there was a huge improvement in all aspects of Higher Education, and the strategic goal is for Cyprus to become a regional centre for quality educational services.

In the meantime, and following the lines of the Bologna Process, the State and the University of Cyprus examine the prospect of developing and establishing a Quality Assurance Body for all institutions of HE, state or private.

Gemma Rauret

A Step by Step Approach for Quality Assurance Demonstration in HE.

Quality Assurance in HE is based on three pillars: documented policy and procedures for quality assurance of institutions and programmes, quality assessment of institution or programmes, and continuous improvement. To demonstrate quality, not only is internal assessment needed but an external one as well, carried out by an independent organism who is responsible for publishing a report or for granting a stamp recognising the fulfilment of a previously established set of quality standards.

To reach successfully the quality objectives, a set of tools is required, including the following: an adequate structure inside the universities, quality culture development, an external independent body, assessment methodology, internationally recognised set of standards and policy initiatives at administration and university level to make easy the achievement of quality goals and, consequently, to be able to demonstrate them.

Policy initiatives, directed towards an adequate context-building, must be implemented step by step to support quality culture increase. Such initiatives must come from the administration or from universities. Best results are obtained when both actors promote complementary activities such as: to foster quality assurance units inside universities and quality assurance agencies, to promote internal quality assurance and external assessment, to promote innovative teaching projects or motivate teaching evaluation systems and to establish stimulating assessment consequences. The step by step approach deals with linking together these internal and external quality assurance initiatives. In the round table, some of these policy initiatives will be presented.

Eugene Rice

"The Collegial Culture versus the Managerial Culture: Sources of Resistance to the Accreditation Process"

This presentation will focus on the growing tension between the collegial culture and the managerial culture in universities in the United States. The argument will be made that in the U.S. experience both accreditation and assessment efforts were originally grounded in the collegial commitment to professional integrity. I will also contend that the future vitality of academic institutions will depend in large major on our ability to build on the strengths of both cultures and our willingness to move toward greater collaboration. The present tension between the two (both in the U.S. and elsewhere) is a serious liability in the quest for quality--both intellectually and procedurally.

Christian Thune 

“Agreed European Standards and Guidelines for Quality Assurance”

The communiqué from the Berlin meeting 21 September 2003 of the Ministers of the Bologna Process signatory states invited ENQA through its members, in cooperation with the European University Association (EUA), the European Association of Institutions in Higher Education (EURASHE), and the National Unions of Students in Europe (ESIB), to explore ways of ensuring an adequate peer review system for quality assurance and/or accreditation agencies or bodies, and to report back through the Bologna Follow-Up Group to Ministers in 2005. The Ministers also asked ENQA similarly to develop an agreed set of standards, procedures and guidelines on quality assurance.

ENQA fulfilled the mandate and submitted in February 2005 its report on standards and guidelines for quality assurance in the European Higher Education Area to the Bologna Follow-Up Group with the endorsement of EUA, EURASHE and ESIB.

The main results of the report are:

  • There will be European standards for internal and external quality assurance, and for external quality assurance agencies.
  • European quality assurance agencies will be expected to submit themselves to a cyclical review within five years;
  • There will be an emphasis on subsidiarity, with reviews being undertaken nationally where possible;
  • A European register of quality assurance agencies operating in Europe will be produced;
  • A European Register Committee will act as a gatekeeper for the inclusion of agencies in the register;
  • A European Consultative Forum on Quality Assurance will be established.

The report also makes it clear that its submission should not be viewed as the end of the project. What has been set in motion by the Berlin mandate will need continuing maintenance and coaxing if it is to provide the fully functioning European dimension of quality assurance for the European Higher Education Area. However, a European Higher Education Area with strong, autonomous and effective higher education institutions, a keen sense of the importance of quality and standards, good peer reviews, credible quality assurance agencies, an effective register and increased co-operation with other stakeholders, such as employers, is now possible and the proposals contained in the report will go a long way towards making that vision a reality.

Mark Wilson

“Assessment - A key aspect of teaching and learning”

A recent National Research Council Committee report, "Knowing what students know", has outlined the crucial role that assessment plays in teaching and learning.  This presentation will review the background to the report, give an overview of the Committee's survey of the current state of the scientific foundations of assessment, and discuss implications of the Committee's views for assessment in higher education. These points will be illustrated with examples from work on an assessment project in higher education that the presenter is currently engaged in.

Erno Zalai

“Some Aspects of the ‘ Bologna’ Transition and the Quality in Higher Education in Hungary”.

Education in general, and higher education in particular, retained its traditionally high social prestige in Hungary during the communist/socialist period. It did not, however, go through the salient structural changes that took place in the more developed Western world. A relatively small number of students were admitted into higher education, it retained its traditional institutional and degree structure, and it was not given enough resources for maintenance and modernization.

After the collapse of the communist system, higher education became exposed to the opportunities and challenges of the transition. New programs, curricula and institutions appeared, especially in the area of social and economic sciences, the number of students more than doubled in less than a decade, transitional international financial support and exchange opportunities opened up for the universities and colleges, due to Tempus, US Aid and other projects.

At the same time, higher education suffered from serious governmental mismanagement (forced institutional integration, continuous reduction in governmental financial support) and the lack of clear educational policy, in general. Also, as a result of the appearance of financially more rewarding positions abroad, as well as of business or political positions at home, many left the academic institutions, and in most areas it became difficult to maintain and recruit staff properly qualified for research and teaching. After the first decade of confused changes, higher education had to face a new challenge posed by the ill-prepared and centrally dictated transition to a new, two(three)-cycle degree system implied by the so-called Bologna Declaration.

The Hungarian Accreditation Committee, formally established in 1993, started and continued its operation among several and serious challenges. The HAC is an independent professional organization and an independent legal entity responsible for evaluating and accrediting the quality of teaching and research at higher education institutions in Hungary, by stating its opinion to the Minister of Education. Since October 2000, Hungarian higher education institutions have submitting their annual internal quality reports to the HAC, which form the basis for their future accreditation. The HAC accredits both programs and institutions, and is recently engaged in evaluating several hundreds of new bachelor programs.

Some issues related to the above changes and the role of the HAC will be presented and discussed.

Robert Zemsky

Markets and Quality in Higher Education: A Closing Comment

Discussions of quality have become ubiquitous across higher education. In nearly every country, there are new demands that higher education demonstrate its commitment to quality. Outside the United States, Quality Audits have become an increasingly standard procedure used to evaluate and fund universities. And even in the United States, William Massy is having some success in persuading the faculty of one big state university (the University of Missouri) to participate in an American style academic audit.

The other ubiquitous force in higher education today is the market. Not just in the United States, but across the globe, governmental agencies are turning to the market to raise the necessary funds to support their universities. The question this presentation addresses is whether quality and markets can co-exist — or, even better, can market forces play a significant role in determining a university’s real as well as perceived quality? What would have to happen for markets to play this kind of role? And how likely are universities in general and their faculty in particular to resist letting the market be a determinant of quality?

Designed by EWORX