In this summary I would like to raise a few objections about the evaluation system that has been in use at the University of Iceland for a few decades now. The objections fall into four main categories. First are concerns about the role and output of Universities. Second is the problem of trying to measure the unmeasureable. Third is the increased corporatization of western universities and the fourth concerns the specifics of the Icelandic evaluation system. My conclusion is that the evaluation system used at the University of Iceland is fundamentally broken, should be disbanded and a new structure put in place to evaluate the performance of the teachers/researchers at the University.

I. Roles of Universities.

First I would like to highlight the roles of Universities in the modern age. Scholars, like our former rector Pall Skulason, have categorized three major roles for Universities. Skulason identifies the French (Napoleonic) university, a utilitarian institution aimed at serving the nation, solving problems at hand (concerning health, agriculture, industry, army) – often with top down administration, the German (Humboltian) university which is concerned with gathering knowledge for its own sake – letting basic research run free so to speak – obviously with the scholars them selves in charge of administration, and the English (Newtonian) university, aimed at providing the government with skilled personel to run an empire (administrators, officers, priests, lawyers, bankers etc) – the board of these universities obviously respond to the needs of governments.

The University of Iceland, like most other universities, tries to serve all of these functions. Most of teachers – that I have spoken with at least – conduct, knowingly or not, their research programs under the premise of the German model and explore questions out of pure academic interest. But research (discoveries and data summarized in articles) is just one of the functions of universities. Most of the faculty agree that educating students is also important, and in this regard adhere to the French or the English models. Educating undergraduate or graduate students is actually the main function of universities. Thus our service to society, industry, mankind is both through the pursuit of knowledge and educating people. A system that counts only articles (and assumes ISI rankings and impact factors (IF) capture quality of research) and at the same time ignores the educational and mentoring aspect of our work – is fundamentally flawed. Also, in small country like Iceland many university professors (or other teachers) are called upon to serve the country or help the press. Many of our geologists help with emergency planing and reactions in responses to eruptions, earthquakes and floods, talk to the press (national and international) and affected communities. Other colleagues are called upon to address and review various other issues in our society. Many of the members of the Faculty of Life and Environmental Science have been called for senate committees, work with governance, talk to the press or just curious citizens on various issues from environmental to new discoveries in genetics. Again those functions are not evaluated by the system, but they are important nevertheless. To conclude, Im not arguing that the system should be expanded to count all them pebbles, but a system that focuses almost exclusively on quantity and bibliometrics is bad.

II. We can not measure the unmeasurable.

This brings us to the second point, the functions (or “products”) of universities are hard to measure. How do we evaluate the impact of a researcher or a teacher? Should we be counting articles, using bibliometrics, counting students, the students average grades or salaries after graduation, number of academic offspring etc. And even if we agree on what to measure or count, how do we calculate an aggregate score – does one Nature publication equal one academic offspring landing a faculty position? Then are the indirect effects, the impact teachers have on undergraduate students or graduate student whose committees they serve on, the services done as reviewers, editors, vocal critics of practices in their own universities (hehe – a joke). This can be summarized in the statement, how do you measure the unmeasureable? You must acknowledge that some things can not be measured directly, and accept that we have to trust people working in Universities. It has been documented that bibliometric and rat-race encouraging evaluation systems induce stress in university staff. Treating and evaluating faculty like workers on a production line, where every action is stereotypic and measurable is a major fallacy. And it undermines the concept of free research and academic education. How are the teachers supposed to educate and expand the minds of young people if they are scrutinized at every step by a big-brother like master evaluation system.

III. Evaluation systems represent the corporate corruption of academia.

The concepts of the modern university are under threat, by capitalistic mindset and economic models. The model for distributing funds to Icelandic Universities relies on enrollment numbers (with few exceptions). There have been increased calls for developing university – industry ties, to foster innovation. This is despite quite a number of warning signs from abroad. The corruption of higher education by the capitalistic mindset is greatly summarized by Jennifer Washburn in the book University Inc. She outlines multiple cases and forms of this corruption, where departments and researchers have let money trump academic values, where conflict of interest of faculty fails puts their graduate students at risk, where tenured faculty are laid off to make room for temporary workers and where innovation becomes tangled in patent forests. The consequences is worse education of students, closed academic programs, compromised student careers and erosion of academic freedom. A system that gives direct financial gain to researchers for publishing articles and more money for more articles, is another example of economic models being used for governing universities. The extreme version could be a Chinese University (I remember reading about but cant name) where faculty got $10.000 for a published ISI article, but $100.000 for high impact article (Science or Nature). Direct financial incentives for publishing puts academics in an uncomfortable position, particularly if they are not paid well to begin with (which is certainly the case in Iceland and possibly China). Im not stating that such incentives will generate scientific fraud, but the fact is that we are all humans and our values are shaped by intrinsic properties, our upbringing and the environment we work in. Barbara Redman has pointed out that the rise in academic misconduct, particularly in the biomedical sciences, may be a consequence of the highly competitive atmosphere in those fields. Thus factors that change the academic environment from a collegial, community oriented place, to a cut-throat, dog-eat-dog competition between Universities, groups, PI’s and even students within groups, is bound to lead to more people cutting corners and do sloppy if not outright fraudulent science. In the context of education, adoption of the corporate mindset in universities leads to people (“players”) being viewed as employers (“administrators”), employees (“teachers”) and customers (“students”). This is in stark contrast to the peer-run German style University, where the academics decide together. And more importantly it views education differently from the more classical mentor – disciple arrangement, or where the bachelor student explores a field or questions of interest through interactions with many teachers. This shift in the view of people in universities is not in cultural isolation. American bachelor students now view their education more in terms of monetary value, education being something you buy, in order to get a better life or salary (pay for degree for with tuition). In the middle part of last century, students went to university to search for meaning for themselves, learn about the world, explore the dimensions of the human spirit and knowledge. Monetary gain was secondary aim. The reasons for this are multiple, from general mood of western society (from community spirit to individualism), perhaps consequences of economic forces on a global scale, to policies enacted in the western Universities. In my mind this path is a dangerous one, and will erode the education the present and future generations will get. The greatest achievements of the human species are have been the result of cooperation, exchanges of ideas and goods, and communal spirit. I understand that administrators like numbers to estimate the output of the University in order to be able to compare it with other institutions. The endless competition among Universities, to satisfy the criteria of the various companies compiling lists, Times higher education, US news and report, QS, Shanghai Ranking, etc is also detrimental. It also generates the wrong incentives for institutions of higher learning. They start competing like sports clubs for talent (researchers), donors (grants or philanthropists) or bright/rich (students) etc. At the University of Iceland, researchers at the Heart association and Decode genetics were handed symbolic professorships, mainly to bolster the estimated research output of the University of Iceland. This seems to have paid of because couple of years ago, the University landed on the Times higher education list, in part due to the decode articles and high number of citations they draw. The Division of science at the University will deny this, but Kari Stefansson the CEO bragged in an interview about Decode pulling the University of Iceland into top 300. In my opinion Universities should not be run in response to external and quite artificial metrics designed by these companies. In sum, the evaluation system at the University of Iceland is another manifestation of the corporate corruption of the modern university, which seriously undermines the practice of science and may eventually harm the reputation of science among the public. If that becomes the case, then western societies may be heading back to the dark ages.

IV. The ultra individualized evaluation system at the University of Iceland.

Our fourth concern is with the specifics of the evaluation system at the University of Iceland. This section will be brief as my colleagues in experimental biology and I have written about the weaknesses and impact of the Icelandic evaluation system, see letter from Petur Petersen and colleagues accompanying the report. Here I iterate or elaborate on a few points.

The Icelandic evaluation systems started out as a bonus system to settle a salary dispute. The government could not raise salaries for the University teachers (because then other unions would call for similar raises), and the solution was to put money into this bonus system. The proponents of the system, and some of my colleagues in the social sciences have said the system was god-sent, because it encouraged researchers to publish internationally and stirred some sleeping dinosaurs into action. In my mind, this system was not necessary for that. Yearly individual interviews with researchers, outlining progress in teaching, research, societal impact, would achieved the same result. The general objections to the system are the following. 1) A system that measures everybody with the same metric, is unjust. Fields vary in tradition of publication and the amount of work needed for a good study (publication). 2) We humans respond to incentives, the metrics used by a systems will influence behavior of the academics, for instance lead them to neglect their teaching at the expense of research. 3) The system can not measure excellence, only general output. Thus it should not be used to rank researchers – if anything a bean-counting-device like this system can be used to see if people are inactive in research (however usually – those individuals are known to their peers and department head). 4) The system started out as bonus system, but has now been expanded as a general tool for distributing goods to individuals within the University. It has become all over-reaching and infiltrates all major decisions. 5) System of this kind generates friction between researchers and departments/disciplines. There are noticeable frictions within the University because the system is currently used to distribute research funds, and more importantly funds for PhD students. The system favors certain researchers who follow a high output model, some sort of factory belt science, where multiple similar or nearly identical papers are produced, where only one small thing is changed. I will state this here openly though I know those individuals will not be pleased (which is an example of the friction the system generates) – where the same survey is done in different years, but each published as a separate unit, or where the same analyzes are done on 10 related compounds and each published as a separate paper. The University administrators, the division of science and innovation in particular, have implemented and polished the system. They are reluctant to drop their baby, which in an understandable sentiment. But it must be stressed that all university systems should serve the greater society and meet the functions and roles of the university. Every call to reform the system has been met with resistance by the administration, at several different levels. The current review is the first positive step in this direction, though the wording of the document suggest strong influence of the values and stance of the current University administration.

V. Possible solutions to the problem.

Certainly it is harder to solve a problem than describing it. My view on this system and its implications have evolved in the 9 years I have worked at the University. Now I am firmly against this system, and bibliometric systems in general, on the grounds outlined above. I think the system has to be abandoned. I know this involves renegotiating the salary agreements of the government and the professors and teachers unions. An interim step might be however to disentangle the system from distribution of goods within the university to minimize its detrimental effects. A future solution could be a two tiered evaluation of the overall performance of each teacher. First, there should be yearly interviews with the department head where the teacher outlines her/his focus in teaching, research and mentoring of graduate students, and in the relevant cases community/societal outreach and/or innovation. This interview needs to hit all of those posts, as our jobs are multifaceted. The second tier would be external evaluation, done at the department level every 4-6 years where each faculty has to put together written summaries of their work (in the same categories as above). International committee would then be brought in to evaluate each department. For the research this would not list all papers, but highlight 1-5 major outputs or discoveries. This allows the faculty to focus their work on major topics, and not have to worry about publishing “bread-and-butter” publications and allow them to focus on projects of substance. This would also allow faculty to review their future education and research aims and discus and debate them with capable minds. Such an arrangement is inspired in part by the changes implemented by the University of Utrecht, see for instance piece by Benedictus, Miedema and Ferguson in Nature 2016.

To summarize, Iceland has developed a hugely biased and wide reaching individual based evaluation system that threatens the core values of academic education in the country. The debate about the system, its functions and flaws has been largely unstructured and in some cases discouraged by the higher levels, with all changes in the system being in the direction of extending its reach. Thus, the input of the external professionals is extremely important, and we sincerely hope they can offer concrete suggestions on how to unravel the mess.