JU - Still within the scope of research quality assessment, are the usual impact and relevance measures the most appropriate, considering Unicamp's mission?
Tom Joe - Not entirely, as an important part of research quality assessments has a more focused focus on productivity measured by the number of publications per capita. As previously mentioned, this does not necessarily take into account the quality of the research and the impact of its results on the communities, whether local, national or international, in which the university is located. It is worth noting that, in the context of current measures, Brazil in general and Unicamp in particular have evolved a lot in recent years, with Brazil already accounting for 2% of the total volume of articles published in the world. It is positive data, without a doubt, but it does not reflect the impact of Brazilian research on the destinations of those carried out around the world.
Another important aspect is that many of these usual impact and relevance measures are based on parameters that do not explicitly consider differences between areas of knowledge. This leads to a tendency to relegate to a position of lesser importance the areas of knowledge whose research relevance is less measurable by these models. In this way, some researchers are necessarily attracted to research themes that are not necessarily linked to the country's interests, but rather to those developed in countries with a greater scientific and technological tradition. The ranking resulting from these evaluation methodologies generates frustration in part of the academic community, as it lists some research as very important and others as not so relevant, roughly based on the citation indexes of the articles. This directly impacts, in turn, the definitions of access to research funding.
Despite an undeniable historical role in the evolution of scientific activity, in Brazil and around the world, this now classic model of research quality metrics also presents flaws and injustices. A current example was the neglect with which messenger RNA research was treated for many years. This basic research, considered unimportant and of low relevance, allowed the development of high-performance vaccines against the coronavirus, in just 10 months, and should allow the world to emerge from a pandemic that has already caused the loss of thousands of human lives. . In the near future, the results of this basic research will be used to develop treatments for numerous illnesses.
The research carried out at a quality public university, such as Unicamp, is diverse in origin: it ranges from basic science, focused on fundamental questions about our existence on the planet, to the most sophisticated research. These include those that can be used to solve specific problems in different industrial sectors; or the proposal of public policies in the areas of health, education, transport, food, security, management of urban spaces; or many other topics relevant to generating quality of life on a local, national and global scale.
Global research has evolved a lot in the last 50 years and, in general, the most relevant and impactful problems require answers that involve the involvement of many areas of knowledge. We have observed very often that the more challenging the question to be answered, the greater the level of intertwining of several different areas of knowledge, and the greater the intertwining of different areas of knowledge, the better the quality of the research results and the greater its impact. Scientific production at a public university must lead to greater knowledge of society and reflection on the varied processes that characterize it, and the appropriate metrics for its evaluation must reflect this purpose.
One of the proposals embraced by the ticket Unicamp: Building Tomorrow aims to stimulate academic interaction between the various areas of research at Unicamp. We intend to implement a research program that encourages interdisciplinarity, internationalization and that contributes to increasing the impact of research developed by Unicamp. Greater joint activity between researchers from different units will naturally lead to a broader understanding and mutual appreciation between research areas, expanding their horizons of action. With this, we also intend to stimulate internal discussions with a view to developing an institutional evaluation model that allows measuring the relevance and quality of research developed at Unicamp, respecting the particularities of each research area and its own social impact, and including items that allow it to be mirrored with international models. The fundamental objective is to help Unicamp maintain its leading role as a generator of knowledge and manager of scientific policy with local, national and international impact.
Mario Saad - Recently, Unicamp underwent an institutional assessment in which it was well evaluated in terms of research and innovation. The evaluation criteria were comprehensive and relevant, exposing Unicamp's clear vocation and prominence in research in different areas of knowledge. However, there are challenges to be overcome, one of which is precisely increasing Unicamp's transformative potential and the university's impact and relevance on the international scene. The part of Planes dedicated to research activities was largely directed towards the search for greater insertion. Some of the planned actions aim to meet these needs, but it is up to the new management to continually evaluate the progress of the strategies proposed in Planes and create ways to make them viable. It is also up to the new management to adapt the evaluation mechanisms and planned actions based on new ways of perceiving impact and relevance. Our commitment is precisely this. And, as previously stated, we think that a way to better assess the impact and relevance of Unicamp's research activities is to promote the visibility and extramural reach of these activities in conjunction with Extension.
Sergio Salles-Filho - To answer this question it is necessary to understand Unicamp’s mission. As explicit in our Management Program, we understand that Unicamp must evolve in a balanced and interactive way in its three constitutional missions: teaching, research and extension. Furthermore, we understand that it is not enough to state the inseparability of missions, it is necessary to practice it as an institutional policy, including all areas of knowledge.
Another foundation of our proposal is the University's two-way engagement with different segments of society. By associating these two fundamentals, we can understand that such engagement occurs primarily through the University's three missions, and then think about the implications for the practice of evaluation at Unicamp.
The first of these is the need for evaluation not to be restricted to research. In other words, if we want a balanced evolution of the three missions, we must also consider the weights of assessments and understand teaching and extension in their specificities, so that we are able to translate and measure the relevance of these activities.
Special attention is paid here to extension, an area in which there is less development in terms of indicators and evaluation methods compared to the areas of teaching and research. In other words, extension evaluation is a more recent practice and no less complex than teaching and research evaluations, as it covers a diversity of actions and relationships with different social groups and lacks comprehensive and available databases.
That said, we can say that from the perspective of institutional evaluation at Unicamp, the evaluation balance of the three missions seems clearer. From the perspective of evaluating teaching and research careers, a review is necessary that allows us to see the effects of each mission and the interaction between them. We know that original production of knowledge is essential, it remains to be seen how this is amplified and fed into training and extension actions.
The second implication of the foundations raised is the need for evaluations to consider the measurement of impacts beyond the more immediate results of their activities. If engagement with society is so dear to us, we need to understand how we are fed by society's demands and the effects of our activities on the University itself and in different social spheres.
In the previous question, this aspect was discussed for the research area. It is necessary that the evaluation of research goes beyond traditional indicators limited to the universe of science and technology (publications, citations, impact factor) and is capable of measuring the contribution of research to solving problems and proposing solutions.
The same goes for teaching and extension. In addition to accounting for actions – subjects, students, agreements, courses, services –, we need to evolve to better identify results and measure impacts.
An important aspect of the University's impact on society comes from its teaching mission. Unicamp trains almost 8 thousand students every year in technical colleges, undergraduate courses, stricto and lato sensu postgraduate programs and extension courses. The recent study carried out with graduates between 2009 and 2018 in the Institutional Assessment showed that they work in more than 400 areas of the Classification of Economic Activities (CNAE/IBGE), with a concentration in general public administration establishments in the three spheres of government and in entities higher education, thus training new professionals. The impacts of the training offered by Unicamp deserve to be better known.
An action already underway at Unicamp that goes in this direction is the mapping of graduates from technical secondary, undergraduate and postgraduate courses, with an emphasis on analyzing their professional trajectories. Expand this mapping to graduates of extension courses, understand the transformations in the routines of organizations with which Unicamp collaborates (be they public administration, private companies, non-governmental organizations), measure the contribution of the Unicamp hospital complex in facing critical health issues (as in the case of the Covid-19 pandemic) are some examples of how progress can be made in this direction and how evaluations can help Unicamp to support its planning and, not least, to establish more effective communication with the society.
A third implication of our foundations for assessment is the consideration of interactions between teaching, research and extension. Measuring the effect of research and extension activities on the quality of teaching or even how much our extension activities impact the research agenda are very promising initiatives in strengthening integration between Unicamp's missions.
Therefore, we reinforce measures of impact and relevance in evaluation practice, going beyond the research universe, so that we can move more effectively towards a single Unicamp.
