It turns out that a certain exhaust was made in the puddle.
Those who received the status in 1994-1997 are the first to fall under the rink. They will take the exam at the National Agency for Quality Assurance in Higher Education (NAME).
In April 2018, the rectors of these free economic zones received a letter of happiness from the Ministry of Education and Science with the task to prepare and fill out the appropriate temporary form. (Osvita.ua shared it on its website). At that time the NAME did not exist and it somehow did not go. Now there is a NAME, and there are reports on its website.
However, there are no evaluation rules yet. The project is there, and it was even posted on the MES website and discussed, but there are no rules.
According to the analytical title NAME, the process of creating clear “rules of the game” for assigning, confirming and revoking the status of “national” higher education institutions has not yet been completed. However, in order to implement the Resolution of the Cabinet of Ministers of November 22, 2017 # 912, in 2019 NAME for the first time organized a collection of annual reports on the criteria for granting and confirming the status of national higher education institution, and forms of self-analysis of criteria for granting and confirming national status (for the formation of the monitoring case) “.
Sounds like a good thing. And we should pay tribute to the authors of the analytical report that they worked well. But, as usual, there is a lot of “but”. Have thousands of man-hours been spent idle and tons of paper recycled?
What’s wrong with the methodology?
And not so that they are made under the wrong methodology. It is not that this methodology has no official status while remaining a project.
Universities are invited to fill in the relevant tables and calculate 14 comparative indicators. We will not waste time on everything. Let us dwell only on some examples of a naumetric nature.
Take Indicator 8. This is the hospital average. To assess the scientist’s activity, the Hirsch index is used (according to Wikipedia, if a scientist is the author of five publications, three of which are cited three times, and the other two – once, then his h-index is 3. If the scientist is the author of n publications, one of which is cited 100 times, and the other four – once, then its h-index is 1).
So, Indicator 8 is calculated as follows: we take all NPNP of the university from Scopus and WoS databases, sum their Hirsch indices and divide by the number of all university researchers, ie including those who have never been published in journals from these databases and have no index at all.
What does Indicator 8 show? What conclusions can be drawn from it? How will we treat the patient?
Figure 9 is relatively simple and less strange. This is the number of NSAIDs that have at least five publications in journals from Scopus or Web of Science, other scientometric databases recognized by the Ministry of Education and Science, divided by the total number of NSAIDs of the university. The figure may be less strange, but one that allows a creative approach to its calculation. A researcher can have the same number of publications in both a journal with Scopus and a journal with WoS (in addition, there are journals that are present in both databases at the same time). So why not count it twice.
Fantasy can also spread due to the creation of the Ministry of Education and Science – “Other scientometric bases”, because no one knows what they are. Well, and another question: why multiply the value obtained by 100 scientists? In order that the received number was not too small and did not blind the eye? Then it should be multiplied by 1000.
By the way, as an example and a reason for reflection. Scientists of KNU. In total, about 16,000 articles were published in Taras Shevchenko’s journals with Scopus. Seven researchers from the university have 2,000 publications in this database of scientific content. Hospital average again?
Indicator 10 seems to be correct, but … Why the number of journals should be correlated with the number of specialties from which higher education students are trained at the university. Again, what other scientometric bases are in question? And why with a non-zero coefficient of influence, if the very fact of including a journal for indexing in Scopus or Web of Science is already an achievement.
And what is the difference between a little non-zero and zero? Let’s then take into account the quarter of the magazine in the database. And what is the coefficient of influence in general? And what does “issued by a higher education institution” mean? That is, the university may not be the founder, may not be related to the editorial board of the journal, but perform the role of publisher and have indicators? How many questions and where are the answers? Apparently, they were especially inquisitive by telephone from the Ministry of Education and Science.
What’s wrong with the reports?
Of course, not everyone should be under one comb, but there were creative people among those who prepared the relevant reports at universities. Unfortunately, in the NAME, during the preparation of their report, this work was not noticed, although they indicate that the basic verification of data was carried out. And she gave just glaring results.
The National University of Life and Environmental Sciences of Ukraine approached the calculation of the mentioned indicators 8, 9 and 10 most creatively. Thus, they have the highest rate in the mid-hospital Hirsch index – 2.26.
This is despite the fact that in Kharkiv National University named after Karazin – 1.53, Chernivtsi National University named after Fedkovych – 1.23, Kyiv National University named after Shevchenko – 1.64 …
You can, of course, use the services of the Scopus itself and try to establish the overall picture, but it makes no sense. This will be the same waste of time as when creating reports by universities. But it is enough, for example, to go to the site of such a project as “Bibliometrics of Ukrainian Science” and see that the leaders in the number of profiles in Scopus are the National Technical University “Kharkiv Polytechnic Institute” (381), Kharkiv National University of Radio Electronics (245), Uzhgorod National University (240), National Technical University of Ukraine “Kyiv Polytechnic Institute named after Igor Sikorsky” (228) … It seems that they had the best chance to become leaders here.
According to the number of scientists with the number of decent publications, more than five of them (in NULES) the indicator is 56.47, and in the mentioned colleagues – 12.91; 2.88 and 3.15, respectively.
The leader in this was the Bogomolets National Medical University, which showed a score of 83.65. How they did it did not surprise anyone. In second place – Lutsk National Technical University – 79.31. Also, isn’t that weird?
As for magazines, NULES does not lag behind here either. This figure is equal to 0.53. And this is the second place. And this despite the fact that almost all are zero or close to zero. In the first place National Technical University “Dnieper Polytechnic” with a score of 0.92. Here, as Stanislavsky would say, I believe. But why not check.
Sad zeros for their own journals in Scopus and Web of Science are demonstrated by Kyiv National Shevchenko University, Dnipro National University named after Honchar, Bogomolets National Medical University, etc.
At the same time, the indicator for such journals at Dluzhenko Glukhiv National Pedagogical University is 0.11, Kyiv National Linguistic University has 0.33, Kyiv National University of Trade and Economics – 0.32, Luhansk Shevchenko National University – 0.14, Lviv National Academy of Arts – 0.14 , Mykolayiv National Agrarian University – 0.11, Sumy National Agrarian University – 0.24 (?).
It is impossible not to pay attention to the National Academy of Management of Culture and Arts. According to the average index, the Worst Academy shows a result of 1.3, according to the number of scientists with more than five publications – 11.8, according to journals – 0.1. Here, a journal that was in the Web of Science, no doubt, led them to the leaders.
And as good in the reports as you like. There would be a desire and time to look.
The title in the analytical report states that scientometric indicators did not have such an impact on the overall result of the free economic zone as the number of foreign students and intellectual property rights. They didn’t, but it’s a matter of principle.
As a result, we have reports and indicators that cannot be used for anything. Unless the universities have gained some experience. But what conclusions will they draw from it? Especially those who believed in the project and honestly (as far as possible with such a methodology) processed all the data and (!) Found themselves among the outsiders. Now we have a reputation tarnished by comparative data, because who will check and analyze.
And who is to blame – universities or methodology developers? The question is rhetorical, although in the NAME report we read the following: “Assessment of compliance of free economic zones with the criteria of national status is carried out on the basis of data provided by the institutions themselves, according to the form of self- assessment.
This approach contains a number of risks in terms of data quality and reliability, as it fully assigns the task of entering information to users by national freelancers.
It turns out that a certain exhaust was made in the puddle. And characteristically, there was no noise from national educational institutions. You wrote nonsense to us, we made alaverdi. Now we will wonder whether someone will be deprived of this high status? What do you think? And what to do?
Everything is simple. Use the experience you have just gained. Create a small group of methodology developers (let its composition be known to the scientific community). Be sure to include those familiar with scientometrics. Elaborate, make changes in regulations. Get rid of paper reports. Get rid of unnecessary indicators and texts. Make appropriate electronic forms to be filled in by universities on the MES website. And all this is urgent.
And to gain the strength of will and spirit of state and educational leaders and deprive the status of the national half of the Free Economic Zone. And even better – remove it from everyone and start with a clean slate.
Sergey Kozmenko, Doctor of Economics, Professor
Lyudmila Ostapenko, Candidate of Economic Sciences for Osvita.ua
Scopus ratinguniversities of Ukraine
There is another panic among researchers. The reason – the order “About publication of results of dissertations on reception of scientific degrees of the doctor and the candidate of sciences”
Dissertation panic: what after it?
The order stipulates that at least 20/5 of publications in scientific (in particular electronic) professional publications of Ukraine and other countries must be available on the topic of the dissertation for the degree of doctor / candidate.