How a Former Rector of the Kharkiv University of Radioelectronics Became a Scourge for Students

On May 21, seven thousand senior students of Ukrainian medical universities have taken the test “KROK-2” (“STEP-2”) to verify their professional knowledge. Students have been taking this examination for 20 consecutive years already; but they clearly disliked the 2019 version of the test: the questionnaire included a number of very difficult questions.

During the next few weeks students picketed the Ministry of Health, wrote petitions to the President of Ukraine and actively discussed on Telegram Channel Eduard Rubin, the newly appointed chief of the Testing Center at the Ministry of Health. In the meantime, Rubin kept reading the discussion in which 3,500 people took part and tried to figure out how to deal with that anxious young crowd.

“We weren’t taught those things!” the students complained in the chat. Allegedly, the questions were borrowed from an American IFOM test for students that the Ministry of Health was trying to forcibly impose on the students (VoxUkraine presented the details of that story in the article “Don’t Want to Study”).

Eduard Rubin, former rector of the Kharkiv University of Radioelectronics and co-owner of the Kharkiv IT company “Telesense,” became the director of the Testing Center at the Ministry of Health just a couple of weeks before the beginning of the student unrest. “He hardly knows anything about medicine! He is in no position to test our knowledge!” were the main reservations about Rubin (apart from those concerning the difficulty of examination questions). The students’ greatest fear was that they would not be able to pass the test, while successful passing is a prerequisite for obtaining a doctor’s certificate.

On June 5, the Testing Center announced the results of the examination: a mere 6.8 % of the students failed to pass it; questions that proved to be too hard to answer were excluded from the assessment. The wave of indignation subsided. And Rubin continued to ponder on ways to introduce the American test in a situation when students are categorically opposed to it.

Have you already been appointed as a fully empowered Director of the Testing Center, or is a competition yet to be held?

I am an acting Director; a competition will follow in which I can participate. And that’s what I will do.

What is the Testing Center like right now; and what are the primary tasks entrusted to you?

The Center keeps developing at the same rate as it did 20 years ago – it still publishes booklets with tests, sends them via courier mail to universities for conducting examinations; such obsolete technologies provide a fertile ground for corruption at all stages of the examinations. Such a situation can be changed by modern technologies – online testing using artificial intelligence for identification and also creation of an electronic National Bank of Tests. This will make it possible to eliminate the human factor in conducting state examinations.

When I went to work at the Center, I was fully aware that many would be against me. For example, the former director (Iryna Bulakh, whose contract expired in 2019 – ed. note). Even though we talked to each other on the telephone and exchanged letters – I explained to her that I had not been after her job, that it was a state institution rather that her own private company… And I came not to steal anything but to run a state organization. I do not conceal my income – and the declaration submitted by me as a public person is absolutely open.

Anyway, I don’t think there was any kind of corruption in the Center, nor do I have any information to that effect. Bulakh rode subway trains; and I can see that the employees are people of modest means – no one has jeeps or Mercedeses.

What are your primary tasks?

I came to conduct, for the first time in Ukraine, an international examination in the foundations of medicine, IFOM, this is the trademark of the American testing center NBME which exists for more than one hundred years and is recognized as a worldwide leader. As I understand, the former Director did not want to do that – or she thought she wouldn’t be able to. That’s her mentality, “Let us change everything fast. But do it slowly.”

Suprun – she is a different kind of person and she won’t wait; she moves on, leaving behind those who do not want to go with the same speed. For a reformer, this is good and proper, but for many people it’s bad. They are not ready to live at such a pace. I communicated with doctors who understand that this is necessary – a majority of them feel that way. I support it (the medical reform – ed. note) myself – I lived in Europe for quite a while, I understand how medical insurance works. But many doctors do not understand it and do not want to accept the reforms. Because of age or mentality, they are not ready to get retrained, to “drive at full speed”; and some see the danger of the reform in that they will no longer be able to capitalize on the problems of medicine; in fact such people are the source of sabotage (of the medical reform – ed. note) at the bottom level. We just need to live it out, to brave through this transitional period. And to hold on.

My main task for the next few months is to conduct KROK and IFOM properly.

So how did KROK-2 in May go?

Excellent. What really matters here, however, is not whether it was good or bad but how the community (of students – ed. note) perceived the examination that was conducted.

The test was not passed by 6.8% of the students (they failed to score 60.5% out of 100%). Those were 480 persons out of a little more than seven thousand. It’s the result of “soft” counting. If, however, rigid criteria had been applied, like they do it in the USA, then 17% would have failed to pass it. But we realized that the figure would cause a social uproar, since in many aspects it was the fault not only of the students but of the entire education system. Our task is not to break down the system in general but, within several years, with the universities, the Ministry and the Center moving in parallel steps, to change the curricula and the attitude of all the parties involved towards education in a way that will make them competitive. So that students not only from Asia and Africa, but from Europe as well come to study here.

Students say that they are ready to pass the American exam. But they haven’t been trained to pass it.

Well, excuse me, the standards are set by the Ministry of Health; the Testing Center is to develop tests matching those standards. Therefore, the Testing Center must not be guided by what is taught at institutes of higher education; instead, institutes of higher education must be guided by education standards.

Some three years ago the Ministry of Education kept creating standard curricula in which strict recommendations were made. Then the law on higher education came into force, providing for the autonomy of institutes of higher education in developing curricula.

For many years already, the KROK tests steadily continue to be simplified – so that it is easier for students to pass the examinations and so that the universities’ statistics are not spoiled. That is a bad tendency; so now we must change fast, increasing the demands on the level of curricula and on the teaching methods.

What is the meaning of “rigid criteria” in testing results evaluation?

We chose the most difficult questions that were answered by less than 25% of the students – a total of 18 questions – and asked experts to review them. The experts acknowledged that three questions were really hard. We also sent those questions to the universities; they found 15 questions difficult and retained three. It turned out that at least 15 questions from a test consisting of 200 had to be dismissed. Also, when calculating the results we took into account the measurement error and eventually got a failure rate of 6.8%. And that figure is not much higher than it was last year. Then, 4.8% of the students failed to pass the test.

Currently, we are analyzing all the answers; and the most important thing to do is to send them to the universities and indicate the problematic points, so that they will focus on improving students’ knowledge in that aspect.

Students were probably most indignant about the fact that those difficult questions were taken from the American IFOM test. Later, they found them in the Internet, in a generally accessible database. Is that so?

One or two questions were similar. They are really similar in essence to American ones. The reason is that the Ukrainian authors of the test items and the experts must bring our KROK tests closer to the American ones – so that the departments and the students will gradually begin to study medicine more deeply, in a higher quality way, closer to Western education standards.

How do the authors for the tests get chosen – and how do they create the database of questions?

The authors are teachers from institutes of higher education, as a rule, professors and associate professors, who practice medicine in parallel with teaching it. We do not disclose their names, so as to avoid pressure on them. The database is updated each year – it is supplemented with new questions while some old ones are removed.

Students don’t like it that in the past the list of authors was open but it is no longer available. They cannot find out who compiled the database.

This decision took effect from the beginning of 2019 for all examinations and is actually aimed at avoiding pressure on the authors of the test items.

But eventually nearly all of the students passed the KROK-2 test. What was the use of worrying?

I think it was a hysterical outburst from students that was heated up by some teachers. Some teachers are afraid of IFOM, because they lose control over the course of the examinations; cases are known – and unfortunately there were many of them – when it was possible to substitute an outsider for a student taking an exam, to make an arrangement with the observers, etc. IFOM will be conducted online and all data will be stored on an American server. Testing will be conducted in the language in which the student was taught at the institute of higher education – it’s Ukrainian for domestic students or Ukrainian, Russian and English for foreign students. By the way, there will be a paper version in Russian in July. All students failing to pass IFOM will be allowed a second online attempt in August.

Will that require setting up computer classes for testing?

Oh, that was one of the first things in letters addressed to me: “You have no premises for conducting the testing! There are not enough computers at the institutes of higher education.” Well, I spread the word to IT companies and they – some for money, some for free – simply knowing me and realizing the importance of the matter – adapted their classes for the testing of the medics. The examinations will be administered on a double-shift basis, so that they can be completed in no more than three days.

Did you take part in preparing the questions for the test?

No; that is a process lasting for many months – and the Director of the Center does not choose the questions to be included in the test. The experts compiling the test are good ones: it pains them to see the troubles of the state and the profession. They did what they had to do and probably went too far with the difficulty. But I will repeat myself: it’s not a matter of difficulty. It’s a matter of quality of education.

Why do students fill out the tests with a pencil?

Using a pencil with an eraser is standard international practice allowing the students to correct their own mistakes during the test. The Testing Center has been using this practice from the beginning of its work.

But if a student is dissatisfied with the testing result (as a rule, those are students that failed to pass it), he/she can appeal against it and then the examination paper will be given to the student for verification of the correctness of the calculated score.

An appeal does not mean that the exam can be retaken; an appeal means comparison of the student’s answers form with the form-processing protocol on the basis of which the score is calculated.

Abroad, students can take a look at their scanned works and check once again what they did right and what they did wrong…

Compiling these test items is a huge amount of work. And if they are spread over the network, everything will again turn into rote learning.

Therefore, if I become a fully empowered Director of the Center, we will put everything into the computer. We will create a large database which will randomly select questions for a student, for example 200 out of 2,000.

The database – will it be IFOM or KROK? Or both at once?

Our maximum goal is not to introduce IFOM forever but to bring our Ukrainian tests up to the world level.

The most important thing that I want to introduce is online testing. That is, to get rid of all those logistics involving pencil, notebook and so on. And use artificial intelligence which will determine that the individual is alone in the premises and which can see whether that person is cheating. Also, it is necessary to start creating a training system. One relying on a clear principle.

For us, it is now important to change the test development methodology and engage more authors and experts.

So students think that KROK-2 was too difficult, don’t they? It takes 8 hours to pass the American test. There are seven 45-minute blocks and 10-minute intervals. That is, you keep sitting and writing all day long. That is the kind of studying required to become a doctor. A doctor must be able, among other things, to keep standing for 4 hours while performing surgery, when there is no time for a smoking break.

There is a fundamental problem in Ukraine – students are separated from sources of quality knowledge. Teachers don’t know anything; doctors prescribe pseudo medicines; and you are given a test in which you must provide correct answers according to American standards. How can students study in such conditions?

They must demand that their teachers teach them properly. When I was acting rector at the Kharkiv University of Radioelectronics, students would come up to me and complain about the quality of lectures. I would say to them, “Go to the department and tell them: We demand that the curriculum be changed to a modern one; we’ve had enough of being taught old things.”

But how can students understand what kind of teaching is the right one?

I don’t come from medicine; I come from IT – but in our sphere third-year students already understand everything clearly. Teachers’ failure to keep up with the new technologies is their problem. Back in KNURE (Kharkiv National University of Radioelectronics – ed. note), I linked an IT company to each department and the heads of the departments began to work.

Some also say that American manuals are expensive.

If you want to study, you will find the opportunity. For example Coursera provides free courses if you have no money. Moreover, there are grants. Come to think of it, free information can be downloaded using Google.

Will IFOM be conducted this year?

Yes, on July 1-3. And it will be fine if students score at least 30% out of 100% in that test. Those exceeding the American criterion will receive an American certificate along with a Ukrainian one. Those reaching the criterion specified by the Ministry of Health will only receive the Ukrainian certificate.

In March, experimental testing was conducted; now the results are calculated and the value to which the limit must be reduced will be determined, so that students of our level can pass it. It is up to the Ministry of Health to draw conclusions.

What are your other tasks at the Training Center?

I don’t dismiss anyone; and the staff are not going to quit their jobs in large numbers. The first day that I came here, I got them all together and told them everything about me – how many children I have, what I had done in my life. “I have told you everything about myself; and I will get to know you after talking to you. My goal is to conduct IFOM and continue to work. If you have other goals, it’s up to you to decide.” No one left; they all stayed. We go on working.

How about the Center’s technical support?

There, the technologies are old; but that is not critical. Yes, the programs were developed 15 years ago; but they are expanded, modified; they are functional. For KROK, it is necessary to count the forms and to scan them – it’s between 7 and 13 thousand forms. That’s quite a lot; but after all, it’s not External Independent Evaluation for 100,000 people. The technique works normally and, for the time being, I see no reasons to replace it.

But if I stay, then everything will develop in a different way – we will replace the program and create complete online testing.

You are a man from business who has been working in poorly paid spheres for a rather long time. First the rector of a university in Kharkiv, then the director of the Training Center. Not much of an income. What are you living on?

If it were not for my own business, it would be hard to work in such positions. When I came to run a college, I was aware that the reason why a teacher living on 3,000 hryvnias per month accepts money from students was not greed but poverty. As a university rector, I was paid 20,000 hryvnias.