How the Ministry of Healthcare is Trying to Make Ukrainian Medical Students Take American Exams

The screen shows a 20-something boy walking briskly along the halls of a Soviet-era building, putting on a white doctor’s coat as he goes. We hear solemn music. “IFOM, International Foundations of Medicine. It is an exam developed by an exam board from the United States. <…> This now concerns us, third year students, as well,” says the off-screen voice.

The young man comes out into the yard, a neat line of young medical students in white coats behind him. “But are we ready for this?” the protagonist asks the viewers. “Does the curriculum for the 1-3 year of studies correspond to that in the US?” The video clip ends with an appeal to the Ministry of Healthcare to review the conditions of the new exam.

In six months after being uploaded to YouTube, this inconspicuous 2-minute long video clip, called #stopIFOM, gathered 50,000 views. Not bad for a channel with 155 subscribers and a single video.

Over the past year, the Ministry of Healthcare was trying to implement obligatory IFOM exam in all Ukrainian medical colleges. For over than 90,000 Ukrainian medical students and 19 medical colleges this novelty may become a professional verdict.

No “false medicines”

Deputy Minister of Healthcare Oleksandr Linchevskyi is eager to talk about the healthcare reform even on his day off. A cardiothoracic surgeon himself, and currently one of the least popular MoH officials, is talking quickly and intricately.

“If we don’t conduct a medical education reform, it will discredit the transformation of medical reform in general”, he claims. According to him even if we increase salaries for doctors and supply hospitals with medications, some doctors will continue prescribing outdated and false medicines to their patients.

In MoH newspeak “false medicines” are medications with unproven effect. The research of Anti-Corruption Action Centre and the International Renaissance Foundation showed that in 2017 every fourth UAH spent by Ukrainians on medicines accounted for medications with no duly proven efficiency and safety. Top 10 most popular medicines include:

• Actovegin (made of calf blood; according to manufacturer helps after strokes, but in Ukraine it is often prescribed to pregnant women)
• Heptral (amino acid, according to the manufacturer it restores liver)
• Rheosorbilact (a mix of Sorbitol and sodium chloride; according to the manufacturer it is used to improve capillary blood flow)
• Essentiale Forte N (phospholipids of soy beans; according to the manufacturer, also helps restore liver).

According to research conducted by AcAC and Renaissance “these medicines have no scientific foundations for clinical use that are recognized in the world”.

At the same time, in 2017 Ukrainians spent almost 1 bln UAH on these four medications. Actovegin has been in the top 3 most sold drugs for 5 years in a row now.

The popularity of false medicines among Ukrainian doctors is definite proof of poor knowledge they have received in medical college, Mr. Linchevskyi believes. “Some doctors also get 7-10% of the price of medications they have prescribed,” Viktoriya Tymoshevska, Director of Public Health programme at Renaissance says.

American or Ukrainian exam? You will have to pass both

In order to check the quality of knowledge gained by Ukrainian medical students, in 2017 MoH conducted a trial American test among 4,906 interns. Able to pass the test, i.e. get more than 70.5 points out of 100, were only 3% of Ukrainian interns. Test results only reaffirmed MoH management in their intention to introduce the American test.

The test was called The United States Medical Licensing Examination (medical licensing exam, USMLE) – an exam you need to take to get a doctor’s license in the USA. The test has been developed by two non-commercial American organizations: National Board of Medical Examiners (NBME) and Federation of State Medical Boards (FSMB). NBME is also developing and running a test similar to USMLE called International Foundations of Medicine (IFOМ). This is the exam that MoH plans to introduce in Ukraine.

Could it be that no one has been checking the knowledge of medical students before and the results of American test have become a surprise for MoH officials?

Ukrainian medical students have been passing KROK test since 1999. While studying general medicine students pass three different KROK tests: KROK 1 at the end of third year, KROK 2 during the last year of studies before they enter internship and KROK 3 when finishing internship.

KROK tests have been developed by Testing Centre affiliated with the MoH of Ukraine. It was set up 20 years ago and deals solely with tests for medical students. As the web site of the Centre says its aim is “to protect the community from unqualified specialists”.

Every year, 50-60 thousand students pass KROK 1, 2, or 3 in different areas – general medicine, dentistry, pharmacy etc. For instance, in 2018, KROK 1 in General Medical Training was passed by 7,805 Ukrainian students. 1,491 students or 19.1% did not get the passing mark (i.e. got less than 60.5 points out of 100).

The highest index of poor performance was demonstrated by International Academy of Ecology and Medicine (Kyiv): 84.4% of third year students could not pass KROK at their first try. The best results were demonstrated by Dnipro Medical Academy: poor performance index among the students was only 4.9%.

KROK is passed not only by Ukrainian students but also by the international students. The test has Russian and English-language versions. In 2018 Russian-language test was taken by 665 persons, out of whom two thirds were not able to get the passing mark. Another 3,050 students passed the English version of the test, out of whom 48.2% did not pass.

Average fail index for KROK 1 during the first try for students of all categories was almost 30% in 2018. KROK results show that medical students in Ukraine are not too keen on studying. So why does MoH believe that Ukrainian tests are not enough to check the level of knowledge of the future doctors?

For MoH, the American test is an additional opportunity to check both the students and the Testing Centre which organizes KROK tests. Besides, MoH has questions to the Centre.

Last year MoH received depersonalized data on KROK test since 2009 and transferred it for analysis to Kyiv School of Economics. Kyiv School of Economics together with VoxUkraine have analyzed the data and saw strange things on the charts. In some years (2009, 2011, 2015) the number of students who passed KROK 1 with a passing mark reached several hundreds, while those who got slightly under or over the passing mark reached up to 150 students. It is an anomaly from the point of view of statistics.

We shall explain this on an example. In 2015, around 400 students got a passing mark (60.5 points out of 100) in KROK 1. At the same time, several dozens of students got 59 points (less than passing mark) and around one hundred students got 61-62 points (more than passing mark). Similar peaks in the numbers of those who got a passing mark were also observed for KROK 2 and KROK 3 in 2016 and 2015 respectively. See the complete research at VoxUkraine web site under the link.

Former Head of Testing Centre, who was there from the start, Iryna Bulakh explains this statistical anomaly on her Facebook page. In 2015, in the heat of military conflict, students from two universities from the East of Ukraine (134 persons) were allowed to pass KROK 1 on the basis of previous year tasks. All such students got minimal passing mark, even if they in fact got more. Bulakh did not explain how many students passed the test and how many failed. She also did not explain why a similar thing was observed in other years.

The explanation may partly be found in the documents and reports of the Testing Centre. For instance, students could retake the KROK 1 test. In such a case, they are received a passing mark. In addition, the Testing Centre could add the statistical error of 0.5-2 points to marks below the passing mark (i.e. to 59 or 60). Such an approach is applied to KROK 2 tests.

“If the Testing Centre has a really reliable system of assessment which excludes human interference, how is it possible for the Centre to “add on” the points for students to reach the passing mark? Did they do it manually? Does it mean that interference is possible, after all?” Linchevskyi asks. VoxUkraine journalist was not able to get answers neither from Testing Centre nor from Iryna Bulakh.

In March, MoH ran a trial IFORM test among the medical students. Filled out tests were sent to NBME main office in the USA for assessment. It is already apparent that the test is unavoidable for all students together with KROK tests. In 2019, the Cabinet of Ministers approved the Single Qualification Exam, consisting of four components, including “international exam in fundamentals of medicine”. The fate of test will then be decided, i.e. when to introduce it into the curriculum. There is little doubt, however, that the exam will become obligatory for all students sooner or later: the Cabinet of Ministers approved a respective Decree a year ago.

Yet, there is one thing. “IFOM will potentially enhance the quality of KROK tests, if it turns out to be not enough, and then the American test may be cancelled,” Inna Sovsun assumes, who is adviser to the Minister of Healthcare and Vice-President of Kyiv School of Economics.

Students against IFOM

Sergiy Suprun, that same guy in the white coat from the YouTube video, is a student of Ternopil Medical University. The university indices are far from being the worst in the country: last year’s KROK 1 was failed by only 11% of Ukrainian-speaking students of the university and 20% of foreigners.

The student, who has the same last name as Heathcare Minister, is really against the obligatory American test. “During the first three years of studies we are getting ready for KROK during classes, we are having pre-KROKs (i.e. trials tests – editor’s note), rector’s (tests – editor’s note),” Sergiy Suprun says, “We are physically and morally ready for the test. IFOM is very hard even for our four-year students (they are planning to introduce IFOM starting from the third year – editor’s note)”.

Alina Moroz, a student of Bohomolets National University, believes that the American test does not correspond to the Ukrainian curriculum. “It has many questions in clinical medicine, while during the first three years we only study the theory! Clinical medicine issues (connected to specific disease – editor’s note) are hardly discussed,” the student says. She is convinced that we first need to change the curriculum in medical education and only then introduce new exams. “But of course, it would be nice to study according to American standards”, she summarized.

In the USA, medical students pass the USMLE exam. They pass it three times during studies. The first test is taken during the second year of studies. Both tests (the international IFOM and the American USMLE) indeed check basic medical knowledge and skills and are very similar.

Why won’t the MoH first change the curriculum and then introduce new exams? The answer to this question is available on MoH web site: “medical education institutions have complete autonomy in the development of curriculum pursuant to the Law of Ukraine On Higher Education.” It means that under the Law, university curriculum shall be based on Ministry of Education and Science standards but the curricula themselves are not approved, while compliance with MoH standards is checked only during university accreditation stage.

Sviatoslav Linnikov is one of the few who supports IFOM and who is skeptical about the Ukrainian test. He is a teaching assistant at the Department of Social Medicine, Public Health and Medical Law of Odesa National Medical University. According to him, the main principle of preparation for KROK is to “cram the tasks of the previous years because around half of them are used again in the next years.” Those who learn the tests by heart pass and it does not matter how many books they have read or how many years of practice at a clinic they have had. “This is where the main difference in the tests lies. To pass KROK you need to cram, while IFOM requires knowledge and logical thinking,” Linnikov summarizes.

A young female doctor, who refused to name herself, has a similar opinion. “Regarding the video appeal of Ternopil students, I find it disgraceful, mildly speaking,” she says.

Linchevskyi has a simple answer to all claims about IFOM: medicine is the same in the whole world and there is no specifically American or Ukrainian anatomy or principles of treatment. “The task of medical colleges is to drop those who have poor academic performance and whose low level of knowledge may harm the life and heath of patients. And they are not doing that!” Deputy-Minister expresses his indignation.

Besides, at the beginning the IFOM passing mark was not high. “The most important thing is to teach students about serious preparation for exams and the need to read Western books which will also require them to improve their English,” Linchevskyi says. What is more, IFOM will be conducted in Ukrainian. IFOM can be retaken only once just as other components of Single Qualification Exam for Medics (SQEM). See more about SQEM below.

What does MoH want? Brief reference about Single Qualification Exam for Medics

Several facts about SQEM

• To be introduced in 2019
• Designed for 3rd and 4th year students only
• Each of the four SQEM components can be retaken only once at your own cost
• Those who failed a SQEM component twice, shall be expelled from the university

Obligatory SQEM components

1. KROK test
2. Practical exam for medics
3. International exam in fundamentals of medicine
4. English language test for medics

Source: Cabinet of Ministers Ukraine, Ministry of Healthcare

What will happen to medical colleges income

Increased demands for students can seriously impact their numbers and, as result, decrease the income of medical colleges. And those are very small as compared to Western colleges.

Let us look at the financial statistics of the biggest medical colleges – Bohomolets National University In 2017 the University special fund made up 441 mln UAH – that is what the university earned by itself. Another 216 mln UAH were given by the state (page 103 of the report). As of January 01, 2018 the number of students of the university was 13, 180. Total budget of the National University per student was 49,800 UAH.

This is obviously a pittance in comparison to the budget of the largest medical university in Sweden – Karolinska, which earned 6.9 bln SEK or 20.7 bln UAH. In 2017, the number of students was 6,079 (full-time studies). Total: 3.4 mln UAH annually per student. For Swedish citizens education is free, for people from countries outside the EU it costs between 165 and 200 thousand SEK per year (or 495-600 thousand UAH). Ukrainian universities, in the meantime, charge between 13,000 UAH (part-time studies in pharmacy) and 35,000 UAH (full-time dentistry programme).

Ukrainian medical education is significantly cheaper than in the West, that is why we are having many local and international students. In academic year 2017/2018, 19 medical universities and colleges had 153,500 students. Every year 26.6 doctors per 100, 000 people graduate medical colleges (MoH data, VoxUkraine calculations), while in Sweden or in Poland the number is 10.2 and 10.5 (OECD data, 2016).

Requirements for Ukrainian applicants are yet not as high as in the Western countries. In 2017, pursuant to the Decree of MoH, passing mark when entering a medical college was increased from minimal 100 to 150. But the top result in Independent External Assessment (ZNO) is 200 points.

“To enter a medical college in Sweden, one needs to get the highest point in an exam analogous to Ukrainian ZNO,” Nataliia Levchenko, a radiologist, says, who moved from Kharkiv to Stockholm many years ago and graduated from Karolinska. If we take the Ukrainian ZNO, it would mean that when entering medical college a Swedish applicant should get 200 out 200 points. So our 150 points are drastically too low to enter any medical college in Sweden.

“Those who enter medical colleges in Sweden are geniuses and the graduates are the best of geniuses,” Ms. Levchenko says.

Will the introduction of American test change the quality of Ukrainian education after all? “The problem of Ukrainian medical education lies not only in tests. We are also talking about: a) corrupt teachers; b) teachers being absolutely indifferent about student education; c) lack of modern clinical bases for teaching medical students,” Valeriy Zukin says, Director of Nadiya clinics and Vice-President of the Ukrainian Association of Reproductive Medicine. According to him, a doctor is a “manual’ profession and even the best tests alone will not help train a good doctor.” Everywhere in the world, medical training is based not only on tests but also on formation of clinical thinking and practical skills. This must be the most important thing,” he summarizes.