Can Everything be Ranked? The Fallacies of Ranking Labour Regulations

We live in the era of rankings. However, if poorly constructed or misinterpreted, composite indicators on which international rankings are based can lead to simplistic and even wrong conclusions. Moreover, it may not be appropriate to rank such topics as labour market regulations. This post explains why justifying labour reforms with the aim of improving Ukraine’s position in aggregate rankings should be by all means avoided. Instead, labour reforms should be undertaken with the view of actual needs of Ukraine.

Rankings flourish. In fact, our age is a golden era of rankings. They exist virtually in every domain, ranging from facebook likes of personalities, youtube and instagram likes of videos and photos, restaurant and hotel tripadvisor likes, to more sophisticated rankings of democracy, freedom, corruption, or institutions, based on comprehensive composite indicators measuring these phenomena. The latter rankings and composite indicators, the “serious” ones, are created by internationally renowned institutions and think-tanks with long-lasting reputation. Creating rankings even became the main businesses for some of them. So surely they know what they do?

In my recent post [link to post1], I examined the quality of a selection of quite renowned composite indicators in the area of labour market regulations, constructed by Fraser Institute, the World Economic Forum, and the IMD, and showed that they suffer from important methodological drawbacks. Those drawbacks included arbitrary choice of components, use of questionably representative opinion surveys, trivial aggregation of de jure and de facto measures, and frequent methodological changes. I argued that these drawbacks render composite indicators data, and rankings based on them, incomparable over time. Moreover, even if one works with one year of data, depending on the composite indicator used, one may get a very different outlook of labour regulations. Indeed, if poorly constructed or misinterpreted, composite indicators and rankings can send misleading policy messages and lead to simplistic – if not altogether wrong – conclusions. The policy advice you are going to give depends – and can be manipulated by – the choice of the indicator on which this advice will be based.

But the issue with rankings is also broader than methodological technicalities. After all, those can be corrected. The real question is: can everything be ranked? To what extent is it appropriate to rank topics such as labour regulations? To what extent can we say that, if Ukraine scores “bad” on labour regulations, reforms should be justified? The question is all the more important, as Ukraine is currently preparing a new project of labour reform, one of the objectives of which is to improve its rankings on international indicators.

Before we proceed, let’s agree on something. If we ask a representative sample of citizens in several countries “How many bribes did you give this year to public authorities?”, then average the responses by country, and order countries, we will obtain a useful and genuine ranking of countries from best to worst bribing practices. Right? Based on this, there will be clearly scope for action. But is it the same with labour regulations?

It appears that even the simplest baseline indicators of labour regulations are more difficult, and not necessarily appropriate, to rank.

For example, take severance pay – a payment that a worker should obtain from the employer if dismissed. Can we say that a higher payment is better than a lower one, or vice versa? The answer is that it depends. It depends for whom: a worker, an employer, a society at large? It also depends on other regulations that are in place: if unemployment benefits exist and they are sufficiently generous, then perhaps it’s ok to have low severance, but if unemployment benefits are absent, then higher severance may be preferable. If composite indicators focus only on outcomes for employers, and do not account for such other institutions, then they represent a one-sided picture of labour market regulations.

More generally, regulations such as severance pay are put in place not only to protect workers in case of dismissal, but also to redress an unequal bargaining power that exists between a worker and an employer in an employment relationship. Thus, it is not possible to say that they represent only a cost to an employer (and hence rank countries on the basis of the cost), without recognizing their beneficial role for workers, but also – somewhat surprisingly – for employers themselves. Indeed, recent research shows that employers do have long-term productivity gains when laws encourage relationships that are more durable. Such laws help reducing transactions costs associated with recruiting frequently rotating personnel, they create incentives to invest in personnel’s’ training, stimulate in-house know-how and creativity, and hence improve firms productivity and performance.

Thus, one definitely can order countries on the amount of severance pay that workers receive in case of dismissals, but higher position in such ordering cannot be seen neither as a better, nor as a worse outcome for a country. In other words, it cannot be called a ranking. Whether severance is bad or good, in which amount, and for whom – is an empirical question, the answer to which should be established in an empirical manner, accounting for other relevant factors, but not just by pure ordering. Conversely, having some mid-range of severance pay may actually be “better” than too much or none at all.

This deliberation is not an abstract philosophical exercise. It was in fact taken quite far with the World Bank Employing Workers Indicators – which are part of Fraser, WEF, IMD, and Heritage Foundation composite indices (as shown in our recent paper with Sandrine Cazes). Originally, when the World Bank started publishing Doing Business Indicators, it included some aspects of labour regulations into overall country rankings. Those sub-indicators were aggregated into Employing Workers Index (EWI), which itself was also used to rank countries on easiness to hire and fire workers.

Based on EWI indicators, the World Bank Doing Business Report (2006) applauded Georgia’s 2006 reform, which concerned numerous aspects of labour relationships. It qualified Georgia’s reform as “the most far-reaching reform of labour regulation”, named Georgia as a “top-performer”, and praised the reform for “help[ing] workers move to better jobs” (World Bank, 2006). In reality, this reform provoked an outcry of both the national and international community, including the ILO and the European Union, as its adoption was imposed without tripartite social dialogue. The new Labour Code contravened International Labour Organisation conventions which the country had ratified, and stripped down most of the country’s provisions on worker protection issues. The years that followed were the years of close collaboration between the ILO, the Government of Georgia, and the social partners. As a result, Georgia substantially amended its Labour Code in 2013, reintroducing the omitted articles on the grounds and procedures for terminating employment legislation, and bringing it to greater compliance with International Labour Standards.

As to the World Bank EWI indicators, they received a very strong criticism from academia, civil society, and other international organizations (see notably Berg and Cazes, 2008). The criticism pointed out that the EWI is one-sided; it measured flexibility in employment regulations but excluded other key dimensions of employment policies, such as worker protection, and considered labour regulations as a pure cost, without seeing its social value. The indicator ignored institutional design features, as well as such issues as how many workers in an economy are actually concerned by the provisions, how many workers and employers comply with these provisions, and how effective the enforcement mechanisms are. Most importantly, EWI did not allow distinguishing economies that chose to have low level of formal protection while providing other adjustment mechanisms and overall protection from economies that have inadequate labour regulations.

For example, the US received the “best” place in the ranking, because the indicators showed that it virtually had no legal regulations of dismissals, and assigned “zero” costs to dismissal laws. However, the indicators did not take into account the system of anti-discriminatory laws in the US, as well as the efficient functioning of its courts, which make contested unlawful dismissals extremely costly for an employer. Such systems and laws either do not exist, or do not effectively function in other countries, including Ukraine. These countries chose statutory ways of protecting workers, the ways that were reflected in the indicators. This means that, if such other countries “relax” their protection levels to reach the levels of the US, based on the existing indicators, but at the same time do not install and enforce anti-discriminatory laws and efficient labour court systems; they will be ripped off any effective protection levels – in contrast to the US.

The criticism was so important that the World Bank EWI had to undergo a subsequent independent evaluation in 2008, an examination by a consultative group in 2011, and a review by an independent panel (World Bank, 2013). These evaluations acknowledged that the EWI contains “the problems inherent in measuring only the costs of labour-market regulation and not the benefits”, and approved the Bank’s reasoning that “a comprehensive approach in advice on labour market policies is needed” (ibid). As a result, the World Bank decided to suspend the use of EWI data in the calculation of the aggregate Ease of Doing Business indicator, cancelled the ranking of the EWI, and barred its use in formulating policy advice. The World Bank also advised its staff not to “include recommendations based on the EWI in Country Assistance Strategies / Country Partnership Strategies, Economic and Sector Work, Doing Business Reform memoranda, policy notes and other strategy or analytical work”. It further instructed its staff not to “use the EWI as a target or performance monitoring indicator when designing development policy, investment and technical assistance loans, even where the relevant project documents refer to the EWI” (World Bank, 2009).

Despite this, there is still a lack of adequate awareness about the debates and adopted changes. The WB EWI series are no longer part of Doing Business, but the data are still collected. Thus, EWI are still systematically reproduced in other datasets – Fraser Institute Economic Freedom of the World Index, IMD World Competitiveness Index, World Economic Forum Global Competitiveness Index, or Heritage Foundation Index of Economic Freedom. They are used to construct other aggregate indices and to rank countries, thus disregarding significant recommendations and the decisions taken by the World Bank itself, and perpetuating data misuse. So shall Ukraine fall into the same trap as Georgia? Or shall it adopt those reforms that it really needs, regardless of where they take it in erroneous rankings?

References