October 21, 2013
Beforehand, we addressed the confusing ETC language on indicators and argued that the context indicators are usually defined by the national statistical services and used to measure an objective to be met, a resource to be mobilized or an effect to be obtained and are highly contingent with their environment. Anheier et. al summarize this observation as a “methodological nationalism”, meaning that the indicators tend to refer to the nation state as the appropriate unit for analysis while having the tendency to ignore the specificities of areas with multi-level governance (2013, 117).
As a result, data used in the Hu-Ro Programme are mainly aggregated at national level and ignore the specific figures from the border area. This situation is caused mostly by the incapacity of the local stakeholders to produce up to date policy documents able to reflect their aspirations as organizations and to infer more operational cross-border objectives into the guidelines of the Programme. Moreover, it emphasizes the weak capacity of both Hungary and Romania to enforce the principle of subsidiary and to oppose centralistic tendencies as those inherited from the communist regime.
The first programme level indicator, the GDP in (the) eligible area, is a key economic indicator defined as the GDP/inhabitant in PPS (EU25=100), % and has the baseline of 35.7% of the EU’s average.
Here are some pitfalls for its applicability in a cross-border area.
First, only the Hungarian Central Statistical Office (2005) is mentioned as a source for the GDP figures. Apparently data were collected at national level and just from one side of the border.
Third, the Operational Programme simply adds up the GDPs of the eight border counties and then divides the total amount by two. According to this logic, the value of the GDP in the HU-RO cross-border area is 35.7% of the EU average.
Forth, there is no clear correlation and methodology for horizontal measurements to connect the GDP indicators with the other two economic indicators Gross added value by sources – agriculture, industrial sector, services – calculated at the price level from 2003 (OP: 2007, 112) and FDI/inhabitant, having the values from 2003/2004. Calculations behind these indicators are inaccurate and they ignore, for example, the level of investments having a cross-border impact.
Uncertainties linger also on the procedures used to collect data for the analysis of the target area. Some of the conclusions of the OP are backed by data from the end of the 90’s and do not account for the specificities of a border region, such as different types of stakeholders, local strategies or EU Regulations. These conclusions are then use to seemingly forecast the evolution of the border area for the next 10 years.
Nonetheless, no clear re-evaluation stages are set up in order to account for changes such as the EU accession, the 2008 financial crises that affected the world’s economies, the adoption of a new EU Treaty in 2009 or the failure of the Lisbon Agenda in 2010. Without these basic instruments capable to ensure a flexible measurement system, the GDP indicator has no capacity to reflect the changes taking place in the border region generated by the implementation of the Programme.
Good governance is directly related with risk assessment, exit options strategy and accountability, things that are hardly identifiable as being part of the Programmes monitoring and evaluation system. Under these circumstances, we can further dispute the capacity of the GDP indicator to comply with the provisions of articles 6 and 19th of the EU Regulation 1080/2006 on the need for joint cross-border, economic, social and environmental strategies for sustainable territorial development. To a certain extent, we can also question the relevance of the GDP indicators towards the general objective of the Programme and its capacity to reflect strategic improvements on the competitiveness and attractiveness of the border area. For example, the World Bank and OECD are usually using this indicator together with other similar indicators as measurement units for the economic performance of the developing countries (WB: 2012, 38).
Therefore, despite the fact that the GDP indicator was most probably added to guide the Programme according to the convergence objective of the Cohesion Policy – to improve conditions for growth and employment in regions where per capita GDP is below 75 % of the European average – in the absence of a sound methodology for measurement, feedback and correction it is difficult to understand both its relevance and utility for the border area and its overall contribution to the EU territorial cooperation. For example, it is not clear why a target value for the GDP in 2013 was not included in the OP or why the interim evaluations of the Programme omit to address drawbacks in institutional cooperation, unfavourable legislation or system failures in data collection.
In the absence of such instruments there is no transparency and flexibility to successfully adapt the Programme to the changes taking place in border areas over a period of seven years. Furthermore, the target area is deprived of a very useful instrument for policy dialog among governments, local public administration, civil society and its citizens.
It is through these types of binding instruments that cross-border governance can be build. A stronger correlation has to be provided also with several other economic indicators that can certify the validity of the measurement provided by the GDP indicator. Such indicators must be up to date and refer to the people living in the target area, surface, density, adult literacy rate (including knowledge of the other country’s language), trade (exchange of goods), or the overall investment in research and development (R&D in the business sector)
The second programme level indicator is also categorized as a context indicator Level of unemployment in eligible area and defined by a baseline of 4.6% unemployment rate at the level of 2007.
To begin with, this indicator is wrongly defined. Typically the employment and unemployment indicators fit the category of long term outcomes and contribute to the achievement of context indicators. In statistics, these types of indicators are also known as lagging indicators, mainly because they change after macroeconomic conditions have already changed. More to the point, the lagging indicators offer the possibility to double check trends that have already been predicted by GDP indicators because they are available before complete national statistical data[i].
No target value is set up for 2013 and no methodology is available to measure the achievements of this indicator or its direct relationship with the other programme level indicators. To quantify such an indicator one should first overcome the possible confusions that can arise from analysing employment and unemployment indicators altogether in the same table and in the absence of clear definitions and relevant data sources (O.P: 2007, 113).
Indicators must be clear, transparent, and easy to understand for all the parties involved. Likewise, they must be based on clear definitions and relevant data sources for the target group. Similar with the GDP indicator, the Programme uses data aggregated at national level to calculate the level of unemployment in the border area. International players such as the WB, OECD and the International Labour Organization (ILO) usually define these indicators separately. This does not mean that these two indicators should not be correlated; it only implies that no useful data can be obtained if we link them without checking their values against the ones provided by similar indicators.
By unemployment the ILO understands the share of the labour force without work but currently available for paid work and looking for employment or self-employment. Characteristics of labour force and unemployment may differ by country according to criteria such as “age limits, reference periods, and criteria for seeking work, treatment of persons temporarily laid off and of persons seeking work for the first time”[ii].
In a cross-border context several other criteria add up to these general reference points: work mobility, transport infrastructure, business cooperation or even hidden unemployment. While the OP of the HU-RO Programme manages to identify some of these challenges, it fails to argue for the ways in which they can be tackled through the common contribution of Hungary and Romania. In our specific case, the unemployment level of the eligible area should be correlated with some clear data provided by indicators such as long term unemployment (% of total unemployment, total, male, female), unemployment by educational attainment (% of total unemployment, primary, secondary, tertiary and higher) or regional and cross-border mobility of the labour force.
The employment is defined by the ILO as “those persons having a working place comprise all persons above a specific age who during a specified brief period, either one week or one day were in paid employment or self-employment”.
One indicator used by the OP to further express some of the narratives behind the unemployment/employment indicators is the Average monthly net earrings of male / female employees in euro*, having the target area as a dependent variable and the male/female, manual/non-manual workers categories as independent variables. Besides the fact that its relevance for the general objective of the Programme is questionable, the measurement of such an indicator is most likely to fail because it involves a high levels of flexibility and cost-effectiveness, features that the actual cross-border cooperation programme does not poses. It is not clear, for example, how the admission of Hungary into Schengen or the Romanian accession to the EU affected the values of this indicator.
Concerning the last programme level indicator, Impact Level of economic co-operation defined as % growth in the share of Romania in the annual foreign trade turnover of the Hungary and vice versa, we can see that it starts with the base value of 0% and targets an increase of 5% by the end of 2013.
First of all, this indicator does not build on the impact of previous experiences because it ignores the results of the previous IPA programme. Thus, in the absence of a clear definition and transparent data sources and valid measurements, one cannot argue for causality between the positive evolution taking place at the level of the target area and this indicator.
In order to assess the impact of the Programme, i.e. whether and how much policy measures have modified the outcomes, the causality between means and ends has to be clearly established. A public policy cannot be justified and implemented outside the basic rule saying that it has to deliver better results than the previous policy and with no higher costs.
The ex-ante evaluation of the OP concludes, back in 2007, that the indicator system contains only one impact indicator which is not able to integrate the outcomes of the two Priority axes (OP, 2007, 157).
More exactly, in idealistic circumstances the above indicator could reflect, to a certain point, the improvements of the key conditions for joint and sustainable development of the cooperation area (priority axis 1) but cannot capture the consolidation of social and economic cohesion in the border area (priority axis 2).
According to the same ex-ante evaluation, the authors of the OP concede that there were no available data to set up a base value for this indicator. In other words, no empirical evidence can act as a baseline for the 5% target. Thus, it impossible to measure this indicator and to pinpoint its relevance for the overall objective of the Programme or the EU Regulations that refers to quantified targets, particularly in terms of impact in relation to the baseline (Regulation 1083/2006, art. 48, 2).
Moreover, all the three programme level indicators do not comply with the basic requirement of the provisions of the article 19 of the 1080/2006 EC Regulation, which conditionally links the financial support for a project from the fulfilment at least two of the following indicators: joint development, joint financing, joint staffing and joint implementation. One cannot measure extent to which these indicators are actually achieved.
The actual development of cross-border indicators requires a common theory and a mechanism. Otherwise, the relation between programme level indicators and the needs of the border area can be spurious. Furthermore, I consider that some of the inconsistencies identified at the above three main indicators come from what is known in statistics as data mining. For example, on the same logic with the formulation of the GDP indicator, the unemployment indicator was set up in order to meet specific requirements of the second objective of the EU Cohesion Policy – to support the economic and social conversion of areas experiencing structural difficulties.
Technically, data mining is the process of finding correlations or patterns that can support a hypothesis or, in this specific case, an indicator. The challenge is to come up with a solution to formulate the needs of the cross-border area into relevant objectives that can be further operationalized through well adapted empirical indicators.
Working with objectives that are not capable to capture the real needs of the border area has a direct impact on the definition of the context and outcome indicators. Such situation may further affect the overall effectiveness, utility and sustainability of the Programme and its contribution to the goals of the ETC. Nevertheless, this tendency can be counterbalanced by constantly checking if the given values of the Programme level indicators can meaningfully be interpreted in terms of the objective they seeks to operationalize.
After more than six years since the beginning of the HURO Programme and five years away from the first call for proposals I consider that available data are at hand for such an undertake.
To this end, Adcock and Collier, quoting A. Kaplan, call this process as approximation and conclude that “proper concepts are needed to formulate good theory, but we need a good theory in order to arrive at the proper concepts… The paradox is resolved through a process of approximation: the better the concepts, the better the theory (Adcock and Collier: 2001, 532).
It implies testing the objectives to their logical consistency, matching and contrasting them with other objectives from local, regional and EU level, in order to improve them with the help of empirical examples. Thus, accurate policy formulation, achievable objectives and measurable indicators can provide a safer way to capture the needs of the border area.
For the moment, the Programme level indicators are not able to provide a convincing narrative, supported by quantified information on the fulfilment of the regional needs and their compliance with the goals set up by the EU 2020 Strategy. Therefore, the Commission is able to aggregate only a limited set of quantified data in terms of Programme’s inputs, payments and outcomes. These quantified data give an EU overview, albeit if limited, on the implementation of the Programme. Nevertheless, they cannot provide a real input for the formulation of a next generation of cross-border indicators.
 purchasing power standard (PPS)
 11.5% out of 21.170 E is 2434.55 E and not 2426 E; 58.6% out of 21.170 E is 12 405.62 E and not 12. 402 E
 Which is again wrong as 58.6% + 11.5% = 70.1% divided by 2 equals 35.05 % GDP
 Gross domestic product (GDP) is the sum of value added by all resident producers plus any product taxes (less subsidies) not included in the valuation of output. Growth is calculated from constant price GDP data in local currency. World Bank Development Indicators, 2012, pp 38-39
 More exactly, this indicator is calculated at the average exchange rate from 2005: HUF/EUR(ECU): 248,05, RON/EUR: 3,6234 Sources:Magyar Nemzeti Bank, Banca Nationala Romania. The actual figures are HUF/EUR: 300 and RON/EUR: 4.5
ANHEIER, Helmut, STANIG, Piero, KAYSER, Mark, Introducing a New Generation of Governance Indicators, The Governance Report 2013, Oxford University Press, 2013
ADCOCK, Robert, COLLIER, David, Measurement Validity: A Shared Standard for Qualitative and Quantitative Research, American Political Science Review, Vol. 95, No. 3, 2001