Funded by the European Investment Bank - University Research Sponsorship Programme (EIBURS)

Project

And here many people will raise a question: which practical consequences came or will come from such an increase in our knowledge about the inner structure of the matter
(Enrico Fermi, 1930)

The measurement or assessment of the cost/benefit of research, development and innovation (RDI) infrastructure is notoriously difficult. And yet global policy discussions increasingly focus on innovation and the knowledge economy as a driver for long-term sustainable growth. A multitude of methods and indicators are used to study socio-economic impacts. However, the current methodology to assess the cost/benefit of RDI infrastructure is deficient.

The aim of the project is to build on the current international practice and produce a cost/benefit analysis (CBA) model for decision making able to assess the potential future net social benefits generated by a research infrastructure and the uncertainty and risks associated to them. The model should be academically credible, robust but at the same time easy to apply and easy to assess. In order to achieve this, the research team will have to review the current state-of-the-art literature in the field, assess willingness-to-pay models from different countries in relation to the various benefits which typically accrue from such projects, develop an understanding of market values where applicable associated with such potential benefits, and develop an understanding of the typical type of RDI infrastructure projects that come forward for funding. All of this will be within an appropriate theoretical framework.

The analytical framework will adapt the standard CBA model to the specificities of research infrastructures, which pose a number of methodological challenges, namely:

  • Intangible nature of benefits
  • The intangible nature of benefits is usually dealt with in a CBA framework by assessing the users’ willingness-to-pay (WTP) through implementation of contingent analysis or looking at revealed preference methods or with benefit transfers approach. In the case of RDI the main intangible output produced by this sector is pure knowledge, whose value is, by its own nature, indefinable. Some attempts have been made to capture spillovers and externalities generated by the research activities (see for example Autio E., Bianchi-Streit M. and Hameri A-P., 2003 or Bach, L. And Lambert, G., 1993), by estimating the value of new commercial products and technological progress achieved by downstream industries. In order to capture benefits of both fundamental and applied research infrastructures, a combination of methods (including an exploration of using bibliometric data and forecasts, such as impact factors and downloads from online electronic archives, reading time, and opportunity cost of researchers’ time as proxies for marginal social value of published knowledge) will be tested, also by looking at sectors and typology of infrastructures sharing the same problems.
  • Long time span of benefits to occur
  • The long time span for benefits to occur is taken into account by the theory of CBA with the choice of an appropriate social discount rate (SDR), which reflects the social view on how benefits and costs are to be valued against present ones. Yet, CBA is generally used for projects involving shorter time horizons: in the EC guide, for example, a reference time horizon of 10 years in the case of industrial projects and 30 years for some transport and environmental infrastructures are recommended. While the standard use of CBA is based on exponential discounting, in order to take into account that certain effects may appear after a longer time, a different discounting model should be adopted. Since investments in the RDI sector often take place long before returns are realised at a very uncertain point of time, the SDR should thus be chosen so as to give more importance to the consequence of RDI projects in the distant future, and so to future generations.
  • Uncertainty and risk
  • Another feature intrinsically related with RDI investment and which should be considered when applying CBA is the high uncertainty and risk related to the achievement of direct and/or indirect effects. Research, development and innovation involve the generation of new ideas and the probability of success depends on many factors, not all of which under the direct control of RDI projects’ promoters and easy to be forecasted. At this point it is convenient to resume the traditional distinction between uncertainty and risk (Knight, 1921). On the one side, risk relates to quantities susceptible of measures, for which it is possible to estimate a probability distribution for a certain event/state of the world to occur. On the other side, we refer to uncertainty when it is not possible or reasonable to assign a probability to a given event. Accordingly, it is apparent that risk, but not uncertainty, is subject to empirical measurement, and can be analysed and possibly managed. Risk assessment is usually performed in a CBA exercise and consists of studying the probability distribution of certain variables and parameters. However, in the case of RDI (but also for other sectors, such as environment if we think for example of implications of climate change), the risk may be particularly high. Moreover, there may be a certain margin of not measurable risk, i.e. uncertainty, when having to do with intangible impacts, for which the determination of probability of occurrence and magnitude may be simply impossible to forecast. To this end, the model developed will explicitly state the extent to which the measurability of benefits can be dealt in a CBA framework and to what extent it has instead to take into account alternative methods (e.g. peer reviewing) to assist the decision-making process.
  • Spillovers and externalities
  • Benefits of research infrastructures, especially for pure research activities whose direct output is actually an increase in the stock of knowledge, may generate many and different effects spreading widely into different sectors and technological contexts, that causal links and attribution are not straightforward.

The research proposal is structured along three Research Lines, aimed at answering the following research questions:

Research Line 1 – Development of the theoretical model:

What is the theoretical conceptual model assessing the economic benefits of research, development and innovation infrastructures?

  • What are the potential market and non-market benefits associated with RDI infrastructures?
  • Which are the most suitable CBA techniques to evaluate the effects of RDI infrastructures?
  • What is the best way to bring out and quantify the long-term effects and the benefit of the future generations?
  • How could the risk and uncertainty of RDI effects been dealt with?
Concept

The first area of this Research Line is to review the existent literature on the benefits associated with RDI infrastructures in order to develop a theoretical framework for their assessment.

Infrastructures in the RDI sectors are recognised as essential ingredients of excellence, creativity and innovative ideas. They are considered instrumental for the advancement of science and stimulate important innovation, learning and education processes that are highly relevant for society and the economy. Beneficiaries of RDI infrastructures are a large and varied range of actors, including researchers, industries and citizens. Such facilities attract a broad range of expertise, by providing scientists, students and researchers with the opportunity of meeting and interacting. Since activities carried out in these infrastructures often lie at the frontiers of science, they contribute at stimulating the interest of young people and motivate them to embrace scientific careers. Moreover, the construction and maintenance of research infrastructure create relevant supply and demand effects, stimulating industrial impact and interdisciplinary contacts between researchers and industry. The linkages between science and industry are particularly strong where technology clusters of associated industry or so-called technological parks exist. More in general, facilities in the research, development and innovation sector have the ability to bring together people and to contribute to research, education and innovation.


The way how RDI investment, particularly infrastructures, may affect socio-economic development has been recently studied by the RAMIRI project, funded by the European Commission under FP7, which distinguishes between the following impacts:
  • Scientific impact, taking the form of new knowledge generated by pioneering scientific research;
  • Social and human capital impact, i.e. benefits from the creation of formal and informal social networks as a result of the variety of stakeholders interested in the development of RDI infrastructures, and accumulation of human capital (via training and learning effects);
  • Direct and indirect economic impact, linked to the generation of employment and spin offs;
  • Societal impact thanks to the contribution to solving major challenges, such as energy sustainability and climate change or resolving treatments of serious diseases.
After having identified the main benefits related to RDI infrastructures, the team will elaborate upon the specific CBA approaches to be applied when appraising RDI projects, so as to build a comprehensive theoretical model for assessing RDI infrastructure effects.

As to the nature of RDI benefits it is convenient to distinguish between pure research and applied research/development/innovation infrastructural projects. Such distinction relies on the type of output generated, which may be more or less quantifiable in economic terms. Actually, the possibility of quantifying RDI effects is larger as the infrastructure produces more tangible outputs, while is limited when the produced outputs are intangibles. For instance, the revenues generated from putting onto the market an innovating product or technology or for taking out a patent, which are the output of applied research, development and innovation activities, could be economically evaluated through a nearly standard CBA. In particular, shadow prices of these benefits could be computed, by applying:
  • The Little and Mirrlees (1974) shortcut of using border prices as proxies of shadow prices for traded goods;
  • The long-run marginal cost rule for non tradables;
  • The willingness-to-pay/willingness-to-accept approach for some externalities.

On the other hand, the output of basic research is in the form of new ideas, which are something intangible and have no immediate monetary value. An example of research output for which benefit quantification may prove to be impossible is the World Wide Web, borne as a technological spin-off of CERN.

For other intangible discoveries, which produce pure knowledge and have no or only minor fallouts in the real life, one may attempt to indirectly quantify the benefit, perhaps on the basis of the concrete outputs of the discovery, i.e. working papers, presentation of results at conferences, publications etc. As to the papers, in particular, although the market value of these outputs is not significant, the willingness-to-pay for producing them could be proxied. A possible approach (inspired from Solla Price, 1963) could be that of applying a statistical construct taking into account the number of times a paper is read, the time needed for reading it and the opportunity cost of time.

Activities

In order to answer the research questions related to Line 1, the research activities will be:
  • Literature review: inspection of the theoretical and empirical literature concerning the economic impact of RDI infrastructure and identification of the existing gaps;
  • Development of the theoretical model: elaboration of the conceptual framework of analysis to understand the economics of RDI infrastructures and assess their net benefits.

Research Line 2 – Reviewing the state-of-art in RDI infrastructure project appraisal:

What is the practical experience developed at EU and international level in the appraisal of research, development and innovation infrastructure?

  • What kind of benefits are typically identified and evaluated?
  • What are the approaches actually used to give a conventional welfare metrics in money terms to the direct and indirect effects of RDI infrastructures?
  • What are the approaches actually used to deal with uncertainty and risk of RDI effects?
Concept

CBA has been used in Cohesion Policy since the 1990s and has been explicitly required since 2000 by the Structural Funds regulations on major projects. The European Investment Bank also regularly applies the CBA method in appraising infrastructure projects. In particular, a CBA must be included, among other documents, in the project dossier for major projects applying for co-financing and, the EU financing decision is informed also by the results of the CBA.

CBA methodology has therefore become central to project appraisal in EU Member States and aspiring EU Member States. It has been used mostly for projects in the transport, energy, heath, waste and environment sectors. The extent of utilisation has spurred the evolution of the methodology itself and has greatly increased the demands being made of CBA throughout the EU Member States.

Cost-Benefit Analysis is largely utilised also by some governments when programming national public expenditure. In the United Kingdom, for example, the Green Book, published by the HM Treasury (2003), gives guidelines for investment appraisal by departments and executive agencies. The rules and methods identified are applied without discretion to projects, programmes and public choices and there is no minimum size, in terms of investment costs, that limits the number of projects to be subject to preliminary evaluation, although these guidelines have no compulsory nature. While some Departments, notably the Department of Transport, have developed a fairly comprehensive CBA approach, others have renounced to use CBA. E.g. the Department of Health uses a quali-quantitative multicriteria cost effectiveness method to appraise hospital building projects. As to the scientific and industrial research, limited evidence about the application of CBA is available.

Another country with long tradition in the use of CBA are the United States. The US do not only apply CBA in the traditional sector, such as transport and environment, but also developed specific guidelines for assessing net benefits in others less conventional fields. The US Army Cost-Benefit Analysis Guide (2010), for example, is available for the defence sector. Moreover, in some cases CBA has been used as a methodological framework to evaluate RDI projects. In particular, the US Advanced Technology Programme – ATP (National Institute of Standards and Technology, 2007) applied Cost-Benefit Analysis to assess the economic impact of two ATP-funded green process technology projects, with applications in food packaging, textile, metals, aerospace, and automotive industries. Both projects return positive net present values, ranging approximately from USD 20 to USD 50 million, with the reduction of greenhouse gas emissions being one of the main benefits achieved and quantified.

As part of Research Line 2, the experience of different institutions and countries with the appraisal of RDI projects through the CBA approach will be reviewed and critically analysed. The scope of analysis will not be limited to the EU, but will include also other international experiences considered relevant in this regard. The aim is to detect to what extent the standard CBA approach, as identified in Research Line 1, has been applied and how the major challenges in assessing net benefits in the RDI sector (concerning, for example, the uncertainty of results or the quantification of spillover/externalities) have been tackled.

Activities

The activities which will be carried out to develop Research Line 2 are:
  • Data gathering: review of RDI infrastructure project appraisals available at the EC, at the EIB and other national agencies;
  • Data processing: classification of research infrastructures by typology, identification of typical benefits and of methodologies of evaluation, calculation of ranges of benefit estimates.
  • Cross-check with the theoretical model: comparison of the CBA practice in the RDI sector with the assumptions gathered from the literature review and the general theory of Cost-Benefit Analysis.

Research Line 3 – Testing and fine tuning the CBA model

What is the suggested model to assess RDI infrastructure projects and support the decision-making process?

  • What are the strengths and limitations of the proposed CBA model shown by testing it with concrete case studies?
  • Which approaches should complement the CBA techniques to help the decision-making process?
Concept

Research Line 3 is the core of this research project. It involves the test of the theoretical model by carrying out a CBA on two selected case studies, with the final objective of developing a new comprehensive model for CBA to be applied to the RDI sector.

Case studies are a popular evaluation approach, for which documentary evidence and interviews are generally the main source of information. Case studies are a methodological tool for evaluating investment projects used by different institutions. The advantage of testing the model on real cases is the chance to adjust and adapt the theoretical framework developed in order to ensure the realism of the tool developed.

We propose to carry out two related, in-depth case studies using the CBA model, one concerning a fundamental research infrastructure and one about an applied research infrastructure. The justification for implementing two related case studies relies on the need to perform the test on different kind of potential outputs a facility may produce at different stage of the research activity: respectively, mostly intangible outputs for fundamental research and mostly tangible ones for applied research.

The findings gathered from the implementation of both the case studies will allow developing guidelines for project analysts, institutions and national agencies aiming at applying CBA in the whole research, development and innovation sector.

Activities

The activities to be implemented within Research Line 3 are:
  • Identification of two case studies of research infrastructures on which to perform a pilot CBA;
  • Testing the theoretical framework with a CBA exercise;
  • Development of a comprehensive CBA model for RDI.

The Research Line will produce the two case studies reports and a Final Report including recommendations and policy implications. In addition, an international conference with decision makers will be organised in order to present the preliminary findings and to discuss the policy implications of the proposed model.