International Transport Forum (ITF) Round Table on Broadening the Scope of Transport Appraisal to Capture the Full Impact of Investments, Organisation for Economic Co-operation and Development (OECD), Paris 29th September–1st October 2021.

Purpose

Held over three days, 47 participants from 14 ITF member countries attended this virtual Round Table, whose aim was to reflect upon recent trends in transport appraisal. With Cost Benefit Analysis (CBA) and its limitations a key theme there was debate over whether it is a tool that is so compromised by distortions that it is no longer suitable or is a useful ‘workhorse’ that will always form the core of transport appraisal. A debate that will undoubtedly continue to run. Discussions covered a range of other topics, clustered around three key aspects of this debate: the role of equity and equity of access to the benefits of transport for different places and groups in society, the value of ex-post analysis and the role of stakeholders.

This report represents one perspective on the event and its relation to key issues in mega infrastructure planning. An overall report will be available on the ITF website (see references). Attendance was a good mix of academics and practitioners, also reflected in the presentations. Although the balance was slightly towards the former, many of the speakers either crossed over into practice or brought considerable practical experience. The event was held under Chatham House rules, so the only points attributed to speakers are those reflecting their presentations.

Day one

First, Tom Worsley presented a report on a pre-Covid ITF Round Table on the potential for strategic infrastructure planning. This highlighted recent developments such as the establishment of independent infrastructure planning bodies seen in the UK and many of the larger Commonwealth countries, where they operate at both the national and state level. Such bodies offer the ability to conduct cross sectoral assessment and planning, take a longer term perspective and explore different appraisal methods in order to identify the wider benefits of infrastructure. Their independence, or at least distance, from party politics could also be seen as an opportunity to increase transparency and consult in a more impartial way. But these bodies are advisory, with limited ability to distance themselves from party politics and what Tim Marshall (2013) calls ‘infrastructuralism’ - a bias towards hard infrastructural solutions. They also have limited ability to challenge powerful narratives such as the assumption that infrastructure investment should be directed towards, and will generally promote, economic growth.

The second session dealt with one of the areas where the event began, implicitly and explicitly, to challenge some of the assumptions that monetised benefits alone are a sufficient basis for transport appraisal. On the subject of equity Yoram Shiftan from Technion (the Israel Institute of Technology) and colleagues presented a helpful analysis of what is meant by equity, followed by two case examples of transport planning in Israel, which discussed methods to appraise equity between different socio-economic groups, and spatial or regional equity. The discussion highlighted the lack of an established definition of equity that could be used in appraisal and produced discussion on how and what ethical frameworks can be used to cover subjects such as the impact of transport infrastructure on future generations and ecosystems, with an important point that it may be more helpful to refer to these as questions of justice rather than equity.

In the second presentation, Floridea Di Ciommo shared examples of her work on gender equity at the Madrid-based non-profit organization Cambia Mo/Changing Mobility. As with the Israeli approach, equity assessments sat within a multi-criteria framework alongside conventional CBA. A Madrid case study showed the unequal impact of pollution from the M30 motorway, with women experiencing higher levels. Comparative studies in Spain and also the US, highlighted the different travel needs of men and women, with women travelling much more in order to care for relatives and children. In contrast male travel was predominantly for employment. Since transport systems tend to be configured and appraised upon their ability to facilitate the latter, this indicates that considerable needs remain unmet and may explain the lower levels of satisfaction with transport systems reported by women in both countries. The ‘15-minute city’ currently being promoted by the Mayor of Paris was given as one positive policy development that begins to consider some of these matters. The presentation concluded with calls for greater diversity particularly in organisations that plan, deliver, manage, and regulate transport infrastructure. Also, for greater appreciation of the bigger picture including the subjective experience of vulnerable users of transport, and how this can be addressed by improvements in the organisation of infrastructure and the design of the public spaces it creates.

Day two

The second day’s theme was ex-post transport appraisal, with three presentations each using a real world example of innovations and experiences in this field. Morten Welde and Gro Holst Volden from the Concept Research Programme at the Norwegian University of Science and Technology, gave a Norwegian perspective on ex-post appraisal. They argued the initial concept for a project matters and is as important as delivery time and cost since project success can be much broader than these two dimensions. In the absence of fully comprehensive appraisal offered by CBA, they further argued it needs to be accepted that there are wider economic impacts and wider political impacts which may not always be explicit. Norway has a relatively well developed programme of ex-post analysis, with the Concept Research Programme having evaluated 34 projects since 2012. This uses a standardized process to enable comparison, focusing predominantly on project outcomes, wider impacts and social objectives. This goes beyond monetised benefits to appraise strategic effectiveness as well as intended and unintended project consequences. The differing appraisal of road and rail projects showed how road projects often appeared to perform much better, operationally and in terms of value for money. But much of this could be explained by the contrast between the ambitious objectives set for rail projects as compared with the more modest aims of road schemes. Rail projects are also far more dependent upon factors outside of their control within the wider network and transport system.

The next presentation drew on French experiences. Alain Bonnafous explained how ex-post appraisal was established as a legal requirement under French law. Despite a strong engineering tradition which encouraged a mindset that saw conventional CBA as sufficient, there was a growing awareness throughout the 1970s that there are wider impacts of transport infrastructure that the CBA methodology struggles to capture. In 1982 the Loi d’Orientation sur les Transports Intérieurs (LOTI) made ex-post analysis compulsory for all large transport schemes (currently over €83 million) where public funding is involved, within 3-5 years after completion. Despite this requirement, the French experience has been that many of the potential benefits, particularly those associated with construction, are difficult to establish with a single study at one point in time. Jobs created for the duration of a project are hard to measure once it is complete, data is lost, and memories fade. With these problems in mind a team from the University of Lyon began the experimental establishment of permanent observatories for motorway projects in order to capture data on societal change in real time. Since then, the practice and methods of permanent motorway observatories have become well established. The establishment of permanent observatories for rail projects has been less comprehensive, although Alain shared his own experiences as Chair of the Socio-Economic Observatory, Sud Europe Atlantique High Speed Rail covering the Tours to Bordeaux section of the Paris to Bordeaux high speed rail link and some of the challenges. In particular, delays in project inception and key elements such as rolling stock purchase costs are not always within the control of project promotors. This has meant that not all the expected benefits can be achieved within the 5 year timeframe required for the LOTI ex-post appraisal Despite this, the Sud Europe Atlantique Observatory has been able to capture a wealth of data including construction and transport impacts, tourism benefits, development around stations and the institutional dimension of the project for example how it has shaped the strategies of local actors and organisations as they sought to capitalise on the ‘brand effect’ of a new high speed rail connection.

The third presentation, by Morten Skou Nicolaisen, the Programme Manager for Green Mobility at the city of Aarhus in Denmark, discussed the motivation of ex-post appraisal due to the widely known problems of ex-ante appraisal, and also common problems such as lack of reliable data. Citing examples of ex-post appraisal from OECD countries, there is still insufficient consistency in methods or data to allow an international comparison. His analysis of existing problems with ex-ante appraisal, covered the well-known problems of inaccuracy in demand and cost forecasts, plus some important additional insights such as problems with assumptions about increased demand that motivate the ‘do nothing’ scenarios of many projects, particularly roads. The tendency to overestimate the extent to which congestion will increase, in contrast to forcing transport users to adopt other strategies or even switch transport mode, tends to distort ex-ante appraisal in favour of projects that reduce congestion. The reverse is often the case once a road project is completed, with the induced demand of new faster connections often underestimated. More accurate forecasts of traffic volumes should cover all scenarios and possibly may offer stronger justification for the ‘do nothing’ option. The presentation was illustrated with real cases where factors outside of the project, in these examples the actions of the property development industry, can have a big impact on project success or failure and can even undermine existing strategic infrastructural choices. He also presented examples of how local political actors, with wider political and spatial objectives, are well aware of the importance of appraisal techniques such as CBA even if they are narrowly defined, and seek to ‘game’ the system by creating situations where it is easy to demonstrate time saving benefits, which are generally given a high weight in CBA.

Some important themes came out of the three presentations. The political nature of ex-post appraisal often militates against an ideal process which can both validate or critique the ex-ante appraisal and importantly inform the next round of project development. The latter point is significant given the timescales involved, and changes in political and social norms over time. For mega infrastructure projects that may have planning and construction phases spanning multiple decades, an-ex post appraisal that seeks to address current issues such as spatial, social or gender equity, or issues with increased political importance, such as climate change/CO₂ emissions reduction, will often be analysing projects originally conceived with few objectives in these areas. In addition to the political dimension of ex post appraisal, there is always a powerful inertia to overcome due to the cost and time requirements of data collection and retention.

Day three

Day three covered issues of stakeholder engagement. As Alain Bonnafous previously pointed out, the politics of transport infrastructure have changed, with much greater levels of opposition to new infrastructure particularly, although not exclusively, in Western democratic countries. Ennio Cascetta from Università degli Studi di Napoli Federico II, presented work conducted with a colleague Armando Cartenì on how existing quantitative methodologies can be strengthened by stakeholder engagement. Analysis of transport planning as a process of decision-making offers an opportunity to consider distinct political and technocratic processes within transport planning, which interact with each other. This represents a shift away from a tradition of ‘Decide Announce and Defend’ where decisions are made solely within a project agency/organisation. This may lead to rational decisions in terms of transport or economics but it can create problems of opposition that make implementation of the proposed solutions more difficult. This can also feed what Cascetta and Cartenì term the ‘Penelope Syndrome’, where incoming political administrations have an incentive to gain favour with sections of the public opposed to a project by unravelling the work of their predecessors. His presentation offered an instrumental view of stakeholder engagement, with examples of opposition to transport infrastructure in Italy and other countries. He saw this approach more as a means of refining transport projects, for example to improve some of the issues of regional equity discussed on day one or to ensure projects are ultimately easier to deliver, rather than as a more fundamental challenge to some of the deep assumptions still inherent in planning and transport planning rationalities.

Round table final discussion

Day three concluded with an overall discussion including the critical question of where exactly appraisal fits in transport decision-making. On one hand, it could be seen as a relatively narrow technical process, although it has the capability of accommodating issues like equity and the wider impact of transport infrastructure and therefore informing a broader decision-making process. On the other hand, seeking wider and more comprehensive appraisal moves the appraisal process from a tool for decision-makers to consider alongside the hoped-for political and social objectives. towards a process for deciding whether or not these objectives will be met. The wider the appraisal process is, the more it draws in less clearly defined benefits (and costs) that are much harder to link directly to a single infrastructure investment.

The impacts of Covid 19 and implications for transport project appraisal were mentioned. Impact on reduced passenger numbers using the Sud Europe Atlantique High Speed Line has occurred at the end of the ex ante appraisal timeframe, wiping out many of the Line’s projected benefits. Also, in Tom Worsley’s presentation on day one, it was noted that it has radically altered the nature of scenario planning, requiring a much greater range of possible futures to be considered.

Concluding comments

As a personal reflection on the event, transport project appraisal is still a field dominated by conventional economics, so while calls to widen the basis of transport project appraisal are welcome, it poses question of how wide this basis should be, and how far are bodies like the ITF and OECD able to go in challenging some of the fundamental assumptions inherent within infrastructuralism.

Although there were many indirect references to the role of transport infrastructure in producing emissions and its impact on ecosystems, it might have been useful to cover this more directly alongside the uncertainty that a changing climate injects into the appraisal of infrastructure. When considering wider economic benefits, it is important to also look at the wider costs of transport infrastructure such as perpetuating a society with a carbon and resource intensive basis for transport. Similarly, it is understandable that for most transport planners, questions of spatial equity remain focused on regions or nations, but a fuller appraisal approach may want to consider the justice of OECD countries, with relatively well developed transport infrastructure, investing heavily to achieve often marginal gains in accessibility, when such investment could generate far greater social and economic benefits in the global South.

It would be helpful to see connections in future events between other strands of ITF and OECD work, for example the Decarbonising Transport initiative (ITF, 2021) which engages with the scale of the task for the transport sector, that could account for 40% of global CO₂ emissions by 2030 and currently derives 92% of its energy from oil.

Finally, future events may want to look beyond the conventional disciplines and institutions that are involved in transport infrastructure appraisal. The OECD has begun to look at initiatives that shift power further towards citizens through deliberative fora which are increasingly used in infrastructure planning (OECD, 2020). Or to consider examples such as the Welsh Government’s Future Generations Commissioner (2018) whose recent recommendations that a proposed mega (motorway) infrastructure project should be abandoned in favour of alternative transport and social policy investments, suggests that there might be value in widening transport appraisal further still.

Daniel Durrant*
Bartlett School of Planning of Planning, University College London, UK
[email protected]