The feasibility and profitability of large investment projects are frequently subject to a partially or even fully undeterminable future, encompassing uncertainty and various types of risk. We investigate significant issues in the field of project appraisal techniques, including risks and uncertainties, appropriate risk analysis, project duration as well as the dependencies between (sub-) projects. The most common project appraisal techniques are examined addressing benefits and weaknesses of each technique. Furthermore, the practical use of the different techniques for the public sector is examined, exemplifying this with a small-scale analysis of the risk analysis procedures of the World Bank. Our finding suggest that in particular for the public sector, practical implementation of quantitative techniques like Monte Carlo simulation in the appraisal procedure of investment projects has not fully occurred to date. We strongly recommend further application of these approaches to the evaluation of processes and financial or economic risk factors in project appraisal of public sector institutions.
KeywordsIsk Analysis TechniquesMonte Carlo SimulationInternational Financing Institutions
M. Florio, U. Finzi, M. Genco, F. Levarlet, S. Maffii, A. Tracogna and S. Vignetti, “Guide to Cost-Benefit Analysis of Investment Projects: Structural Funds-ERDF, Cohesion Fund and ISPA,” 3rd Edition, DG Regional Policy, European Commission, Brussels, 2002.
W. J. H. van Groenendaal and J. P. C. Kleijnen, “On the Assessment of Economic Risk: Factorial Design Versus Monte Carlo Methods,” Journal of Reliability Engineering and Systems Safety, Vol. 57, No. 1, 1997, pp. 91-102. DOI:10.1016/S0951-8320(97)00019-7
Environmental Assessment Institute, “Risk and Uncertainty in Cost Benefit Analysis,” Toolbox Paper, Institut for Milj?vurdering, Copenhagen, 2006.
E. A. Parson, V. R. Burkett, K. Fisher-Vanden, D. Keith, L. O. Mearns, H. M. Pitcher, C. E. Rosenzweig and M. D. Webster, “Global-Change Scenarios: Their Development and Use,” Sub-Report, United States Climate Change Science Program and the Subcommittee on Global Change Research, 2007.
R. Lempert, “Can Scenarios Help Policymakers Be Both Bold and Careful?” In: F. Fukuyama, Ed., Blindside: How to Anticipate Forcing Events and Wild Cards in Global Politics, Brookings Institution Press, Washington DC, 2007.
I. M. D. Little and J. A. Mirrlees, “Risk Uncertainty and Profit,” Project Appraisal and Planning for Developing Countries, Heinemann, London, 1994.
S. Devarajan, L. Squire and S. Suthirwart-Narueput, “Reviving Project Appraisal at the World Bank, Policy Research Working Paper No. 1496,” Public Economics Division, Policy Research Department, World Bank, Washington DC, 1995.
J. P. Gittinger, “Economic Analysis of Agricultural Projects,” 2nd Edition, The Johns Hopkins University Press, Baltimore, Maryland, 1982.
Overseas Development Administration, “A Guide to Road Project Appraisal,” Overseas Road Note No. 5, London, 1988.
D. G. Davies, (Ed.), “The Economic Evaluation of Projects,” Papers from a Curriculum Development Workshop, World Bank, Washington DC, 1996.
C. Kirkpatrick, “The Rise and Fall of Cost Benefit Analysis: The Role of Project Appraisal in Managing Development Projects,” In: F. Analoui, Ed., The Realities of Managing Development Projects, Avebury, Aldershot, 1994.
J. Weiss, “Project Failure: The Implications of a 25 Percent Rule,” In: C. Kirkpatrick and J. WEISS, Cost-Benefit Analysis and Project Appraisal in Developing Countries, Edward Elgar, Cheltenham, 1996.
R. Fao and M. Howard, “Use of Monte Carlo Simulation for the Public Sector: An Evidence-Based Approach to Scenario Planning, the Market Research Society,” International Journal of Market Research, Vol. 48, No. 1, 2006, pp. 27-48.
Y. H. Kwak and L. Ingall, “Exploring Monte Carlo Simulation Applications for Project Management,” Risk Management, Vol. 9, 2007, pp. 44-57. doi:10.1057/palgrave.rm.8250017
K. Balcombe and L. Smith, “Refining the Use of Monte Carlo Techniques for Risk Analysis in Project Planning,” Journal of Development Studies, Vol. 36, No. 2, 1999, pp. 113-135. doi:10.1080/00220389908422623
F. H. Knight, “Risk, Uncertainty and Profit,” Houston Mifflin, Boston, 1921.
J. F. Weston, “The Profit Concept and Theory: A Restatement,” The Journal of Political Economy, Vol. 62, No. 6, 1954, pp. 152-170. doi:10.1086/257499
G. Stigler, “Frank Knight,” In: J. Eatwell, M. Milgate and P. Newman, Eds., The New Palgrave Dictionary of Economics, Stockton Press, New York, 1987.
M. Friedman, “Price Theory: A Provisional Text,” Aldine, Chicago, 1976.
S. F. le Roy and D. Singell, “Knight on Risk and Uncertainty,” The Journal of Political Economy, Vol. 95, No. 2, 1987, pp. 394-406. doi:10.1086/261461
A. Jaafari, “Management of Risks, Uncertainties and Opportunities on Projects: Time for a Fundamental Shift,” International Journal of Project Management, Vol. 19, No. 2, 2001, pp. 89-101. doi:10.1016/S0263-7863(99)00047-2
N. Piyatrapoomi, A. Kumar and S. Setunge, “Framework for Investment Decision-Making under Risk and Uncertainty for Infrastructure Asset Management,” Research in Transportation Economics, Vol. 8, No. 1, 2004, pp. 199- 214. doi:10.1016/S0739-8859(04)08010-2
D. Vose, “Risk Analysis—A Quantitative Guide,” 2nd Edition, John Wiley & Sons Ltd., Chichester, 2000.
H. van Greuning and S. Brajovic-Bratanovic, “Banking Risk Analysis and Management,” Irecson Publishing House, Bucharest, 2003. DOI:10.1596/0-8213-5418-3
S. Savvides, “Risk Analysis in Investment Appraisal,” Project Appraisal, Vol. 9, No. 1, 1994, pp. 1-30.
C. B. Chapman and S. C. Ward, “Project Risk Management: Processes, Techniques and Insights,” 2nd Edition, John Wiley & Sons, Chichester, 2003.
D. B. Hertz and H. Thomas, “Risk Analysis and Its Applications,” John Wiley, Chichester, 1983.
I. Dikmen, M. T. Birgonul, C. Anac, J. H. M. Tah and G. Aouad, “Learning from Risks: A Tool for Post-Project Risk Assessment,” Automation in Construction, Vol. 18, No. 1, 2008, pp. 42-50. doi:10.1016/j.autcon.2008.04.008
D. Vose, “Risk Analysis—A Quantitative Guide,” 3rd Edition, John Wiley & Sons Ltd., Chichester, 2008.
M. Florio, “Cost-Benefit Analysis and the Rates of Return of Development Projects: An International Comparison,” Centro Studi Luca d’Agliano Development Studies Working Papers No. 182, 2003.
B. Jafarizadeh and R. Khorshid-Doust, “A Method of Project Selection Based on Capital Asset Pricing Theories in a Framework of Mean-Semideviation Behaviour,” International Journal of Project Management, Vol. 26, No. 6, 2007, pp. 612-619.
A. Saltelli, et al., “Global Sensitivity Analysis: The Primer,” John Wiley & Sons Ltd., Chichester, 2004.
A. E. Boardman, D. H. Greenberg, A. R. Vining and D. L. Weimer, “Cost Benefit Analysis: Concepts and Practice,” 3rd Edition, Prentice Hall, Upper Saddle River, 2006.
T. Merna and D. V. Storch, “Risk Management of Agricultural Investment in a Developing Country Utilising the CASPAR Program,” International Journal of Project Management, Vol. 18, No. 5, 2000, pp. 349-360. doi:10.1016/S0263-7863(99)00050-2
A. H. Russell, “Cash Flows in Networks,” Management Science, Vol. 16, No. 5, 1970, pp. 357-373. DOI:10.1287/MNSC.16.5.357
R. C. Grinold, “The Payment Scheduling Problem,” Naval Research Logistics Quarterly, Vol. 19, No. 1, 1972, pp. 123-136. doi:10.1002/nav.3800190110
S. E. Elmaghraby and W. S. Herroelen, “The Scheduling of Activities to Maximize the Net Present Value of Projects,” European Journal of Operational Research, Vol. 49, 1990, pp. 35-49. doi:10.1016/0377-2217(90)90118-U
R. Etgar, A. Shtub and L. J. Leblanc, “Scheduling Projects to Maximise Net Present Value—The Case of Time-Dependent, Contingent Cash Flows,” European Journal of Operational Research, Vol. 96, No. 1, 1995, pp. 90-96. doi:10.1016/0377-2217(95)00382-7
R. Graves, “Open and Closed: The Monte Carlo Model,” PM Network, Vol. 15, No. 12, 2001, pp. 37-41.
T. Williams, “The Contribution of Mathematical Modelling to the Practice of Project Management,” IMA Journal of Management Mathematics, Vol. 14, No. 1, 2003, pp. 3-30. doi:10.1093/imaman/14.1.3
W. C. Baum and S. M. Tolbert, “Investing in Development. Lessons of World Bank Experience,” Oxford University Press, Oxford, 1985.
G. Koop, D. Poirier and J. Tobias, “Bayesian Econometric Models,” Cambridge University Press, Cambridge, 2007.
T. Bedford and R. Cooke, “Probabilistic Risk Analysis: Foundations and Methods,” Cambridge University Press, Cambridge, 2001.
R. A. Brealey and S. C. Myers, “Principles of Corporate Finance,” 6th Edition, Irwin McGraw-Hill, Boston, 2000.
L. Trigeorgis, “Real Options: Managerial Flexibility and Strategy in Resource Allocation,” MIT Press, Cambridge, 1997.
P. N. Smith, S. Sorensen and W. R. Wickens, “Macroeconomic Sources of Equity Risk,” Discussion Papers 03/13, Department of Economics, University of York, York, 2003.
N. M. L. Barnes, “How to Allocate Risks in Construction Contracts,” International Journal of Project Management, Vol. 1, No. 1, 1983, pp. 24-28. doi:10.1016/0263-7863(83)90034-0
S. G. Kahn, “Partners in Risk,” Cost Incentive Development Contracts, ESA Bulletin, Vol. 26, 1981, pp. 48-53.
J. Lintner, “Valuation of Risky Assets and the Selection of Risky Investments in Stock Portfolios and Capital Budgets,” Review of Economics and Statistics, Vol. 47, No. 1, 1965, PP. 13-37.
W. F. Sharpe, “Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk,” Journal of Finance, Vol. 19, No. 3, 1964, pp. 425-442. doi:10.2307/2977928
A. Huchzermeier and C. H. Loch, “Project Management under Risk: Using the Real Options Approach to Evaluate Flexibility in R & D,” Management Science, Vol. 47, No. 1, 2001, pp. 85-101. doi:10.1287/mnsc.47.1.85.10661
W. E. Walker, “Policy Analysis: A Systematic Approach to Supporting Policymaking in the Public Sector,” Journal of Multi-Criteria Decision Analysis, Vol. 9, No. 1-3, 2000, pp. 11-27. doi:10.1002/1099-1360(200001/05)9:1/3 3.0.CO;2-3
P. W. F. van Notten, J. Rotmans, M. B. A. van Asselt and D. A. Rothman, “An Updated Scenario Typology,” Futures, Vol. 35, No. 5, 2003, pp. 423-443. doi:10.1016/S0016-3287(02)00090-3
R. Bradfield, G. Wright, G. Burt, G. Cairns and K. van der Heijden, “The Origins and Evolution of Scenario Techniques in Long Range Business Planning,” Futures, Vol. 37, 2006, pp. 795-812. doi:10.1016/j.futures.2005.01.003
P. Schwartz, “The Art of the Long View,” Doubleday/Currency, New York, 1991.
K. Van der Heijden, “Scenarios: The Art of Strategic Conversation,” John Wiley and Sons, Chichester, 1996.
L. B?rjeson, M. H?jer, K. Dreborg, T. Ekvall and G. Finnveden, “Scenario Types and Techniques: Towards a User’s Guide,” Futures, Vol. 38, No. 7, 2006, pp. 723- 739.
P. Bishop, A. Hines and T. Collins, “The Current State of Scenario Development: An Overview of Techniques,” Foresight, Vol. 9, No. 1, 2007, pp. 5-25. doi:10.1108/14636680710727516
European Environment Agency, “Looking Back on Looking Forward: A Review of Evaluative Scenario Literature,” EEA Technical Report, No. 3, 2009.
G. Wright, M. J. Lawrence and F. Collopy, “The Role and Validity of Judgment in Forecasting,” International Journal of Forecasting, Vol. 12, No. 1, 1996, pp. 1-8. doi:10.1016/0169-2070(96)00674-7
G. Rowe and G. Wright, “The Delphi Technique as a Forecasting Tool: Issues and Analysis,” International Journal of Forecasting, Vol. 15, No. 3, 1999, pp. 53-375.
H. A. Linstone and M. Turoff, “The Delphi Method: Techniques and Applications,” Addison-Wesley, London, 1975.
A. Lock, “Integrating Group Judgments in Subjective Forecasts,” In: G. Wright and P. Ayton, Eds., Judgmental Forecasting, Wiley, Chichester, 1987.
F. J. Parenté and J. K. Anderson-Parenté, “Delphi Inquiry Systems,” In: G. Wright and P. Ayton, Eds., Judgmental Forecasting, Wiley, Chichester, 1987.
T. R. Stewart, “The Delphi Technique and Judgmental Forecasting,” Climatic Change, Vol. 11, 1987, pp. 97-113. doi:10.1007/BF00138797
G. Rowe, G. Wright and F. Bolger, “The Delphi Technique: A Re-Evaluation of Research and Theory,” Technological Forecasting and Social Change, Vol. 39, No. 3, 1991, pp. 235-251. doi:10.1016/0040-1625(91)90039-I
N. C. Dalkey and O. Helmer, “An Experimental Application of the Delphi Method to the Use of Experts,” Management Science, Vol. 9, No. 3, 1963, pp. 458-467. doi:10.1287/mnsc.9.3.458
W. C. Wedley, “New Uses of Delphi in Strategy Formulation,” Long Range Planning, Vol. 10, No. 6, 1977, p. 70. doi:10.1016/0024-6301(77)90011-5
F. J. Parenté, J. K. Anderson, P. Myers and T. O’Brien, “An Examination of Factors Contributing to Delphi Accuracy,” Journal of Forecasting, Vol. 3, No. 2, 1984, pp. 173-182. doi:10.1002/for.3980030205
G. Rowe and G. Wright, “The Impact of Task Characteristics on the Performance of Structured Group Forecasting Techniques,” International Journal of Forecasting, Vol. 12, No. 1, 1996, pp. 73-89. doi:10.1016/0169-2070(95)00658-3
L. L. Leape, M. A. Freshour, D. Yntema and W. Hsiao, “Small Group Judgment Methods for Determining Resource Based Relative Values,” Medical Care, Vol. 30, No. 11, 1992, pp. NS28-NS39. doi:10.1097/00005650-199211001-00003
F. C. Miner, “A Comparative Analysis of Three Diverse Group Decision Making Approaches,” Academy of Management Journal, Vol. 22, No. 1, 1979, pp. 81-93. doi:10.2307/255480
C. S. Huang, Y. Lin and C. Lin, “Determination of Insurance Policy Using a Hybrid Model of AHP, Fuzzy Logic, and Delphi Technique: A Case Study,” WSEAS Transactions on Computers, Vol. 7, No. 6, 2008, pp. 660-669.
H. Sackman, “Delphi Critique,” Lexington Books, Lexington, 1975.
M. J. Bardecki, “Participants’ Response to the Delphi Method: An Attitudinal Perspective,” Technological Forecasting and Social Change, Vol. 25, 1984, pp. 281- 292. doi:10.1016/0040-1625(84)90006-4
J. O. Berger, “Statistical Decision Theory and Bayesian Analysis,” Springer, New York, 1985.
H. Bühlmann and A. Gisler, “A Course in Credibility Theory and Its Applications,” Springer, Berlin, 2005.
S. T. Rachev, J. Hsu, B. Bagasheva and F. Fabozzi, “Bayesian Methods in Finance,” Wiley, New York, 2008.
P. V. Shevchenko and M. V. Wüthrich, “The Structural Modelling of Operational Risk via Bayesian Inference: Combining Loss Data with Expert Opinions,” Journal of Operational Risk, Vol. 1, No. 3, 2006, pp. 3-26.
S. Trück and C. Truong, “Catastrophic Risks and Optimal Climate Change Adaptation Strategies at the Local Level,” Working Paper, Macquarie University, Sydney, Australia, 2009.
B. Clemson, Y. Tang, J. Pyne and R. Unal, “Efficient Methods for Sensitivity Analysis,” System Dynamics Review, Vol. 11, No. 1, 1995, pp. 31-49. doi:10.1002/sdr.4260110104
T. G. Eschenbach and R. J. Gimpel, “Stochastic Sensitivity Analysis,” The Engineering Economist, Vol. 35, No. 4, 1990, pp. 305-321. doi:10.1080/00137919008903024
D. M. Hamby, “A Review of Techniques for Parameter Sensitivity Analysis of Environmental Models,” Environmental Monitoring and Assessment, Vol. 32, No. 2, 1994, pp. 135-154. doi:10.1007/BF00547132
K. J. Lomas and H. Eppel, “Sensitivity Analysis Techniques for Building Thermal Simulation Programs,” Energy and Buildings, Vol. 19, 1992, pp. 21-44. doi:10.1016/0378-7788(92)90033-D
S. G. Tzafestas, N. Theodorou and A. Kanellakis, “Recent Advances in the Stability Analysis of Multidimensional Systems,” Information and Decision Technologies, Vol. 14, 1988, pp. 195-211
D. J. Pannell, “Sensitivity Analysis of Normative Economic Models: Theoretical Framework and Practical Strategies,” Agricultural Economics, Vol. 16, No. 2, 1997, pp. 139-152. DOI:10.1016/S0169-5150(96)01217-0
F. Canova, “Sensitivity Analysis and Model Evaluation in Simulated Dynamic General Equilibrium Economies,” International Economic Review, Vol. 36, No. 2, 1995, pp. 477-501. doi:10.2307/2527207
G. W. Harrison and H. D. Vinod, “The Sensitivity Analysis of Applied General Equilibrium Models: Completely Randomised Factorial Sampling Designs,” The Review of Economics and Statistics, Vol. 74, No. 2, 1992, pp. 357-362. doi:10.2307/2109672
P. Javanovic, “Application of Sensitivity Analysis in Investment Project Evaluation under Uncertainty and Risk,” International Journal of Project Management, Vol. 17, No. 4, 1999, pp. 217-222. doi:10.1016/S0263-7863(98)00035-0
G. E. P. Box, W. G. Hunter and J. S. Hunter, “Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building,” John Wiley & Sons, New York, 1978.
Z. M. Christopher, “Monte Carlo Simulation, Quantitative Applications in the Social Sciences,” Sage University Papers Series, No. 116, 1997.
L. Y. Pouliquen, “Risk Analysis in Project Appraisal,” Staff Occasional Paper, The World Bank, Washington DC, 1970.
M. Javid and P. Seneviratne, “Investment Risk Analysis in Airport Parking Facility Development,” Journal of Construction Engineering and Management, Vol. 126, No. 4, 2000, pp. 298-305. doi:10.1061/(ASCE)0733-9364(2000)126:4(298)
S. Button, “Project Duration Prediction Using a Monte Carlo Simulation of the Periodic Output of the Project Resources,” Monte Carlo Methods and Applications, Vol. 9, No. 3, 2003, pp. 217-225. DOI:10.1515/156939603322728987
W. Hurley, “On the Use of Martingales in Monte Carlo Approaches to Multiperiod Parameter Uncertainty in Capital Investment Risk Analysis,” The Engineering Eco- nomist, Vol. 43, No. 2, 1998. DOI:10.1080/00137919808903195
R. Clarke and A. Low, “Risk Analysis in Project Planning: A Simple Spreadsheet Application Using Monte- Carlo Techniques’,” Project Appraisal, Vol. 8, No. 3, 1993.
M. M. Akalu, “The Process of Investment Appraisal: The Experience of 10 Large British and Dutch Companies,” International Journal of Project Management, Vol. 21, No. 5, 2002, pp. 355-362. doi:10.1016/S0263-7863(02)00051-0
S. Reutlinger, “Techniques for Project Appraisal under Uncertainty,” World Bank Staff Occasional Papers No. 10, Johns Hopkins Press, Baltimore, 1970.
P. Belli, J. R. Anderson, J. A. Dixon and J. P. Tan, “Economic Analysis of Investment Operations: Analytical Tools and Practical Applications,” World Bank Institute Development Studies, Washington DC, 2001.
S. Szekeres, “Considering Uncertainty in Project Appraisal,” EDI, World Bank, Washington DC, 1986.
M. Dailami, I. Lipkovich and J. van Dyk, “INFRISK, a Computer Simulation Approach to Risk Management in Infrastructure Project Finance Transactions,” World Bank Policy Research Working Paper, the World Bank Economic Development Institute, Washington DC, 1999.
World Bank, “Risk & Uncertainty Analysis, Transport Economics,” Transport Note No. TRN-7, Policy and Poverty Thematic Group, 2005.
G. Pohl and D. Mihaljek, “Project Evaluation and Uncertainty in Practice: A Statistical Analysis of Rate-of-Return Divergences of 1015 World Bank Projects,” World Bank Economic Review, Vol. 6, No. 2, 1992, pp. 255-277. doi:10.1093/wber/6.2.255
Asian Development Bank, “Guidelines for the Economic Analysis of Guidelines for the Economic Analysis of Projects,” Economics and Development Resource Centre, On-Line Publication from ADB Homepage Site, 1999.
R. Baltussen, R. Hultubessy, D. Evans and C. Murray, “Uncertainty in Cost-Effectiveness Analyses: Probabilistic Uncertainty Analysis and Stochastic League Tables,” International Journal of Technology Assessment in Health Care, Vol. 18, No. 1, 2002, pp. 112-119.
M. Skitmore and S. Ng, “Analytical and Approximate Variance of Total Project Cost,” Journal of Construction Engineering and Management, Vol. 128, No. 5, 2002, pp. 456-460. doi:10.1061/(ASCE)0733-9364(2002)128:5(456)
P. Lorterapong and O. Moselhi, “Project-Network Analysis Using Fuzzy Sets Theory,” Journal of Construction Engineering and Management, Vol. 122, No. 4, 1996, pp. 308-318. doi:10.1061/(ASCE)0733-9364(1996)122:4(308)
J.-G. Cho and B.-J. Yum, “Functional Estimation of Activity Criticality Indices and Sensitivity Analysis of Expected Project Completion Time,” Journal of the Operational Research Society, Vol. 55, No. 8, 2004, pp. 850-859. doi:10.1057/palgrave.jors.2601739