The big debate: Best-practice application of Monte Carlo simulation in probabilistic cost risk estimation

29 August 2017

Aquenta: It’s not surprising the Monte Carlo method is often a topic of vibrant discussion amongst our Aquenta planning, estimating and risk teams. As you’ll see below, the process itself looks relatively simple at first (as most things do in risk management and project controls). Although, there are various tricks and tactics often used along the way that lead to different approaches and most significantly, different outcomes – there is a great debate around best practice Monte Carlo modelling and simulation, particularly for probabilistic scheduling and cost estimation.

From memory, the first time I used the Monte Carlo method was to assess the range of possible afflux impacts (in hydrology, afflux is defined as a rise in the water level immediately upstream of and due to a natural or artificial obstruction, like a new bridge) while completing my first master degree in Hydraulic Design. It then took me another 10-15 years before I realised I needed to go back to UNSW to fully understand the quantification of risks and simulation modelling in delivery of major projects. You should never underestimate the need for long hours of manual iterations to fully understand the process.

So what is the Monte Carlo method? It is best described as a problem solving technique used to approximate the probability of certain outcomes by running multiple trial runs, called simulations, using random variables. Typically, a simulation will consist of 2,500 to 10,000 iterations in order to reach a steady-state result. The results of the simulation include risk-adjusted estimates and corresponding statistical estimate distributions; these provide the decision maker with a range of possible outcomes with a minimum and maximum value. The method has been successfully used in scientific applications for about 70 years in various applications, approaches and interpretations, and is recognised by ISO 31000 as a standard in risk management and risk assessment.

Stanislaw Ulam is often credited with inventing the Monte Carlo method in 1946 while pondering the probabilities of winning a card game of solitaire! The Polish born mathematician worked for John von Neumann on the United States’ Manhattan Project during World War II, and is primarily known for designing the hydrogen bomb with Edward Teller in 1951. He published his first paper on the Monte Carlo method in 1949.

In 2017, our schedule and cost risk assessments are vastly more complex, including economic factors such as rate uncertainties, cost estimating errors, and statistical uncertainty inherent in the estimate/schedule. Cost estimating risk assessment takes into account the cost, schedule, and contingent risks that are then planned back into the cost estimate.

A number of good industry practices state that the contingency element of any cost estimate needs to account for the likelihood and cost impact of three factors:

  1. Specific risks, or measured uncertainties,

  2. Defined but unmeasured uncertainties, around the estimate

  3. Unknown uncertainties, that at a given time are not known or understood

For projects that are relatively self-contained, such as projects within the resource sector, contingency is primarily concerned with the first two of these. However, in the early stages of large infrastructure projects, such as projects within the transport sector, a significant proportion of the risk exposure comes from the latter. These may derive from complex interfaces with the physical environment into which the infrastructure is to be built, as well as the unpredictable responses and requirements of stakeholders affected by the siting or performance of the infrastructure assets. A good application of Monte Carlo method should ensure the model addresses all these three factors.

At a high level, the process associated with the Monte Carlo method is as follows;

  1. Generate / obtain the ‘Most Likely Point Estimate’

  2. Identify and quantify Inherent Risks (e.g. cost estimating uncertainty)

  3. Identify and quantify Contingent Risks (e.g. technical risks)

  4. Quantify correlation/s

  5. Run simulation

  6. Review, adjust and re-run simulation

  7. Allocate appropriate contingency (e.g. to the WBS, Delivery Package etc)

As I mentioned, the process may look simple but due to varying approaches, and subsequent results, it is anything but and a common topic of debate. Some of the key areas of difference between the diverse approaches to Monte Carlo modelling are:

 

--ENDS--

 

Source:  Aquenta - www.aquenta.com.au

Contact:  N/A

External Links:  N/A

Recent news by:  Aquenta

 

 

ONLINE PROFILE