The big debate: Best-practice application of Monte Carlo simulation in probabilistic cost risk estimation
29 August 2017
Aquenta: It’s not surprising the Monte
Carlo method is often a topic of vibrant discussion amongst our Aquenta
planning, estimating and risk teams. As you’ll see below, the process
itself looks relatively simple at first (as most things do in risk
management and project controls). Although, there are various tricks and
tactics often used along the way that lead to different approaches and
most significantly, different outcomes – there is a great debate around
best practice Monte Carlo modelling and simulation, particularly for
probabilistic scheduling and cost estimation.
From memory, the first time I used the Monte Carlo method was to assess
the range of possible afflux impacts (in hydrology, afflux is defined as
a rise in the
water level immediately upstream of and due to a natural
or artificial obstruction, like a new
bridge) while completing my first
master degree in Hydraulic Design. It then took me another 10-15 years
before I realised I needed to go back to UNSW to fully understand the
quantification of risks and simulation modelling in delivery of major
projects. You should never underestimate the need for long hours of
manual iterations to fully understand the process.
So what is the Monte Carlo method? It is best described as a problem
solving technique used to approximate the probability of certain
outcomes by running multiple trial runs, called simulations, using
random variables. Typically, a simulation will consist of 2,500 to
10,000 iterations in order to reach a steady-state result. The results
of the simulation include risk-adjusted estimates and corresponding
statistical estimate distributions; these provide the decision maker
with a range of possible outcomes with a minimum and maximum value. The
method has been successfully used in scientific applications for about
70 years in various applications, approaches and interpretations, and is
recognised by ISO 31000 as a standard in risk management and risk
assessment.
Stanislaw Ulam is often credited with inventing the Monte Carlo method
in 1946 while pondering the probabilities of winning a card game of
solitaire! The Polish born mathematician worked for John von Neumann on
the United States’ Manhattan Project during World War II, and is
primarily known for designing the hydrogen bomb with Edward Teller in
1951. He published his first paper on the Monte Carlo method in 1949.
In 2017, our schedule and cost risk assessments are vastly more complex,
including economic factors such as rate uncertainties, cost estimating
errors, and statistical uncertainty inherent in the estimate/schedule.
Cost estimating risk assessment takes into account the cost, schedule,
and contingent risks that are then planned back into the cost estimate.
A number of good industry practices state that the contingency element
of any cost estimate needs to account for the likelihood and cost impact
of three factors:
-
Specific risks, or measured uncertainties,
-
Defined but unmeasured uncertainties, around the estimate
-
Unknown uncertainties, that at a given time are not known or understood
For projects that are relatively
self-contained, such as projects within the resource sector, contingency
is primarily concerned with the first two of these. However, in the
early stages of large infrastructure projects, such as projects within
the transport sector, a significant proportion of the risk exposure
comes from the latter. These may derive from complex interfaces with the
physical environment into which the infrastructure is to be built, as
well as the unpredictable responses and requirements of stakeholders
affected by the siting or performance of the infrastructure assets. A
good application of Monte Carlo method should ensure the model addresses
all these three factors.
At a high level, the process associated with the Monte Carlo method is
as follows;
-
Generate / obtain the ‘Most Likely Point Estimate’
-
Identify and quantify Inherent Risks (e.g. cost estimating uncertainty)
-
Identify and quantify Contingent Risks (e.g. technical risks)
-
Quantify correlation/s
-
Run simulation
-
Review, adjust and re-run simulation
-
Allocate appropriate contingency (e.g. to the WBS, Delivery Package etc)
As I mentioned, the process may look simple but due to varying approaches, and subsequent results, it is anything but and a common topic of debate. Some of the key areas of difference between the diverse approaches to Monte Carlo modelling are:
-
Validation of Most Likely Point Estimate
-
Quantification of Inherent Risks (e.g. applying ranging to cost line items, critical items and uncertainties that may drive the cost estimate)
-
Quantification of Contingent Risks (e.g. likelihood vs probability, risks with multiple consequences)
-
Correlations to be assessed (e.g. cost items, WBS levels, contingent risks, risk-factor ranges)
-
Simulation (e.g. probability distribution functions, number of iterations)
-
Allocation (e.g. based on dollar value, risk profile or risk exposure)
--ENDS--
Source: Aquenta - www.aquenta.com.au
Contact: N/A
External Links: N/A
Recent news by: Aquenta