Looking under a model’s hood

One of the biggest difficulties for a regulator in assessing the accuracy of groundwater model predictions is that the models themselves are generally not made available and in any case cannot be readily examined by non-expert modellers.  Instead, regulators are presented with a report about the model and its outcomes, but these reports often do not accurately convey all of the important information about how the model has been constructed, how key assumptions have been made and what uncertainties remain.  It is too easy and too tempting to hide a model’s weak points or a modeller’s uncertainty about how to simulate a groundwater regime by omission in a report if you are confident that the model will not be examined.

This situation is somewhat improved if the model has been independently peer-reviewed as recommended by the Australian Groundwater Modelling Guidelines (Barnett et al, 2013) and their equivalents in other countries, but as a regulator I have been frequently disappointed by the tick-box quality of these peer reviews.  I’m aware of numerous instances where professional peer-reviews have been dutifully undertaken in models that later turned out to be very poor for their purpose.

One solution to this impasse is for the model report to be accompanied by a set of exported model layers and other information in a standardised electronic format that would enable the regulator (or any other well-informed stakeholder) to visualise how the model has been constructed and whether key outputs appear sensible.  The model layers and information would sometimes need to be tailored depending on the decision(s) that the model is seeking to inform, but it is still possible to nominate a “routine” set of model layers to export as a starting point.

I’m not sure why this hasn’t been attempted as far as I’m aware, but one reason is how much format matters.  The format for these export files needs to be sensibly restricted to enable the regulator/ stakeholder to import them into GIS-compatible visualisation software, such as xxx.  I have had some experts provide some guidance on formatting requirements and will make that available to anyone interested by email, or maybe in a future post if I can get the advice debadged.

An informed observer can quickly learn a lot from the layers and information set out below.  By importing the required layers into visualisation software, one can see for example how aquifer recharge has been spatially interpreted and whether this makes sense in view of the known topography and geology.

The following is a list of my nominations for the “Minimum Model Export” list which I hope one day to get into government regulations – they can of course be used by anyone and can be adjusted for local/regulatory/stakeholder needs.

The list is broken into two tables below; Table 1 contains the metadata and information that can be better communicated outside a graphical environment, whilst Table 2 lists those attributes that are better understood in a spatial context, and are able to be imported into a groundwater visualisation package.

 

Table 1. Standard model information to be provided in electronic text or spreadsheet files

Property/ Parameter Description
Model Objectives A clear statement of the purpose of the model, as discussed in the Australian Modelling Guidelines (Barnett et al, 2012).
Model Metadata Single text file that stores the high level model metadata as a .json object file in accordance with formatting requirements (separately stipulated).  Any number of attributes may be specified in this file, e.g. model/run name, created date, author etc.
Grid Definition Multiple text files exported as .json files in accordance with formatting instructions , describing the 3D grid used by the model and its position in space.
Groundwater Pumping/Extraction Rates Applied Excel table summarising predicted groundwater pumping or other discharge rates applied in model
Scenario Properties Excel table explaining variables used in each scenario presented in report.
Receptors Excel table summarising locations and types of key receptors considered in modelling
Water Budget Excel table summarising key volumetric flows (water budget components) across the model space at key time slices*, both within model domain and through external boundaries.

 

Table 2 presents the layers and parameters which could be routinely exported from the model.

Table 2. Minimum model layers and parameters to be exported from model

Property Parameter Description
Stratigraphy 3D representation of key stratigraphic layers (with labels based on Geoscience Australia stratigraphic names)
Hydraulic Properties Key hydraulic properties generated following calibration, specifically vertical and horizontal hydraulic conductivity (Kv & Kh), storativity [s] at key time slices*.
Rainfall & Evapotrans A map summarising annual rainfall and evapostranspiration rates if areally varied.
G’water Recharge Map layer showing inferred recharge rates and spatio-temporal variability (if any applied) for shallowest aquifer for each main time step*.
Borehole Data Stratigraphic or lithological logs and temporal water level measurements of key monitoring wells (see Tab C for format requirements).
Structural Geology Major faults, shear zones or joint sets incorporated into model (if any applied).
Topography Surface topography (presented as a digital elevation model)
Boundary Conditions Graphical representation of the boundary conditions applied to each edge of the model.
Surface Water Features Stream/lake bottom elevations and heads, streambed conductance.
Surface Water Interaction Fluxes at base of stream/lake/drain features at key time slices*.
Receptors Map layer showing location of main receptors, with legend denoting receptor types.
Predicted Heads Separate contoured layers showing groundwater (and surface water if relevant) heads at pre-mining and current conditions, and predicted at key time slices*.
Certainty Level of model certainty/confidence associated with spatial data.  As described in Section 8.5.7 and Figure 8-6 of the Australian Groundwater Modelling Guidelines (Barnett et al, 2012), one option is to present certainty as a colour density or by varying the transparency of a layer to indicate the level of uncertainty.

* Note 

Where the parameters are requested in terms of key time slices, the default set of events for which those parameter should be provided are as follows:

  1. Predicted conditions at commencement of the activity
  2. Predicted conditions at “peak impact” for that parameter
  3. Predicted conditions at completion of the activity

Predicted conditions once steady-state water conditions have returned following project closure.

Author: glassearth

Looking hard at how we can share resources sustainably through better regulation

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s