This chapter explains the risk assessment method the Regulator uses to consider applications for DIR and DNIR licences. This method is also applied to consideration of other matters, for example preparing advice on the classification of GMO dealings as exempt or notifiable low-risk dealings (sections 74, 75 and 140–143 of the Act), on an emergency dealing determination (section 72B(2)(c)) or on the GMO Register (section 79). The purpose of the risk assessment is to identify and characterise risks to the health and safety of people or to the environment from dealings with GMOs posed by or as the result of gene technology.
Risk assessment can be usefully viewed as a narrative that answers a set of key questions (see Figure 4.1), namely:
- What could go wrong? (Risk identification) Initially, a broad range of circumstances is considered, whereby the proposed dealings with a GMO are postulated to give rise to harm to people or the environment (risk scenarios). Each risk scenario describes a plausible causal linkage between the GMO and harm.
- How serious could the harm be? (Risk characterisation—consequence assessment) An identified risk is subjected to an assessment of the seriousness of potential harm via the particular risk scenario.
- How likely is the harm to occur? (Risk characterisation—likelihood assessment) An identified risk is also assessed with regard to the chance of the occurrence of a series of individual steps in a risk scenario that may lead to harm. The assessment will derive the chance of harm from the overall series of individual steps.
- What is the level of concern? (Risk evaluation) The level of risk is evaluated as negligible, low, moderate or high by considering a combination of the seriousness of harm and the likelihood of it occurring. Risk evaluation determines whether or not mitigation measures to reduce risk are required.
Figure 4.1: Considerations for risk assessment
In practice, the risk assessment process tends to be iterative and the steps depicted in Figure 4.1 can be viewed as part of a repeated cycle. The risk assessment steps may be repeated:
- as a result of ongoing accumulation of information (such as data requested from the applicant, expert advice, consultation, or literature searches)
- as a result of development of more specific consequence criteria when substantive risks are identified and considered in more detail
- as a result of consideration of potential interactions between postulated risk scenarios, or
- in response to the monitoring and review process (see Chapter 5).
The degree of consideration given to each cycle of the process should correlate with the degree of risk; greater consideration should be given to risks that are potentially more substantial.
The results obtained in the risk assessment process are used to prepare the risk management plan (see Chapter 5).
Risk identification considers what could go wrong from activities with a GMO. It is the ‘process of finding, recognising and describing risk’. Risks are identified within the context established for the risk assessment (see Chapter 3), taking into account the proposed dealings with the GMOs, controls or limits for DIRs, or containment measures for DNIRs, relevant baseline information on the parent organism and/or other suitable comparator; and the receiving environment.
Postulating risk scenarios
Initially, risk identification considers a wide range of circumstances where potential harm to people or the environment could be credibly linked to exposure to the GMO or GM product, or the introduced genetic material (risk scenarios).
A risk scenario can be viewed as a ‘what if’ statement that describes a possible set of circumstances that might give rise to harm in the future. It is an hypothesis constructed from three essential components (Figure 4.2).
- A risk source. A new or altered property/trait of the GMO.
- A potential harm to people or the environment.
- A plausible causal linkage between components 1 and 2.
Figure 4.2: Components of a risk scenario
However, the relevance or importance of a risk scenario will depend on the context. The effects of a novel GM trait need to be considered in the context of the whole organism. Also, the plausibility of a causal linkage to harm will depend on a broad range of external factors such as the type of containment or confinement, availability of sexually compatible relatives, likely environmental conditions or the nature of nearby land use/functions.
Many possible risk scenarios can be formulated, but only those risks that may be greater than negligible are considered in detail in the risk assessment. For example, Hayes et al. (2004) proposed almost 200 risk scenarios for the commercial release of a hypothetical herbicide-tolerant GM canola. However, only four risk scenarios were considered substantive for two commercial releases of herbicide-tolerant canola (OGTR 2003a; OGTR 2003b).
In addition, interactions between risk scenarios may give rise to synergistic, additive or antagonistic effects. For instance:
- synergism arises when the combined effects are greater than the sum of the individual effects
- additive effects may occur when different scenarios lead to the same adverse outcome, which could increase the negative impact
- antagonistic effects may occur when the introduced trait alters the characteristics of the organism in opposing ways.
Top of page
The techniques available for developing a comprehensive set of risk scenarios range from checklists and brainstorming to targeted analysis. Techniques the Regulator uses may include previous agency experience, reported international experience, consultation, scenario analysis and inductive reasoning (fault and event tree analysis). The handbook that accompanies ISO 31000:2009 (Standards Australia 2012) and Hayes (2004) contains details of a range of other structured decision-making techniques that may be useful in postulating risk scenarios for proposed dealings with GMOs.
The type of information used to establish the risk assessment context includes the genotype and phenotype of the GMO, the proposed dealings, the parent organism, the receiving environment, and any relevant previous releases. Information on other factors might also be applicable to postulating risk scenarios, but not all will be relevant to all risk assessments or require the same degree of consideration. The factors include:
- altered biochemistry
- altered physiology
- unintended change in gene expression
- production of a substance that is toxic or allergenic to humans
- production of a substance that is toxic to other organisms
- survival and persistence at the release site
- survival and persistence outside the release site
- gene flow by sexual gene transfer
- gene flow by horizontal gene transfer
- expression of an introduced gene that may alter the infectivity or pathogenicity, host range, transmissibility, pathogen load or vector specificity of a disease agent
- interaction of introduced genes or products related to pathogenicity with other pathogens
- unintended effects on an existing non-GM weed, pest or pathogen
- secondary effects (such as development of herbicide resistance in related species as a result of gene flow)
- altered production (such as farming) practices
- alteration to the physical environment, including biogeochemical cycles that partition chemical elements and compounds between the living and non-living parts of the ecosystem
- unauthorised activities, including vandalism and terrorism.
Identifying risks that require further characterisation
Risk identification should be comprehensive and rigorous; however, care should be taken to avoid over-emphasising insubstantial risk scenarios. Risks that warrant detailed consequence and likelihood assessments to determine the level of risk they pose to human health and safety or to the environment are generally identified by considering the questions:
- Is the potential harm attributable to gene technology? Any harm not posed by or resulting from the use of gene technology should not be considered.
- Is there a plausible and observable pathway linking the proposed dealings to the potential harm? In cases where no plausible or observable pathways link the proposed dealings to the potential harm, the risk scenario should not be considered further.
- Is the risk substantive? After an initial consideration of the chance and seriousness of harm, does the risk scenario warrant more detailed consideration?
It is important to avoid underestimating or missing substantive risks. Therefore, the Regulator takes a cautious approach, postulating and considering an extensive list of potential risk scenarios. As a result, some identified potential risks can subsequently be classified as negligible risks after more detailed consequence and likelihood assessments.
The approach the Regulator uses also includes consulting a number of people with relevant expertise in the risk assessment process and by extensive internal review, and in the case of DIRs, external review of the risk assessment.
Risk characterisation determines the seriousness of harm (consequence assessment) and the chance of harm (likelihood assessment) from a GMO. The likelihood and consequence assessments are based on inferences from the available scientific and technical information, and include consideration of uncertainty. In the process of more detailed characterisation of identified risks, the generic criteria for the nature and types of consequences described in Chapter 2 are continually updated and become more clearly elaborated to allow evidence-based characterisation of a specific risk scenario.
Quantitative and qualitative assessment
Likelihood and consequence assessments can be either quantitative (reporting risks numerically) or qualitative (reporting risks descriptively). For instance, likelihood can be expressed as a relative measure of either probability (from zero to one, where zero is an impossible outcome and one is a certain outcome) or as frequency (the number of occurrences per unit of time). For qualitative assessments, likelihood is expressed in terms of highly likely, likely, unlikely and highly unlikely.
Quantitative risk assessment determines the conditional probabilities of risk and the associated statistical error (uncertainty). This type of analysis can be used where there is a history of accumulated information, such as with chemical and industrial manufacturing. Quantitative risk assessments are most useful for addressing narrowly defined risks with relatively simple pathways, leading to well-specified adverse outcomes. However, some forms of structured decision making (eg Bayesian belief networks) attempt to quantify probabilities in more complex situations.
Quantitative assessments use numerical values, which may be derived from:
- experimental data
- extrapolation from experimental studies on related systems
- historical data, or
- inference from models used to describe the system and its interactions.
Qualitative assessments use relative descriptions of likelihood and consequences, and can combine data derived from various sources, including quantitative data, if available. By using qualitative assessments, the maximum amount of information can be used in describing likelihood and consequence.
Use of qualitative or quantitative approaches depends on the amount, type and quality of available data; the complexity of the risk scenario under consideration; and the level of detail needed to make a decision. Some of the relative merits that distinguish the two approaches are listed in Table 4.1 (Hart 2001).
Top of page
Table 4.1: Relative merits of qualitative and quantitative risk assessments
Type of assessment
For GMOs, qualitative risk assessments are, in most instances, the most appropriate form because:
- there may be limited long-term experience with particular organisms and/or introduced genes/traits
- there is an absence of demonstrated harm
- potential harm relating to human health and safety and the environment is highly varied
- environmental effects manifest within highly complex systems that have many incompletely understood variables
- harm may occur in the long term through indirect routes, for example through interaction with impacts from climate change, and is therefore difficult to quantify.
The weaknesses of qualitative assessments described in Table 4.1 can be controlled and minimised in several ways, including the use of different terms for the various levels of likelihood, consequences and risk to reduce ambiguity. Potential variations between assessors can be reduced through quality control measures such as internal and external review and sourcing of expert advice. Differing viewpoints, perspectives and biases can be reduced through stakeholder input via effective consultation. Validation of findings can be supported by the monitoring and review processes.
Nevertheless, there is a requirement for testable and repeatable scientific evidence to support qualitative estimates of likelihood and consequences according to measurable, observable criteria of harm to human health and safety or to the environment. For example, toxicological or epidemiological data may be used in cases where harm is postulated to arise from the presence of toxins, allergens or other chemicals, such as enzyme inhibitors or anti-nutrients.
Consequence is ‘harm to protection goals from an activity’; in particular, harm to people or to the environment. A consequence assessment determines the potential degree of seriousness of harm (see Table 4.2). The seriousness of harm is dependent on the scale at which impacts are considered. Harm to humans is usually considered significant at the level of an individual, whereas harm to the environment is usually considered significant at the level of species, communities or ecosystems.
The presence of vulnerable, including rare or endangered, individuals, populations, species, communities or ecosystems is also considered. For example, if a genetic modification resulted in production of a protein with allergenic properties, some people may have no reaction to that protein, others may react mildly, while others may be severely affected.
Assessing the seriousness of harm to people or to the environment may include consideration of:
- What is the magnitude of each potential adverse impact: does it cause a large change over baseline conditions?
- What is the spatial extent or scale of the potential adverse impact?
- What is the temporal occurrence of the impact, namely, the duration and frequency? Does it cause a rapid rate of change? Is it likely to occur in the short or long term? What is the duration (day, year, decade) for which an impact may be discernible, and the nature of that impact over time? Is it intermittent and/or repetitive, if so, how often? Will it disappear?
- Can the adverse impact be reversed and, if so, how long will this take?
- Is the exposed species rare or endangered?
Table 4.2 provides a descriptive scale for the seriousness of harm in relation to the health of people and in relation to the environment. The explanations are relatively simple so as to be applicable to the wide range of possible licence applications and potential risks. The variety of potential risks may be affected by different factors (magnitude, scale, time, reversibility) that may contribute to the significance of adverse outcomes. For specific risks, these descriptors may be defined in more detail.
Table 4.2: Consequence assessment scale
|Consequence assessment||Degree of potential harm to the health of people and the environment due to gene technology relative to the parent organism|
|Marginal||Minimal or no increase in illness/injury to people.|
Minimal or no increase in harm to desirable components of the environment.
|Minor||Minor increase in illness/injury to people that is readily treatable.|
Minor increase in damage to desirable components of the environment that is reversible and limited in time and space or numbers affected.
|Intermediate||Significant increase in illness/injury to people that requires specialised treatment.|
Significant increase in damage to desirable components of the environment that is widespread but reversible or of limited severity.
|Major||Significant increase in severity of illness/injury to people, or large numbers of people affected, and generally not treatable.|
Major increase in damage to desirable components of the environment, with extensive biological or physical disruption to whole ecosystems, communities or an entire species, which persists over time.
In some cases, these qualitative descriptors may be supported by quantitative descriptors for certain harms. For example, the adverse impact of a GMO to reduce the establishment of desirable vegetation would be considered marginal, if the GMO does not affect the germination and seedling survival of desired plants (eg regenerating pasture, sown crops, planted trees, regenerating native vegetation); minor, if the GMO stops the establishment of less than 10% of desired plants; intermediate, if the GMO stops the establishment of between 10% and 50% of desired plants; and major, if the GMO stops the establishment of more than 50% of desired plants.
Desirable organisms or components of the environment that should be protected (or undesirable counterparts that should be controlled) are determined by legislation, government policies, national and international guidance material, and widely accepted community norms.
The consequences are assessed with respect to the impact of gene technology. Where there is no appropriate comparator parent organism, such as may be the case for some products of synthetic biology, then the generic consequence assessment scale (Table 3.1) can be used.
Likelihood assessmentTop of page
The likelihood assessment determines the chance that harm will occur, and is expressed as highly likely, likely, unlikely or highly unlikely (see Table 4.3). If the chance of harm is close to zero, then risk is considered minimal and needs no further analysis. However, care needs to be exercised when considering the remote possibility of risks that may have extreme adverse impacts.
Table 4.3: Likelihood assessment scale
|Likelihood of harm from gene technology|
|Highly unlikely||Harm may occur only in very rare circumstances|
|Unlikely||Harm could occur in some limited circumstances|
|Likely||Harm could occur in many circumstances|
|Highly likely||Harm is expected to occur in most circumstances|
Factors that are important in considering the likelihood of harm occurring are those related to plausible linkages between a dealing with a GMO and potential harm to people or susceptible entities in the environment from exposure to the GMO, the introduced gene(s) or products of the introduced gene(s).
Identifying all steps in a causal pathway leading to harm is important for deriving an overall assessment of the chance that harm may occur. For example, a causal pathway leading to increased harm (eg weediness or pathogenicity) may involve many steps, including transfer of the introduced genetic material from the GMO into a sexually compatible relative; survival and increased fitness of the recipient species; followed by spread and persistence of the recipient species, which then results in harm (eg reduced establishment of native plants in a protected area). If several steps have only a small chance of occurring, then the overall pathway has an extremely limited chance of occurring due to the combination of several low probability steps. Alternatively, one step may have almost no chance of occurring (eg the co-occurrence of a sexually compatible relative is not expected due to incompatible climate requirements between the GMO and its relative), resulting in a very low overall probability even if all other steps have a reasonable chance of occurring.
Assessing likelihood is more difficult for complex pathways. For instance, horizontal gene transfer from a GM plant or animal to a pathogenic micro-organism requires a large number of events to occur in sequence. However, occurrence of the gene transfer does not necessarily result in harm. Further steps are necessary, including the ability of the newly modified micro-organism to survive, replicate, display a selective advantage over the parent organism and give rise to some identifiable harm, such as increased virulence. In such cases, the overall likelihood of an adverse outcome occurring will be substantially lower than the likelihood of any individual step.
In contrast, scenarios that outline a simpler route to a potentially adverse outcome, such as a gene product that is toxic to non-target organisms, usually allow more robust estimates of likelihood, particularly as there is often a direct correlation between the dose of toxin and the severity of the adverse outcome, and the mechanism of action may have been experimentally verified.
For limited and controlled releases there is a fixed period for the intentional release, but any potential for adverse effects beyond this period must also be considered. As with any predictive process, accuracy is often greater in the shorter term than in the longer term.
In the case of DNIRs with pathogens in containment, the first step in developing plausible causal pathways to potential harm involves activities with the GMO that could give rise to infection of a laboratory worker or release from the containment facility, leading to spread and/or persistence of the GMO in the environment. This first step is considered to have only a small chance of occurring for GMOs in PC2 facilities due to the containment and some work practice requirements, and only a rare chance for GMOs in PC3 or PC4 facilities due to even greater requirements for protection that apply to these facilities.
Quality of evidence
Evidence used in risk identification and risk characterisation is obtained by a thorough review of the relevant scientific literature and from information supplied by the licence applicant. In addition, evidence is obtained in the form of advice from GTTAC and other prescribed agencies in the case of DIR licence applications and certain DNIR licence applications. The Regulator may also consult other relevant experts to gain information.
An applicant must supply information as prescribed by the Regulations (if any) and as specified in writing by the Regulator (section 40) (eg in the application forms). In the absence of adequate information, the Regulator may not consider the application or may request further information from the applicant.
It is important to consider the quality of the evidence (WHO 2008), including how much and what type of data are needed. Determining the quality of the evidence includes consideration of:
- appropriateness—the degree to which the data are relevant and applicable to the risk assessment question
- reliability—the accuracy and integrity of experimental design, methodology and statistical analysis used to report data and conclusions
- transparency—the clarity and completeness with which all key data, methods and processes, as well as the underlying assumptions and limitations, are documented, available, reproducible and capable of independent verification
- expertise—the standing of the author(s) or expert(s) presenting the data
- strength—how much data there is to support the conclusion in the scientific literature; whether there is conflicting data and the strength of the conflicting data
- robustness—whether data from disparate sources, experiments or researchers support similar conclusions.
Factors that may influence the relevance and value of the information include whether the:
- subject of the experiment is identical, similar or different from the GMO being assessed
- experiment is addressing a question relevant to the risk assessment
- experiment was performed in Australia or overseas.
Peer-reviewed papers are often regarded as high-value evidence, but they are not automatically accepted and used in the risk assessment without further evaluation. Their appropriateness, transparency and robustness are all factors in determining how much reliance can be placed on each piece of evidence.
Figure 4.3 illustrates how the Regulator may view the value of some different types of information. Information may be ranked low in one criterion but high in others. The overall value of the data for the risk assessment is open to the Regulator’s judgment.
Figure 4.3: Some types of information and their relative value as evidence
|Validated studies conducted according to international protocols meeting defined standards.|
Peer reviewed literature—strongly supported reports, models, theories.
Peer reviewed literature—single report, model, theory.
Opinion of an expert familiar with the GMO, parent organism, modified traits, ecology.
General biological principles.
Other technical reports, specialist literature, government reports, etc.
Experience of no reports of a problem.
|Experimental data on the GMO and/or parent organism in the Australian environment.|
Experimental data on the GMO and/or parent organism overseas.
Experimental data on modified traits in other organisms.
Experimental data on related surrogate systems.
The combined weight of evidence may also influence the risk assessment: a single strong piece of information (as judged by the above criteria) may stand on its own, or a number of weaker pieces of evidence may support each other, enabling the Regulator to have sufficient confidence in the information. In addition, judgment is needed to determine the sufficiency of the data to achieve a reliable and robust evaluation of risk, including consideration of residual uncertainty. Collection and assessment of unnecessary or excessive data is an inefficient use of resources for applicants and the Regulator.
Where another Australian regulatory agency or a regulatory agency of another country has made an assessment of the same or a similar GMO, their findings are taken into account during the Regulator’s risk assessment (regulation 10(1)(a)). The Regulator has established links with relevant agencies that facilitate exchange of information. The Regulator also participates in work by international agencies, such as the OECD, to produce documentation that contributes to harmonisation of regulatory activities between countries, which simplifies consideration of other countries’ assessments.
It is important to consider not only the available information, but also uncertainty associated with the evidence. For example, if data regarding a proposed dealing with a GMO are unavailable, inconsistent or incomplete, the significance of that absence, inconsistency or incompleteness will be considered in the risk assessment process.
Risk evaluationTop of page
Risk is evaluated against the objective of protecting the health and safety of people and the environment to determine the level of concern and, subsequently, the need for controls to mitigate or reduce risk. Risk evaluation may also aid consideration of whether the proposed dealings should be authorised, need further assessment, or require collection of additional information.
Factors used to determine which risks need treatment may include:
- risk criteria
- level of risk
- uncertainty associated with risk characterisation
- interactions between potential risks.
Figure 4.4: Risk matrix used to estimate the level of risk
Table 4.4: Evaluation of the level of risk
|Level of risk||Risk evaluation definitions|
|Negligible||Risk is of no discernible concern and there is no present need to invoke actions for mitigation.|
|Low||Risk is of minimal concern, but may invoke actions for mitigation beyond standard practices.|
|Moderate||Risk is of marked concern and will necessitate actions for mitigation that need to be demonstrated as effective.|
|High||Risk is of considerable concern that is unacceptable unless actions for mitigation are highly feasible and effective.|
Risk matrices should generally keep the number of risk categories within the matrix to a minimum and the inherent sources of uncertainty associated with formulation of the risk matrix should be reduced (Cox 2008).
The Regulator applies a set of distinct descriptors to the consequence assessment (Table 4.2), likelihood assessment (Table 4.3) and level of risk (Table 4.4) to reduce ambiguity of terminology used in qualitative risk assessments. Application of these descriptors to identified risks must be considered in the context of the proposed dealings, including the introduced trait, the parent organism and the receiving environment.
Typically, the method used for preparing a risk assessment in relation to licences is an iterative process that places increasing focus on risks that are more substantive and usually require more information, more detailed characterisation and a closer examination of uncertainty (see Figure 4.5). Many potential risks are considered initially, but most of these will be insubstantial. Therefore, fewer risks will remain that require a more detailed assessment and even fewer risks that will warrant consideration for risk treatment.
Figure 4.5: Summary of approach used for preparing a risk assessment for DIRs and DNIRs
In the case of DIRs, after preparing the risk assessment the Regulator considers whether one or more dealings proposed to be authorised by the licence may pose a significant risk to the health and safety of people or to the environment under section 52(2)(ba) of the Act. If the Regulator determines there is a significant risk, a longer period of consultation is mandated.
Although determination of significant risk is made on a case-by-case basis, it is expected that in most cases risk would be considered significant if the risk requires control or mitigation measures. These risks correspond to a level of risk that the Regulator has estimated as either moderate or high. In some cases, risks estimated to be low, but evaluated as requiring risk treatment, may also be determined as significant. In contrast, risks considered to not need mitigation (that is, negligible risks) would not be expected to be considered significant.Top of page