Quality assurance and quality control
Overview
Quality assurance and quality control procedures were incorporated into all aspects of the monitoring programs, including:
- sampling (equipment preparation, sample collection and sample storage and transport)
- laboratory analysis
- data management
- community reporting.
The results of these assessments are discussed in this section.
Definitions
In general, quality control refers to all actions, procedures, checks and decisions undertaken to ensure the representativeness and integrity of samples, and the accuracy and reliability of analytical results. Quality assurance refers to all actions that monitor and measure the effectiveness of quality control procedures.
Program reviews
DECC staff conducted quality reviews with council staff to ensure that sample collection and data management were conducted in accordance with the recommended and accepted procedures.
The review involved an audit of water sampling techniques in the field, as well as an audit of documentation, data management and community reporting procedures at the council office.
The quality assurance reviews provided the opportunity for training of council staff and recommendations for program improvement.
All councils performed very well in the reviews, with no major issues identified. Council officers had excellent knowledge of their sites, potential pollution sources and sampling protocols.
Data management
All councils transferred water quality data to DECC on a regular basis for centralised storage on the Beachwatch water quality database (BACTO). In some cases, data were e-mailed directly from the analysis laboratory, otherwise from the council.
Quality assurance procedures for the storage of data on the centralized database followed a rigorous protocol that was developed as part of the Beachwatch program. This included data validation procedures to identify anomalous results.
Validated data were uploaded to the SoEdirect website by Beachwatch staff. This website ( www.soedirect.nsw.gov.au) provides online access to several important environmental monitoring data resources and assists councils to prepare their State of the Environment reports.
Laboratory analysis
Laboratory accreditation
Most laboratories employed in the BPP were accredited by the National Association of Testing Authorities (NATA). This included the accreditation of relevant laboratory methods, including analysis of indicator bacteria.
Interlaboratory comparison methodology
In order to test the quality of sample analysis of each laboratory, an inter-laboratory comparison study was undertaken. This involved the submission of five test samples to each laboratory during the 2007–2008 summer season.
Interlaboratory comparisons undertaken during the 2003–2004 summer season revealed that there were differences between laboratories (DEC 2004b). A potential cause of the variation in environmental samples was thought to be delays in sample analysis owing to overnight delivery to regional areas. This delay provided an opportunity for bacterial die-off or growth, causing variation in results. In order to overcome this problem, BioBalls® have been used to assess laboratory performance during the last four years.
BioBalls® are freeze-dried balls that contain an accurate number of organisms, either faecal coliforms or enterococci. They are supplied in glass vials and can be stored at –20ºC for up to 12 months. Once defrosted, the balls are aseptically added to 100 mL of sterile water, filtered, incubated and analysed in accordance with the laboratory method for the bacteria.
Five samples covering a range of microbial densities were sent by overnight courier to each laboratory. Samples were stored on ice. If samples were not analysed immediately, the vials were placed in the laboratory freezer.
Data collected in the interlaboratory comparison study were analysed in two ways: deviation from the known number of colony forming units/100 mL (density) and analysis of variance.
Deviation from the known bacterial density
The interlaboratory comparison results for faecal coliforms are presented in Figure 14. The results from the eight laboratories that undertook faecal coliform analysis are presented for each sample in grey and the known faecal coliform density of each sample is in black.
The results are tightly clustered around the known densities of samples 1 and 3, indicating a high level of certainty. However, there is greater variation in results for samples 2, 4 and 5, where faecal coliform levels were mostly underestimated by several laboratories.
Results that are within 0.3 log-units of the known value (equivalent to a halving or doubling of density on a linear scale) are considered to be acceptable. Seven of the eight laboratories consistently reported results within the acceptable range for faecal coliforms.
The laboratory that reported results outside the acceptable range did so for only one sample. Although this laboratory under-reported the bacteria count, the result was from a sample with low bacterial density, with the reported number and known density well below the median guideline.
Figure 14: Interlaboratory comparison results for faecal coliforms
The interlaboratory comparison results for enterococci are presented in Figure 15. The results from the seven laboratories are presented for each sample in grey, and the known enterococci density of each sample is in black. There is a high level of certainty for the enterococci results from samples 1, 2, 3 and 4, with data from most laboratories clustered around the known enterococci density. There is greater variation around sample 5, where enterococci levels were underestimated by all laboratories.
Figure 15: Interlaboratory comparison results for enterococci
Four of the seven laboratories reported enterococci values that were within the acceptable range of 0.3 log units from the known value (equivalent to a halving or doubling of density on a linear scale). Two laboratories reported a single result each that was just outside the range. One laboratory reported four results outside the acceptable range, all of which were under-underestimated. All laboratories were notified of their results.
Laboratory performance during 2007–2008 was similar to that recorded over the previous three summers, with most laboratories reporting results within 0.3 log units of the known values for most samples.
Analysis of variance
The results generated by each laboratory were statistically analysed to determine whether there were significant differences between results from different laboratories.
This analysis found that there were no significant differences between laboratories for enterococci and faecal coliforms.
Conclusion
Most laboratories reported results within the acceptable range for most samples. One laboratory reported a result that was outside the acceptable range for faecal coliforms and three laboratories reported results outside the range for enterococci. Although one laboratory did underestimate four enterococci samples, all faecal coliform results for that laboratory were within the acceptable range. The number of laboratories that generated acceptable results in 2007–2008 was an improvement from the previous year.
The clustering of results was similar to the previous year. This is because four of the eight laboratories generated results for both bacterial indicators all within the acceptable limits. Of the remaining laboratories, all but one generated only one result for either faecal coliforms or enterococci just outside the acceptable range.
In general, the results from the 2007–2008 laboratory quality assurance program were good, and with no significant difference found between the laboratories tested, confidence can be placed in all data in this report. Although the use of BioBalls® has minimised variation in results and provided a more accurate assessment of laboratory performance, they are not without problems. Poor results can be obtained if the preparation of the BioBall® samples is not undertaken with great care.
Page last updated: 26 February 2011
