Tiltaksovervakingen: opsjon for kvalitetskontroll av analysedata
This repository contains code for quality-checking and visualising data from external labs (VestfoldLAB and Eurofins) by comparing against historic data exported from Vannmiljø.
Project background
Since 2012, the “Tiltaksovervakingen” project has collected data to assess the effects of liming at more than 200 stations across southern Norway. NIVA was responsible for analysing the water chemistry data from this project between 2012 and 2015, then VestfoldLAB took over from 2016 to August 2020, and from September 2020 onwards the analyses will be carried out by Eurofins.
Data from 2012 to 2019 have been previously quality checked and are available in Vannmiljø. Data from VestfoldLAB from January to August 2020 were provided by VestfoldLAB in an Excel spreadsheet, and we have agreed with Miljødirektoratet that future data from Eurofins will be provided using the same template.
The aim of this workflow is to quality assess the “new” data and to highlight “outliers” for further investigation and reanalysis.
Stations
An interactive map showing station locations is here.
Quarterly quality checking
Each quarter, the lab submits data for assessment. The initial dataset is labelled “v1”. Various tests are performed to identify data issues and the feedback provided to the lab. Following reanalysis, a second dataset may be provided (“v2”). When available, this is checked again to ensure problems highlighted in v1 have been fixed.
Note: In many cases, reanalysis by the lab will confirm the original extreme values. In such cases, many of the outliers will still be present in v2 of the dataset. The aim of this workflow is to highlight outliers and possible bad data, but it is up to the lab (not NIVA) to decide which data are ultimately submitted to Vannmiljø.
Eurofins 2020 Q4 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2021 Q1 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2021 Q1 (v2)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2021 Q2 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2021 Q2 (v2)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2021 Q3 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2021 Q3 (v2)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2021 Q4 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2021 Q4 (v2)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2022 Q1 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2022 Q2 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2022 Q3 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2022 Q4 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2023 Q1 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2023 Q2 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2023 Q3 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2023 Q4 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2024 Q1 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Eurofins 2024 Q2 (v1)
- Summary of basic checks
- Distribution plots
- Time series outlier plots
- Table of time series outliers
- Water sample level outliers (table and plot)
- Distributions for aluminium
Workflow details
The notebooks below are run each quarter.