CREED data
Records assessment purpose statement, relevant data summarised from above tables, relevance/reliability scores, and final assessment of data quality.
Source:vignettes/CREED_data.qmd
Introduction
Criteria for Reporting and Evaluating Exposure Datasets (CREED) is a framework for evaluating the relevance and reliability of a reference object for a given exposure assessment question across a series of questions. Users set criteria for it to be relevant to their question, and use a pre-defined set of reliability criteria.
The object is evaluated based on its data availability, precision, choice of methods, and a final score is calculated for both its relevance and reliability based on its useability at “Gold” and “Silver” levels of quality. This table collects the relevant data from non-CREED tables selected by the user or an automated data population process, the user-appraised scores, and any limitations the user chooses to comments on. In the data export process, the below functions are used to create two .CSV files: one for reliability and one for relevance. Scores are used to calculate a CREED score; the highest value for each Gold and Silver Reliability and Relevance score (0-4) is used to generate the overall score per category.
For example, if a data source scored 1, 2, 2 for Silver/Required Reliability and 4, 2 and 1 for Gold/Recommended Reliability, the final score would be 2 for Silver/Required Reliability and 4 for Gold/Recommended Reliability, which would translate into Silver Reliability “Reliable with Restrictions” and Gold Reliability “Not usable”.
This table is not intended to be used as a final data product, but is included in the format because it provides traceability on how the CREED score was calculated. Because a large part of the evaluation logic is included in the eData app, this table is currently just a container with little relevant information. It is planned to migrate this logic to the eDataDRF in future.
initialise_CREED_data_tibble()
#> # A tibble: 0 × 6
#> # ℹ 6 variables: criterion_id <chr>, criterion_title <chr>,
#> # required_recommended <chr>, relevant_data <chr>, score <chr>,
#> # limitations <chr>Variables
Criterion ID
criterion_id - string, free, mandatory
A alphanumeric serial identifier for CREED’s relevance (RV1 - RV11) and reliability (RB1 - RB19) criteria.
Criterion Title
criterion_title - string, free, mandatory
A short, descriptive title for the domain of the criterion, e.g. “Sample Medium/Matrix”.
Required or Recommended
required_recommended - string, free, mandatory
Whether the criterion is required (in which case it will be used to calculate a silver score), or recommended (gold score).
Relevant Data
relevant_data - string, free, mandatory
Any relevant data provided by the auto-population process or the user. For example:
1 compartment: Aquatic (Freshwater)
3 protocols:
Sampling Protocol - Point
Extraction Protocol - Not reported
Fractionation Protocol - Not reportedScore
score - string, free, mandatory
An assessment score based on the extent to which the data object fulfills each criterion. Criterion scores are stored as a named vector of numbers. A lower score (1) is better, a higher score (4) is worse. Note that there are five possible responses but only four numerical scores - both Not Applicable and Fully Met are considered a full pass (1).
CREED_choices_vocabulary()
#> Not Met Fully Met Partly Met Not Reported Not Relevant
#> 4 1 2 3 1Limitations
limitations - string, free, mandatory
Any limitations, caveats, or other relevant comments identified by the user conducting the assessment.
Example CREED Assessments
Example CREED relevance and reliability assessments of example data are provided below, for reference.
CREED Relevance
example_CREED_relevance_tibble() |>
dplyr::mutate(relevant_data = stringr::str_replace_all(relevant_data, "\n", "<br>")) |>
gt::gt() |>
gt::fmt_markdown(columns = relevant_data)| criterion_id | criterion_title | required_recommended | relevant_data | score | limitations |
|---|---|---|---|---|---|
| RV1 | Sample Medium/Matrix | Required | 1 compartment: Aquatic Sediment | 1 | |
| RV2 | Collection Method/Sample Type | Recommended | 1 protocol: Sampling Protocol - Grab: (Surface sediments collected using Van Veen grab sampler) |
1 | Sampling reported to adequate level of detail. |
| RV3 | Study Area | Required | 2 sites. Countries (1): Norway. Areas (1): Oslofjord. Lowest coordinate precision: 4 | 2 | Limited to two sites within single fjord system. |
| RV4 | Site Type | Recommended | 2 sites. Distinct features: Types: Coastal, fjord, Subtypes: Sediment | 2 | Rationale for specific site selection not provided. |
| RV5 | Sampling Timespan | Required | 2023-03-15 to 2023-04-20 (n=2) | 1 | |
| RV6 | Sampling Frequency | Required | 2023-03-15 to 2023-04-20 (n=2) | 2 | Single sampling event per site. |
| RV7 | Temporal Conditions | Recommended | Relevant data not collected by app. Please complete manually. | 4 | Season, weather conditions not reported. |
| RV8 | Analyte | Required | 1 protocol: Fractionation Protocol - Total |
1 | Total metal concentration appropriate for purpose. |
| RV9 | Sensitivity/LOD/LOQ | Required | 1 protocol: Analytical Protocol - ICP-MS: (Inductively coupled plasma mass spectrometry) |
1 | ICP-MS provides adequate sensitivity for environmental assessment. |
| RV10 | Summary Statistics Type | Recommended | Uncertainty types: Standard Deviation | 2 | Mean +/- SD reported. |
| RV11 | Supporting Parameters | Recommended | 1 protocol: Fractionation Protocol - Total |
1 | Total metals appropriate; no additional supporting parameters required. |
CREED Reliability
example_CREED_reliability_tibble() |>
dplyr::mutate(relevant_data = stringr::str_replace_all(relevant_data, "\n", "<br>")) |>
gt::gt() |>
gt::fmt_markdown(columns = relevant_data)| criterion_id | criterion_title | required_recommended | relevant_data | score | limitations |
|---|---|---|---|---|---|
| RB1 | Sample Medium/Matrix | Required | 1 compartment: Aquatic Sediment 2 samples. Grain size: <63um sieved fraction. |
1 | |
| RB2 | Collection Method/Sample Type | Recommended | 1 protocol: Sampling Protocol - Grab: (Surface sediments collected using Van Veen grab sampler) |
1 | |
| RB3 | Sample Handling | Recommended | Samples stored frozen at -20 degrees C until analysis. Dried at 60 degrees C before digestion. | 1 | |
| RB4 | Site Location | Required | 2 sites. Countries (1): Norway. Areas (1): Oslofjord. Lowest coordinate precision: 4 | 1 | |
| RB5 | Date and Time | Required | 2023-03-15 to 2023-04-20 (n=2) | 1 | |
| RB6 | Analyte(s) Measured | Required | 1 protocol: Analytical Protocol - ICP-MS: (Inductively coupled plasma mass spectrometry) |
1 | |
| RB7 | Limit of Detection and/or Limit of Quantification | Required | LOD: 0.1-0.3 mg/kg; LOQ: 0.5-1.0 mg/kg | 1 | |
| RB8 | Accreditation/Quality Management System | Required | Relevant data not collected by app. Please complete manually. | 4 | Justification: Laboratory accreditation status not reported in source. |
| RB9 | Method | Required | 4 protocols: Sampling Protocol - Grab Extraction Protocol - Acid digestion: (HNO3/HCl microwave-assisted) Fractionation Protocol - Total Analytical Protocol - ICP-MS |
1 | |
| RB10 | Lab Blank Contamination | Recommended | Method blanks analysed with each batch. | 2 | Blank values not explicitly reported. |
| RB11 | Recovery/Accuracy | Recommended | CRM MESS-3 analysed. Recovery: 95-102%. | 1 | |
| RB12 | Reproducibility/Precision | Recommended | Triplicate analyses performed. RSD <10%. | 1 | |
| RB13 | Field QC | Recommended | 1 protocol: Sampling Protocol - Grab |
4 | No evidence of field blanks or duplicates. |
| RB14 | Calculations | Recommended | Uncertainty types: Standard Deviation | 1 | Not relevant - no normalisation applied. |
| RB15 | Significant Figures | Recommended | Measured parameter values reported to 3 significant figures | 1 | |
| RB16 | Outliers | Recommended | Relevant data not collected by app. Please complete manually. | 1 | No outliers reported or removed. |
| RB17 | Censored Data | Required | Relevant data not collected by app. Please complete manually. | 1 | One value reported as <LOQ with LOQ provided. |
| RB18 | Summary Statistics Procedures | Recommended | Uncertainty types: Standard Deviation | 1 | |
| RB19 | Supporting Data Quality | Recommended | Relevant data not collected by app. Please complete manually. | 1 | No supporting parameters required. |
CREED Bibliography
Adam Peters, Michael Beking, Leonard Oste, Mick Hamer, Jeanne Vuaille, Andrew J. Harford, Thomas Backhaus, Stephen Lofts, Claus Svendsen, Charles Peck, Assessing the relevance of environmental exposure data sets, Integrated Environmental Assessment and Management, Volume 20, Issue 4, 1 July 2024, Pages 1004–1018, https://doi.org/10.1002/ieam.4881
Graham Merrington, Lisa H. Nowell, Charles Peck, An introduction to Criteria for Reporting and Evaluating Exposure Datasets (CREED) for use in environmental assessments, Integrated Environmental Assessment and Management, Volume 20, Issue 4, 1 July 2024, Pages 975–980, https://doi.org/10.1002/ieam.4899
Carolina Di Paolo, Irene Bramke, Jenny Stauber, Caroline Whalley, Ryan Otter, Yves Verhaegen, Lisa H. Nowell, Adam C. Ryan, Implementation of the CREED approach for environmental assessments, Integrated Environmental Assessment and Management, Volume 20, Issue 4, 1 July 2024, Pages 1019–1034, https://doi.org/10.1002/ieam.4909
Michelle L. Hladik, Arjen Markus, Dennis Helsel, Lisa H. Nowell, Stefano Polesello, Heinz Rüdel, Drew Szabo, Iain Wilson, Evaluating the reliability of environmental concentration data to characterize exposure in environmental risk assessments, Integrated Environmental Assessment and Management, Volume 20, Issue 4, 1 July 2024, Pages 981–1003, https://doi.org/10.1002/ieam.4893