Measurement, Modeling and Reproducible Evidence

By the end of this unit students should be able to (a) diagram relevant stocks, flows, feedbacks and boundaries; (b) identify at least two mechanisms compatible with incomplete data; (c) compute the unit-consistent screen and recognize its simplifying assumptions; (d) design a measured contrast or control; (e) discuss the difference between evidence for a process and the ethical choice of response; and (f) prepare a short decision note with uncertainty, stakeholder impacts and specific follow-up actions.

7.1 Sampling design and representativeness

Core mechanism and governing scale. Random, systematic, stratified and targeted samples answer different questions. Sampling hotspots intentionally increases the chance of detecting a localized problem but cannot estimate regionwide prevalence without correcting selection. Accessibility, seasonality and budgets can determine what is missing. This is a physical claim about a set of processes, not a description of a single visible feature. To use it rigorously, distinguish the quantity being measured from a proxy and draw an explicit spatial and temporal boundary. If the relevant mass or energy can cross that boundary, the investigator must consider transfers as well as any material produced or consumed internally. A strong explanation predicts how a change in forcing would alter at least two independently observable consequences. Identify those predictions before seeing the sample data, so an initially appealing interpretation can genuinely be rejected.

Site investigation. Design a fictional 30-site groundwater survey spanning agricultural, urban and protected land with a separate sentinel network near a suspected plume. Write the site description in a way that another geoscientist can reconstruct the comparison. Record location, elevation or depth, coordinate system, local time, sampling interval, instrument settings, quality-control flags and any reason observations were excluded. A sample label should distinguish a point from a mixture of points, and a laboratory result should identify the phase analyzed: filtered water, whole water, sediment, porewater or another material. For a process that varies after rainfall, one visit gives a snapshot rather than a stable baseline. Use pre-event and post-event observations where the mechanism makes a temporal prediction.

Working with competing explanations. More samples taken in one convenient neighborhood may add less information than fewer well-located samples. Shared laboratory batch errors can make observations correlated. Formulate two alternative mechanisms that could plausibly produce the observation, then specify a measurement that would discriminate them. Merely collecting more measurements of the same ambiguous quantity may increase precision without increasing explanatory power. Test whether the apparent change can arise from sensor drift, dilution, sample composition, changed boundary conditions or an unrelated common driver. A transparent analysis reports which explanations were evaluated and which remain unresolved. Numerical agreement between a model and data is not a sufficient test if the model parameters were selected using those same data.

Decision interpretation. Register the population of inference, selection method, sample size and exclusion rules before collecting data. State the practical action that could change under the result, then calculate how an error in the parameter would affect that action. For a preliminary screening study, use conservative but explicit uncertainty bounds rather than claim exact compliance. Distinguish a threshold from a causal mechanism: exceeding a screening value may justify follow-up, while remaining below it does not automatically prove no ecological or human-health impact. Prioritize measures that interrupt a credible exposure pathway if an urgent hazard is identified, and refer regulatory or public-health determinations to competent authorities.

Worked quantitative screen (7.1). sampling. Use the wholly invented parameters sampled=30 locations, total=120 cells. The teaching expression is sampled / total × 100; substitute consistently converted units to obtain 30 ÷ 120 × 100 = 25, or approximately 25 percent spatial cells sampled. The arithmetic illustrates the chosen model; it should not be mistaken for a field observation or a calibrated regulatory prediction. Ask whether doubling one input would double the output. If it would not, explain the nonlinearity. Name one missing parameter that could reverse a site decision despite this seemingly precise result.

7.2 Calibration, precision and uncertainty

Core mechanism and governing scale. Accuracy concerns closeness to a reference value; precision concerns repeated measurement spread. Calibration standards, blanks, duplicates, drift checks and detection limits each detect different weaknesses. Reporting extra decimal places does not reduce uncertainty. This is a physical claim about a set of processes, not a description of a single visible feature. To use it rigorously, distinguish the quantity being measured from a proxy and draw an explicit spatial and temporal boundary. If the relevant mass or energy can cross that boundary, the investigator must consider transfers as well as any material produced or consumed internally. A strong explanation predicts how a change in forcing would alter at least two independently observable consequences. Identify those predictions before seeing the sample data, so an initially appealing interpretation can genuinely be rejected.

Site investigation. Track a fictional conductivity probe over a week using daily checks against standards and repeated measurements at a stable reference station. Write the site description in a way that another geoscientist can reconstruct the comparison. Record location, elevation or depth, coordinate system, local time, sampling interval, instrument settings, quality-control flags and any reason observations were excluded. A sample label should distinguish a point from a mixture of points, and a laboratory result should identify the phase analyzed: filtered water, whole water, sediment, porewater or another material. For a process that varies after rainfall, one visit gives a snapshot rather than a stable baseline. Use pre-event and post-event observations where the mechanism makes a temporal prediction.

Working with competing explanations. A small coefficient of variation can coexist with a systematic bias. Measurement uncertainty should be combined with sampling uncertainty where relevant. Formulate two alternative mechanisms that could plausibly produce the observation, then specify a measurement that would discriminate them. Merely collecting more measurements of the same ambiguous quantity may increase precision without increasing explanatory power. Test whether the apparent change can arise from sensor drift, dilution, sample composition, changed boundary conditions or an unrelated common driver. A transparent analysis reports which explanations were evaluated and which remain unresolved. Numerical agreement between a model and data is not a sufficient test if the model parameters were selected using those same data.

Decision interpretation. Provide enough raw observations and quality flags to recreate the reported conclusion. State the practical action that could change under the result, then calculate how an error in the parameter would affect that action. For a preliminary screening study, use conservative but explicit uncertainty bounds rather than claim exact compliance. Distinguish a threshold from a causal mechanism: exceeding a screening value may justify follow-up, while remaining below it does not automatically prove no ecological or human-health impact. Prioritize measures that interrupt a credible exposure pathway if an urgent hazard is identified, and refer regulatory or public-health determinations to competent authorities.

Worked quantitative screen (7.2). cv. Use the wholly invented parameters sd=0.4 μS/cm, mean=50 μS/cm. The teaching expression is sd / mean × 100; substitute consistently converted units to obtain 0.4 ÷ 50 × 100 = 0.8, or approximately 0.8 percent coefficient of variation. The arithmetic illustrates the chosen model; it should not be mistaken for a field observation or a calibrated regulatory prediction. Ask whether doubling one input would double the output. If it would not, explain the nonlinearity. Name one missing parameter that could reverse a site decision despite this seemingly precise result.

7.3 Geospatial data, scale and resolution

Core mechanism and governing scale. A map pixel is an observation or model estimate over an area, not a point truth. Coarse elevation products can miss berms and drainage channels, while high-resolution imagery can still have positioning errors. Coordinate reference systems, datums and time stamps must be compatible across layers. This is a physical claim about a set of processes, not a description of a single visible feature. To use it rigorously, distinguish the quantity being measured from a proxy and draw an explicit spatial and temporal boundary. If the relevant mass or energy can cross that boundary, the investigator must consider transfers as well as any material produced or consumed internally. A strong explanation predicts how a change in forcing would alter at least two independently observable consequences. Identify those predictions before seeing the sample data, so an initially appealing interpretation can genuinely be rejected.

Site investigation. Overlay a fictional flood depth raster with school locations and drainage culverts. Check the effect of shifting coordinates by one pixel. Write the site description in a way that another geoscientist can reconstruct the comparison. Record location, elevation or depth, coordinate system, local time, sampling interval, instrument settings, quality-control flags and any reason observations were excluded. A sample label should distinguish a point from a mixture of points, and a laboratory result should identify the phase analyzed: filtered water, whole water, sediment, porewater or another material. For a process that varies after rainfall, one visit gives a snapshot rather than a stable baseline. Use pre-event and post-event observations where the mechanism makes a temporal prediction.

Working with competing explanations. The modifiable areal unit problem and ecological fallacy can produce misleading associations between aggregated exposure and disease counts. Missing rooftops in a digital terrain model may matter to flood routing. Formulate two alternative mechanisms that could plausibly produce the observation, then specify a measurement that would discriminate them. Merely collecting more measurements of the same ambiguous quantity may increase precision without increasing explanatory power. Test whether the apparent change can arise from sensor drift, dilution, sample composition, changed boundary conditions or an unrelated common driver. A transparent analysis reports which explanations were evaluated and which remain unresolved. Numerical agreement between a model and data is not a sufficient test if the model parameters were selected using those same data.

Decision interpretation. Publish map scale, positional uncertainty and source metadata next to the visualization. State the practical action that could change under the result, then calculate how an error in the parameter would affect that action. For a preliminary screening study, use conservative but explicit uncertainty bounds rather than claim exact compliance. Distinguish a threshold from a causal mechanism: exceeding a screening value may justify follow-up, while remaining below it does not automatically prove no ecological or human-health impact. Prioritize measures that interrupt a credible exposure pathway if an urgent hazard is identified, and refer regulatory or public-health determinations to competent authorities.

Worked quantitative screen (7.3). pixels. Use the wholly invented parameters area=2.25 km², pixel=0.0009 km². The teaching expression is area / resolution area; substitute consistently converted units to obtain 2.25 ÷ 0.0009 = 2500, or approximately 2,500 number of raster cells. The arithmetic illustrates the chosen model; it should not be mistaken for a field observation or a calibrated regulatory prediction. Ask whether doubling one input would double the output. If it would not, explain the nonlinearity. Name one missing parameter that could reverse a site decision despite this seemingly precise result.

7.4 Inverse modeling and equifinality

Core mechanism and governing scale. Environmental observations are often explained by multiple parameter combinations. For example, a contaminant plume may appear similar under higher dispersivity and lower source strength. Parameter fitting finds models that match data, but equifinality means good fit alone does not establish mechanism. This is a physical claim about a set of processes, not a description of a single visible feature. To use it rigorously, distinguish the quantity being measured from a proxy and draw an explicit spatial and temporal boundary. If the relevant mass or energy can cross that boundary, the investigator must consider transfers as well as any material produced or consumed internally. A strong explanation predicts how a change in forcing would alter at least two independently observable consequences. Identify those predictions before seeing the sample data, so an initially appealing interpretation can genuinely be rejected.

Site investigation. Fit a fictional decay curve using two parameter sets and compare predictions at an unmeasured downstream well. Write the site description in a way that another geoscientist can reconstruct the comparison. Record location, elevation or depth, coordinate system, local time, sampling interval, instrument settings, quality-control flags and any reason observations were excluded. A sample label should distinguish a point from a mixture of points, and a laboratory result should identify the phase analyzed: filtered water, whole water, sediment, porewater or another material. For a process that varies after rainfall, one visit gives a snapshot rather than a stable baseline. Use pre-event and post-event observations where the mechanism makes a temporal prediction.

Working with competing explanations. An optimized solution may be physically implausible or overfit measurement noise. Validation requires independent observations not used to tune parameters. Formulate two alternative mechanisms that could plausibly produce the observation, then specify a measurement that would discriminate them. Merely collecting more measurements of the same ambiguous quantity may increase precision without increasing explanatory power. Test whether the apparent change can arise from sensor drift, dilution, sample composition, changed boundary conditions or an unrelated common driver. A transparent analysis reports which explanations were evaluated and which remain unresolved. Numerical agreement between a model and data is not a sufficient test if the model parameters were selected using those same data.

Decision interpretation. Disclose model structure, priors or bounds, calibration objective and predictions sensitive to disputed assumptions. State the practical action that could change under the result, then calculate how an error in the parameter would affect that action. For a preliminary screening study, use conservative but explicit uncertainty bounds rather than claim exact compliance. Distinguish a threshold from a causal mechanism: exceeding a screening value may justify follow-up, while remaining below it does not automatically prove no ecological or human-health impact. Prioritize measures that interrupt a credible exposure pathway if an urgent hazard is identified, and refer regulatory or public-health determinations to competent authorities.

Worked quantitative screen (7.4). residual. Use the wholly invented parameters observed=14.2 mg/L, predicted=11.5 mg/L. The teaching expression is observed − predicted; substitute consistently converted units to obtain 14.2 − 11.5 = 2.7, or approximately 2.7 mg/L residual. The arithmetic illustrates the chosen model; it should not be mistaken for a field observation or a calibrated regulatory prediction. Ask whether doubling one input would double the output. If it would not, explain the nonlinearity. Name one missing parameter that could reverse a site decision despite this seemingly precise result.

7.5 Open workflows and evidence provenance

Core mechanism and governing scale. Reproducible analysis preserves input data, units, cleaning decisions, versioned code, numerical methods and figure-generation steps. A workflow can be computationally reproducible yet scientifically invalid if sampling is biased or a physical assumption is wrong. Documentation must permit critical reanalysis without exposing private individuals. This is a physical claim about a set of processes, not a description of a single visible feature. To use it rigorously, distinguish the quantity being measured from a proxy and draw an explicit spatial and temporal boundary. If the relevant mass or energy can cross that boundary, the investigator must consider transfers as well as any material produced or consumed internally. A strong explanation predicts how a change in forcing would alter at least two independently observable consequences. Identify those predictions before seeing the sample data, so an initially appealing interpretation can genuinely be rejected.

Site investigation. Create a hypothetical data dictionary, versioned script and decision log for a contaminated-well investigation; write a unit test for mass-balance closure. Write the site description in a way that another geoscientist can reconstruct the comparison. Record location, elevation or depth, coordinate system, local time, sampling interval, instrument settings, quality-control flags and any reason observations were excluded. A sample label should distinguish a point from a mixture of points, and a laboratory result should identify the phase analyzed: filtered water, whole water, sediment, porewater or another material. For a process that varies after rainfall, one visit gives a snapshot rather than a stable baseline. Use pre-event and post-event observations where the mechanism makes a temporal prediction.

Working with competing explanations. An undocumented correction of outliers can change the apparent trend. A synthetic teaching dataset must not be presented as a real survey. Formulate two alternative mechanisms that could plausibly produce the observation, then specify a measurement that would discriminate them. Merely collecting more measurements of the same ambiguous quantity may increase precision without increasing explanatory power. Test whether the apparent change can arise from sensor drift, dilution, sample composition, changed boundary conditions or an unrelated common driver. A transparent analysis reports which explanations were evaluated and which remain unresolved. Numerical agreement between a model and data is not a sufficient test if the model parameters were selected using those same data.

Decision interpretation. Publish de-identified datasets with limitations, licensed sources, reproducible calculations and an explicit distinction between measurements and scenario assumptions. State the practical action that could change under the result, then calculate how an error in the parameter would affect that action. For a preliminary screening study, use conservative but explicit uncertainty bounds rather than claim exact compliance. Distinguish a threshold from a causal mechanism: exceeding a screening value may justify follow-up, while remaining below it does not automatically prove no ecological or human-health impact. Prioritize measures that interrupt a credible exposure pathway if an urgent hazard is identified, and refer regulatory or public-health determinations to competent authorities.

Worked quantitative screen (7.5). completeness. Use the wholly invented parameters valid=162 records, collected=180 records. The teaching expression is valid / collected × 100; substitute consistently converted units to obtain 162 / 180 × 100 = 90, or approximately 90 percent valid records. The arithmetic illustrates the chosen model; it should not be mistaken for a field observation or a calibrated regulatory prediction. Ask whether doubling one input would double the output. If it would not, explain the nonlinearity. Name one missing parameter that could reverse a site decision despite this seemingly precise result.

Research brief. Design a network with two laboratory methods, a field meter, satellite data, a well inventory and an open analysis repository. Work with the accompanying datasets/unit-07-synthetic.csv file. The data are deliberately synthetic and not observations from a real study. Students must not use the outputs to locate actual pollution or certify safety. Before drawing a graph, write down the spatial and temporal extent of the question, the target population of inference, and the causal alternative being examined. If these differ across competing hypotheses, explicitly state the difference rather than changing the question after plotting the data.

Initial scenario evidence. A fictional seasonal trend appears in routine station data but not in a one-off volunteer survey. The network fails to sample several communities near a low-lying floodplain. An appropriate notebook records the stated evidence separately from new calculations and speculative explanations. Construct a table with source, transport medium, receptor, measurement, unit, time stamp and quality flag. Do not automatically interpolate through a gap. Describe what direct observations could distinguish spatial gradients from a measurement artifact. Think carefully about whether measurements made on the same day are independent: weather, laboratory batches and access constraints can introduce common disturbances.

Hypothesis comparison. Investigate sampling selection, sensor drift, reference-frame mismatch and missing data before inventing an environmental mechanism. Express each hypothesis as a short process chain, from cause to transport or transformation and then to observable consequence. For each chain, describe a finding that would weaken it even if the original observation remained true. This is more demanding than simply asking students to find supporting evidence. Write a four-row evidence matrix containing current observations, ambiguous clues, missing observations and tests capable of discrimination. Use a mixture of field observations and existing institutional records rather than putting the entire burden on the laboratory assay.

Measurement design. Select a reference location, a likely impact location and at least one location that tests the suspected transport direction. Plan a repeat period long enough to capture the dominant forcing, with a predeclared justification for the number and location of samples. Document the measurement unit, calibration procedure, field duplicate, transport method, chain-of-custody practice when needed and appropriate data-handling permissions. Explain what a non-detect means in relation to the reporting limit. If the accessible sampling locations omit vulnerable groups, state how that limitation affects the conclusion rather than quietly treating data absence as exposure absence.

Quantitative analysis. Read the CSV and calculate the mean, range and a simple temporal contrast for one indicator. Plot indicator against the forcing variable and inspect outliers on the original scale before considering transformation. Report the result with units and avoid inferring causation from one correlation coefficient. Test sensitivity to removal of one observation, alternative grouping of sites and plausible systematic error. If a variable is coded as a toy index, keep it clearly labeled as such; there is no physical unit hiding behind a derived ranking. A good report includes a notebook where a reader can reproduce every number without proprietary data access.

Uncertainty and decision. Draft reproducible outputs: map metadata, dataset dictionary, quality-control summary, code version and a statement of populations not represented. List which uncertainties are reducible through new measurements and which depend on social values or future land-use choices. Produce at least two recommendations: a near-term low-regret action and a longer-term investigation or remediation option. For each, state the measurable trigger that would alter the recommendation and the organization that must act. An unexplained rank ordering is weaker than a transparent comparison across environment, health, fairness, time, cost and implementability. If a real-world hazard were suspected, qualified professionals and local authorities would have to replace these toy values with regulatory procedures.

Deliverables and review rubric. Submit a labeled site diagram; a one-page hypothesis table; a reproducible calculation file; a figure whose axis labels include units; a paragraph describing alternative mechanisms; a decision table with at least three choices; and an evidence-quality note. Award 20% for physically plausible mechanisms, 20% for transparent data provenance, 20% for unit-consistent arithmetic and uncertainty checks, 20% for discriminating experiment design and 20% for decision clarity and treatment of affected stakeholders. A polished figure alone receives no substitute credit for missing science.

Construct a two-page dossier for a regional environmental observatory operating sensors, laboratories and satellites. Start with a map or cross-section of the physical environment, then list stocks, fluxes or forces that operate on multiple time scales. Use one instrument observation, one derived physical quantity, one sensitivity test and one clearly identified decision assumption. Organize claims by evidence quality and specify what additional field or laboratory result would change the conclusion. A scientifically defensible exercise may conclude that the available fictional evidence is insufficient for strong attribution.

For peer review, exchange dossiers and ask a partner to locate unsupported causal leaps, missing mass-balance terms, numerical unit mistakes and hidden stakeholder costs. The reviewer should mark every model parameter as measured, calibrated, hypothesized or fixed for teaching. In a revision note, explain what changed and what remains unknown. Do not substitute an unverified AI summary or a single plotted trend for this explicit review process.

Mastery test: Define the core process in one paragraph, show one correct calculation with units, draw two plausible mechanisms capable of producing the same initial observation and give a measurement that separates them. Make a separate recommendation that identifies who benefits or bears costs; explicitly label which part is scientific inference and which part is a value-based decision.