Unit 7: Instrument Transformers: Current Transformers (CT), Potential Transformers (PT), and Phase Errors

Unit 7 — Sensors, signal conditioning and calibration

Why this unit matters. Translate strain, temperature, current and displacement into electrically measurable quantities.

Application setting. Build mathematical calibration curves before relying on a sensor as evidence. The examples are synthetic, deliberately low-energy, and do not establish safe procedures for handling energized high-power systems.

Concept map

A sensor is not identical to its electrical output. It is a chain from physical measurand to material response, transducer, excitation, wiring, amplifier and digitizer. Temperature, stress direction, spatial averaging and mounting create biases that cannot be removed by digital arithmetic alone. Calibration establishes a relation over a specified range rather than proving every physical mechanism understood.

A small sum of squared residuals is not enough. Test independent inputs, rising and falling trajectories, different environmental conditions and repeated runs. Compare interpolations against extrapolations. Hysteresis or creep means that the same instantaneous input can produce multiple outputs depending on history, defeating an unconditional one-to-one calibration curve.

Lessons and reading sequence

• Lesson 7.1 — [Strain gauges and bridge conditioning](../lessons/U07-L01-strain-gauges-and-bridge-conditioning.md)
• Lesson 7.2 — [Temperature sensors: RTD and thermistor](../lessons/U07-L02-temperature-sensors-rtd-and-thermistor.md)
• Lesson 7.3 — [Magnetic current sensors and calibration](../lessons/U07-L03-magnetic-current-sensors-and-calibration.md)
• Lesson 7.4 — [Least-squares calibration, hysteresis and validation](../lessons/U07-L04-least-squares-calibration-hysteresis-and-validation.md)

After reading the lessons, complete the [unit assessment](../assessments/unit-07.html), [research-style project](../projects/unit-07.html), and [interactive software model](../simulations/model-07.html). Follow the sequence for conceptual continuity, but revisit earlier examples when error analysis changes the meaning of a later result.

Five essential terms

• Gauge factor: Strain sensitivity of a resistive element.
• RTD: Resistance temperature detector.
• Hysteresis: History-dependent difference for same input.
• Holdout: Independent calibration-validation sample.
• Self-heating: Sensor temperature change from excitation power.

Cross-lesson example: competing explanations

Assume two engineers report conflicting results for the same notional specimen. One engineer repeats a measurement ten times and obtains a very narrow cluster. The second uses another range, reference, sample timing or wiring topology and obtains a systematic difference. The first engineer's repeatability does not prove that the second engineer is wrong. In this unit, list the physical differences between instrument chains and predict which could create a reproducible offset or scale change. Identify one controlled experiment that would separate the causes without relying on a higher number of decimal places.

Unit laboratory notebook template

Use fields for measurand, connection topology, symbol definitions, numeric input sources, model assumptions, instrument settings, raw observations, rejection criteria, derived values, uncertainty budget, limitations and conclusion. Include a citation for each manufacturer specification if a real instrument is discussed. Reserve a separate field for software version and simulator parameters; a teaching simulator is not a calibrated artifact.

Reader checkpoint

1. Explain the difference between an instrument's reported indication and the intended measurand.
2. Give one case in which making the display resolution finer does not improve the measurement result.
3. Identify one quantity in the unit simulator that must be experimentally calibrated before using the equation for a real system.
4. Define the most important limitation of the idealized model and a practical observation that could falsify it.

Answer guidance. A sound response links the observed quantity to the physical chain and names a bias or uncertainty source instead of vaguely saying 'human error.' It distinguishes a model prediction from independent evidence. Complete safety qualification is outside the scope of this book.

Safety note: The examples are synthetic and intended for analysis or software-only simulation. Do not connect measuring instruments to mains, high-energy equipment, CT secondaries or exposed conductors without authorized training, appropriately rated equipment and site-specific procedures. Instrument-category markings, isolation ratings and current limits must be verified in the manufacturer documentation.

7.1 — Strain gauges and bridge conditioning

Unit 7: Sensors, signal conditioning and calibration Level: undergraduate, with prerequisite circuit theory, calculus and introductory probability. Estimated study: 70–100 minutes, including the numeric investigation. Tools: calculator, spreadsheet or the original offline simulator in this book. Evidence: examples are synthetic unless explicitly identified otherwise.

A problem that a measurement must answer

A bonded strain gauge changes its resistance under deformation; gauge factor converts fractional length strain to fractional resistance. A Wheatstone quarter-bridge detects the small change but is sensitive to temperature, lead effects and excitation drift. Amplifier input offset may exceed the signal if the gain is chosen casually.

The application for this unit is: Build mathematical calibration curves before relying on a sensor as evidence. The meaningful output is not necessarily the number displayed by a meter. First decide whether it is the unperturbed circuit voltage, a time average, a reference-scale resistance, a physical temperature or a confidence-qualified estimate. That decision determines the topology, calibration approach and acceptable model. In an instrumentation report it is better to leave a result undecided than to replace the unmeasured quantity with a convenient proxy without disclosure.

Mathematical model and physical assumptions

Central relation: GF=(ΔR/R)/ε; for a small-change quarter-bridge |Vo|≈(Vs/4)GF|ε|.

Derivation strategy

Start by writing the quantity and its reference state as a sentence. Identify every input that is actually measured and every value that is assumed. If a parameter came from a manufacturer specification, label it as a specification rather than calling it an independent observation. Use the equation to map those inputs onto the desired output. For a laboratory record, store instrument mode and environmental conditions alongside the number. In this case the key input evidence is the reading or physical parameter introduced above, not the final calculated answer. An experimenter should separate constants taken from a definition, instrument parameters taken from a datasheet and conditions measured on the day of use. If an assumed parameter is poorly characterized, a small model residual does not make the resulting prediction precise. The uncertainty of a quantity calculated from a product, quotient or nonlinear relation must follow its sensitivity to each contributing input, not a simple count of display digits.

Why the relation is useful

Transfer functions of sensors A sensor is not identical to its electrical output. It is a chain from physical measurand to material response, transducer, excitation, wiring, amplifier and digitizer. Temperature, stress direction, spatial averaging and mounting create biases that cannot be removed by digital arithmetic alone. Calibration establishes a relation over a specified range rather than proving every physical mechanism understood. The formula above is one of the operational links between a physical system and a numerical record. It is not an endorsement of any particular manufacturer's ratings, which must be checked independently. For instance, ideal resistance or sampling calculations disregard temperature-dependent and frequency-dependent behavior unless those are included explicitly.

Fully worked numerical case

For Vs=5 V, GF=2 and ε=500 μstrain, output magnitude ≈5×2×500e−6/4=1.25 mV. A 2 mV amplifier offset would overshadow the intended signal without calibration.

Uncertainty, nonidealities and instrumentation bias

Nonuniform strain, adhesive creep, transverse sensitivity and thermal expansion affect physical interpretation. A stable output voltage can conceal mechanical creep if the calibration reference is unchanged. The first concern affects the credibility of the numerical parameter; the second suggests a rival explanation of the observed behavior. A useful uncertainty analysis therefore goes beyond listing a collection of percentages. It defines the output measurand, classifies major sources as additive or multiplicative, evaluates correlations when necessary, and identifies the sensitivity of the output to each one. An uncertainty budget should preserve documentary support for Type B inputs and raw observations for Type A estimates. If the model is an approximation, give a bounded validity statement in addition to any statistical interval.

A counterexample to overconfidence

Validation beyond regression A small sum of squared residuals is not enough. Test independent inputs, rising and falling trajectories, different environmental conditions and repeated runs. Compare interpolations against extrapolations. Hysteresis or creep means that the same instantaneous input can produce multiple outputs depending on history, defeating an unconditional one-to-one calibration curve. This contextual distinction provides a way to design a discriminating check. For an instrument-chain problem, change input range, source impedance, frequency or wiring to see whether the measurement follows the predicted dependence. For a data-analysis problem, change sample rate, observation length or independent reference while holding the physical input constant. If a conclusion persists only because the same assumption is reused in every run, that persistence supplies little additional evidence.

Engineering use and what would be required to trust it

Choose compensation gauges or full-bridge patterns when temperature and bending modes matter. Calibration records, environmental limits and independent checks belong in the method, not just in appendices. Choose an experiment that isolates the main influence while staying within authorized, properly rated equipment and low-energy pedagogical conditions. A formula computed with fictional values is evidence that arithmetic was performed, not that a real device passed a standard. When the output feeds a safety, certification or financial decision, the appropriate reviewer and agreed acceptance rule must be identified before collecting measurements.

Study exercise 1 — reconstruct the baseline

Derive or justify GF=(ΔR/R)/ε; for a small-change quarter-bridge |Vo|≈(Vs/4)GF|ε|. from the relevant conservation law, circuit relation, transfer model or statistical definition. Recompute the worked example, label every unit and distinguish the physical value from what an actual instrument could have displayed. Identify one data item whose uncertainty is not quantified by the example. Explain what would happen if that datum were systematically too high by a small amount.

Study exercise 2 — sensitivity experiment

Use the same physical configuration but change a controlled parameter by 10% in the direction that would expose a plausible failure mechanism. Compute or predict the response using the stated equation, and state whether the output sensitivity is linear, inverse, nonlinear or conditional. Now specify one nonideal effect, grounded in this lesson, that could cause a real observation to differ. A technically sufficient answer must compare two predicted numerical or symbolic outputs; a verbal statement that the error “increases” is not enough where values are available.

Study exercise 3 — measurement design

Design a low-energy or software-only verification that could distinguish the main mechanism from this competing explanation: A stable output voltage can conceal mechanical creep if the calibration reference is unchanged. Specify the independent variable, the measured variable, the reference standard or alternate observation, the expected trend and a stop criterion for an invalid run. Describe what metadata would be saved for another engineer to reproduce the analysis without interviewing the experimenter.

Solution and grading guide

Exercise 2. The independent variable is changed while the model assumptions remain fixed. For a linear proportional dependency, +10% in the input produces +10% in the relevant output contribution; for a reciprocal dependence the output becomes 1/1.1=0.9091 of the original; for a squared dependence it becomes 1.1²=1.21. Apply only the form actually supported by GF=(ΔR/R)/ε; for a small-change quarter-bridge |Vo|≈(Vs/4)GF|ε|.; do not choose a generic factor without checking the equation. Explain how nonuniform strain, adhesive creep, transverse sensitivity and thermal expansion affect physical interpretation. could invalidate an extrapolation.

Exercise 3. The strongest design isolates one cause using a second evidence pathway. A stable output voltage can conceal mechanical creep if the calibration reference is unchanged. should lead to a different prediction from the central mechanism. For example, changing source impedance while fixing source voltage is an effective loading test; changing sample rate while holding a physical sine frequency constant reveals aliasing; testing increasing and decreasing excitation reveals hysteresis. The appropriate case must match the instrument and phenomenon studied here.

Simulator and extension

Use [interactive Unit 7 model](../simulations/model-07.html) to explore the unit's central instrumentation mechanism. The simulator may focus on a related quantity rather than exactly this lesson's formula. Save the default model inputs and at least two modified cases; attach a screenshot or numerical result table. Compare one manually calculated point with the model output, and state a physical mechanism that the simulated equations omit. The original example dataset is not field evidence and must not be cited as such.

Safety note: The examples are synthetic and intended for analysis or software-only simulation. Do not connect measuring instruments to mains, high-energy equipment, CT secondaries or exposed conductors without authorized training, appropriately rated equipment and site-specific procedures. Instrument-category markings, isolation ratings and current limits must be verified in the manufacturer documentation.

7.2 — Temperature sensors: RTD and thermistor

Unit 7: Sensors, signal conditioning and calibration Level: undergraduate, with prerequisite circuit theory, calculus and introductory probability. Estimated study: 70–100 minutes, including the numeric investigation. Tools: calculator, spreadsheet or the original offline simulator in this book. Evidence: examples are synthetic unless explicitly identified otherwise.

A problem that a measurement must answer

A platinum RTD offers a relatively predictable resistance-versus-temperature relation over a specified range; a thermistor is more nonlinear but highly sensitive. Lead resistance is significant for RTDs, especially in two-wire arrangements. Self-heating produces a bias that depends on measurement current and thermal contact.

The application for this unit is: Build mathematical calibration curves before relying on a sensor as evidence. The meaningful output is not necessarily the number displayed by a meter. First decide whether it is the unperturbed circuit voltage, a time average, a reference-scale resistance, a physical temperature or a confidence-qualified estimate. That decision determines the topology, calibration approach and acceptable model. In an instrumentation report it is better to leave a result undecided than to replace the unmeasured quantity with a convenient proxy without disclosure.

Mathematical model and physical assumptions

Central relation: For a Pt100 local approximation R(T)≈100Ω[1+0.00385 T] near 0°C, with T in °C.

Derivation strategy

Begin with the lowest-complexity circuit or signal model that explains the key effect, then state the approximations it makes. For a loaded circuit, preserve current paths and terminal definitions; for a sampled signal, preserve sample timing and analog bandwidth; for a statistical model, preserve independence assumptions. A circuit may obey Kirchhoff laws perfectly while the inferred physical quantity is still wrong because the sensor transfer function is uncharacterized. In this case the key input evidence is the reading or physical parameter introduced above, not the final calculated answer. An experimenter should separate constants taken from a definition, instrument parameters taken from a datasheet and conditions measured on the day of use. If an assumed parameter is poorly characterized, a small model residual does not make the resulting prediction precise. The uncertainty of a quantity calculated from a product, quotient or nonlinear relation must follow its sensitivity to each contributing input, not a simple count of display digits.

Why the relation is useful

Transfer functions of sensors A sensor is not identical to its electrical output. It is a chain from physical measurand to material response, transducer, excitation, wiring, amplifier and digitizer. Temperature, stress direction, spatial averaging and mounting create biases that cannot be removed by digital arithmetic alone. Calibration establishes a relation over a specified range rather than proving every physical mechanism understood. The formula above is one of the operational links between a physical system and a numerical record. It is not an endorsement of any particular manufacturer's ratings, which must be checked independently. For instance, ideal resistance or sampling calculations disregard temperature-dependent and frequency-dependent behavior unless those are included explicitly.

Fully worked numerical case

Using the linear approximation, a Pt100 reads 119.25 Ω at 50°C. Two 1 Ω leads measured in series give 121.25 Ω, which would suggest ≈55.19°C under the same model—a 5.19°C lead-induced error.

Uncertainty, nonidealities and instrumentation bias

Actual calibrated RTD curves use more complete relationships; the linear formula is a restricted teaching approximation. A sensor mounted far from a heat source may indicate its own housing temperature rather than the desired process measurand. The first concern affects the credibility of the numerical parameter; the second suggests a rival explanation of the observed behavior. A useful uncertainty analysis therefore goes beyond listing a collection of percentages. It defines the output measurand, classifies major sources as additive or multiplicative, evaluates correlations when necessary, and identifies the sensitivity of the output to each one. An uncertainty budget should preserve documentary support for Type B inputs and raw observations for Type A estimates. If the model is an approximation, give a bounded validity statement in addition to any statistical interval.

A counterexample to overconfidence

Validation beyond regression A small sum of squared residuals is not enough. Test independent inputs, rising and falling trajectories, different environmental conditions and repeated runs. Compare interpolations against extrapolations. Hysteresis or creep means that the same instantaneous input can produce multiple outputs depending on history, defeating an unconditional one-to-one calibration curve. This contextual distinction provides a way to design a discriminating check. For an instrument-chain problem, change input range, source impedance, frequency or wiring to see whether the measurement follows the predicted dependence. For a data-analysis problem, change sample rate, observation length or independent reference while holding the physical input constant. If a conclusion persists only because the same assumption is reused in every run, that persistence supplies little additional evidence.

Engineering use and what would be required to trust it

Compare self-heating power I²R against sensor thermal dissipation constant before selecting excitation. Calibration records, environmental limits and independent checks belong in the method, not just in appendices. Choose an experiment that isolates the main influence while staying within authorized, properly rated equipment and low-energy pedagogical conditions. A formula computed with fictional values is evidence that arithmetic was performed, not that a real device passed a standard. When the output feeds a safety, certification or financial decision, the appropriate reviewer and agreed acceptance rule must be identified before collecting measurements.

Study exercise 1 — reconstruct the baseline

Derive or justify For a Pt100 local approximation R(T)≈100Ω[1+0.00385 T] near 0°C, with T in °C. from the relevant conservation law, circuit relation, transfer model or statistical definition. Recompute the worked example, label every unit and distinguish the physical value from what an actual instrument could have displayed. Identify one data item whose uncertainty is not quantified by the example. Explain what would happen if that datum were systematically too high by a small amount.

Study exercise 2 — sensitivity experiment

Use the same physical configuration but change a controlled parameter by 10% in the direction that would expose a plausible failure mechanism. Compute or predict the response using the stated equation, and state whether the output sensitivity is linear, inverse, nonlinear or conditional. Now specify one nonideal effect, grounded in this lesson, that could cause a real observation to differ. A technically sufficient answer must compare two predicted numerical or symbolic outputs; a verbal statement that the error “increases” is not enough where values are available.

Study exercise 3 — measurement design

Design a low-energy or software-only verification that could distinguish the main mechanism from this competing explanation: A sensor mounted far from a heat source may indicate its own housing temperature rather than the desired process measurand. Specify the independent variable, the measured variable, the reference standard or alternate observation, the expected trend and a stop criterion for an invalid run. Describe what metadata would be saved for another engineer to reproduce the analysis without interviewing the experimenter.

Solution and grading guide

Exercise 2. The independent variable is changed while the model assumptions remain fixed. For a linear proportional dependency, +10% in the input produces +10% in the relevant output contribution; for a reciprocal dependence the output becomes 1/1.1=0.9091 of the original; for a squared dependence it becomes 1.1²=1.21. Apply only the form actually supported by For a Pt100 local approximation R(T)≈100Ω[1+0.00385 T] near 0°C, with T in °C.; do not choose a generic factor without checking the equation. Explain how actual calibrated rtd curves use more complete relationships; the linear formula is a restricted teaching approximation. could invalidate an extrapolation.

Exercise 3. The strongest design isolates one cause using a second evidence pathway. A sensor mounted far from a heat source may indicate its own housing temperature rather than the desired process measurand. should lead to a different prediction from the central mechanism. For example, changing source impedance while fixing source voltage is an effective loading test; changing sample rate while holding a physical sine frequency constant reveals aliasing; testing increasing and decreasing excitation reveals hysteresis. The appropriate case must match the instrument and phenomenon studied here.

Simulator and extension

Use [interactive Unit 7 model](../simulations/model-07.html) to explore the unit's central instrumentation mechanism. The simulator may focus on a related quantity rather than exactly this lesson's formula. Save the default model inputs and at least two modified cases; attach a screenshot or numerical result table. Compare one manually calculated point with the model output, and state a physical mechanism that the simulated equations omit. The original example dataset is not field evidence and must not be cited as such.

Safety note: The examples are synthetic and intended for analysis or software-only simulation. Do not connect measuring instruments to mains, high-energy equipment, CT secondaries or exposed conductors without authorized training, appropriately rated equipment and site-specific procedures. Instrument-category markings, isolation ratings and current limits must be verified in the manufacturer documentation.

7.3 — Magnetic current sensors and calibration

Unit 7: Sensors, signal conditioning and calibration Level: undergraduate, with prerequisite circuit theory, calculus and introductory probability. Estimated study: 70–100 minutes, including the numeric investigation. Tools: calculator, spreadsheet or the original offline simulator in this book. Evidence: examples are synthetic unless explicitly identified otherwise.

A problem that a measurement must answer

A Hall sensor responds to magnetic field and can provide galvanic isolation when engineered appropriately. A current shunt is often simpler but is intrusive; a current transformer cannot measure steady DC. Hall offset and temperature-dependent sensitivity can dominate small-current signals.

The application for this unit is: Build mathematical calibration curves before relying on a sensor as evidence. The meaningful output is not necessarily the number displayed by a meter. First decide whether it is the unperturbed circuit voltage, a time average, a reference-scale resistance, a physical temperature or a confidence-qualified estimate. That decision determines the topology, calibration approach and acceptable model. In an instrumentation report it is better to leave a result undecided than to replace the unmeasured quantity with a convenient proxy without disclosure.

Mathematical model and physical assumptions

Central relation: Ideal linear Hall-channel output Vout=V0+S I, where S is sensitivity in V/A.

Derivation strategy

Classify each influence by its pathway to the output. Consider bias corrections separately from dispersion. A standard uncertainty is not necessarily equal to a specified maximum tolerance; a rectangular bounded-error model, for example, converts half-width a into u=a/√3 only if that probability model is justified. In a multivariable equation, derivatives and covariance decide how input uncertainty propagates. In this case the key input evidence is the reading or physical parameter introduced above, not the final calculated answer. An experimenter should separate constants taken from a definition, instrument parameters taken from a datasheet and conditions measured on the day of use. If an assumed parameter is poorly characterized, a small model residual does not make the resulting prediction precise. The uncertainty of a quantity calculated from a product, quotient or nonlinear relation must follow its sensitivity to each contributing input, not a simple count of display digits.

Why the relation is useful

Transfer functions of sensors A sensor is not identical to its electrical output. It is a chain from physical measurand to material response, transducer, excitation, wiring, amplifier and digitizer. Temperature, stress direction, spatial averaging and mounting create biases that cannot be removed by digital arithmetic alone. Calibration establishes a relation over a specified range rather than proving every physical mechanism understood. The formula above is one of the operational links between a physical system and a numerical record. It is not an endorsement of any particular manufacturer's ratings, which must be checked independently. For instance, ideal resistance or sampling calculations disregard temperature-dependent and frequency-dependent behavior unless those are included explicitly.

Fully worked numerical case

A fictional sensor has V0=2.50 V and S=0.040 V/A. At 8.0 A it predicts 2.82 V; if a measurement gives 2.78 V, the inferred current is (2.78−2.50)/0.04=7.0 A. Offset uncertainty 10 mV corresponds to 0.25 A.

Uncertainty, nonidealities and instrumentation bias

Real sensors can saturate or respond to external fields; an isolation label is not a general safety approval. Temperature-driven offset shifts can look like a small real DC current. The first concern affects the credibility of the numerical parameter; the second suggests a rival explanation of the observed behavior. A useful uncertainty analysis therefore goes beyond listing a collection of percentages. It defines the output measurand, classifies major sources as additive or multiplicative, evaluates correlations when necessary, and identifies the sensitivity of the output to each one. An uncertainty budget should preserve documentary support for Type B inputs and raw observations for Type A estimates. If the model is an approximation, give a bounded validity statement in addition to any statistical interval.

A counterexample to overconfidence

Validation beyond regression A small sum of squared residuals is not enough. Test independent inputs, rising and falling trajectories, different environmental conditions and repeated runs. Compare interpolations against extrapolations. Hysteresis or creep means that the same instantaneous input can produce multiple outputs depending on history, defeating an unconditional one-to-one calibration curve. This contextual distinction provides a way to design a discriminating check. For an instrument-chain problem, change input range, source impedance, frequency or wiring to see whether the measurement follows the predicted dependence. For a data-analysis problem, change sample rate, observation length or independent reference while holding the physical input constant. If a conclusion persists only because the same assumption is reused in every run, that persistence supplies little additional evidence.

Engineering use and what would be required to trust it

Use bipolar calibration when possible to separate offset from gain and external-field bias. Calibration records, environmental limits and independent checks belong in the method, not just in appendices. Choose an experiment that isolates the main influence while staying within authorized, properly rated equipment and low-energy pedagogical conditions. A formula computed with fictional values is evidence that arithmetic was performed, not that a real device passed a standard. When the output feeds a safety, certification or financial decision, the appropriate reviewer and agreed acceptance rule must be identified before collecting measurements.

Study exercise 1 — reconstruct the baseline

Derive or justify Ideal linear Hall-channel output Vout=V0+S I, where S is sensitivity in V/A. from the relevant conservation law, circuit relation, transfer model or statistical definition. Recompute the worked example, label every unit and distinguish the physical value from what an actual instrument could have displayed. Identify one data item whose uncertainty is not quantified by the example. Explain what would happen if that datum were systematically too high by a small amount.

Study exercise 2 — sensitivity experiment

Use the same physical configuration but change a controlled parameter by 10% in the direction that would expose a plausible failure mechanism. Compute or predict the response using the stated equation, and state whether the output sensitivity is linear, inverse, nonlinear or conditional. Now specify one nonideal effect, grounded in this lesson, that could cause a real observation to differ. A technically sufficient answer must compare two predicted numerical or symbolic outputs; a verbal statement that the error “increases” is not enough where values are available.

Study exercise 3 — measurement design

Design a low-energy or software-only verification that could distinguish the main mechanism from this competing explanation: Temperature-driven offset shifts can look like a small real DC current. Specify the independent variable, the measured variable, the reference standard or alternate observation, the expected trend and a stop criterion for an invalid run. Describe what metadata would be saved for another engineer to reproduce the analysis without interviewing the experimenter.

Solution and grading guide

Exercise 2. The independent variable is changed while the model assumptions remain fixed. For a linear proportional dependency, +10% in the input produces +10% in the relevant output contribution; for a reciprocal dependence the output becomes 1/1.1=0.9091 of the original; for a squared dependence it becomes 1.1²=1.21. Apply only the form actually supported by Ideal linear Hall-channel output Vout=V0+S I, where S is sensitivity in V/A.; do not choose a generic factor without checking the equation. Explain how real sensors can saturate or respond to external fields; an isolation label is not a general safety approval. could invalidate an extrapolation.

Exercise 3. The strongest design isolates one cause using a second evidence pathway. Temperature-driven offset shifts can look like a small real DC current. should lead to a different prediction from the central mechanism. For example, changing source impedance while fixing source voltage is an effective loading test; changing sample rate while holding a physical sine frequency constant reveals aliasing; testing increasing and decreasing excitation reveals hysteresis. The appropriate case must match the instrument and phenomenon studied here.

Simulator and extension

Use [interactive Unit 7 model](../simulations/model-07.html) to explore the unit's central instrumentation mechanism. The simulator may focus on a related quantity rather than exactly this lesson's formula. Save the default model inputs and at least two modified cases; attach a screenshot or numerical result table. Compare one manually calculated point with the model output, and state a physical mechanism that the simulated equations omit. The original example dataset is not field evidence and must not be cited as such.

Safety note: The examples are synthetic and intended for analysis or software-only simulation. Do not connect measuring instruments to mains, high-energy equipment, CT secondaries or exposed conductors without authorized training, appropriately rated equipment and site-specific procedures. Instrument-category markings, isolation ratings and current limits must be verified in the manufacturer documentation.

7.4 — Least-squares calibration, hysteresis and validation

Unit 7: Sensors, signal conditioning and calibration Level: undergraduate, with prerequisite circuit theory, calculus and introductory probability. Estimated study: 70–100 minutes, including the numeric investigation. Tools: calculator, spreadsheet or the original offline simulator in this book. Evidence: examples are synthetic unless explicitly identified otherwise.

A problem that a measurement must answer

A calibration line should be fitted to a documented set of reference inputs and holdout measurements, not just two convenient endpoints. Least squares minimizes squared vertical residuals under its assumed error model; uncertainty in the reference axis may require errors-in-variables methods. Hysteresis reveals path dependence.

The application for this unit is: Build mathematical calibration curves before relying on a sensor as evidence. The meaningful output is not necessarily the number displayed by a meter. First decide whether it is the unperturbed circuit voltage, a time average, a reference-scale resistance, a physical temperature or a confidence-qualified estimate. That decision determines the topology, calibration approach and acceptable model. In an instrumentation report it is better to leave a result undecided than to replace the unmeasured quantity with a convenient proxy without disclosure.

Mathematical model and physical assumptions

Central relation: For y=a+bx, b=Σ(x−x̄)(y−ȳ)/Σ(x−x̄)²; a=ȳ−b x̄.

Derivation strategy

Use both a numerical consistency check and a counterexample. Change one input to challenge sensitivity; change the topology or data-generating assumption to challenge validity. A result is only as robust as the least defensible assumption. When discussing acceptance, state a decision rule, uncertainty and consequences for an incorrect classification. In this case the key input evidence is the reading or physical parameter introduced above, not the final calculated answer. An experimenter should separate constants taken from a definition, instrument parameters taken from a datasheet and conditions measured on the day of use. If an assumed parameter is poorly characterized, a small model residual does not make the resulting prediction precise. The uncertainty of a quantity calculated from a product, quotient or nonlinear relation must follow its sensitivity to each contributing input, not a simple count of display digits.

Why the relation is useful

Transfer functions of sensors A sensor is not identical to its electrical output. It is a chain from physical measurand to material response, transducer, excitation, wiring, amplifier and digitizer. Temperature, stress direction, spatial averaging and mounting create biases that cannot be removed by digital arithmetic alone. Calibration establishes a relation over a specified range rather than proving every physical mechanism understood. The formula above is one of the operational links between a physical system and a numerical record. It is not an endorsement of any particular manufacturer's ratings, which must be checked independently. For instance, ideal resistance or sampling calculations disregard temperature-dependent and frequency-dependent behavior unless those are included explicitly.

Fully worked numerical case

Measurements (x,y)=(0,0.02),(1,1.03),(2,2.04) lie exactly on y=0.02+1.01x. A holdout x=1.5 predicts 1.535; a measured holdout 1.60 leaves residual +0.065 and challenges that fit.

Uncertainty, nonidealities and instrumentation bias

A perfect in-sample fit can arise from too few points; calibration and validation must be separated. Correlation between x and y does not establish that all operating hysteresis or nonlinearities are absent. The first concern affects the credibility of the numerical parameter; the second suggests a rival explanation of the observed behavior. A useful uncertainty analysis therefore goes beyond listing a collection of percentages. It defines the output measurand, classifies major sources as additive or multiplicative, evaluates correlations when necessary, and identifies the sensitivity of the output to each one. An uncertainty budget should preserve documentary support for Type B inputs and raw observations for Type A estimates. If the model is an approximation, give a bounded validity statement in addition to any statistical interval.

A counterexample to overconfidence

Validation beyond regression A small sum of squared residuals is not enough. Test independent inputs, rising and falling trajectories, different environmental conditions and repeated runs. Compare interpolations against extrapolations. Hysteresis or creep means that the same instantaneous input can produce multiple outputs depending on history, defeating an unconditional one-to-one calibration curve. This contextual distinction provides a way to design a discriminating check. For an instrument-chain problem, change input range, source impedance, frequency or wiring to see whether the measurement follows the predicted dependence. For a data-analysis problem, change sample rate, observation length or independent reference while holding the physical input constant. If a conclusion persists only because the same assumption is reused in every run, that persistence supplies little additional evidence.

Engineering use and what would be required to trust it

Run rising and falling sweeps, replicate references and record ambient conditions. Calibration records, environmental limits and independent checks belong in the method, not just in appendices. Choose an experiment that isolates the main influence while staying within authorized, properly rated equipment and low-energy pedagogical conditions. A formula computed with fictional values is evidence that arithmetic was performed, not that a real device passed a standard. When the output feeds a safety, certification or financial decision, the appropriate reviewer and agreed acceptance rule must be identified before collecting measurements.

Study exercise 1 — reconstruct the baseline

Derive or justify For y=a+bx, b=Σ(x−x̄)(y−ȳ)/Σ(x−x̄)²; a=ȳ−b x̄. from the relevant conservation law, circuit relation, transfer model or statistical definition. Recompute the worked example, label every unit and distinguish the physical value from what an actual instrument could have displayed. Identify one data item whose uncertainty is not quantified by the example. Explain what would happen if that datum were systematically too high by a small amount.

Study exercise 2 — sensitivity experiment

Use the same physical configuration but change a controlled parameter by 10% in the direction that would expose a plausible failure mechanism. Compute or predict the response using the stated equation, and state whether the output sensitivity is linear, inverse, nonlinear or conditional. Now specify one nonideal effect, grounded in this lesson, that could cause a real observation to differ. A technically sufficient answer must compare two predicted numerical or symbolic outputs; a verbal statement that the error “increases” is not enough where values are available.

Study exercise 3 — measurement design

Design a low-energy or software-only verification that could distinguish the main mechanism from this competing explanation: Correlation between x and y does not establish that all operating hysteresis or nonlinearities are absent. Specify the independent variable, the measured variable, the reference standard or alternate observation, the expected trend and a stop criterion for an invalid run. Describe what metadata would be saved for another engineer to reproduce the analysis without interviewing the experimenter.

Solution and grading guide

Exercise 2. The independent variable is changed while the model assumptions remain fixed. For a linear proportional dependency, +10% in the input produces +10% in the relevant output contribution; for a reciprocal dependence the output becomes 1/1.1=0.9091 of the original; for a squared dependence it becomes 1.1²=1.21. Apply only the form actually supported by For y=a+bx, b=Σ(x−x̄)(y−ȳ)/Σ(x−x̄)²; a=ȳ−b x̄.; do not choose a generic factor without checking the equation. Explain how a perfect in-sample fit can arise from too few points; calibration and validation must be separated. could invalidate an extrapolation.

Exercise 3. The strongest design isolates one cause using a second evidence pathway. Correlation between x and y does not establish that all operating hysteresis or nonlinearities are absent. should lead to a different prediction from the central mechanism. For example, changing source impedance while fixing source voltage is an effective loading test; changing sample rate while holding a physical sine frequency constant reveals aliasing; testing increasing and decreasing excitation reveals hysteresis. The appropriate case must match the instrument and phenomenon studied here.

Simulator and extension

Use [interactive Unit 7 model](../simulations/model-07.html) to explore the unit's central instrumentation mechanism. The simulator may focus on a related quantity rather than exactly this lesson's formula. Save the default model inputs and at least two modified cases; attach a screenshot or numerical result table. Compare one manually calculated point with the model output, and state a physical mechanism that the simulated equations omit. The original example dataset is not field evidence and must not be cited as such.

Safety note: The examples are synthetic and intended for analysis or software-only simulation. Do not connect measuring instruments to mains, high-energy equipment, CT secondaries or exposed conductors without authorized training, appropriately rated equipment and site-specific procedures. Instrument-category markings, isolation ratings and current limits must be verified in the manufacturer documentation.

Project 7 — Sensors, signal conditioning and calibration: reproducible measurement review

Research question. Can two plausible measurement chains give meaningfully different estimates of the same physical measurand when one ideal-model assumption is violated?

Starting case. Build mathematical calibration curves before relying on a sensor as evidence. This is a software-only project. It must not be implemented by handling energized installations or using the simulation as a substitute for calibration.

Background and model selection

A sensor is not identical to its electrical output. It is a chain from physical measurand to material response, transducer, excitation, wiring, amplifier and digitizer. Temperature, stress direction, spatial averaging and mounting create biases that cannot be removed by digital arithmetic alone. Calibration establishes a relation over a specified range rather than proving every physical mechanism understood.

A small sum of squared residuals is not enough. Test independent inputs, rising and falling trajectories, different environmental conditions and repeated runs. Compare interpolations against extrapolations. Hysteresis or creep means that the same instantaneous input can produce multiple outputs depending on history, defeating an unconditional one-to-one calibration curve.

Select one formula from each of Lessons 7.1–7.4, but model one of them in depth. For the chosen case, clearly define independent input, measured indication, desired output, units, reference conditions and primary sensitivity. Explain why changing a software parameter is a model intervention, not a laboratory measurement.

Task A — baseline and synthetic evidence

Open [Unit 7 simulator](../simulations/model-07.html). Save the four slider values, the computed numerical output, the axes and the model description. Change one input to three different values and record the response. Independently calculate one point. Create a small table including percent deviations from the baseline and at least one limiting-case check. Where a plotted variable is not the same as an instrument's recorded output, document that distinction.

Task B — deliberately introduce error

Choose one missing mechanism from the lesson limitations. The candidates specific to this unit include: Nonuniform strain, adhesive creep, transverse sensitivity and thermal expansion affect physical interpretation. Another candidate is: Actual calibrated RTD curves use more complete relationships; the linear formula is a restricted teaching approximation. Write a small correction term, state its assumed magnitude and investigate how the inferred result changes. Do not assert that an invented error magnitude is representative of a commercial instrument or a deployed installation.

Task C — contrast two explanations

Two technical reports disagree. One report blames calibration; the other blames a physical effect: Temperature-driven offset shifts can look like a small real DC current. Design a low-energy or observational check that would produce different predictions under those explanations. State the independent evidence source and explain how correlated calibration references could defeat the check. If both explanations predict the same observable under your selected settings, your proposed test is not discriminating and must be redesigned.

Task D — quantitative uncertainty and decision

Assign three clearly labeled synthetic standard uncertainties and propagate them through the selected measurement equation. Where appropriate, include a common-reference correlation. Present a result estimate, combined standard uncertainty and an optional expanded uncertainty with specified k. Introduce a fictional tolerance limit and a predeclared decision rule. Show a near-boundary example where the displayed reading by itself would give a different decision from a guard-banded approach. Do not conflate a rejected acceptance decision with proof of nonconformity.

Deliverables

• Reproducible spreadsheet or notebook with units, formulas and a synthetic-data flag.
• At least one parameter-sensitivity plot and one tabulated calculation.
• A short model-assumption statement and a distinguishable rival hypothesis.
• A traceable uncertainty budget with corrections separated from uncertainty contributions.
• A final one-page technical memo explaining what is supported and what remains unmeasured.

Evaluation and extension

A report is complete only when a reader can reproduce the model calculation, recognize synthetic inputs and identify the decisive missing evidence for a real-world conclusion. Additional complexity earns credit only if it addresses the dominant uncertainty, not simply because the diagram becomes more elaborate. For an extension, estimate how much improved reference accuracy or increased acquisition time would actually reduce the dominant combined uncertainty under the stated model.

Safety note: The examples are synthetic and intended for analysis or software-only simulation. Do not connect measuring instruments to mains, high-energy equipment, CT secondaries or exposed conductors without authorized training, appropriately rated equipment and site-specific procedures. Instrument-category markings, isolation ratings and current limits must be verified in the manufacturer documentation.