MAI-Alchemy · Analytical Chemistry, Taught by Practice

The Analytical Column

where ideas resolve
Calibration · No. 15The Linear Range

Calibration — where linearity ends

Where the line stops being straight

A calibration is only good where the response actually tracks concentration. Above that, the curve bends, the top standard falls off the line, and a straight fit forced through it distorts the whole calibration. So you prove where linearity ends — you do not assume it.

By Medrado Analytical Innovations · Foundations, No. 15 · for the working analyst

In this piece · 7 min read
  1. Why the line bends at the top
  2. Linear range is not dynamic range
  3. Two different tests — proportionality, and linearity
  4. When the sample is above the range
  5. Single point, revisited
  6. Where it sits in the family
  7. The judgement

Draw a straight line through your standards, admire the R², and start reading samples off it — and you have made a promise the method may not keep: that response rises in step with concentration all the way up. Real detectors do not. Somewhere near the top of the range the signal begins to lag the concentration — the curve bends over — and a point sitting on that bend is telling the truth about the sample and falling off the line. Fit a straight line through it and you have not extended your range; you have distorted the fit and biased results across it. The linear range is the stretch where the line is honestly straight, and knowing where it ends is as much a part of the method as the slope itself.

Why the line bends at the top

Every detection scheme has a ceiling above which more analyte no longer buys proportionally more signal. In absorbance, Beer–Lambert is linear only while absorbance is modest; higher up the measured absorbance flattens — chiefly because stray light begins to dominate the tiny transmitted beam and caps the reading, with the finite bandwidth of real light adding a gradual deviation wherever the absorptivity is not flat across the bandpass, and chemical effects (association, pH, index) able to push either way.2 Note it is the signal that flattens, not the molecule — the absorber does not "run out." A separate, real ceiling is detector saturation and overload. The mechanisms differ; the shape is the same: straight at the bottom, rolling over at the top. And the linear range is a property of this analyte on this method in this matrix, not a number to look up.

Linear range is not dynamic range

The instrument will keep giving a bigger number well past where the line stopped being straight; that wider span, where response still rises but no longer proportionally, is the dynamic range (from the detection limit up to where response tops out). It is usable — but only with a curve that admits the bend. Reading a curved region off a straight line is the error; treating the whole dynamic range as if it were linear is how people back into it.

Two different tests — proportionality, and linearity

You can find the bend without staring at a plot, but you have to ask the right question. Divide each standard's response by its concentration to get the response factor. If — and only if — the calibration passes through the origin, that ratio is constant across standards, and its drift marks where proportionality fails. But a genuinely straight line with a non-zero intercept, response = m·c + b, has a response factor m + b/c that trends with concentration even though the fit is perfectly linear — so a drifting response factor at the low end is the intercept talking, not curvature. That is the trap: the response-factor test checks proportionality (a line through zero), which is what response-factor and single-point quantitation actually assume; to test linearity of a line that carries an intercept, look at the residuals or run a lack-of-fit test (which needs replicate measurements at each level) — testing constant slope, not constant ratio. As the piece on the fourth nine warned, R² will not tell you either: unweighted, it is dominated by the largest points (weighting by 1/x or 1/x² rebalances that), and even so a high R² sits happily on a gently bent fit — it is not a test of linearity.

Worked example — a benzene curve that rolls over (illustrative)

Benzene is run on a GC with a VUV absorbance detector, quantified through the origin by response factor (response ÷ vol%). Numbers illustrative; the shape is the lesson:

Benzene (vol%)Response factor (a.u./vol%)vs mean of low five
0.5101+1%
1.01000%
1.51000%
2.099−1%
2.51000%
3.095−5%
4.088−12%
The article's benzene standards: the response factor stays within about 1% of 100 through 2.5 vol%, then falls to −5% at 3.0 and −12% at 4.0 vol%. The points are real; they have left the straight line.01234859095100105Benzene (vol%)Response factor (a.u. per vol%)linear range, ≈ 2.5 vol%mean of the low five = 100−5%−12%
The article's benzene standards: the response factor stays within about 1% of 100 through 2.5 vol%, then falls to −5% at 3.0 and −12% at 4.0 vol%. The points are real; they have left the straight line.

Through 2.5 vol% the response factor holds within about 1% — flat, and (since it does not drift at the low end) genuinely through the origin. At 3.0 and 4.0 it falls as the measured VUV absorbance flattens: the analyte is there, the readings are real, but they sit below the proportional line the low points define. Keep those two in a response-factor calibration and the mean response factor is dragged down, so every in-range result then reads high. (Fit an ordinary least-squares line with an intercept through all seven instead and the curve pivots — top pulled down, intercept lifted — a different bias, but still wrong.) Either way the honest move is the same: set the linear range at ≈ 2.5 vol% and keep samples inside it. The top standards did not fail — they were asked to lie on a line they had already left.

When the sample is above the range

Two honest options, and one forbidden move. Dilute into range — the usual, cleanest answer: bring the high sample down to where the line is straight, read it there, and multiply back by a dilution done with care (Class A glassware, and mind whether diluting also changes the matrix). Or fit the curve you actually have — a quadratic, or the detector's known response function, defined by enough standards through the bend and shown to be accurate there, used knowingly (and watched, since a quadratic can turn non-monotonic and hand back two concentrations for one signal). What you may never do is extrapolate above your highest standard, or read a sample off a straight line in a region you have shown is curved. And there is a quiet failure the other way: when the sample itself sits in the curved region, its response has already rolled over, so read against the steeper low-range line it comes back low — a high sample masquerading as an ordinary one.

Single point, revisited

This is the fine print under single-point calibration from the opening piece. A single standard and one response factor are legitimate only inside a proven linear range that passes through the origin — the same through-origin condition the response-factor test needs — and "proven" means you once ran the multi-point curve that established where the straight part is. A single-point method quietly inherits the linear range of the study that validated it; use it on a sample above that range and it fails without a word, because a single point cannot see the bend it is standing on.

Where it sits in the family

Put this beside the piece on detection limits and you have both ends of the honest span. The floor is the LOQ, the lowest you can report with acceptable precision and accuracy; the ceiling is where linearity ends (or, if you fit a validated curve, where that curve stops being accurate). Between them lies the working range — and note it is defined for the job (an assay validated over 80–120% of a target is a working range too), not simply everything between the floor and the top of the straight part. The wide, log-spaced calibration of the opening piece is how you find these edges; external standard, standard addition, internal standard each live inside this window and stop meaning anything outside it.

The judgement

Linearity is a claim, and the top of your curve is where it is most often quietly false. Run standards past where you expect to work, check for the tip-over with the right test — response factor for a through-origin method, residuals or lack-of-fit for a line with an intercept — set the range where it stays honest, and either dilute into it or fit the curve on purpose. The instrument will read a sample above the range without complaint and hand you a confident, wrong number. Knowing where your line stops being straight — and refusing to read past it — is the last of the calibration disciplines: the same habit, at the top of the curve, that detection limits demand at the bottom.

Where this sits. The bookend to detection limits: that piece guards the floor, this one the ceiling, and together they bound the working range every calibration lives inside. With the opener and the three technique pieces — external standard, standard addition, internal standard — the calibration thread is complete: why you calibrate, how, in what matrix, against what companion, and between which two honest limits.

A foundations piece — the reasoning behind linear range, not a validation protocol. For linearity criteria, the number of levels, and how to demonstrate and bracket the range, work from ICH Q2(R2), the applicable ASTM/EPA/USP method, and your laboratory's quality system.

Check yourself

Answer in your head first, then open the answer. Any question can go into your Quiz me.

  1. What chiefly makes measured absorbance flatten at the top?

    Show the answer
    “chiefly because stray light begins to dominate the tiny transmitted beam and caps the reading”

    See it in the article ·

  2. What is the dynamic range, as opposed to the linear range?

    Show the answer
    “that wider span, where response still rises but no longer proportionally, is the dynamic range”

    See it in the article ·

  3. Your sample reads above the linear range. What is the cleanest answer?

    Show the answer
    “Dilute into range”

    See it in the article ·

Glossary

Linear range
The concentration span over which response rises in a straight line (with or without an intercept) — where the calibration is genuinely linear.
Proportional (through-origin) response
The special case of linear with a zero intercept: response = m·c. Only then is the response factor constant, and only then is single-point / response-factor quantitation valid.
Dynamic range
The span over which the detector gives an increasing, usable response — from the detection limit up to where response tops out; wider than the straight part, which is the "linear dynamic range." Beyond linearity it must be calibrated with a curve.
Working range
The interval, defined for the application, over which the method is shown accurate, precise, and adequately linear — from the LOQ up to a demonstrated upper limit; not automatically LOQ-to-ceiling.
Response factor
Response ÷ concentration; constant only for a through-origin response, so its drift tests proportionality — not the linearity of a line that carries an intercept.
Beer–Lambert deviations
Why measured absorbance flattens at high A: chiefly stray light capping the reading, plus bandwidth effects where absorptivity varies across the bandpass, and chemical effects (either direction). The absorber does not saturate.
Detector saturation / overload
A distinct ceiling — a flooded detector, an overloaded inlet or column, crowded ionization — where more analyte no longer gives proportional signal.
Extrapolation
Reading beyond the highest standard, or off a straight line in a curved region — outside the calibration, not allowed.

Sources

  1. ICH Q2(R2) (2023), Validation of Analytical Procedures — linearity and range: demonstrating response over a defined interval and reporting that interval. International Council for Harmonisation.
  2. Skoog, D. A.; West, D. M.; Holler, F. J.; Crouch, S. R., Fundamentals of Analytical Chemistry, and Harris, D. C., Quantitative Chemical Analysis — deviations from Beer's law (stray light dominating the high-absorbance plateau, polychromatic-bandwidth and chemical deviations) and detector linearity. See also ASTM E169 (general UV-Vis quantitative practices).
  3. Miller, J. N.; Miller, J. C., Statistics and Chemometrics for Analytical Chemistry (Pearson) — testing linearity, residual and lack-of-fit analysis, weighting, and why a high correlation coefficient does not prove a straight line.
  4. Response-factor / calibration-factor constancy (%RSD across levels) versus regression, as a calibration and linearity check — e.g. U.S. EPA SW-846 Method 8000 (D) calibration criteria.