[Encyclios]

Quantitative evaluation

Rigorous quantification of resolution error would require a lengthy (and expensive) analysis. Fortunately, in practice, the following considerations come in favor:

• the “limitation to resolution” of highest value makes all others “unresolvable”, so it is sufficient to find this one to know the resolution error of the measurement system;
• for a certain application, the limitations that can be significant are few and always the same;
• the empirical control on the field, during the execution of the measures, highlights which can be the resolution error of the measuring system.
• Below are some useful rules for the evaluation of the resolution error.

[Encyclios]

Reading digital instruments

Digital instruments discretize measurements, that is, starting from an analog input signal, they transform it into a numerical format. It is evident that the discretization of a measure constitutes a limit to its resolution. The classic example of these devices are digital indicators with numerical display.

The resolution error in the reading of a digital instrument is normally equal to the value of the least significant digit of its display. Example: a 4 digits voltmeter, with a full scale of 10 V, has a resolution error on the reading of 0.001 V. Exceptions to this are:

• when the least significant digit varies with an increment other than 1 (typical are increments of 2 in 2, or 5 in 5), in which case the resolution on the reading is equal to the value of the increment;
• when measurements are made during the transition of the digit value, on a measurand that is certain to vary very slowly, in which case the resolution of the reading will be a fraction of a digit, depending on the reading speed of the instrument and the maximum speed of variation of the measurand.

[Encyclios]

Reading analog instruments

The resolution error in reading an analog instrument depends on:

• the value of the format unit;
• the length of the graduation;
• the size of the index;
• the reading ability of the operator.

Example: For a pressure gauge with a unit size of 1 bar, a distance between two graduations of 5 mm and an index thickness of 1 mm, the theoretical limit of resolution is 0.2 bar.

In practice, despite the use of optical aids (lenses, microscopes), the reading resolution of an analog instrument is severely limited by the operator’s ability to discern fractions of graduation, as well as by the parallax error that can be generated (index and graduated scale are not on the same plane). Although trained personnel can discern 1-2 microns when reading a micrometer equipped with a centesimal nonius (10-micron units), a general operator is unlikely to be able to reliably distinguish 1-mm shifts when the index is far from the scale markings.

To adjust for the subjectivity of these considerations, some activity standards or for instruments (e.g., manometers or dynamometers) specifically define how to calculate the reading resolution. In all other cases, common sense and the principle of prudence apply: when in doubt, an operator should at least be able to discern to which notch the index is closest, in which case the reading resolution becomes equal to the unit of format.

Exceptions are the cases:

• when the index is larger than the scale division, in which case the reading resolution is a multiple of the format unit;
• when measurements are made in conditions where there is a coincidence between index and notch, in which case the reading resolution will be equal to the theoretical limit.

[Encyclios]

Background noise and interference

When making measurements, the display may show instabilities that cannot be attributed to real variations in the measurand. This problem is typical of electronic instrumentation, especially when working in very low voltage ranges (< 1 mV) or when very high resolutions (< 0.1 of full scale) are required from an instrument. In these conditions the indicators may read background noise from their own electronics or noise from an external source.

If these noises cannot be shielded or filtered out, the instability found on the reading constitutes a limit to the resolution of the measurement; in this case the resolution of the reading is defined as equal to the amplitude of the oscillation observed.

[Encyclios]

Mechanical Limits

When using instruments equipped with mechanical components (rods, gears, linkages, racks), jerky or jerky readings may be encountered. The most likely cause of these behaviors is a manufacturing defect or damage to the instrument; however, when working with very sensitive instruments, it may also indicate that the instrument is working at the limits of its mechanical capabilities. Under these conditions, one can already see the effects of friction, elastic deformation and inertia that prevent the mechanics involved from “following” the variations in the measurand.

As already mentioned, a rigorous evaluation of these limits to resolution would require a specific analysis by a specialized laboratory. Fortunately, normally these limits are an order of magnitude smaller than the others, requiring only a quick check of the response of the display to changes in the measurand to ensure that this problem is irrelevant.

If the problem is significant, the resolution of the reading is defined as the maximum ” trigger” or ” bottleneck” detected.

[Encyclios]

Measurement discretization

We have already mentioned the problem of the resolution limitation due to the discretization of digital instruments, inherent in their reading; now it is worth pointing out that this is only one aspect of the problem of discretization of measurements. The increasingly massive use of digital electronic instrumentation extends the considerations made with regard to visualization, to “measurement” in the broadest sense.

In fact, beyond the problem of visualization, almost all digital electronic instruments perform an analog-to-digital conversion of signals, and consequently create a relative resolution error. It is important to note that this error is present regardless of the presence or characteristics of the display device adopted: a limiting case is an instrument used to acquire measurements to be stored in files, where there is no real display device, but nevertheless there is a resolution error due to the discretization of the measurand.

In almost all cases, the analog-to-digital conversion works on electrical signals that constitute or represent (following the use of a transducer) the measurand. The conversion is performed by an electronic circuit called ADC (analogue-to-digital converter), whose main characteristic is the size (in bits) of the corresponding digital value; the latter is an indication of the resolution of the conversion: a 10-bit ADC converter is able to encode 1024 different values (\(2^{10}\)) within the measuring range, an 8-bit converter is able to encode 256 values (\(2^8\)).

Similarly, resolution errors are also generated in the case of digital-to-analog conversion: in fact, although in theory it is possible the perfect transformation of a digital signal in the analog equivalent, the very fact that we start from a discrete signal prevents the possibility of generating signals of arbitrary value. The electronic circuit called for this conversion is called DAC (digital-to-analog converter). Three general cases can be envisaged:

1. A/D instruments, instruments that provide a conversion of an analog signal representing the measurand, in order to display and store it in digital format (e.g. digital indicators or measurement acquirers);
2. D/A instruments, instruments that generate an analog signal, starting from the relative command in digital format (as in function generators or calibrators used for the calibration of instrumentation);
3. A/D/A instruments, instruments that perform operations (filtering, amplification, conversion, storage) on analog signals, after converting them into digital format, and then make them available again in analog format (e.g. some signal conditioners or in measuring recorders).

In the absence of precise manufacturer’s instructions, the resolution error can be measured in the laboratory by slowly varying the measurand, and detecting the jump in display due to discretization.