Reply To: Resolution error

  • Encyclios

    May 3, 2023 at 3:14 PM

    Reading digital instruments

    Digital instruments discretize measurements, that is, starting from an analog input signal, they transform it into a numerical format. It is evident that the discretization of a measure constitutes a limit to its resolution. The classic example of these devices are digital indicators with numerical display.

    The resolution error in the reading of a digital instrument is normally equal to the value of the least significant digit of its display. Example: a 4 digits voltmeter, with a full scale of 10 V, has a resolution error on the reading of 0.001 V. Exceptions to this are:

    • when the least significant digit varies with an increment other than 1 (typical are increments of 2 in 2, or 5 in 5), in which case the resolution on the reading is equal to the value of the increment;
    • when measurements are made during the transition of the digit value, on a measurand that is certain to vary very slowly, in which case the resolution of the reading will be a fraction of a digit, depending on the reading speed of the instrument and the maximum speed of variation of the measurand.