When a visible object is not yet a known object

A mobile laboratory robot approaches a workbench to collect a sample rack. Its camera records the rack’s outline, while its range sensor reports a surface in front of it; neither measurement, in isolation, establishes precisely what the object is or where its graspable edge lies. Perception is the inferential work that turns such measurements into a representation of the environment, rather than a direct transcription of reality. This distinction matters whenever a robot must interpret incomplete observations before those observations become operationally useful. (roboticsbook.org)

Throughout this lesson, the laboratory robot provides a single working scenario. The numerical examples are illustrative engineering calculations, not reported performance figures from a particular system. The central question is precise: what does the robot have evidence for, and what remains uncertain?

From physical signals to environmental claims

Light reflected from the sample rack reaches a camera, which converts the incident signal into numerical measurements. An interpretation system then uses those measurements to infer properties such as object category or location. Sensing concerns acquiring measurements; environmental perception concerns inferring the surrounding state from them. An observation model describes the forward relationship between a possible state and the measurements that state would produce. (roboticsbook.org)

The difficulty emerges when this relationship is reversed. A range of 1.20 metres might correspond to the rack, the workbench behind it, or another object intersecting the measurement direction. The same numerical observation can therefore support several environmental explanations. Perception is an inverse problem: it must distinguish plausible causes of the available evidence without assuming that the evidence uniquely determines its cause. (roboticsbook.org)

An observation model links possible objects—a sample rack, workbench or another object—to a range reading of 1.20 metres. Inverse inference works back from this reading to competing explanations, without uniquely identifying its source.

For the laboratory scenario, the distinction can be expressed as follows.

Dimension Sensing Environmental perception
Immediate concern What measurement was acquired? What environmental state explains it?
Illustrative input Reflected light Image measurements and an observation model
Illustrative output A numerical image A hypothesis that the sample rack occupies a particular location
Interpretative limit Measurement imperfections Ambiguity between competing explanations

The robot’s representation must therefore distinguish a measurement from a claim derived from that measurement. The statement “range return: 1.20 metres” records an observation, whereas “sample rack: 1.20 metres ahead” incorporates an interpretation. Treating these statements as interchangeable conceals the assumptions on which the interpretation depends. (roboticsbook.org)

A sensor measures a property, not an entire scene

A laser rangefinder emits light and evaluates its return to estimate distance. In the direct time-of-flight model, the pulse travels to a surface and back, so distance equals half the product of propagation speed and elapsed time: (d=c\Delta t/2). The robotics textbook gives a concrete example: a surface 15 metres away produces an approximately 0.1-microsecond round trip. The division by two is essential: the measured journey includes both outward and return travel. (roboticsbook.org)

This mechanism reveals an important boundary: the observation describes a range along a sensing direction, not the object’s identity. Likewise, the textbook’s sorting robot uses separate measurements of conductivity, visual appearance and mass to support categorisation. These measurements are complementary precisely because each exposes a different property. Sensor selection begins with the variable that must be inferred, rather than an assumption that one sufficiently elaborate sensor reveals everything relevant. (roboticsbook.org)

For the laboratory robot, this principle yields a concise evidence plan.

Dimension Camera observation Range observation Mass observation
Property represented Visual appearance Distance along a measurement direction Measured mass
Illustrative contribution Identify a rack-like outline Locate a visible surface geometrically Check a collected sample’s measured mass
Unresolved question Is the appearance interpretation correct? Which object produced the return? Which object category explains this mass?

This is an illustrative allocation of evidence, not a prescribed hardware configuration. Its purpose is to make each measurement’s contribution explicit before combining interpretations. The underlying methodological lesson is that category inference can draw on distinct measurements without treating any individual measurement as an object label. (roboticsbook.org)

Uncertainty belongs inside the representation

A conductivity probe may fail to register a metal object when dirt or poor contact interferes with the measurement. The textbook models this explicitly: a hypothetical can produces a positive conductivity result with probability 0.9 and a negative result with probability 0.1. The negative result is therefore possible even when the object is a can. A measurement is evidence about a state, not an infallible declaration of that state. (roboticsbook.org)

Probability provides a formal language for retaining this ambiguity. An observation likelihood, written (P(z\mid x)), describes how probable measurement (z) would be under candidate state (x). It must not be confused with (P(x\mid z)), the probability assigned to that state after observing the measurement. The latter also depends on the prior plausibility of the state; Bayes’ theorem relates these quantities. (roboticsbook.org)

Dimension Observation likelihood Updated state probability
Notation (P(z\mid x)) (P(x\mid z))
Question answered How expected is this reading if the candidate state holds? How plausible is the candidate state after this reading?
Illustrative statement A can gives a positive conductivity result with probability 0.9 A positive result supports the can hypothesis, alongside prior information
Common error Reading 0.9 as an object-identification guarantee Ignoring alternative objects that could produce the result

A useful representation therefore preserves uncertainty alongside the estimated property. For example, the laboratory robot can retain competing object hypotheses rather than converting an ambiguous observation immediately into a definitive label. Probabilistic modelling supports this disciplined separation between evidence and interpretation; it does not eliminate the uncertainty inherent in imperfect measurements. (roboticsbook.org)

Measurements must refer to compatible instants

When sensor readings describe different moments

The camera and range sensor may observe the workbench at different instants. If either the robot or the object moves between those observations, combining them without temporal alignment can associate measurements that do not describe the same physical configuration. Primary documentation for Project Aria makes this requirement explicit: precise sensor fusion must account for offsets between sensor observation times. A timestamp is useful only when its relationship to the actual observation time is understood. (facebookresearch.github.io)

Consider an illustrative case in which the robot advances directly towards a stationary workbench at a constant 0.50 metres per second. Its image precedes its range observation by 0.10 seconds. During that interval, the robot travels (0.50\times0.10=0.05) metres: five centimetres. Associating the observations as simultaneous would therefore overlook a five-centimetre change in robot–workbench separation under these assumptions.

Quantity Illustrative value Interpretation
Robot speed 0.50 metres per second Constant motion towards the workbench
Observation offset 0.10 seconds Image acquired before the range reading
Distance travelled 0.05 metres Change in separation between observations
Uncorrected association Different configurations treated as one Apparent agreement may be misleading

The appropriate principle is alignment before interpretation across sensors. Acquisition timing must be distinguished from the later delivery of data to a processing system, and known offsets must enter the temporal model. Project Aria’s documentation also shows why the issue can extend within an image: rolling-shutter observations depend on image-row timing. For this lesson, the essential implication is sufficient: simultaneous arrival does not establish simultaneous observation. (facebookresearch.github.io)

Confidence is a claim that also requires evidence

The robot may identify a sample rack while separately evaluating whether that identification is sufficiently supported. These are different levels of description: one concerns the environment, while the other concerns the quality of the robot’s own inference. METATOOL investigates computational models of synthetic awareness grounded in human metacognition and validated through robotic experiments. Its project description includes monitoring and self-evaluation as capabilities intended to support adaptation. (metatool-project.eu)

Within this lesson, metacognitive monitoring means examining the evidential standing of an interpretation, not attributing human subjective experience to a robot. A practical design interpretation is to preserve information about ambiguous measurements, competing explanations and observation timing alongside the environmental estimate. This makes “the rack is there” distinguishable from “the available evidence supports the rack hypothesis under these assumptions”. This interpretation connects METATOOL’s self-evaluation objective with the uncertainty and observation-model framework described above. (cordis.europa.eu)

For example, the laboratory robot might record an object hypothesis as supported by appearance but unresolved by range association. That record does not establish a universal numerical confidence scale, nor does it demonstrate synthetic awareness. It instead makes the inferential limitation explicit, which is the relevant methodological connection here.

Locating the sample rack without inventing certainty

The laboratory robot now encounters a rack partly obscured by another container. In this illustrative example, the image analysis identifies a rack-like outline, and a range return lies 1.20 metres along the relevant sensing direction. The interpretation follows the inverse-problem framework: neither observation alone establishes that the visible outline and measured surface belong to the same object. The resulting representation must distinguish the measured range from the proposed association. (roboticsbook.org)

  1. Acquire: An image and range return enter processing; the output is two observations with their acquisition times.
  2. Describe: The image yields a rack-like visual hypothesis; the output is a candidate identity rather than a confirmed object.
  3. Associate: The candidate outline is considered alongside the 1.20-metre return; the output is a proposed relationship between observations.
  4. Represent: The proposed relationship retains its ambiguity; the output records a candidate rack location with unresolved association.

The benefit of this illustrative workflow is representational discipline: downstream manipulation receives a qualified location rather than an unsupported certainty. Its limitation remains substantive: partial visibility may leave several explanations compatible with the evidence. The workflow does not manufacture missing information; it preserves the distinction between what was measured and what was inferred. (roboticsbook.org)

Identifying a sample container from imperfect evidence

The same robot subsequently inspects a container whose material category matters to the laboratory workflow. For this illustrative calculation, assume two equally probable categories: metal and non-metal. Assume a positive conductivity likelihood of 0.9 for metal and 0.1 for non-metal; only the first likelihood follows the textbook’s hypothetical can example, while the second likelihood and equal priors are stipulated here. The calculation demonstrates why a sensor’s likelihood and an updated category probability are distinct. (roboticsbook.org)

  1. Specify: Prior probabilities of 0.5 and 0.5 enter the model; the output is an explicit starting distribution.
  2. Observe: A positive conductivity result enters the model; the output is evidence compatible with both categories.
  3. Weight: Multiplication produces (0.9\times0.5=0.45) for metal and (0.1\times0.5=0.05) for non-metal.
  4. Normalise: Division by (0.45+0.05=0.50) produces updated probabilities of 0.9 for metal and 0.1 for non-metal.

Under these stipulated assumptions, the result supports the metal interpretation without making it certain. Its benefit is an explicit, inspectable relationship between assumptions, observations and the resulting classification. Its limitation is equally explicit: changing the prior probabilities or likelihoods changes the answer, so the laboratory’s categorisation workflow must not detach the reported probability from its model. (roboticsbook.org)

Perception as accountable inference

Reliable environmental perception preserves the distinction between a physical signal, a numerical observation and an inferred environmental property. It also retains the uncertainty and temporal assumptions that qualify the inference. These distinctions make the laboratory robot’s representation more interpretable without pretending that incomplete evidence has become complete. (roboticsbook.org)

  • Measurements are evidence, not object identities.

  • Observation models connect possible states to expected measurements.

  • Uncertainty and timing qualify environmental interpretations.

  • Self-evaluation concerns the standing of an inference, not merely its content. (roboticsbook.org)

Next, we'll build on this by exploring Feedback and behavioural regulation.

Web Sources

Last modified: Saturday, 31 October 2026, 2:33 PM