Physical AI Lab // Episode 1.3 // One-pager
Sensors: look, measure, touch
Cameras, depth, force and touch
Three key points
- Cameras (RGB) see colour and shape and recognise objects, but a flat photo can't tell near from far (the colour camera on one depth camera has 12 million pixels, O).
- Depth cameras give every pixel a distance (the same device's depth stream has about 1 million pixels, coarser than colour, O); stereo disparity or timing light can both measure it. Force and touch take over after contact: touching or not, how hard, slipping or not (one force-controlled arm has a torque sensor in every joint, O).
- Proprioception: joint encoders tell the robot where it has turned to. Only several sensors together make the whole picture, and all of them need calibrating and checking against each other.
Evidence: O official · R research · M media
| O | 12 million | pixels: the colour camera on one depth camera (Microsoft Azure Kinect DK) |
| O | 1 million | pixels or so: the same device's depth camera (time of flight; coarser than the colour stream) |
| O | Every joint | of one force-controlled arm has a built-in torque sensor, to estimate external contact forces and sense collisions |
Further reading
- Microsoft · Azure Kinect DK hardware specifications
https://learn.microsoft.com/en-us/azure/kinect-dk/hardware-specification - Franka Research 3 (a torque sensor built into every joint)
https://franka.de/franka-research-3 - DIGIT: a low-cost, high-resolution vision-based tactile sensor (arXiv 2005.14679)
https://arxiv.org/abs/2005.14679