Mon, Jan 13, 2025

Object Recognition on Multitouch Tables: Uses and Technologies

Some interactive experiences have a rare power: they make you want to touch, manipulate, and experiment. In a world saturated with screens, scrolling, and dematerialized content, the need for tangible, embodied experiences has never been greater.

Long before multitouch became mainstream, creators like Étienne Mineur were already exploring the boundary between physical objects and digital content. With projects from Éditions Volumiques, dice, cards, and tokens became full-fledged interfaces, capable of triggering stories, animations, and on-screen interactions.

At PandaSuite, this vision of tangible interaction has long fueled our thinking. The idea that a real object could become a natural entry point into digital experiences, without code or technical complexity, has been part of the platform’s DNA from the start.

Today, thanks to the integration of the TUIO protocol, PandaSuite enables you to transform these concepts into concrete experiences, deployable on interactive screens and multitouch tables. A way to make accessible, without specific development, devices that were long reserved for complex or experimental installations.

In museums, education, events, or entertainment setups, object recognition addresses a clear expectation: reconciling physical gestures with digital experiences.

What is tangible object recognition?

Tangible object recognition allows an interactive screen to detect the presence, position, and movement of physical objects placed on its surface. These objects then become true triggers for digital actions.

Concretely, when an object is placed on a multitouch table, it can:

• display specific content
• launch an animation or video
• modify an interface
• activate an interactive scenario

Users no longer interact solely with their fingers, but with real objects, making the experience more natural, memorable, and collaborative.

What is the TUIO protocol?

Most object recognition devices rely on the TUIO (Tangible User Interface Object) protocol.

TUIO is an open protocol, widely used in professional interactive installations. It transmits essential information about detected objects in real time:

• Position: where the object is placed on the surface
• Orientation: how it is rotated
• Movement: displacement, rotation, appearance, or removal

👉 This protocol plays a key role: it bridges the gap between interactive hardware (multitouch table, screen) and the software responsible for interpreting this data to trigger actions.

Compatible hardware and ecosystem

Some manufacturers offer devices that are natively compatible with TUIO. This includes Displax, a recognized leader in professional multitouch solutions, which provides screens and tables capable of detecting active or passive objects.

Combined with software that can receive and leverage TUIO streams (like PandaSuite), this combination enables you to design robust, precise installations perfectly suited for intensive use (museums, events, public spaces).

Real-world use cases for multitouch object recognition

Museums and interactive exhibitions

Object recognition can transform a visit into an active experience. Visitors manipulate replicas, tokens, or symbolic objects to explore enriched content.

Examples:

• place a figurine on a map to discover its historical context
• position an object on a table to display an interactive timeline
• combine multiple objects to compare information

Trade shows and events

At a booth, tangible objects become entry points to content.

Examples:

• an object representing a product triggers an animated demonstration
• cards activate use cases or customer testimonials
• a token placed on a screen displays personalized data

These interactions strengthen engagement and promote message retention.

Education and training

In educational settings, object manipulation facilitates understanding of abstract concepts.

Examples:

• explore human anatomy using physical pieces
• match tangible shapes to mathematical concepts
• learn through experimentation rather than reading

Games and playful experiences

Object recognition also opens the door to new forms of hybrid games, bridging board games and digital experiences.

Examples:

• an interactive die that influences the course of an on-screen game
• physical cards that trigger animations or events
• collaborative games for multiple users on a multitouch table

Why integrate object recognition into an interactive project?

• Natural interaction: manipulate objects rather than menus
• Increased engagement: physical gestures reinforce involvement
• Collective experience: ideal for multi-user applications
• Unlimited creativity: each object can become a unique interface

This approach is particularly relevant for projects where experience takes precedence over simple information consumption.

How to design an experience with object recognition?

An object recognition project relies on three main building blocks:

1. Compatible hardware Multitouch table or screen capable of detecting objects via TUIO.

2. Tangible objects Tokens, cards, figurines, or custom objects, equipped or recognized by the system.

3. Software capable of exploiting TUIO The software interprets received data and triggers planned interactions.

With no-code software like PandaSuite, these interactions can be designed visually, without specific development, while combining touch gestures and object manipulation.

Toward a new generation of interactive experiences

Tangible object recognition marks a profound evolution in how interactive experiences are designed. At a time when interfaces are becoming increasingly virtual, manipulating real objects provides a concrete answer to a growing need: to live more sensory, more engaging, and more human experiences.

Museums, schools, events, cultural or entertainment installations: contexts where the tangible enriches the digital are multiplying. Where simple touch is no longer enough, the object becomes a powerful mediator between the user and the content.

Pioneers like Étienne Mineur demonstrated the potential of these approaches early on. Current technologies—open protocols, compatible hardware, and no-code tools like PandaSuite, now make it possible to deploy them on a larger scale, with more flexibility and creativity.

In a rapidly changing interactive landscape, experiences that combine physical and digital are no longer curiosities: they respond to a real public expectation.

And the next remarkable tangible experience might just be the one you’re about to imagine.

Start your project for free today on PandaSuite 😊