With computer vision, you can make the physical world programmable. A real-world trigger – for example, the presence of a particular painting in a museum exhibit – can be used to open up a digital experience. One promising use of computer vision is building learning experiences that start in the physical world and bridge to the digital world for further learning.
Consider a scenario where you are in a museum at an art history exhibit. With vision, you can create an application that lets people scan a painting to see more information. Instead of using QR codes, which can be clunky and require adding codes to each exhibit, you can use items in the exhibit themselves as points to scan.
Kunstmuseum Bern, an art institution situated in Bern, Switzerland, is using computer vision to create a more immersive exhibition experience. In this article, we will explore what Kunstmuseum Bern is doing with computer vision and how vision, more broadly, can be used to craft unique, memorable interactive experiences.
How Kunstmuseum Bern Uses Computer Vision
Kunstmuseum Bern (the Museum of Fine Arts Bern) was established in 1879 and maintains a vast collection of art works, from works by Edmond Jean de Pury to Pablo Picasso.
The museum carefully selects the most suitable mediums to engage audiences. Providing immersive, memorable experiences is at the heart of the museum experience. For the museum, this means using text, sound, video, images, and digital tools to allow visitors to engage with exhibitions.
The Chief Curator at Kunstmuseum Bern asked themselves a question: “Can we provide comprehensive information through a digital guide without relying on QR codes?” Their team identified image recognition as a solution, and opted to use Roboflow for use in identifying objects.
Using Computer Vision for Interactive Exhibits
In their recent Katharina Grosse. Studio Paintings, 1988–2022 exhibition, the museum used computer vision as part of a digital guide. The digital guide made available by the museum provides a range of services. This includes a map of the exhibit and thumbnail previews of images. An "Identify image with camera" feature was available, allowing visitors to scan an artwork with their phone to learn more.
With this system, the digital and the physical were seamlessly integrated. Visitors could engage with the physical exhibit and scan to find out more information about the artwork.
To build this system, the team at Kunstmuseum Bern took photos of the artworks in the exhibit collection, annotated them with an appropriate label, and then linked each image to web pages that provided more information about each artwork. The museum worked with NETNODE as a digital partner to decide on project requirements, how to convey and represent information in the digital experience, and the user experience.
Reflecting on the project, the team at Kunstmuseum Bern noted “our visitors could explore artworks simply by pointing their smartphone camera at the artwork of interest.”
Furthermore, visitors could filter artworks by different characteristics, such as the decade in which an artwork was made and the techniques used to create the artwork:
Commenting further on the project, Kunstmuseum Bern noted:
From concept to development to implementation, digital companions transform how we engage with exhibitions and artworks. By utilizing a digital exhibition guide alongside Roboflow's image recognition tool, Kunstmuseum Bern harnesses the potential of the digital realm, offering a contemporary and engaging approach to art.
Computer Vision in Museum Exhibits
Kunstmuseum Bern demonstrates the extent to which vision can be used to enhance art exhibits. Being able to scan a work of art on your phone to find out more information eliminates the need to have an intermediary QR code that visitors need to find and scan. Other projects such as Barnes Focus use computer vision to provide a digital companion.
The perennial problem of a curator is balancing the amount of information in the exhibit with the artworks. Too much text becomes overwhelming; too little and the curiosity of visitors may not be fully satisfied.
With vision, you can bridge this gap: provide text descriptions as part of the exhibit, then allow visitors to scan objects if they want to learn more about a particular exhibit. As part of this, a visitor could bookmark the information for later, so they can continue to learn about a concept after leaving the museum.
The opportunity for technology to pair with museum exhibits is vast. Curators can use digital experiences to provide more information about an exhibit, references, links to other objects in the exhibit or elsewhere in the museum, and more.
Another way you could build a digital companion is to use a Snap Lens. Snap Lenses are interactive vision applications in Snapchat. Using a Snap Lens, you could create an experience that opens up more information about an exhibit when an object comes into view.
Building a Snap Lens gives you access to a range of Snapchat built-in features like easy sharing and the ability to record a video that features the digital exhibit functionality. This may be more engaging to younger audiences.
Any object that you can see can be made programmable in the way Kunstmuseum Bern built their system. Across areas of art, history, and culture – from prehistoric dinosaurs to paintings to objects found in excavations – vision can be employed. If you can see an object, it can be used as a springboard into an immersive experience accompanying an exhibit.