Cultural Heritage Objects: Bringing Them Alive Through Virtual Touch

Abstract

In recent years, several attempts have been made to preserve the cultural heritage sites, including the architectural ruins at Hampi, with the help of digital technology. A user’s experience of interaction with such heritage objects becomes more realistic if we incorporate haptics (a sense of touch) along with audio–visual rendering. Incorporation of haptic feedback allows people to touch and feel the objects irrespective of their locations. In this chapter, we discuss a combined hapto-visual-auditory rendering technique of 3D cultural heritage objects at multiple levels of spatial details. The frontal part of the object is represented as a Monge surface, and the object is shown in the the form of quad mesh. The challenging part of the research is to haptically render a Monge surface at different levels of details in real time during haptic interaction. A proxy-based haptic rendering technique is used to address these issues, with the proxy update frequency around one hundred times faster than the minimum haptic update frequency of 1000 Hz. We also incorporate some additional surface properties such as friction and texture to enhance the user experience. The surface texture information is captured by using a bilateral filter on the 3D depth data. The audio rendering is integrated by incorporation of appropriate position dependent audio clips. When a collision takes place, the corresponding audio note is played. The visualization of digitally preserved heritage structures may include animated characters which are often soft and inhomogeneous in material composition. In this case, the above-mentioned technique does not work as one has to accommodate deformation of the object during haptic interaction. In order to handle such objects, we discuss a physics-based rendering approach based on the Kirchhoff thin plate theory. Experimentations by several different subjects demonstrate that the such rendering techniques greatly enhance a user’s experience.