Temporal consistency among five-sensory events: application to multimedia technology

Abstract

While transmitting multi or five- sensory information after transduction, the concept of temporalconsistency becomes an important issue at the receiving end from the point of view of context. Anyviolation of temporal consistency among the different decoded sensory information can lead to a failure ofcontext, thereby making the multisensory system ineffective in creating unified percept and bringing inreality to any scene. Therefore, when a multisensory information is captured, transduced and the resultingconvergent (fused or mashed) information is transmitted as a multimedia information, and again reverse -transduced and interpreted at the receiving end, the temporal constraints among the different sensoryresponses need to be consistent and preserved. Since a multimedia message containing fused five-sensoryinformation describes patterns of occurrence of events in the real world, the invocation of a partiallyordered sequence of such messages in a given context should take us from one consistent state to anotherconsistent state and checking for context violations. In this paper we describe a cooperating-agents basedalgorithm to detect temporal consistency among the five sensory events and for multimedia five-sensoryprocessing. The theoretical formulation requires elementary principles of temporal logic. The cooperativeaspects of the agent-based algorithm is illustrated using consistent and inconsistent examples and theUML activity diagram. This algorithm will be very useful for applications in Web 2.0 location-basedcontext awareness in multimedia technology.