Researchers in computer science and computer engineering devote a significant part of their efforts on communication and interaction between man and machine. Indeed, with the advent of multimedia and multimodal processing in real time, the computer is no longer considered only as a computational tool, but as a machine for processing, communi cation, collection and control. Many machines assist and support many activities in daily life. The main objective of this paper is to propose a new methodological solution by modeling an architecture that facilitates the work of multimodal system especially for a fission module. To realize such systems, we rely on ontology to integrate data semantically. Ontologies provide a structured vocabulary usedas support for data representation. This paper provides a better understanding of the fission system and multimodal interaction. We present our architecture and the description of the detection of optimal modalities. This is done by using an ontological model that contains different applicable scenarios and describes the environment where a multimodal system exists.
S. Oviatt, “Multimodal Interfaces,” In: S. Oviatt, Ed., The Human-Computer Interaction Handbook, Lawrence Erlbaum Associates Inc., Hillsdale, 2003, pp. 286-304.
A. Zaguia, et al., “Using Multimodal Fusion in Accessing Web Services,” Journal of Emerging Trends in Computing and Information Sciences, Vol. 1, No. 2, 2010, pp. 121-138.
D. Lalanne, et al., “Fusion Engines for Multimodal Input: A Survey,” The 11th International Conference on Multimodal Interfaces, Cambridge, 2-6 November 2009, pp. 153-160.
P. Feiteira and C. Duarte, “Adaptive Multimodal Fusion,” Universal Access in HCI. Design for All and eInclusion, Lecture Notes in Computer Science, Vol. 6765, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, pp. 373-380. doi:10.1007/978-3-642-21672-5_41
C. Lauer, “Contending with Terms: ‘Multimodal’ and ‘Multimedia’ in the Academic and Public Spheres,” Computers and Composition, Vol. 26, No. 4, 2009, pp. 225-239. doi:10.1016/j.compcom.2009.09.001
N. O. Bernsen, “Multimodality Theory,” In: D. Tzovaras, Ed., Multimodal User Interfaces, Springer, Berlin, 2008, pp. 5-29. doi:10.1007/978-3-540-78345-9_2
R. Bolt, “Put-That-There,” Voice and Gesture at the Graphics Interface ACMSIGGRAPH Computer Graphics, Vol. 14, No. 3, 1980, pp. 262-270. doi:10.1145/965105.807503
N. Guarino, D. Oberle and S. Staab, “What Is an Ontology?” In: S. Staab and R. Studer, Eds., Handbook on Ontologies, Springer Berlin Heidelberg, Berlin, Heidelberg, 2009, pp. 1-17.
M. E. Foster, “State of the Art Review: Multimodal Fission,” University of Edinburgh, COMIC Project 2002.
W. A. Carnielli and C. Pizzi, “Modalities and Multimodalities,” Springer, Berlin, 2008.
A. Zaguia, et al., “Interaction Context-Aware Modalities and Multimodal Fusion for Accessing Web Services,” Ubiquitous Computing and Communication Journal, Vol. 5, No. 1, 2010, in press.
A. Jaimes and N. Sebe, “Multimodal Human-Computer Interaction: A Survey,” Computer Vision and Image Understanding, Vol. 108, No. 1-2, 2007, pp. 116-134. doi:10.1016/j.cviu.2006.10.019
D. Costa and C. Duarte, “Adapting Multimodal Fission to User’s Abilities,” In: C. Stephanidis, Ed., Universal Access in Human-Computer Interaction. Design for All and eInclusion, Vol. 6765, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, pp. 347-356. doi:10.1007/978-3-642-21672-5_38
M. E. Foster, “Interleaved Preparation and Output in the COMIC Fission Module,” Proceeding of Workshop on S oftware, Stroudsburg, 2005, pp. 34-36. doi:10.3115/1626315.1626318
P. Grifoni, “Multimodal Fission,” In: P. Grifoni, Ed., Multimodal Human Computer Interaction and Pervasive Services, IGI Global, 2009. doi:10.4018/978-1-60566-386-9.ch006
C. Rousseau, et al., “A Framework for the Intelligent Multimodal Presentation of Information,” Signal Processing, Vol. 86, No. 12, 2006, pp. 3696-3713.
A. Benoit, et al., “Multimodal Focus Attention and Stress Detection and Feedback in an Augmented Driver Simulator,” Personal and Ubiquitous Computing, Vol. 13, No. 1, 2009, pp. 33-41.
A. Zaguia, et al., “Multimodal Fission For Interaction Architecture,” Journal of Emerging Trends in Computing and Information Sciences, Vol. 4, No. 1, 2013, pp. 152-166.
A. Zaguia, et al., “Architecture Générique Pour le Processus de la Fission Multimodale,” 25th Canadian Conference on Electrical and Computer Engineering, Montreal, 29 April-2May 2012.
M. D. Hina, C. Tadj, A. Ramdane-Cherif and N. Levy, “A Multi-Agent Based Multimodal System Adaptive to the User’s Interaction Context,” In: F. Alkhateeb, MultiAgent Systems-Modeling, Interactions, Simulations and Case Studies, InTech, 2011, pp. 29-56.
T. R. Gruber, “Towards Principles for the Design of Ontologies Used for Knowledge Sharing,” Formal Ontology in Conceptual Analysis and Knowledge Representation, Kluwer Academic, Dordrecht, 1993.
S. University, “Ontolingua,” 2013. http://www.ksl.stanford.edu/software/ontolingua
S. University, “OKBC,” 2013. http://www.ksl.stanford.edu/software/OKBC
C. Huajun, W. Zhaohui, W. Heng and M. Yuxin, “RDF/ RDFS-Based Relational Database Integration,” International Conference on Data Engineering, Atlanta, 3-8 April 2006, pp. 94-94.
I. Horrocks, “DAML + OIL: A Description Logic for the Semantic Web,” IEEE Data Engineering Bulletin, Vol. 25, No. 1, 2002, pp. 4-9.
G. Antoniou and F. Harmelen, “Web Ontology Language: OWL,” In: S. Staab and R. Studer, Eds., Handbook on Ontologies, Springer Berlin Heidelberg, Berlin, Heidelberg, 2009, pp. 91-110. doi:10.1007/978-3-540-92673-3_4
H. Knublauch, R. Fergerson, N. Noy and M. Musen, “The Protégé OWL Plugin: An Open Development Environment for Semantic Web Applications,” The Semantic Web, Vol. 3298, 2004, pp. 229-243.
S. Bechhofer, I. Horrocks, C. Goble and R. Stevens, “OilEd: A Reasonable Ontology Editor for the Semantic Web,” Advances in Arts Intelligent, Vol. 2174, Springer Berlin Heidelberg, Berlin, Heidelberg, 2001, pp. 396-408.
Y. Sure, J. Angele and S. Staab, “OntoEdit: Guiding Ontology Development by Methodology and Inferencing,” The Move to Meaningful Internet Systems CoopIS, DOA, and ODBASE, Vol. 2519, Springer Berlin Heidelberg, Berlin, Heidelberg, 2002, pp. 1205-1222.
Z. Huiqun, Z. Shikan and Z. Junbao, “Research of Using Protege to Build Ontology,” International Conference on Computer and Information Science, Shanghai, 30 May-1 June 2012, pp. 697-700.
K. Jensen, “Coloured Petri Nets: Central Models and Their Properties,” Vol. 254, Springer, Berlin, 1987, pp. 248-299. doi:10.1007/BFb0046842
L. Zhu, W. Tong and B. Cheng, “CPN Tools’ Application in Verification of Parallel Programs Information Computing and Applications,” Information Computing and Applications, Vol. 105, Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, pp. 137-143. doi:10.1007/978-3-642-16336-4_19