Multimodal Interaction

Multimodal interaction, experience sharing, and storytelling

The 6th international conference on Design Principles and Practices, 20-22 January, 2012, UCLA, Los Angeles, USA

Presentation. “Multimodal Interaction as Communicative Media of Narrative Form for Experience Sharing” 

Yuan-Yi Fan & Péter Pál Boda (2012), “Multimodal Interaction as Communicative Media of Narrative Form for Experience Sharing”, International Conference on Design Principles and Practices & An International Journal (Journal paper, in review)

 

Bio-inspired Works

BioCymatics – mobile arts [Heart Rate Measurement via Camera, 2D Chladni Plate in OpenGL ES 2.0, Granular Sound Synthesis for iPod Music Library, Bonjour for Interactive Installation]

He+ARTs – mobile biometrics for creative installation [Camera Heart Rate Detection, Bonjour for Interactive Installation]

Device-Server-Plugin for BioPAC – XML-RPC, socket programming, COCOA [it allows streaming physiological signals from BioPAC's AcqKnowledge software to Allosphere's native mapping portal Device Server for developing multimodal content]

BioRhythm - biologically-inspired (spatial) audiovisual installation [Expressing "Harmonics on Spectrum of Arterial Pulse Pressure Waves" and "Resonance - the Missing Phenomenon in Hemodynamics" through Multimedia Arts]

Ambient Vision – Interactive installation that addresses rippled mental images as the product of perceived external stimulus [Kinect sensor (Microsoft)] and internal bodily response [Bionomadix wireless bio-sensors (Biopac System Inc.)]

“Ambient Vision”, Public Installation at End of Year Show Media Arts and Technology University of California Santa Barbara, June, 9th, 2011

 

Electronic Music/Data Sonification

Performance

“ECG-Breathe-Ambient (remix of P.O.M.)” presented at the CREATE concert “Critical Point”, Lotte Lehmann Concert Hall, University of California Santa Barbara on Thursday 13 October 2011.

Composition

“P.O.M.“ ECG/Stereo field recorder sonification/mixing, Honorable Mention in the Electronic-Acoustic category in the 2009-2010 Dorothy and Sherrill C. Corwin Awards for Excellence in Music Composition Competition, Music Department at University of California Santa Barbara, 2010

 

Visualization and Computer Graphics Gallery

Popular Vector – data visualization [3D data visualization of Seattle Public Library transactions, with additional dimension on how Google News API interprets title of Chomsky's books]

Social Voting + Appropriation Arts - website design, social experiment, appropriation arts [Drupal Voting API + Social Media + Visualizing Speech Styles]

Link to my OpenGL/GLSL/C++ Gallery

 

Form of Resonance Series – I

Interactive installation that addresses frequency relation of harmonic pulse spectrum in real time [MP150 PPG (Biopac System Inc.)]

Ryan McGee, Yuan-Yi Fan, Reza Ali, BioRhythm – a Biologically-inspired Audio-Visual Installation, in Proc. of NIME 2011, The 2011 International Conference on New Interfaces for Musical Expression, Oslo, Norway, May 30-June 1, 2011, pp. 80-83.

“BioRhythm”, Public Installation at NIME, Oslo, Norway, June, 1st, 2011

“BioRhythm”, Public Installation at TEDActive2011 AlloSphere Open House, Santa Barbara, CA, Feb., 28th, 2011

“BioRhythm”, Public Installation at Mindshare L.A., Los Angeles, CA, August, 19th, 2010

“BioRhythm”, Invited Presentation at Biopac System Inc., Santa Barbara, CA, June, 7th, 2010

“BioRhythm”, Public Installation and Critique Session, End of Year Show, Media Arts and Technology, University of California Santa Barbara 2010, May, 26th, 2010