Designing Across Senses. A Multimodal Approach to Miasteczko Śląskie

Today we have the ability to connect speech, touch, haptic, and gestural interfaces into products that engage several human senses at once. This practical book explores examples from current designers and devices to describe how these products blend multiple interface modes together into a cohesive user experience.Authors Christine Park and John Alderman explain the basic principles behind multimodal interaction and introduce the tools you need to root your design in the ways our senses shape experience. This book also includes guides on process, design, and deliverables to help your team get started.The book covers several topics within multimodal design, including:New Human Factors: learn how human sensory abilities allow us to interact with technology and the physical worldNew Technologies: explore some of the technologies that enable multimodal interactions, products, and capabilitiesMultimodal Products: examine different categories of products and learn how they deliver sensory-rich experiencesMultimodal Design: learn processes and methodologies for multimodal product design, development, and release Spis treści: Praise for Designing Across Senses [Preface] What Is This Book About? Who Should Read This Book How This Book Is Organized Part I: New Human Factors Part II: Multimodal Design Why Write a Book About Multimodal Design Acknowledgments 1. Returning to Our Senses If a Tree Falls in the Forest The Sound of Violence Experience Is Physical People Have Modalities Devices Have Modes Human Modalities + Device Modes = Interfaces Physical Information: The New Data Sensing, Understanding, Deciding, and Acting: The New Human Factors Sensing Understanding and Deciding Acting Focus: The New Engagement Multimodality Makes a Wider Range of Human and Product Behaviors Possible How Multimodality Affects Design Creating Usability Creating Delight, Trust, and Love Multimodal Design Is Cross-Disciplinary Summary 2. The Structure of Multimodal Experiences The Human Slice of Reality: Umwelt and Sensibility Assembling Multimodal Experiences: Schemas and Models The Building Blocks of Multimodal Experience Summary 3. Sensing The Three Main Categories of Stimuli Electromagnetic Chemical Mechanical Defining the Senses: Dimension, Resolution, and Range Sensory Focus: Selecting, Filtering, and Prioritizing Information Reflexes Our Senses and Their Unique Properties Vision Visual Interfaces Hearing Auditory Interfaces Touch (Somatosensory or Tactile Abilities) Haptic Interfaces (Tactile, Proprioceptive, and Vestibular) Smell (Olfactory Ability) Olfactory Interfaces Taste (Gustatory Ability) Gustatory Interfaces Sixth Senses and More Time and Rhythm Proprioception and the Vestibular System Summary 4. Understanding and Deciding The Foundations of Understanding: Recognition, Knowledge, Skills, and Narratives Aware and Non-Aware: Fast and Slow Thinking Agency: Balancing Self-Control and Problem Solving Motivation, Delight, Learning, and Reward: Creating Happiness Summary 5. Acting About Anthropometrics The Origin of Anthropometrics Task Performance Nonverbal Communication Precision Versus Strength Inferring Versus Designating Intent Summary 6. Modalities and Multimodalities Modalities: How We Use Our Senses Types of Modalities We Shape Our Modalities, and They Shape Us Attributes and Abilities of Modalities Applying Modalities to Design Multimodalities Trusted Version and Performance Optimization Validation Integration A Single Prioritized Sense or Many Together? Across one main sense Across multiple senses How Multimodality Shapes Our Activities and Experiences Attributes and Abilities of Multimodalities Focus Flow Sequence Simultaneity Shift Transition Substitution Translation Proficiency Common Categories of Multimodalities Basic abilities Orientation and scanning Handeye coordination (visuo-haptic integration) Social interaction Performance and athletics Cognition and analysis Applying Multimodality to Design Maintaining Focus Respecting Cognitive Load Overcoming Barriers with Substitutions and Translations Shifts, Interruptions, and Flow Maintaining focus Reinforce Pace Block Dealing with interruptions Safety exits Ease of re-entry Off-switch Allowing shifts Social or ecosystem norms Priming Feedback and Validation Body language and physical engagement Summary 7. The Opportunity: Transforming Existing Products and Developing New Ones Key Applications of IoT: Monitor, Analyze and Decide, Control and Respond Functional Categories Monitor Analyze and Decide Control and Respond Disruptive Technologies Removing SoundAnd Putting It Back Mapping Apps Know Who Is in the Drivers Seat Beginning Inquiry Workflow to Identify Opportunities Assessing user needs Assessing user context Assessing changes to existing product modes Summary 8. The Elements of Multimodal Design Using Physical Information Constructing Knowledge, Interactions, and Narratives Summary 9. Modeling Modalities and Mapping User Experiences Behaviors Shared Between Users and Devices Demanding Contexts and Complex Interactions Call for Alignment Experience Maps and Models for Multimodality Different Maps Cover Different Scopes and Details Key Considerations of Multimodal Design Modalities and Senses Focus Level of Focus and Engagement Depth Continuity Sequence Shifts Flow and Habits Interruptions Substitutions Specialized Integration Knowledge and Skill Key Contexts in Multimodal Experiences Physical and environmental context Social and institutional context Device and information context Example Maps and Models Experience Map: Transitional Flow Ecosystem Map Context Map Focus Model Storyboards and Keyframing Orienter Establisher Initial Prolongation Peak Release Update as Needed Summary 10. Form Factors and Configurations Creating Multimodal Properties Configuring Interface Modes Mapping Modal Behaviors to Modal Technologies Vision Dominant Activities Immersive Activities: Screen-Based Experiences and VR Augmented or Auxiliary Activities: Visual Indicators for Peripheral Information Augmented Reality Versus Augmented Products: Visual Arrays of Control and Choice Automated Visual Capabilities Creating Focal Experiences with Audio and Speech Personal Sound Experiences Social Experiences: Broadcast Conversation Experiences: Speech Creating Haptic Experiences Summary 11. Ecosystems Device Ecosystems Information Ecosystems Physical Ecosystems Social Ecosystems Specialized Ecosystems Cloud Architectures: Distributing Resources Through Connectivity Ecosystem and Architecture: Applying Ecosystem Resources to Multimodal Design Sensing Experiences: Answering the DoorA Doorbell, Ring, and the August Lock Understanding and Deciding Experiences: Determining Distancea Pedometer, Apple Watch, and Lyft App Acting Experiences: Writing and DrawingA Pencil, a Tablet, and the Apple Pencil Summary 12. Specifying Modalities: States, Flows, Systems, and Prototypes Introduction: A Prototype Is a Custom Measuring Tool Practice Makes Perfect The Media of Multimodal Products: Information and Interactions, Understandings and Behaviors The Product Development Process for Multimodal Products Defining Design Requirements User Goals, Scenarios and Storyboards, and Use Cases Pseudocode and Swimlane Logic Flows Specifying Multimodalities Synchronous and Asynchronous Modes Parallel and Integrated Modes Input/Output MAP Summary 13. Releasing Multimodal Products: Validation and Learning Release Is About Process Alpha Release Organizing alpha Learning from Alpha Beta Release Organizing Beta Learning from Beta Public Release Validation and Feedback Important metrics Feedback methods The Out-of-the-Box Experience Summary A. Further Reading B. Glossary Index