Seeing Between the Pixels: Pictures in Interactive Systems

This practical and informative book shows how pictures can be used in computer systems to convey information to users. The authors define a new classification of pictures and use it to analyze a wealth of topical examples.

PartI: Preliminaries.- 1 Introduction.- 1.1 Pictures and Society.- 1.2 Pictures and Language in Computer Science.- 1.3 Dialogue Systems.- 1.4 Pictures as Systems.- 1.5 Organization of the Book.- II: Fundamentals.- 2 Pictures in Computer Systems.- 2.1 Desktop Publishing.- 2.2 Drawing Programs.- 2.3 Hypertext Systems.- 2.4 Image Processing Software.- 2.5 Three-Dimensional Computer Graphics Systems.- 2.6 Computer Simulation.- 2.7 Summary.- 3 Classification of Pictures.- 3.1 Classes of Pictures,.- 3.1.1 Presentational Pictures.- 3.1.2 Abstract-Graphical Pictures.- 3.1.3 Pictograms.- 3.2 Pictures Made of Visual Signs.- 3.2.1 Natural and Non-Natural Signs.- 3.2.2 Other Classes of Signs.- 3.2.3 Are There Visual Signs Independent of Conventions?.- 3.2.4 Arbitrariness of Conventional Signs.- 3.2.5 Summary of the Semiotic Excurse.- 3.3 Classification Scheme for Pictures.- 3.3.1 Characteristics of the Classes of Pictures.- 3.3.2 Contours of the Terms of the Classification Scheme.- 3.4 Discussion.- 4 Picture Processing by Humans.- 4.1 Pictures and the Iconic Code.- 4.2 Perception, Learning and Cognition.- 4.3 Cognitive Theories.- 4.4 Understanding of Pictures: The Communicative Theory Point of View.- 4.5 Human Memory Performance.- 5 Information Flow During Human-Computer Interaction.- 5.1 Information Theory.- 5.2 Information Transfer Between Humans.- 5.3 Transmitted and Transputed Information in Human-Computer Interaction.- 5.3.1 Information Transmitted and Transputed by a Computer.- 5.3.2 Information Transmitted and Transputed by Users.- 5.4 Methods of Transputing Information.- 5.5 Analysis of Some Dialogue Systems.- 5.6 Discussion.- III: Abstract-Graphical Pictures.- 6 Abstract-Graphical Pictures and Some Applications.- 6.1 Characterization.- 6.2 Focus on Relations Between Elements.- 6.3 Focus on Functionality.- 6.4 Focus on Behavior.- 6.5 Focus on Properties of Elements.- 6.6 Abstract-Graphical Pictures for Cognitive Processing.- 6.7 Discussion 1ll.- 7 Analysis of Abstract-Graphical Pictures.- 7.1 Requirements.- 7.2 Methodology.- 7.3 An Application to Pictures in Dictionaries.- 7.4 An Application to Business Graphics.- 7.5 Discussion.- 8 Users’Analysis and Criticism of Abstract-Graphical Pictures.- 8.1 The Role of Animation During Simulation.- 8.1.1 Simulation Methodology.- 8.1.2 Model Validation and User Support.- 8.1.3 Underestimation of Animation.- 8.2 Information Sources for the Computer.- 8.2.1 User Observations.- 8.2.2 Oracles Yesterday and Today.- 8.2.3 Architecture of an Oracle-Based Model Modification.- 8.3 Requirements and Components of an Oracle System for Simulation.- 8.3.1 Typical Oracles in Simulation.- 8.3.2 Knowledge Base.- 8.3.3 Results of the Reasoning Component.- 8.4 A Prototypical Implementation.- 8.4.1 Environment for the Implementation.- 8.4.2 Scenario.- 8.5 Discussion.- 9 Viewpoint Descriptions.- 9.1 Viewpoint Descriptions in Simulation.- 9.2 Information Conveyed by Animations.- 9.3 Formalism of Viewpoint Descriptions.- 9.3.1 Documentation.- 9.3.2 Criticism.- 9.3.3 The Complete Formalism.- 9.3.4 Temporal Aspects and Verbal Descriptions as General Additions.- 9.4 Implementation of the Viewpoint Description for the Simulator Create!.- 9.4.1 How to Use the Viewpoint Description.- 9.4.2 Example of a Viewpoint Description.- 9.5 Applications.- 9.5.1 Addition to the Knowledge Base for Oracle-Based Model Modification.- 9.5.2 Model Documentation with Viewpoint Descriptions.- 9.6 Discussion.- IV: Pictograms.- 10 The Nature of Pictograms and Their Use.- 10.1 Typicality of Pictograms.- 10.2 Pictograms and Their Normalization Demands.- 10.3 Methods for Describing Pictures.- 10.3.1 Frege’s Notation.- 10.3.2 Pictureless Knowledge Processing.- 10.4 Picture Frames.- 10.4.1 Constituent Parts of Picture Frames.- 10.4.2 Interactive Problem Solving with Picture Frames.- 10.4.3 An Application in Computer Aided Instruction of Chemistry.- 10.5 Are Combinations of Pictograms Still Pictograms?.- 11 Pictograms as Words.- 11.1 Choice of Pictograms.- 11.2 Evaluation of Pictograms.- 11.3 Ambiguity of Pictograms.- 11.4 Some Drawbacks of Pictograms in User Interfaces.- 11.4.1 Physiological Stress.- 11.4.2 Cognitive Stress.- 11.5 Speech to Augment Pictograms in User Interfaces.- 11.5.1 Speech in User Interfaces.- 11.5.2 An Experimental Interface.- 11.6 Discussion.- 12 Pictograms as Pictures.- 12.1 Design of Pictograms as Pictures.- 12.2 Pragmatics of Pictograms.- 12.3 Pictograms and Their Context.- 13 Formal Representations and Informal Presentations.- 13.1 Pictures as Write-Only Data Structures.- 13.2 Formal and Less Formal Aspects of Human-Computer Interaction.- 13.2.1 Formal Notations as Extensions to Informal Concepts.- 13.2.2 Formalization in Knowledge-Based Systems.- 13.3 Semi-Formal Representations.- 13.3.1 Characteristics of Semi-Formal Representations.- 13.3.2 The Trichotomy Informal - Semi-Formal - Formal.- 13.4 Discussion.- PartV: Presentational Pictures.- 14 Image Generation.- 14.1 Background.- 14.2 Object Layout.- 14.3 Choosing the Perspective.- 14.4 Choosing Light Sources.- 14.5 Cutaways and Ghosting.- 14.6 Complex Communicative Goals.- 14.7 Animation.- 14.7.1 Cinematic Editing.- 14.7.2 Animation of Complex Processes.- 15 Alternative Rendering of Images.- 15.1 Standardization.- 15.2 Standardization in Computer Graphics.- 15.3 Affecting the Effect of Rendered Images.- 15.3.1 Some Advanced Graphical Presentation Techniques.- 15.3.2 Information Flow.- 15.4 The Sketch-Renderer.- 15.4.1 Software Organization.- 15.4.2 The User Interface.- 15.4.3 Examples.- 15.5 Algorithms for the Sketch-Renderer.- 15.5.1 Rendering.- 15.5.2 Sketching Shadows.- 15.5.3 Drawing Lines and Circles.- 15.6 Special Effects in Line Drawings.- 15.6.1 Leaving Out Detail.- 15.6.2 Representing Movements of Objects.- 15.7 Assessing the Effect of Rendered Images.- 15.7.1 Experiment.- 15.7.2 Results.- 15.7.3 Interpretation of the Results.- 15.8 Discussion.- 16 Tactile Computer Graphics.- 16.1 Introduction.- 16.2 Tactile Output.- 16.3 Blind Computer Users.- 16.4 Visual and Tactile Graphics: A Comparison.- 16.5 Constructing 2D/3D Models.- 16.6 A Rendering/Editing System for Tactile Graphics.- 16.7 Discussion.- 17 Immersive Systems.- 17.1 Reality and Immersion.- 17.1.1 The Role Model.- 17.1.2 Augmented Reality.- 17.2 Degrees of Immersion.- 17.2.1 Low Level of Immersion.- 17.2.2 Head-Tracking and Parallax to Improve Immersion.- 17.2.3 Head-Mounted Displays in Immersive Systems.- 17.3 How Much Immersion Is Enough?.- 17.3.1 Monitors Versus Head-Mounted Displays.- 17.3.2 Spatial Perception of Architects.- 17.3.3 Degree of Presence.- 17.4 Support for Richer Interaction in Immersive Systems.- 17.4.1 Requirements.- 17.4.2 Hyper-Rendering.- 17.4.3 Notes on the Implementation of a Hyper-Renderer.- 17.4.4 Applications of Hyper-Rendering.- 17.5 Discussion.- VI: Epilogue.- 18 Pictures and Language.- 18.1 A Comparison Between Pictures and Language.- 18.1.1 Disadvantages of Presentational Pictures Compared to Language.- 18.1.2 Advantages of Presentational Pictures over Language.- 18.1.3 Common Strengths of Presentational Pictures and Language.- 18.2 Languages for Non-verbal Communication.- 18.3 Going Full Circle.- 19 Quovadis?.- 19.1 Falsification of Pictures.- 19.2 The Grey Area Between Enhancement and Falsification.- 19.3 The Bottom Line.- List of Figures, Tables, and Credits.- List of Petroglyphs.- Author Index.