Text and Context : Document Storage and Processing

Text and Context: Document Storage and Processing describes information processing techniques, including those which do not appear in conventional textbooks on database systems. It focuses on the input, storage, retrieval and presentation of primarily textual information, together with auxiliary material about graphic and video data. There are chapters on text analysis as a basis for lexicography, full-text databases and information retrieval, the use of optical storage for both ASCII text and scanned document images, hypertext and multi-media systems, abstract document definition, and document formatting and imaging. The material is treated in an informal way with an emphasis on real applications and software. There are, among others, case studies from Reuters, British Airways, St. Bartholomew's Hospital, Sony, and HMSO. Relevant industry standards are discussed including ISO 9660 for CD-ROM file storage, CCITT Group4 data compression, the Standard Generalised Markup Language and Office Document Architecture, and the Postscript language. Readers will benefit from the way Susan Jones has brought together this information, in a logical sequence, to highlight the connections between related topics. This book will be of interest to second and third year undergraduates and MSc students in computer science, to B/TEC HTD final year computing and information science students either specialising in IT or taking an IT option, and to students taking courses in IT and in business computing systems.

One Introduction and Overview.- Data Capture.- Storage.- Searching.- Presentation.- Applications.- Two Fundamentals of Text Processing.- Natural Language as Data.- Representing Text.- Computers in Lexicography.- The Cobuild Project.- Obtaining the Corpus.- Basic Text Analysis.- Interactive and Dynamic Use of a Corpus.- Building the Dictionary Database.- Generating a Dictionary Text.- Summary.- Investigations.- References.- Three Information Retrieval I.- Definitions.- Information Retrieval Services.- Query Languages.- Database Design for a Co-ordinate Indexing System.- Word Occurrence Vectors and Document Signatures.- Assessment of IR Systems.- Improving Search Performance.- Thesauri.- Faceting.- Stop-lists.- Dealing with Variant Spellings.- Conflation, Suffix-stripping, Stemming, Lemmatisation.- Proximity Searching.- Ranking Retrieved Documents in Order of Relevance.- Exploiting Connections Within the Database.- Summary.- Investigations.- References.- Four Information Retrieval II.- The Move Towards Full-text Systems.- Reuters Newsbank.- Selection and Indexing.- Validation and Updating.- Searching.- The Status Text Retrieval System.- The ICL CAFS Extension.- Oracle SQL*Textretrieval.- Extensions to the Relational Model.- Extensions to SQL.- Handling Queries.- Text Compression Techniques.- Compression by Substitution.- Run-length Encoding.- Two-dimensional Encoding.- Summary.- Investigations.- References.- Five Introduction to Optical Storage.- The Physical Level.- Investigations.- References.- Six CD-ROM.- Physical Data Representation Methods.- Standards for CD-ROM Logical Structure.- Volumes.- Directories and Path Tables.- Files.- The Standard in Practice.- Example Applications.- Whitaker’s Bookbank.- The Possible Impact of CD-ROM on Libraries.- British Airways Technical Publications.- Background.- The Feasibility Study.- Structure of the Manuals.- System Operation.- Extensions.- Summary.- Investigations.- References.- Seven Worm Disc and Document Image Processing.- Overview of Worm Disc Characteristics.- Logical Data Organisation — Requirements and Strategies.- Worm Disc Applications.- An Optical Storage Archiving and Retrieval System.- Document Preparation.- Scanning.- Compression.- Verification/Processing.- Indexing.- Storage.- Retrieval.- Printing.- Conclusions.- Summary.- Investigations.- References.- Eight Video Disc and Computer-Based Training.- Physical Characteristics.- Video Disc Control Functions.- Video Disc Applications.- Educational Software Overview.- CBT: Authoring Systems.- Example System 1: MacAid.- Frames.- Programming Commands.- Use of Video Disc in MacAid.- Use of MacAid for Video Databases.- Example System 2: Interactive Knowledge System.- IAS: Page Structure Definition.- Video/Audio Production.- Page Editing.- Courseware Presentation.- Summary.- Investigations.- References.- Nine Hypertext Principles.- What is it?.- Hypertext Systems.- Data Models.- Frame-based Systems.- Scrolling Systems.- Textual Relationships and Their Representation.- Hierarchical.- Sequential.- Referential.- Hypertext System Design Issues.- Textual Units or Nodes.- Textual Relationships or Links.- Searching and Browsing.- Authoring Hypertext.- Authoring with HyperCard.- Authoring with Guide.- Preprocessing and Verification.- Large Scale Document Management.- Hypertext in a Broader Context.- Summary.- Investigations.- References.- Ten Describing the Structure of Documents.- The Need for Standards.- General Principles of Document Structuring.- The Standard Generalized Markup Language.- What is Mark-up?.- Defining Documents with Replacement Rules.- Other SGML Language Features.- SGML in Use: Creating and Formatting Documents.- The Oxford English Dictionary.- Her Majesty’s Stationery Office: Statutory Instruments.- Office Document Architecture.- Contrasts with SGML.- Summary of the ODA Document Processing Model.- Defining a Document: Generic and Specific Structures.- The Document Layout Process.- Examples of Object Attributes.- Comments on the ODA Processing Model.- Summary.- Investigations.- References.- Eleven Formatting and Printing Documents.- The Development of Desk-top Publishing.- Models and Metaphors.- Functions of Formatting Software.- Overall Document/Page Design.- Representation of Logical Structures.- Selection of Layout Structures.- Text Filling.- Document Style/Use of Auxiliary Files.- Special Document Elements.- Utilities.- Behind the Scenes.- Troff/Nroff.- Macros, Conditionals, and Traps.- Environments.- Diversions.- TeX and LaTeX.- TeX Formatting.- Exploiting TeX Macro Facilities.- Summary.- Investigations.- References.- Twelve Postscript.- The Postscript Imaging Model.- Stacks.- Fonts.- An Example Program.- Postscript in Practice.- Summary.- Investigations.- References.