This book provides a thorough introduction to the subfield of theoretical computer science known as grammatical inference from a computational linguistic perspective. Grammatical inference provides principled methods for developing computationally sound algorithms that learn structure from strings of symbols. The relationship to computational linguistics is natural because many research problems in computational linguistics are learning problems on words, phrases, and sentences: What algorithm can take as input some finite amount of data (for instance a corpus, annotated or otherwise) and output a system that behaves "correctly" on specific tasks? Throughout the text, the key concepts of grammatical inference are interleaved with illustrative examples drawn from problems in computational linguistics. Special attention is paid to the notion of "learning bias." In the context of computational linguistics, such bias can be thought to reflect common (ideally universal) properties of natural languages. This bias can be incorporated either by identifying a learnable class of languages which contains the language to be learned or by using particular strategies for optimizing parameter values. Examples are drawn largely from two linguistic domains (phonology and syntax) which span major regions of the Chomsky Hierarchy (from regular to context-sensitive classes). The conclusion summarizes the major lessons and open questions that grammatical inference brings to computational linguistics. Table of Contents: List of Figures / List of Tables / Preface / Studying Learning / Formal Learning / Learning Regular Languages / Learning Non-Regular Languages / Lessons Learned and Open Problems / Bibliography / Author Biographies
Jeffrey Heinz received his Ph.D. from the University of California, Los Angeles in 2007, and is currently an Associate Professor at the University of Delaware. His research lies at the intersection of theoretical and mathematical linguistics, theoretical computer science, and computational learning theory, with specializations in phonology, linguistic typology, and grammatical inference. His work in these areas has appeared in the journals Linguistic Inquiry, Phonology, Theoretical Computer Science, Topics in Cognitive Science, Transactions of the Association of Computational Linguistics, and Science, among others. His current research interests are on establishing language-theoretic, automata-theoretic, model-theoretic, and logical characterizations of subregular classes of formal languages and transductions in order to better characterize the computational nature of phonological grammars and to better understand how they can be learned. He has served as part of the executive committee of the Association for Computational Linguistics Special Interest Group in Computational Morphology and Phonology (ACL-SIGMORPHON) since 2007. He has been a member of the steering committee of the International Community in Grammatical Inference (ICGI) since 2012. Moving forward, he would like to also support and strengthen the work of the Association for Mathematics of Language (MOL) and the Association for Logic, Language, and Information (FoLLI).
Colin de la Higuera received his Ph.D. at Bordeaux University, France, in 1989. He has been an Associate Professor at the University of Montpellier, a Professor at Saint-Etienne University, and is now a Professor at Nantes University. He has been involved in a number of research themes, including algorithmics, formal language theory, and pattern recognition. His chief interest lies in grammatical inference, a field in which he has been the author of more than 50 reviewed research papers and a monograph, Grammatical Inference: Learning Automata and Grammars, published in 2010. He has developed algorithms, studied learning models, and has been trying to link classical formal language frameworks with alternative ways of defining languages, inspired by linguistic considerations or techniques developed in pattern recognition. He has been chairman of the International Community in Grammatical Inference (2002-2007) and president of the SIF: The French Informatics Society (2012-2015). He is currently a trustee of the Knowledge for All foundation and working toward the usage of technology for an open dissemination of knowledge and education.
Menno van Zaanen received his Ph.D. from the University of Leeds, UK in 2002. He holds Master degrees in both computer science and linguistics. He is currently an Assistant Professor at Tilburg University, the Netherlands. His research concentrates on empirical grammatical inference and its applications. He worked and is still working on several projects dealing with structure in different modalities, multi-modal information retrieval, question answering, and symbolic machine learning for language and music. He has taught courses on a range of topics, including digital heritage, natural language processing, language and speech technology, social intelligence, and information search. He has published on several systems that deal with both clean and noisy linguistic data, such as language independent syntactic structure induction, boundaries in compounds (of different languages), spelling checkers, part-of-speech tagging of Twitter messages, and the identification of patterns in music and text. He is a founding member of the International Community in Grammatical Inference and was chairman between 2007 and 2010. He is International Advisory Committee member of the ACL Special Interest Group on Finite-State Methods (ACL-SIGFSM), editorial board member of the CLIN journal, and Associate Editor of the Computational Cognitive Science journal.
