[Infowarrior] - CFP: ³ Text Annotation for Political Science Research ²

[Richard Forno]

Call for Papers A special issue of the Journal of Information Technology &
Politics http://www.jitp.net <http://www.jitp.net/>

³Text Annotation for Political Science Research²
http://www.jitp.net/files/cfp_text_issue.pdf

Text is an important data source for political science research. Large,
digitized text collections are becoming increasingly common. Yet most
political scientists have little familiarity with the language-processing
methodologies available to support research using these collections.
Specifically, we are interested in methodologies from the fields of
information retrieval, natural language processing, and machine learning.
These techniques facilitate the automatic searching, organizing,
categorizing, and extracting of information from digitized text.

At a high level, the goal of language-processing is to provide one or more
semantic annotations on the text. The political science question of interest
can then be explored using these annotations. Text annotation techniques
vary not only according to the type of semantic annotation required, but
also according to the degree of manual intervention involved in the
annotation process: text annotation tasks can be accomplished entirely
manually (i.e., via human content coding), entirely automatically (e.g. via
keyword-based search or text clustering algorithms), automatically after a
manual training period (i.e. via "supervised" machine learning methods), or
semi-automatically (e.g. via "weakly supervised" machine learning methods
that acquire automatic annotation systems from very small amounts of
manually labeled text).

Although the potential of text annotation methods for political science
research is enormous, it is understandably difficult for researchers to know
where to start. In addition, in contrast to other research methodologies in
the social sciences, the criteria for evaluating social science results that
rely on automatic text annotation systems are not widely known, accepted, or
appreciated. Keyword searches, for example, are commonly used to trace
changing political attention across time, but rarely is attention given to
their reliability or accuracy, raising doubts about the validity of
researcher inferences.

The aim of the special issue is to solicit and publish papers that provide a
clear view of the state of the art in text annotation and evaluation,
especially for political science. How do the techniques map onto major
questions addressed by political scientists? What kinds of problems have
been addressed in existing work and what text annotation methods have proven
most successful? Are standard statistical measures of accuracy, recall, and
precision adequate for evaluating the performance of the text annotation
technique, or are new evaluation procedures needed that simultaneously
consider the social science question being investigated?

Given these interests, we therefore encourage submissions in the following
areas: tutorial-style surveys of state-of-the-art techniques in human
language technologies and text annotation; surveys of the state-of-the-art
in the application of language-processing techniques in the social sciences,
particularly in political science; comparisons of competing text annotation
methodologies on the same corpus/corpora; innovative evaluation and
diagnostic methods; studies of text annotation methods that try to limit the
amount of costly, manually annotated data for training automatic annotators,
e.g. active learning; specific applications and evaluations of
language-processing and text annotation techniques; applications of the
text-processing techniques on non-social science problems that point the way
to innovative social science applications; and surveys of the available
language-processing tools and resources with guidance for when to use them.

All submissions must be prepared according to the submission guidelines
available at: http://www.jitp.net <http://www.jitp.net/> .

Authors must submit via:
http://www.criticalmath.com/method/sm.php?org_id=12789

The initial submission is due by November 1, 2007

The guest editors for the special are:

Claire Cardie, Professor Computer Science and Information Science 4130 Upson
Hall Cornell University Ithaca NY 14853-7501 cardie at cs.cornell.edu

John Wilkerson, Associate Professor Department of Political Science 101
Gowen Hall University of Washington Seattle WA 98195-353530
jwilker at u.washington.edu