Mapping the multimodal genres of traditional and electronic newspapers

John Bateman¹, Judy Delin², Renate Henschel³

¹University of Bremen (bateman@uni-bremen.de),
²Enterprise IDU (judy.delin@enterpriseidu.com),
³University of Bremen (rhenschel@uni-bremen.de)

In this paper we describe the design and use of the richly structured corpus of multimodal print and electronic documents that we have been constructing as part of the GeM project (Genre and Multimodality: Delin and Bateman, 2002, Bateman et al., 2002; http://www.purl.org/net/gem). The GeM project defines an annotation scheme for describing the multimodal contributions made within documents—focusing particularly on the contributions of text, graphical and diagrammatic content and their combination in layout. Our annotation scheme requires further that we give due attention not only to the finished artefacts—the documents themselves—but also to the practical conditions of production which constrain the shape of the final product

As one of the areas of application of this corpus, we are investigating the changes and continuities in generic identity exhibited by British newspapers in their move to provide electronic web-based editions. An exact multi-layered annotation of such documents allows us to investigate empirically the design decisions employed and how these have been distributed across the presentation modes available. As a consequence, we can articulate quite finely the progression from print to electronic version and see something of how these two delivery forms are mutually changing one another.

Our more general aim with this approach is to extend the traditional linguistic notion of ‘genre’ (cf. Eggins and Martin, 1997) so as to take in generalizations ranging over diverse modes of presentation, not simply the linguistic. This therefore falls within the research paradigm set out by, for example, Kress and van Leeuwen (2001), while at the same time emphasizing the central role to be played by corpus-based empirical study. For future work it will be necessary to extend both the scope and depth of our corpus materials. We outline some of the directions that this must take, considering in particular documents belonging to other genres and to other cultures.

References

Delin, J. L,   & Bateman, J. A. (2002). Describing and critiquing multimodal documents. Document Design, 3,  140--155.

Bateman, J. A., Delin, J. L. & Henschel, R. (2002). A brief introduction to the GEM annotation schema for complex document layout. In Proceedings of the 2nd Workshop on NLP and XML (NLPXML-2002) --- Post-Conference Workshop of the 19th International Conference on Computational Linguistics (COLING-2002) (G. Wilcock, N. Ide and L. Romary, eds.).  Academica Sinica, Taipei, Taiwan: Association of Computational Linguistics and Chinese Language Processing.

Eggins, S. & Martin, J. R. (1997). Genres and Registers of Discourse. In Discourse Studies: a multidisciplinary introduction. Discourse. Discourse as Structure and Process (D. T. A. va, ed.) (pp. 230-256).  London: Sage.

Kress, G. & van Leeuwen, T.  (2001). Multimodal discourse: the modes and media of contemporary communication. London: Arnold.