How much rhetoric does an account of multimodal/hypermedial design need?

John Bateman, University of Bremen (bateman@uni-bremen.de)

In document design of almost all kinds it is now quite common to see appeals to 'rhetoric' in order to motivate particular design decisions over others (cf. Schriver, 1997 and many others). It is assumed, with considerable justification, that an appropriate rhetorical organization is a prerequisite for effective communication. There is, however, considerably less consensus concerning just what notions of rhetoric this might involve. This issue extends from decisions in practice through to theoretical and experimental discussions of, for example, automated hypermedia design where an explicit statement of the rhetorical tools employed is essential. Here positions differ: some researchers (e.g., André and Rist, 1993; Feiner and McKeown, 1993; Lindley et al., 2001) adopt some version of a computationally instantiated modern account of rhetoric such as Mann and Thompsons's Rhetorical Structure Theory (RST), others have suggested that such a rich and differentiated rhetorical account is not necessary and that a more purely structural and lean account of discourse coherence may suffice. This is an important issue since the adoption of RST is a substantial theoretical investment which complicates both resulting models and implementations considerably.

In order to investigate the extent to which a sophisticated, explicit and highly differentiating view of rhetoric is necessary in understanding document design, we have in the GeM project (http://www.purl.org/net/gem) been collecting and annotating a corpus of multimodal documents from a variety of genres—including newspapers, instruction manuals, electronic newspapers, magazines and informational books—that combine media, particularly text, graphics and diagrams. In this paper we focus on the use of this corpus for exploring the relationships between content, rhetorical structuring and layout more systematically. The GeM annotation scheme employs a slightly extended variant of Mann and Thompson's RST and so allows us to investigate both theoretically and empirically the extent to which successful designs draw on rhetorical structure. This paper reports on this work and argues that it is not possible to avoid a rich and explicit rhetorical structure such as is offered by RST when dealing with design decisions in general. The extent to which the rhetorical distinctions are taken up in layout decisions is, however, very flexible. Illustrations of this from our corpus will be given.  We also set out some of the theoretical issues involved in transferring the initially text-based notion of rhetorical structure of RST to layout.

References

André, E. & Rist, T. (1993). The design of illustrated documents as a planning task. In Intelligent Multimedia Interfaces. In M. T. Maybury (op.cit.), pp. 94—116.

Feiner, S. K.  & McKeown, K. R. (1993). Automating the generation of coordinated multimedia explanations. In M. T. Maybury (op.cit.), pp. 117—138.

Maybury, M. T. (1993). Intelligent Multimedia Interfaces. Cambridge, Massachusetts : AAAI Press and MIT Press.

Lindley, C., Davis, J.,  Nack, F. & Rutledge, L. (2001). The application of rhetorical structure theory to interactive news program generation from digital archives. CWI Report INS-R0101, Amsterdam.

Schriver, K. A. (1997). Dynamics in document design: creating texts for readers. New York: John Wiley and Sons.