Analysing Multimodal Documents: Course Schedule
John Bateman - WS06/07
   
1

23.10.2006

Orientation:
Goals of the course: what are multimodal documents and why should we analyse them?

2

30.10.2006

What are the parts of a multimodal document?

Task:

  • Look for examples of interesting 'multimodal documents' and bring them along for analysis
  • Pages that were discussed in class and for next time (pdf).
  • A checklist of potential parts (the base units) of a multimodal page (pdf).

Reading to prepare for this session:

Len Unsworth (2001) Teaching multiliteracies across the curriculum: changing contexts of text and image in classroom practice. Open University Press. Chapter 3: "Describing visual literacies", pp71-112.

Anthony Baldry & Paul J. Thibault (2006) Multimodal transcription and Text Analysis. London: Equinox. Extract: §1.2 "Cluster analysis and the transcription of static multimodal texts". pp21-34.
3

6.11.2006

Genre and multimodality:
how can we apply the linguistic notion of genre usefully to the multimodal case?

Task:

  • Look for examples of interesting 'multimodal documents' that can be grouped into distinctive types, or genres.

Reading to prepare for this session:

John Swales (1990) Genre analysis. Cambridge University Press. Chapter 3: "The concept of genre", pp33-67.

Robert Waller (1999) 'Making connections: typography, layout and language'. In: Proceedings of AAAI Fall Symposium on Using Layout for the Generation, Understanding, or Retrieval of Documents.AAAI Press.
4

13.11.2006

The historical development of multimodal genres:
What influences? what effects?

Tasks:

  • Look for examples of multimodal genres as we have defined them that have changed over time.
  • How have they changed?
  • Why have they changed?

Reading to prepare for this session:

Patrick Allen, John A. Bateman & Judy Delin (1999) 'Genre and layout in multimodal documents: towards an empirical account'. In: Proceedings of the AAAI Fall Symposium on Using Layout for the Generation, Understanding, or Retrieval of Documents, Richard Power & Donia Scott (eds.), pp27--34. AAAI Press (pdf)
5

20.11.2006

The rhetorical analysis of multimodal documents:
How to do it? Why do we need to?

Reading to prepare for this session:

William Mann & Sandra Thompson (1988) Rhetorical Structure Theory: towards a functional theory of text organization. Text 8(3):243-281. (online page scans)

Elisabeth André (1994) Ein Plan-basierter Ansatz zur Generierung multimedialer Präsentationen. PhD thesis. Universität des Saarlandes. Chapter 3: "Die Struktur Multimedialer Präsentationen". (pdf file: 4Mb)

and/or

Elisabeth André & Thomas Rist (1993) The design of illustrated documents as a planning task. In: Mark Maybury (ed.) Intelligent Multimedia Interfaces. MIT Press, pp94-116.
6

27.11.2006

The rhetorical analysis of multimodal documents:
The text-image relation

Reading to prepare for this session:

Rolland Barthes (1977) Image - Music - Text. London: Fontana. Chapter: "Rhetoric of the Image", pp32-51. The picture referred to in this text can be seen here.

Radan Martinec & Anthony Salway (2005) A system for image-text relations in new (and old) media, Visual Communication 4(3):337--371 (online)
7

4.12.2006

Analysis and Critique: applying the tools

Example of critique:

Judy Delin & John Bateman (2002)'Describing and critiquing multimodal documents'. Document Design Journal 3(2):140--155. (pdf)

Tasks:

  • Select some texts/documents for analysis
  • Can they be analysed for their layout?
  • Can they be analysed for their rhetorical intent?
  • How well do the two match?
8

11.12.2006

Composition and Multimodality

Reading to prepare for this session:

Guenther Kress & Theo van Leeuwen (1996) Reading Images: a grammar of visual design. Chapter 6: "The meaning of composition". pp181-229.
9

18.12.2006

New Media and Variations

Reading to prepare for this session:

John Bateman, Judy Delin & Renate Henschel (2006) 'Mapping the multimodal genres of traditional and electronic newspapers' In: Royce, T. & W. Bowcher. (eds.) New Directions in the Analysis of Multimodal Discourse. New Jersey: Lawrence Erlbaum Associates. (pdf file)

M. Shepherd and C. Watters (2004) The evolution of cybergenres, In: Proceedings of the 31st Annual Hawaii International Conference on System Sciences, Volume 2, pp97-109. Los Alamitos: IEEE Computer Society Press. (online)

Materials used in class

News: online and off (pdf file;1.6Mb)

 
Christmas break
  
10

8.1.2007

Application Areas and Determination of Projects

 
11

15.1.2007

Multimodal classrooms

Reading to prepare for this session:

Guenther Kress, Carey Jewitt, Jon Ogborne & Charalampos Tsatsarelis (2001) Multimodal teaching and learning. London: Continuum. Chapter 1: "Rhetorics of the science classroom: a multimodal approach", pp10-41.

Carey, Jewitt (2002) The move from page to screen: the multimodal reshaping of school English. Visual Communication 1(1): 171-195. (online)

Presentation of analyses, projects, difficulties and results
12

22.1.2007

Multimodal meanings

Reading to prepare for this session:

Jay Lemke (1998) 'Multiplying meaning'. In: J. Martin & R. Veel (eds.) Reading science: critical and functional perspectives on discourses of science. London: Routledge. Chapter 5, pp87-113. (online version)
13

29.1.2007

Empirical validation

Reading to prepare for this session:

Jana Holsanova, Henrik Rahm & Kenneth Holmqvist (2006) Entry points and reading paths on newspaper spreads: comparing a semiotic analysis with eye-tracking
measurements. Visual Communication 5(1):65--93. (online)

John A. Bateman, Judy L. Delin & Renate Henschel (2004) Multimodality and empiricism: preparing for a corpus-based approach to the study of multimodal meaning-making. In: Eija Ventola, Cassily Charles & Martin Kaltenbacher (eds.)
Perspectives on Multimodality, Amsterdam: Benjamins}, pp65--87. (pdf file)
14

5.2.2007

Review