Kategorie-Archiv: Allgemein

Masterclass Djonov/Tseng | Transmedial Narratives | 11. Oktober, 16.15

Wir freuen uns, eine weitere Masterclass zum Thema Multimodalität ankündigen zu dürfen, zu der wir gleich zu Beginn des neuen Semesters alle interessierten Studierendem und KollegInnen einladen möchten:


Emilia Djoniv & Chiao-I Tseng

am 11. Oktober2018
um 16.15 Uhr
in GW2 A3570

A large body of research has established the need for systematic investigations into learners’ ever more pervasive engagement with new technologies and their competence in accessing, interpreting and authoring multimedia texts (cf. Buckingham 2003, Finch & Arrow 2017; Marsh et al. 2017). As a step towards addressing this need, we consider the question of how engagement with different media can be effectively harnessed for teaching multimodal literacy. Ourpresentationopens with a quick overview of definitions of and approaches to multimodal literacy. Inspired by studies that recognisethe value of studying ‘transmedialnarratives’ for both multimodality (e.g. Tseng & Bateman 2018) and critical multiliteracies education (e.g. Unsworth 2014), we then examinethe potential of engaging with adaptations of the same narrative in different media to build on young children’s awareness of the affordances of different media and the modes they support. Specifically, we integrate:

  • selected results of a study in which 16 dyads of university-educated mothers and their 4-5-year-old children, each read the narrative The Fantastic Flying Books of Mr. Morris Lessmoreas an interactive picture book app (MoonbotStudios 2011) and as a traditional-format picture book (Joyce 2012), and
  • the systematic, comparative multimodal analysis of action and event development (based on methods presented in Tseng 2013, 2018) in three versions of that narrative: the award-winning animated short film (Joyce 2011), the book app and traditional-format book.

We argue that comparing basic aspects of the materiality (e.g. presence or absence of interactive hotspots) and semantics (e.g. constructions of events; depictions of and interactions between characters) of adaptations of the same narrative in different media is a method that can help young children enhance their existing multimodal awareness, and support teachers in both evaluating transmedia narratives and later adopting and introducing, in age-appropriate ways, a metalanguage for discussing more complex semantic systems (e.g. ‘focalisation’– see Unsworth, 2014).

Buckingham, D. (2003).Media Education: Literacy, Learning and Contemporary Culture. Cambridge, UK/ Malden, MA, USA: Polity Press.
Finch, B., & Arrow, A. W. (2017). Digital technologies in the literate lives of young children. In C. J. McLachlan & A. W. Arrow (Eds.), Literacy in the Early Years: Reflections on International Research and Practice(pp. 221-238). Singapore: Springer Singapore.
Joyce, W. (Producer), & Joyce, W., & Oldenburg, B. (Directors). (2011). The Fantastic Flying Books of Mr Morris Lessmore[Motion Picture]. USA: Moonbot Studios.
Joyce, W. (2012). The Fantastic Flying Books of Mr. Morris Lessmore (Illustrations by William Joyce and Joe Bluhm). New York: Moonbot Books.
Marsh, J., Hannon, P., Lewis, M., & Ritchie, L. (2017). Young children’s initiation into family literacy practices in the digital age. Journal of Early Childhood Research, 15(1), 47-60. doi:doi:10.1177/1476718X15582095
Moonbot Studios (2011). The Fantastic Flying Books of Mr Morris Lessmore(Version 1.4). [Mobile Application Software].
Tseng, C.-I. (2013). Analysing characters‘ interactions in filmic text: a functional semiotic approach. Social Semiotics, 23(5), 587-605. doi:10.1080/10350330.2012.752158
Tseng, C.-I. (2018). Unravelling the myth of multiple endings and the narrative labyrinth in Mr. Nobody(2010). In S. Zhao, E. Djonov, A. Björkvall & M. Boeriis (Eds.), Advancing multimodal and critical discourse studies: Interdisciplinary research inspired by Theo Van Leeuwen’s social semiotics(pp. 131-145). London/New York: Routledge.
Tseng, C., & Bateman, J. A. (2018). Cohesion in comics and graphic novels: an empirical comparative approach to transmedia adaptation in City of Glass. Adaptation.
Unsworth, L. (2014). Point of view in picture books and animated film adaptations: Informing critical multimodal comprehension and composition pedagogy. In E. Djonov & S. Zhao (Eds.), Critical Multimodal Studies of Popular Discourse(pp. 202-216). London/New York: Routledge.

Masterclass Carman NG | Thursday 28 June 2018 | 14.15

Bremen Masterclass on Multimodality
Carman Ng, Bremen
Videogame Criticism: A Socio-Semiotic Approach
Thursday, 28 June 2018
GW2 A3570

This presentation illustrates a socio-semiotic approach to videogame criticism, in attempt to examine underexplored dynamics between two connected paradigms in game studies via multimodality. A focus on microlevel analyses of digital game forms and content over macro-level accounts of networked production and policies remains (Kerr 2017). While game scholarship has assembled methods of critique focused on aesthetics, culture, and design (Jagoda 2017), multimodal analyses situate on the fringes. To discuss dialectics between the two facets and highlight values of multimodality-enriched game criticism, this presentation combines a gameplay analysis framework and annotation for a case study of a genre-setting war-themed Japanese stealth videogame series Metal Gear Solid (MGS, Kojima 1998 -), which is internationally recognized for its anti-nuclear rhetoric, self-reflexive gameplay, and narrative complexity.

Analyses are three-fold, focused on empirical intervention into socio-semiotic multimodality (Bateman, Wildfeuer, and Hippala 2017; Kress 2010) for fine-grained, multi-layered, analyses of game designs andsemiotic resources; and organized around the series narrative concluding MGS4: Guns of Patriots (2008). First, I apply multimodal annotations to elaborate how selected gameplay sequences connect interactivity and narrativity to create immersive gameplay experiences; and motivate anti-war discourse semantics on both installment and series dimensions. Second, I contrast changes in interface designs and their gameplay effects (MGS3 and MGS4) to address gamic influences on the use, design, and social implications of digital media (Ash 2015; Galloway 2012). Third, in discussing the MGS series’ exemplary critique of virtual war play (MGS2 and MGSV), I detail how videogames systematically construe ethical motifs and rhetoric via gameplay mechanics (Bogost 2007; Sicart 2013).

The presentation examines this through the series’ portrayal of human-technology dynamics within a speculative, networked war economy, exploring its post-WWII sentiments, reduced anti-Americanism, and transnational socio-political influences on videogames as a creative cultural industry. Ultimately, this case study emphasizes considering both materiality and political economy of videogames for comprehensive criticism; that a multimodality-driven methodology justly investigates meaning-making in videogames, informing further inquiries in areas such as popular culture, transmediality and intermediality research, and social impact game design.


Masterclass Prof Kay O’Halloran | Wednesday 16 May 2018 | 12.15

Prof Kay O’Halloran, Curtin University, Australia
„Words and Images of Contradition:
The Art of Being a Space Invader“
Wednesday, 16 May 2018, 12.15-1.45 pm
GW2 A3570

The proliferation of media channels enables individuals and political sub-communities to manage and control the creation and dissemination of information, including advertisements that are personalized according to user profiles and false news stories which have been found to spread faster and more widely than true news stories in platforms such as Twitter. In this presentation, I consider the semiotic mechanisms through which contradiction flourishes in these media channels.
I first analyse Donald Trump’s domination of media coverage during the US presidential campaign. In presenting himself and his agenda, he made himself the focus of attention through strategies that invaded various semiotic spaces of others. The effects of this were that what he did and what he said captured the majority of media attention at the expense of his opponents which, aided by other campaign strategies, enabled him to win the election.
Following this, I present a multimodal approach to big data analytics aimed at tracking patterns of recontextualization across media. The approach is illustrated with examples from online extremist communications. The aim of the discussion is to highlight key questions of how meanings arising from the integration of language, images and videos can undercut or repoint fragments of discourses to promote more loaded, extremist and often contradictory formulations.