Multimodal Literacy Development in English Studies Contexts - linguistic and pedagogical perspectives on multimodal meaning-making and knowledge-building in classrooms and museum exhibitions

Abstract

This thesis reports on research whose aim is to explore the possibilities of multimodal literacy development in higher education L2 contexts. Our meaning-making practices have always relied on a variety of modes such as writing and image on the page, moving image and sound on the screen, and speech, gesture, gaze and posture in embodied interaction. However, language has a special role among semiotic modes: it is a complex semiotic system interlinked with human cognition and knowledge-building (e.g., Halliday, 1978). In recent SLA research, the Douglas Fir Group (2016) introduced significant themes which build on the ideas introduced above. One of the themes concerning the latest developments in transdisciplinary SLA studies states that “Language Learning Is Semiotic Learning” (p. 27). Moreover, theme number four further reinforces the idea of multimodality as one of the most relevant approaches in SLA, i.e., “Language Learning Is Multimodal, Embodied, and Mediated” (p. 29). With these ideas in mind, we can observe that a variety of meaning-making resources have become easily accessible in our everyday and academic lives recently, after the centuries-long dominance of written text in education and communication.

The thesis specifically addresses, from the perspectives of sociocultural theories of learning languages (e.g., Lantolf, 2000, 2001), social semiotics (Halliday, 1978), multimodality (e.g., Kress and van Leeuwen, 1996) and pedagogies informed by Systemic Functional Linguistics (Martin and Rose, 2008; Rose and Martin, 2012), the issue of integrating scaffolding pedagogical approaches to L2 and multimodal pedagogy. It also introduces the Specialization and Semantics dimensions of Legitimation Code Theory (Maton, 2013, 2014) as analytical and pedagogical tools. It aims at introducing innovation and affecting change at the level of classroom discourse and course design. The title of the course designed to explore multimodal literacy and L2 development was Making Meaning with Visual Narratives. Chapters 1-6 introduce the research aims and overview the relevant literature in disciplinary areas in connection with this research. Chapters 7-11 present the research methodology and four empirical studies focusing on four different aspects of multimodal literacy development based on my classroom research. Chapter 12 summarizes the findings of the thesis and discusses future directions for research.