Multimodality

Rowsell, Jennifer. Working With Multimodality:  Rethinking Literacy in a Digital Age. Routledge, 2013, 166 pp. ISBN 978-0-415-67623-6.

Teachers are often caught between the 20th and 21st-century expectations of literacy instruction knowing that multimodality should be a part of their lesson plans, yet not sure where or how to begin.  In reading this short paperback, I found not only answers to many of my questions but practical classroom suggestions that I could implement immediately.

Roswell uses an ethnographic lens to interview thirty successful professionals (whom she calls producers) who utilize the affordances of multimodality in their careers. These producers grant the reader the opportunity of a behind the scenes view of how multimodality works. The modes frame each chapter and detail how the producers use the modes when they are in the design phase of production.  The book ends each chapter with classroom takeaways that bridge the gap between heady research tomes and the hit or miss of classroom discovery.

As educators, we understand that literacy is much more than words on paper. We know that real-world literacy experiences (such as books, websites, films) combine many different elements for the author to communicate a message successfully.  To be prepared to read and write in the professional world, students need to have the skills necessary to create and understand the multimodal experiences in which they live. But, as Rowsell writes, “there remains a veil of secrecy around what experts in productions, design, and multimodality know and do and a discrepancy between that and the conventions that we teach students when they produce texts at school.”  Reading this book caused me to make curricular changes so that my students would be better prepared for a life of meaningful and productive work. Watching these producers combine modes in specific ways and leverage their affordances to achieve particular effects invigorated my teaching strategies.

What are modes and how do the work?

Rosewell prepares the reader by explaining that a mode is defined as a unit of expression and representation.  As Jewitt (2009) purposed, it is the outcome of the cultural shaping of material. It is socially and materially situated.  Halliday (1978) broadened the definition to extend “far beyond mere physicality, to encompass ephemeral, immaterial qualities that are materialized through physical features such as color, heft, light, angle, and gaze.”  Rosewell gives precise examples of how these modes can work together in different ways. Transmodal elements reach across the modes like the visuals and sound in films. Intermodal elements are when the modes can exist separately but can cross-reference each other. For example, in a book, the illustrations may work in sync with the font.  And lastly, there are intramodal elements that join together to make meaning. An example of this may be the choice of fabric and color that work together to create a particular fashion statement.

Who are the producers and why should we emulate them?

Roswell interviewed successful filmmakers, illustrators, web designers, journalists, actors, and architects, each of whom is a producer who designs. “Design physically manifests ideas, beliefs, and values through modes,” writes Rowsell, adding that  “we must examine the way producers evoke, combine and privilege one or more modes in a text if we are to determine how we can use a mode that best suits a composition when we produce texts.” The value of this learning lies not in just the modes themselves, but in how the modes are used and manipulated by the producers. Our desire as classroom teachers is not that our students become experts in textiles, but instead learn the practices that these modes offer these producers.

Film and Visuals

A key concept in film production is differentiating between representation and communication.  Gunther Kress claims that “representation focuses on my interest; communication focuses on the assumed interest of the recipient of the sign: (Kress, 2010). The three takeaways that I found interesting was the need to isolate the mood of films, the importance of perspective, and the deep sense of director agency in controlling the created message. “Framing stories through visuals and sound effects is a skill,” Rowsell states, that could “serve as a framework for educators to think about assigning moving-image projects on topics.”

In visuals, the producers spoke of perspective, color, and detail.  One consideration these producers offered was when visuals might be incorporated into the text and when sometimes the text is good enough.  Another critical idea they offered was that not everything could or should be depicted visually. Good readers and writers know that texture plays an integral part of good writing.  Bringing these modes to the front of the stage may enable literacy learners to appreciate the nuances of the selections they read or enable them to develop their writing more fully.

Sound

Sound can be a subjective experience as in the case of cultural experiences such as church or holiday music, and it can be objective when used to “create more general if not universal feelings of joy, suspense, or fear, as in the case of movie soundtracks.”  Literacy students would benefit from knowing what affordances sound has. One way of using sound in the classroom might be to analyze how different songs may achieve different moods. However, as Roswell states, the value of what we learn about sound can transcend the aural elements.  Both of the interviewed producers talked about the value of knowing how to combine information to convey meaning. What can be learned through an analysis of sound is directly applicable to writing, where student-created combinations might be “words that students use to create sentences, the evidence they offer in an argument, choices for syntax, etc.”

Interface

As an educator, this chapter was by far the biggest game changer for me.  At this point in the book I realized that if I wanted to prepare my students for their future professions, I needed to make changes in my attitudes and behavior in my classroom.  Rowsell points out that interface has moved beyond the mere function of how a user interacts with a given system to a consideration of aesthetics. Instead of spending hours wordsmithing, searching for the perfect word, we must look at interface to prepare our writers for their future professions. Where remixing privileges mixing and melding together of previously existing texts, discourses, and “stuff” (Gee, 1999), convergence privileges uniting technologies and functions, thereby gathering dispersed networks (Sheridan and Rowsell, 2010).  Creating digital media helps students become more critical readers, writers, and thinkers (Peppler and Kafai, 2007). Using and creating digital material are the skills my students need to know.

Rowsell continues with chapters each detailing modes of movement, textile, word, and space, granting the novice multimodality student a glimpse into the possibilities of a genuinely multimodal classroom.

In conclusion, Rowsell writes that “it is critical for students to develop the same brand of creativity and innovation that producers talk about and demonstrate – creative and innovative habits that are distinguished from the demonstrations of competence that characterize traditional monomodal, print-based writing practices because the professional and economic landscape of their future will demand that they be inventive and creative.”  If our students are to be prepared for their futures, we as teachers must demand that our classrooms offer more of an emphasis on designing and making than on the final product. And, we must come to believe that “words may be vital, but they are by no means the only competence of value and worth and words and language are usually coupled with other modes.”

Works Cited

Gee, J.P. (1999). An Introduction to Discourse Analysis:  Theory and Method. London: Routledge.

Halliday. M. (1978).  Language as a Social Semiotic.   London: Edward Arnold.

Jewitt, C. (ed.) (2009). The Routledge Handbook of Multimodal Analysis.  London:  Routledge.

Kress, G. (2003). Literacy in the New Media Age.  London: Routledge.

Sheridan, M. and Rowsell, J. (2010). Design Literacies.   London:  Routledge.