Assessing Multimodal Texts

What key theoretical insights about multimodality are required to effectively assess multimodal texts?

When considering the assessment of multimodal texts one needs to come back to the basics of literacy and considering the purpose, form and audience. Then layered on top of this is the multimodal aspects that have been chosen to add depth of understanding to the piece of work, the semiotics:

  1. Linguistic: comprising aspects such as vocabulary, generic structure and the grammar of oral and written language
  2. Visual: comprising aspects such as colour, vectors and viewpoint in still and moving images
  3. Audio: comprising aspects such as volume, pitch and rhythm of music and sound effects
  4. Gestural: comprising aspects such as movement, speed and stillness in facial expression and body language
  5. Spatial: comprising aspects such as proximity, direction, position of layout and organisation of objects in space.,31522.html?issueID=12141

Finally, it is the overall meaning that is gained and whether or not each semiotic component supports and adds to the purpose and meaning. As has been pointed out by Thomas (2008, 8) in relation to inter-semiotic meanings “Modal complexity occurs when the same narrative meaning is replicated across all modes simultaneously.” Kress and van Leeuwen (2006) also express the importance of the importance of meaning being supported by all aspects of the visual content. Linking it to the similarities that exist in music, it is the composition of a piece of work that is vital to the message delivered. Poor composition, that is semiotic components that are disconnected, provide a meaning that appears confused or lacks the desired power and intensity.

In summary, when assessing a piece of work that is multimodal one must unpack all the layers (semiotic components) and determine whether or not they are supporting the meaning of the work. This requires more than a quick look, it requires a well created measuring tool, and in some cases this might be different for each piece of work assessed. Also a  sound understanding of the interplay between semiotics to create the story/meaning is required as they cannot be assessed in isolation.


Kress, G., & van Leeuwen, T. (2006). Reading images: the grammar of visual design.New York: Routledge.

Thomas, A. (2008). Machinima: Composing 3D Multimedia Narratives. In: Unsworth, L. [Ed]. New Literacies and the English Curriculum: Multimodal Perspectives. London, Continuum.,31522.html?issueID=12141

Posted in Uncategorized | Leave a comment

Creating Multimodal texts

How easily can you (and your students) create multimodal texts?

My simple response to this is, it is not hard. In fact it is part of using social media. The hardest thing is creating good multimodal texts. The one thing that I have observed through creating multimodal texts, whether I am making them or students are, is that they become very focussed on their audience and how they will receive it. In fact, through the creation of them there appears to be greater involvement of others in this part of the process and the consequent drafting and refining. To the point where refining continues after the product is “published” bringing about the question of when is something finished. In considering this question I came across Learning with ‘e’s blog post on Everyone’s a Critic. One particular paragraph stood out to me:

Assessment should no longer be confined to the written tests or essays that were so prevalent in the last century, but might be extended to podcasts, blogs, wikis, videos, image collections and combinations of these in other media. What teachers now need to avoid is replicating old practices within new media. The opportunities to create new ways of assessment are there to be exploited. The only real limitation is imagination. (Wheeler, 2012)

When I dig a bit deeper into my thoughts I actually find creating my own multimodal text extremely challenging, even frightening. I don’t view myself as creative and certainly not one for writing stories that are engaging. Further to this, I have a very high expectations of myself when it comes to the end product – I want it to look professional not clunky and this is due to the audience that can view it. Putting this aside I have been creating multimodal texts for sometime. But do they go beyond written text and static pictures? Are they of high quality and do I want an unfiltered audience to view them?

The animoto presentation I created for this unit is a example of multimodal as it incorporates visual, linguistics, audio and spacial aspects. I have also used the Art Maker App from ABC to create stories with students, they love this as it immediately taps into roleplaying to create their story. This App allows you to use, linguistics, spacial, visual and audio. I have dabbled with Story Kit, Toon Doo, Creative Book Builder, blurb, the concept of machinima and a few others.
With the above question in mind I found the following post by Boyd (2012) interesting –

Knowing that texts in a wider variety of forms, other than the written word, have been in existence for more than a hundred years, one does need to question why are we now only just linking them into the concept of literacy….

Obviously, the prevalence of multimodal texts in the classroom is becoming more wide spread due to the nature of technology providing easy avenues for their creation and use. What impact should this then have on education and what is happening in the classroom…

I am not sure whether this question was meant to raise more questions, but it has for me.


Boyd, B. (2012)

Wheeler, S. (2012)

Posted in Uncategorized | Leave a comment

Viewing Multimodal Texts

“What makes a multimodal story successful?

When creating a mulitmodal story greater consideration needs to be taken as you immediately immerse the reader into a world where words do not exist on their own (Bill Cope  ) The meaning is interpreted  through the use of at least two semiotic systems:

There are five semiotic systems in total:

  1. Linguistic: comprising aspects such as vocabulary, generic structure and the grammar of oral and written language
  2. Visual: comprising aspects such as colour, vectors and viewpoint in still and moving images
  3. Audio: comprising aspects such as volume, pitch and rhythm of music and sound effects
  4. Gestural: comprising aspects such as movement, speed and stillness in facial expression and body language
  5. Spatial: comprising aspects such as proximity, direction, position of layout and organisation of objects in space.

Examples of multimodal texts are:

  • a picture book, in which the textual and visual elements are arranged on individual pages that contribute to an overall set of bound pages
  • a webpage, in which elements such as sound effects, oral language, written language, music and still or moving images are combined
  • a live ballet performance, in which gesture, music, and space are the main elements.

Multimodal texts can be delivered via different media or technologies. They may be live, paper, or digital electronic.,31522.html?issueID=12141

Woven into this is the multiplicity of the information or story, fitting with or to a particular group in society. When putting this into perspective a multimodal story has many layers that need to support each other to create a fluid story. I wonder whether this makes creating a story using a multimodal format more challenging? Does it change the way we draft and edit to create a final product? Does this engage a greater level of literacy? Or is it engaging the new literacies, rather than a greater level of literacy? It certainly builds in a new level of critique and one that changes as technologies become more refined. I gave Inanimate Alice to a grade 9 student to look at. This student is very familiar with creating and making meaning through ICT tools. He felt that there were areas that could be built on to make the story more fluid with regard to its interactivity hence enabling greater immersion into the story. This is something that is determined by two factors, the knowledge of the user regarding the technology used and the level of the technology used (keeping in mind that technology is continually changing, growing and refining). The student could see the relevance of Inanimate Alice for a primary audience with regard to themes and the story presentation. He also felt that it was a good entry point for teachers with little background in gaming and more complicated technologies. Further to this it has inspired him to contemplate his own multimodal story – Inanimate Alice is achieving its purpose on many levels.

As a result of sharing this with the student, he immediately sent me which needs to be viewed in google chrome. The possibilities are growing…

Posted in Uncategorized | Leave a comment

Learning in a Virtual World

What are some of the affordances of a virtual world for education?

Having had the opportunity to be introduced to Second Life (SL) a number of years ago by a colleague I consider myself one of the lucky ones as I have been able to play, dabble and explore SL for a range of reasons. These reasons have been just for fun, as part of a conference attendee and presenter (this required branching off to a Minecraft server), as a learner/student. Each role has provided me with a different level of understanding that SL can take from an educational perspective.

The thing that I have found most powerful in each situation is the immersion into a world that provides a realistic feel. On two separate occasions I have entered into Virtual McBeath, as part of Professional Learning as a teacher and as a student as part of this course, on each occasion I have become more aware of the life of McBeath, the inner workings of his mind and the external influences of his life. I am in no way a scholar of Shakespeare, but from these moments of immersion I have learnt more than I have via other more traditional forms. Maybe it is the combination of the visual imagery and audio combined, maybe it is the interpretation that has been provided by its creator or just maybe it is the fact that I could run off and explore what I was ready for? It provided me the opportunity to play, which possibly was slightly frustrating to the one leading the tour, but play and intern investigation is so important to learning especially when it is packaged in a way that allows a feeling of reality for the senses all at once. That is through the use of an avatar I am actually part of the situation being explored, I need to develop a better understanding of the place to be able to participate effectively.

Another area of SL I have entered into on a number of occasions is that of a Story Quest which allowed you to look into the life of a man impacted by HIV. This was a powerful expedition as it allowed you to sit in his house and contemplate all aspects of his life – almost like stepping into his shoes. It left me feeling privileged as it felt more personal than reading someones edited diary entries that may form a biography later on.

So what does this provide for education…. interesting question. The value of the learning that can be provided through this medium is unquestionable. It is more about whether or not we are ready to employ it. Opening up connections to such media is at times difficult and requires an appreciation beyond educators to those that are able to provide access. I currently have a student who has been developing an area on Jokaydia Grid however, as it is for just one student, the access cannot be opened up at a school level at this time. So it is now about seeking out partnerships that will open up opportunities.

Posted in Uncategorized | Leave a comment

What are the so called “new literacies”?

This question led me to think about what is literacy? Immediate thoughts sprung to mind about reading and writing and the underlying principles that enable one to be efficient at these within our society. And that is the key, within our society or the context in which they are used. Lankshear and Knobel (2011) discuss literacy and its connection to social and cultural contexts. When I think about travel and being immersed in another culture that relies on a language that is vastly different from my own I start to dig deep into the literacy skills that I have to be able to make sense of the environment. I begin to place high importance on decoding and interpretation of information to be able to participate. Being immersed in a language that is germanic in its roots is much easier for me to connect with rather than one that is not…. so how does this connect to literacy and new media and more specifically new literacies?

When looking at the Australian Curriculum it states that literacy in effect develops “students’ ability to interpret and create texts with appropriateness, accuracy, confidence, fluency and efficacy for learning in and out of school, and for participating in Australian life more generally. ”

Participation is central to new literacies. There is now a stage for all to perform on, contribute to and critique if one wishes. Web 2.0 provides a platform for its users to be participatory in every aspect of its usage. For the first time anyone can put on the cap of an author to an unknown or known audience using a range of tools. Understanding the tools, their purpose and intent, used is vital to ensure that that the message is understood, not to mention the audience and the general environment or culture that it sits within.

Participating within this environment means that you are not anonymous as you can be in the “real” world. Everything you do has your handle, nickname, or other form of identity connected to it. It soon becomes evident what your purpose is and the role you choose to hold within the space you chose to participate in. This awareness is directly connected to being literate here:

WoW from aram bartholl on Vimeo.

My thoughts tend toward new literacies being that of participatory in a digital world that has few boundaries (filters on who is accessing your work), unless you create them yourself. New literacies place greater importance on “old” literacies as the very nature of the instant and live platform that the internet provides consequently requires greater literacy efficiency. This includes the ability to transfer and use literacy skills across the many different platforms be it blogging, participating in creating zines, virtual worlds, gaming, twitter, facebook and youtube to name a few. Being more savvy is a requirement regarding text type, understanding purpose, form and audience along with knowing the protocols that exist. New literacies are directly connected with the creation of your web presence or identity.

In furthering my search and understanding of new literacies I stumbled across Howard Rheingold who points out that new literacies are:



Network know how

Critical consumption




Henry Jenkins et al, Confronting the Challenges of a Participatory Culture: Media Education for the 21st Century. , pp.3-23.

Colin Lankshear and Michele Knobel, New Literacies: Everyday Practices & Social Learning. Third Edition(Maidenhead: Open University Press, 2011). Part One: “New Literacies: Concepts and Theories”, pp.1-92 viewed 1/4/12  viewed 4/4/12

Posted in Uncategorized | Leave a comment

Growing up digital

Key question: What does it mean for schools and educators to have students in their classrooms who have grown up “digitally”

Watching the video, Digital Media – Learners of the 21st Century, was interesting as there was one rather powerful statement that stood out for me. It indicated that the use of technological tools – whether they be computers, devices, programs etc – should be seamless, non intrusive, they should just be. This is something that I am finding with my minecraft kids. They constantly use a range of technologies to enable their learning. This learning takes on a collaborative form that relies on peer based learning.

From a schooling perspective what does this mean? As with any large organisation the uptake of change is slow and consequently can seem out of touch with where individuals are. It is interesting to listen to John Seely Brown who talks about learning in a digital age. He talks in depth about the need for tinkering and its power to enable today’s students to embrace change. He also discusses peer based learning, the need for social learning and the importance of play, the notion that it is okay to keep trying to bring about the perfecting of a skill, concept etc… When I consider the notion of social learning it sparks the question in me as to whether we as teachers are providing opportunities for this to occur. Do some of our imposed school policies reduce the potential of social learning at the moment due to our fear of the unknown? I feel that sometimes we are in lock down mode within the school system, blocking anything that could be potentially harmful.

Lyman, Thorne and Carter (2009) discuss the concepts of “hanging out, messing around, and geeking out” This morning I was fortunate to see this start to occur with some students who were participating at the VWBPE conference. To see a student who is 14 years old compliment a student who is 11 years old in relation to her skills and knowledge, with regard to minecraft, highlights the importance of going beyond the rigid notion of the classroom and allowing it to become a place where students are able to participate in peer based learning. The 14 year old has a lot to offer the younger student but equally the 11 year old has much to offer back.

Our practice as teachers in the classroom with students who are digitally connected does need to evolve to a position that is more focused on being a facilitator, mentor and role model of and for learning. We need to start to allow ourselves the freedom to experience the immersion into the digital tools/new media literacies that engage our youth. Not from a superficial perspective, but from one that allows us to participate and understand the needs and issues faced in these environments. This leads to grasping the concept of identity, ethics and responsibilities. There are incredible opportunities for young people to have a voice using social media, be creative using a range of digital tools along with developing connections with those that share their interests, there are also perils. Hence the importance for those in an educative role to support the learning required to navigate new media and in the process provide real moments for students to learn the importance of being ethical, responsible and developing more than just an awareness of how this creates their identity on line. We need to allow them to be in an online environment. Accepting that a young persons identity exists in both the online and physical world is required to enable young people to participate in learning in today’s society. A young person determines their identity in both and as the use of technology is ubiquitous so is the way they move between their identities.

The beauty of new media literacies is that they offer an authentic learning space, so the learning that goes on is real; it has an audience and as a result has incredible power for engagement.

This video is an example of social learning. The student had not attempted videoing before and consequently developed his skills and understandings to produce it using a peer-learning network that he collaborated with and of course an audience that gave feedback. The video was created by one student and is their personal work; it is not the work of those that he engaged with to build his skills and knowledge. This video provides and example of how this student is developing his identity in the online environment. The unfinished product was placed on YouTube for peer review. During this process other students learnt powerful lessons about their own online identity through they way they phrased up feedback.

We have so much to learn from our students in relation to their learning. Maybe we need to participate in social learning and peer based learning with them.

James, C., Davies. K., Flores, A., Francis, J., Pentingill, L., Rundle, M. and Gardner, H. Young people, ethics, and the new digital media. pp.1-62. Available:

Lyman, P., Ito, M., Thorne, B. and Carter, M. (2009). Hanging out, messing around, and geeking out: kids living and learning with new media. Cambridge: MIT Press/MacArthur Foundation.

Palfrey, J. and Gasser, U. (2008). Born digital. Philadelphia: Basic Books.

Posted in Uncategorized | Leave a comment