More than Just Remixing: Uptake and New Media Composition

This article turns to genre theory's recent explorations of uptake, broadly defined as the ways genres interact, as a resource for sketching a pedagogy of shuttling between genres. Using uptake, I intend to reconceptualize multimodal compositions as a means of participating in rhetorical ecologies that consist of transactions between genres instead of thinking of remixes as an end in themselves. In this article, I first define the concept of uptake in detail and discuss its use in rhetorical genre studies. After further illustrating uptake through an analysis of transactions between YouTube parodies and the 2005 German language film Downfall, I discuss existing scholarship in multimodal composition that draws on genre but not the idea of uptake in order to lay a foundation for a pedagogy that highlights the links, feedbacks, and rules that coordinate genres. My aim in the last section is to sketch possibilities for how teachers and students can deploy the concept of uptake as a rhetorical tool to strengthen their awareness of genre and multimodality. In doing this, I hope to reposition multimodal projects as beginnings or midpoints that lead to students’ emersion into public discourse rather than culminations or end goals in themselves. Integrating studies of uptake into writing curricula in this way will help students to make sophisticated rhetorical decisions in the age of media convergence.     

Sound engineering: Toward a theory of multimodal soundness

The activity-based multimodal theory of composing presented here offers us a way to better understand how, when, and why students might choose to explore the affordances of sound (oral, aural) in their work. Importantly, however, insofar as it resists attempts to bracket off individual senses and the uptake of select semiotic resources, it presents a more robust, integrated approach to theorizing, researching, and teaching multimodal production, one that facilitates the move toward multimodal soundness. An activity-based multimodal approach to composing provides us, but perhaps more importantly still provides students, with strategies for attending to the complex ways that a greater variety of senses, semiotic resources, and rhetorical positionings might be taken up and brought together, if only briefly, and if only in sound-for-now ways, to help them accomplish specific kinds of work in specific contexts. To illustrate how an activity-based multimodal theory of composing achieves these ends, I present two accounts of first-year composition students who explored sound's potential in their work after determining that the uptake of sound could help them, at least in part, accomplish the work they hoped to do.     

Tuning the Sonic Playing Field: Teaching Ways of Knowing Sound in First Year Writing

In recent years there has been a call within composition to include sound, among other modes, such as word and image in writing. Some of this call relies on a movement to multimodal composition in order to capture both the richness of rhetorical possibility and the reality of communities of practice, and some is in response to a perceived shift in writing due to digital media tools and environments. Regardless of the impetus for including the auditory realm in the composition classroom, it is important for the field of composition and rhetoric to develop further pedagogies of sound so that students are not simply offered the opportunity to produce diverse texts, but instead, are invited to enter “the playing field.” In order to do this I first explore an approach to teaching auditory rhetoric based on ways of knowing sound from an acoustics and musicology perspective, then I consider a phenomenological approach based on listening, and finally I construct a model of “tuning the sonic playing field” that draws on the literal, material practice of tuning as a metaphor for how sound may be taught in composition. The “tuning” approach to teaching sound draws on attention, embodiment, listening, and negotiation. Rather than simply offering students opportunities to use sound in rhetorically sensitive ways, this final approach asks instructors to become “attuned” to how different auditory epistemologies influence students’ ability to design and compose in sound.     

The Rhetorical Work of Multimedia Production Practices: It's More than Just Technical Skill

Calls for multimodal communication are being heard with increasing frequency in composition and professional communication. Oftentimes, however, teaching multimedia production is viewed by those outside of the field as simply a matter of imparting technical skill rather than facilitating development of diverse and significant literacies. A closer look at these practices reveals how the complex choices made during production regarding audience, content, technology, and media can dramatically affect the final text and its reception by users. Rather than viewing multimodal production work as just technical skill, I argue that it requires careful attention to both traditional and technological rhetorical considerations. Better understanding these varied rhetorical practices specific to new media supports us in helping students to appreciate the constraints and possibilities of composing and offers support for the value of our work with multimedia to colleagues in other areas of English studies.     

Composition in the Dromosphere

Speed is one way to distinguish how we approach rhetorical situations within and outside of the academy. Texts and arguments move quickly in any rhetorical situation, and rhetoric and composition has begun to theorize ways to make sense of these speedy situations. Drawing on Paul Virilio's discussion of the dromosphere, our contemporary environment of speed, this essay continues that work by examining the work of two DJs: DJ Spooky and Girl Talk. The compositional methods of DJs have received a great deal of attention in computers and composition scholarship, and much of that work has focused on how DJs can help us rethink intellectual property. This essay examines DJs for a different purpose, in order to examine two approaches to the difficulties of speed: scholê and dromos. I discuss DJ Spooky's remix of the film The Birth of a Nation, and I apply Girl Talk's methods to provide my own mashup of that same film. This essay argues that the mashup, in all of its various forms, is a particularly useful example of dromological writing, writing that is attuned to the problem of speed.     

“Growing Smarter Over Time”: An Emergence Model for Administrating a New Media Writing Studio

This article reports on efforts to create an administrative structure for learning and teaching multimodal composing that depends not on the leadership of a new media writing expert but on the collaboration of relative novices organized according to principles of emergent learning. Based on four years’ experience in a grant-funded program that supports new media composing in multiple disciplines, I report on the benefits of a bottom-up, emergent approach while raising questions about the long-term sustainability of such an approach. I conclude by describing efforts to create more top-down support for new media composing, while remaining committed to serving the teachers and students who are the real agents of change.     

Part 1: thinking out of the pro-verbal box

As the World Wide Web and other visual media gain prominence in students’ lives, we, as teachers of composition, have to re-evaluate our strict adherence to the verbal medium. If our classrooms focus on a single mode of representation—the verbal—then the concurrent implication is that only one voice deserves to be heard. In such a classroom, students will not be able to recognize that verbal forms and visual forms—or better yet their combination—carry an equal degree of complexity, representative richness, and rhetorical power. Basing composition almost exclusively on verbal instruction counters the very nature of literacy education, because our current verbal-based education system produces illiterates in our highly visual and multimodal modern society. Drawing on composition scholarship published in three major composition journals, this article demonstrates composition’s verbal bias, argues that this bias is both politically and rhetorically suspect, and calls for a composition pedagogy that integrates verbal instruction with visual instruction.     

Show, not tell: The value of new media scholarship

In this article, I consider the changing nature of publications in relation to technology and tenure, presenting a taxonomy of scholarly publications: online scholarship, scholarship about new media, and new media scholarship. I offer a focused definition of new media texts as ones that juxtapose semiotic modes in new and aesthetically pleasing ways and, in doing so, break away from print traditions so that written text is not the primary rhetorical means. By applying this definition to scholarly online publications, readers can be better prepared to recognize and interpret the meaning-making potential of aesthetic modes used in new media scholarly texts. I conclude by offering an analysis of a scholarly new media text, “Digital Multiliteracies.”     

Integrating Assessment and Instruction: Using Student-Generated Grading Criteria to Evaluate Multimodal Digital Projects

In this article, I explore how we can link assessment to instruction and the multimodal composing process by inviting students to generate the grading criteria for new media assignments, and I show how this approach influenced students’ composing and understanding of multimodal texts. I first detail the scaffolding processes I took to help the class learn to construct the evaluative criteria for a digital video project, describing the course curriculum, instructional approaches, and assignments. Then drawing from extended interviews with three learners, I present their perceptions on how the collaborative construction of grading standards affected their learning and comprehension of new media rhetoric. I close with pedagogical recommendations for instructors who teach multimodal digital writing and who seek to integrate the collaborative construction of grading criteria into their classroom.     

The opposite of control: a Deweyan perspective on intrinsic motivation in “After 3” technology programs

A central feature of the design of many “After 3” technology programs is the assumption that student learning and motivation requires that they have choice and control of their activity. Similarly, the dominant cognitive-rational perspective of motivation portrays effective learners as having control of themselves and their environment. In this article, we build on Dewey's (1934. Art as experience. New York: Perigree.) aesthetics and epistemology — as most fully developed in “Art as experience” — to suggest that to be deeply engaged in learning, to be truly moved, requires not only control, but also the “opposite of control”. In “Art as experience” Dewey proposed that aesthetic experience — compelling, transformative experience — requires doing (acting on the world), reflection (standing back from the world), and undergoing (being acted upon by the world). Furthermore, grasping the meaning of these experiences emerges through a qualitative sense in addition to intentional analysis and reflection. Thus, intrinsic motivation, or what we shall call transformative experience, finds a balance between control and its opposite. We elaborate our conception of the “opposite of control” and discuss how this idea helps us appreciate heretofore unilluminated qualities of intrinsic motivation in “After 3” technology programs.     

Multimodal Composition in a College ESL Class: New Tools, Traditional Norms

Research has noted that multimodal writing allows for better communication of knowledge and expression of personal identities through various modes of representation. Studies of multimodal composition have tended to examine separate modes, or have looked at connections between only a few selected available modes. Less attention has been devoted to multimodal composition from a holistic perspective. Drawing on the concept of synaesthetic semiosis [Kress, Gunther. (1998). Visual and verbal modes of representation in electronically mediated communication: The potentials of new forms of text. In Ilana Snyder (Ed.), Page to screen: Taking literacy into the electronic era (pp. 53-79). London: Routledge; Kress, Gunther, & Van Leeuwen, Theo. (2006). Reading images: The grammar of visual design (2nd ed.). New York: Routledge], the present study examines how ESL writers in a freshman composition class used available modes in multimodal argumentative essays posted on the World Wide Web. The findings indicate that word-dominated discourse was the primary factor in selecting available modes. Non-linguistic modes were primarily used to illustrate written essays. However, students also used non-linguistic modes to project cultural and national identities and to express emotional connections with their topics. The ways in which the students synthesized multiple meaning-making modes represented the social practices of learning multimodal genres in which they were engaged. The paper concludes with suggestions for pedagogy and research in multimodal composition.     

一种基于背景知识的本体注释方法

本体采用基于语法词汇的表述方式,使本体自身表示可能存在模糊性、错误理解等问题,部分本体的概念可以通过自身的上下文信息推测出其含义,但是有些本体根据已有信息不能清晰表达其概念的确切含义.针对这个问题,提出基于背景知识的本体注释方法,对本体本身进行注释和澄清.包括基于WordNet和Web搜索引擎的注释方法,利用WordNet查找本体概念的正确词义,利用Web搜索引擎搜索本体概念的snippets,分别将词义和snippets作为其属性注释到本体中.实验表明本体注释率达到99.12％,表明本文方法的是可行的,本体注释正确率达到80.76％,比同类方法更高.     

Context-driven encrypted multimedia traffic classification on mobile devices

The Internet has been experiencing immense growth in multimedia traffic from mobile devices. The increase in traffic presents many challenges to user-centric networks, network operators, and service providers. Foremost among these challenges is the inability of networks to determine the types of encrypted traffic and thus the level of network service the traffic needs to maintain an acceptable quality of experience. Therefore, end devices are a natural fit for performing traffic classification since end devices have more contextual information about device usage and traffic. This paper proposes a novel approach that classifies multimedia traffic types produced and consumed on mobile devices. The technique relies on a mobile device’s detection of its multimedia context characterized by its utilization of different media input/output (I/O) components, e.g., camera, microphone, and speaker. We develop an algorithm, MediaSense, which senses the states of multiple I/O components and identifies the specific multimedia context of a mobile device in real-time. We demonstrate that MediaSense classifies encrypted multimedia traffic in real-time as accurately as deep learning approaches and with even better generalizability.     

Iconographic Tracking: A Digital Research Method for Visual Rhetoric and Circulation Studies

Drawing on recent scholarship in the disciplines of rhetoric and composition/writing studies and communication, the author advocates for generating new methodologies and methods for studying rhetorical circulation. The author introduces iconographic tracking—a research method that employs traditional qualitative and inventive digital research strategies to investigate the circulation, transformation, and consequentiality of images across genres, mediums, and contexts. As evidence of what this method can afford, the author presents findings from a five-year long case study that employs iconographic tracking to trace Shepard Fairey's now iconic Obama Hope image. To help readers understand some of the theories and philosophies that undergird the method of iconographic tracking, the author also briefly introduces a new materialist approach to rhetorical study. As such, the author points in new directions for visual rhetorical study and circulation studies at large.     

Just For Fun: Writing and Literacy Learning as Forms of Play

This article focuses on Ink, a Multiplayer Online Game (MOG) being developed at Michigan State University. The design of Ink reflects the developers’ understanding of writing pedagogy and rhetorical theory. Ink allows players to enter into complex rhetorical situations that include exigencies, audiences, and rhetorical purposes. The developers of Ink hope that placing players in these rhetorical situations will facilitate literacy learning while simultaneously providing a satisfying game experience. Players will hopefully learn while having fun. In order to test the effectiveness of Ink as a game and learning environment, the authors designed a small-scale preliminary study with a focus group of student playtesters. The study was designed to answer three fundamental questions: Will players write? Will players have fun? Will players learn? The study generated some evidence that the answer to all three questions is “yes.”     

Choosing Rhetorical Structures To Plan Instructional Texts

This paper discusses a fundamental problem in natural language generation: how to organize the content of a text in a coherent and natural way. In this research, we set out to determine the semantic content and the rhetorical structure of texts and to develop heuristics to perform this process automatically within a text generation framework. The study was performed on a specific language and textual genre: French instructional texts. From a corpus analysis of these texts, we determined nine senses typically communicated in instructional texts and seven rhetorical relations used to present these senses. From this analysis, we then developed a set of presentation heuristics that determine how the senses to be communicated should be organized rhetorically in order to create a coherent and natural text. The heuristics are based on five types of constraints: conceptual, semantic, rhetorical, pragmatic, and intentional constraints. To verify the heuristics, we developed the spin natural language generation system, which performs all steps of text generation but focuses on the determination of the content and the rhetorical structure of the text.     

The aesthetic turn in sonification towards a social and cultural medium

The public release of datasets on the internet by government agencies, environmental scientists, political groups and many other organizations has fostered a social practice of data visualization. The audiences have expectations of production values commensurate with their daily experience of professional visual media. At the same time, access to this data has allowed visual designers and artists to apply their skills to what was previously a field dominated by scientists and engineers. The ‘aesthetic turn’ in data visualization has sparked debates between the new wave and older more scientifically grounded schools of thought on the topic. Sonification is not as well known or commonly practiced as visualization. But sound is a naturally affective, aesthetic and cultural medium. The extension of the aesthetic turn to sonification could transform this field from a scientific curiosity and engineering instrument into a popular mass medium. This paper proposes that a design approach can facilitate an aesthetic turn in sonification that integrates aesthetics and functionality by dissolving divisions between scientific and artistic methods. The first section applies the design perspective to the definition of sonification by replacing the linguistic concept of representation with non-verbal concept of functionality. The next section describes applications of the TaDa design method that raised aesthetic issues particular to sonification practice. The final section proposes a pragmatic aesthetics that distinguishes sonification from the auditory sciences and sonic arts. A design perspective may lead to a future where the general public tunes into pop sonifications for listening enjoyment as well as useful information about the world.     

空白的精神——水彩画中的留白

在水彩画的创作中,"留白"无疑是创作的难点之一。一方面:作为技法和媒材的表现,留白不仅涉及到创作主体的创作体验、技巧表达,也包含对媒材自身的研究,以及对其表现张力的利用和再塑造。另一方面:"留白"作为一种审美观,还包括更为深层的东方绘画审美的文化范畴的表达。因此,本文将从水彩本体语言和东方化的审美观念等美学范畴入手深入地阐述其重要性。同时论及"留白"在当代水彩画创作中的重要意义。     

Remediating Knowledge-Making Spaces in the Graduate Curriculum: Developing and Sustaining Multimodal Teaching and Research

Functional, critical, and rhetorical training in multimodal teaching and research must span all aspects of professional development within the graduate curriculum in order to best prepare emerging professionals for success in the age of digital media. The authors call upon multiple perspectives of their home program and offer a range of benchmarks for other programs that want to promote digital teaching and research as integral, sustainable components of their knowledge-making spaces.