Touch as techne: rethinking digitality

Abstract

This article introduces Pulse Project (2011–), a doctoral performance research project that engages in critical discourse between art and science through the creation of digital soundscapes that weave together artistic, medical, technological, ancient and modern methodologies. Pulse ‘reading’, case histories, notations of pulses and programming soundscape compositions are all used together as methods for exploring the cultural encounter between artist, participants and diverse medical practices. Pulse Project seeks to provide an examination of the means with which the temporal materiality of touch can be used together with audio programming to form a translation and synthesis of different ecologies and disciplines, e.g. medicine and art, Eastern and Western practices, touch and digitality, etc. Drawing upon my experience as a clinical acupuncturist (with training in biomedicine), I use Chinese pulse diagnostics together with SuperCollider (an audio synthesis programming language) to inform the composition process of each soundscape. In this way, Pulse Project interrogates the aesthetic and philosophical axioms underpinning contemporary medicine, technology and cognitive embodiment through the exploration of their corollary ‘others’—traditional Chinese medicine and music theory—in order to generate a fresh approach to embodiment and soundscape composition.     

Practical approaches towards interactivity in soundscape composition

Abstract

The article discusses the role of technology in integrating acoustic instruments within soundscape composition in a live and interactive context, thus encouraging an engagement with the sonic environment. The approaches to two compositions that combine instruments with live electronics and pre-composed soundscapes – Cold Wood (bass trombone) and Arcando (alto saxophone) – are considered. These pieces create unique live performance scenarios in order to investigate how acoustic instruments might be positioned within, and reframed by, soundscape composition practice. Potential pitfalls inherent in mixed media works are explored, proposing technological solutions through sound processing and use of microphones, whilst ensuring that 'liveness' remains more aesthetic than procedural. Aspects of field recording practice influence the behaviour of the instruments, and improvisation is encouraged in both pieces to ensure unpredictability. Consequently, it is also proposed that the process of listening to, capturing, and processing the sound of the instrument in a live performance scenario is analogous to the practice of field recording, in which there is little control over the sound events.     

Bring in the Arts and Get the Creativity for Free: A Study of the Artists in Residence Project

In the literature on creativity and innovation, there is a tendency to idolize great contributors to the liberal arts as extraordinarily creative individuals. Such ex post facto accounts of ‘great men and women’ are, however, of limited value for everyday practices in organizations. This paper reports insights from a Swedish project named Artists in Residence (AIRIS) wherein artists, including musicians, painters, actors and directors, dancers and choreographers, collaborated with a regular company or workplace during a ten‐month project, aimed at helping the co‐workers think in new and creative ways. The study concludes that there are many benefits from making the world of artists and the world of work intersect, but there is also a demand on the participant to fully commit to the project. Even though the culture project was positively received among co‐workers, only a limited effect on workplace climate could be reported for the 2005 evaluation, while the 2006 evaluation indicates more positive effects. Still, the amount of texts advocating artists' creative skills and experiences outnumber the cases of actual projects bringing the two groups together. Studies of projects like AIRIS show that there is a great potential in bringing artists into industry.     

Gestural systems for the voice: performance approaches and repertoire

ABSTRACT

Singing, like dance, emerges directly from the body. The voice, in combination with whole body movement, constitutes a potent form of self-expression. Gestural systems offer a specialized context in which to explore the intersection between voice and movement. The practice-based investigation presented in this article charts the development of an original musical work, Intangible Spaces, which gives form to the invisible aspects of voice and movement through gestural control, physical modelling synthesis and visual feedback. I draw on embodied and performative autoethnographic methods to capture the felt sensations and sound-movement associations that arise during the composition process. I also explore the performance approaches of key practitioners in the area to gain a broader understanding of the ways in which musicians leverage existing performance skills to uncover novel connections between movement and voice in gestural performance.     

Programs as visual,interactive documents

We present a novel approach to combined textual and visual programming by allowing visual, interactive objects to be embedded within textual source code and segments of source code to be further embedded within those objects. We retain the strengths of text‐based source code, while enabling visual programming where it is beneficial. Additionally, embedded objects and code provide a simple object‐oriented approach to adding a visual form of LISP‐style macros to a language. The ability to freely combine source code and visual, interactive objects with one another allows for the construction of interactive programming tools and experimentation with novel programming language extensions. Our visual programming system is supported by a type coercion‐based presentation protocol that displays normal Java and Python objects in a visual, interactive form. We have implemented our system within a prototype interactive programming environment called ‘The Larch Environment’. Copyright © 2013 John Wiley & Sons, Ltd.     

Communication and Context: Collective Tacit Knowledge and Practice in Japan’s Workplace ba

In contrast to Schumpeter’s “perennial gale of creative destruction” (Schumpeter 1976: 84), government–coordinated economic development in post–1945 Japan has owed more to informal (but binding) “rules of the game” (North 1990) that situate working, learning and innovation within the spaces delineated by tightly bounded company–as–family workplace organisations or ‘ba’ (which roughly means ‘place’ or ‘interaction field’). Horizontal keiretsu groupings, together with fixed trading–patterns in supply and distribution chains, continue to support an interlocking ‘steady state’ economic structure in which new technologies tend to emerge from existing organisations. Shared experience within workplace ba generates tacit knowledge that is held in common by colleagues and retained as a potent tool for shaping future practice. It plays a vital role in facilitating ‘friction free’ communication amongst insiders, who can act as a group to ostracise and retaliate against agents who break their code. Long–term obligations link salaried male employees to their workplace ba. Consequently, autonomous boundary–spanning communities of practice, together with industry–university collaboration and other transient associations with outsiders, lack legitimacy. Cook and Brown’s (1999) pluralist epistemology is used to compare Western interpretations of Mode 1 and Mode 2 knowledge (Gibbons et al 1994) with the privileged role that Japan’s workplace ba accord to insider collective–tacit knowledge, which we tentatively call ‘Mode 3’ knowledge.     

Timing analysis of scenario‐based specifications using linear programming

Scenario‐based specifications (SBSs), such as UML interaction models, offer an intuitive and visual way of describing design requirements, and are playing an increasingly important role in the design of software systems. This paper presents an approach to timing analysis of SBSs expressed by UML interaction models. The approach considers more general and expressive timing constraints in UML sequence diagrams (SDs), and gives a solution to the reachability analysis, constraint conformance analysis and bounded delay analysis problems, which reduces these problems into linear programs. With the synchronous interpretation of the SD compositions, the timing analysis algorithms in the approach form a decision procedure for a class of SBSs where any loop in any path is time‐independent of the other parts in the path. These algorithms are also a semi‐decision procedure for general SBSs with both the synchronous and asynchronous composition semantics. The approach also supports bounded timing analysis of SBSs, which investigates all the paths in the bound limit one by one, and performs the timing analysis for each finite path by linear programming. A tool prototype has been developed to support this approach. Copyright © 2010 John Wiley & Sons, Ltd.     

A study of the graphical mediating representations used by collaborating authors

Abstract

Though they have access to sophisticated authoring support systems, many authors use conventional media for the earliest stages of generating and organising ideas. This paper gives an overview of a study of six pairs of collaborating authors who were videoed in aprewriting task and who also underwent a structured post-interview concerning their use of external representations in the study and in their everyday work. Pairs were studied so that the behaviour was more externalised, there were natural verbal protocols and these protocols could be interpreted by the interlocutor rather than by the researcher. The analysis focuses on the part that the mediating representations play in the cognitive task and in coordinating cooperative cognition. The properties of the marks produced and the affordances of conventional media are considered. The study is interpreted within a ‘shared cognitive’ framework which draws on insights from distributed cognition, socially shared cognition, situated action and most importantly from Soviet psychology. It is suggested that the mediating representations perform two important and highly related functions. First, they support ‘idea sketching’ which is a form of reflexive communication where the individual mediates her own creative cognition through external representations. Second, they mediate cooperative cognition, providing the common grounding necessary to coordinate shared thought.     

Conditional composition

Generalizing the notion of function composition, we introduce the concept ofconditional function composition and present a theory of such compositions. We use the theory to describe the semantics of a programming language with exceptions, and to relate exceptions to theIF statement.Supported in part by Air Force Office of Scientific Research grant number 91-0070. Now at DEC Systems Research Center, Palo Alto, CA.     

Constructures: supporting human ingenuity in software

Existing media authoring software programs are generally regarded as offering little support for the earliest stages of the creative process, which have characteristics and needs that are distinct from the later stages. As a step toward addressing this, we present our constructural design philosophy: a set of principles that can be applied to software in any creative medium and under which we believe effective, early-stage creativity tools can be created. We then illustrate the application of these principles with Wheelsong, a complete, constructurally designed, early-stage music composition exploratorium and discuss a selection of musical sketches resulting from its use.     

A reactive extension of the OpenMusic visual programming language

Objectives: OpenMusic (OM) is a domain-specific visual programming language designed for computer-aided music composition. This language based on Common Lisp allows composers to develop functional processes generating or transforming musical data, and to execute them locally by demand-driven evaluations. As most historical computer-aided composition environments, OM relies on a transformational declarative paradigm, which is hard to conciliate with reactive data-flow (an evaluation scheme more adequate to the development of interactive systems). We propose to link these two evaluation paradigms in the same and consistent visual programming framework.Methods: We establish a denotational semantics of the visual language, which gives account for its demand-driven evaluation mechanism and the incremental construction of programs. We then extend this semantics to enable reactive computations in the functional graphs.Results: The resulting language merges data-driven executions with the existing demand-driven mechanism. A conservative implementation is proposed.Conclusions: We show that the incremental construction of programs and their data-driven and demand-driven evaluations can be smoothly integrated in the visual programming workflow. This integration allows for the propagation of changes in the programs, and the evaluation of graphically designed functional expressions as a response to external events, a first step in bridging the gap between computer-assisted composition environments and real-time musical systems.     

Object-oriented analysis and design of the Message Passing Interface

The major contribution of this paper is the application of modern analysis techniques to the important Message Passing Interface standard, work done in order to obtain information useful in designing both application programmer interfaces for object-oriented languages, and message passing systems. Recognition of ‘Design Patterns’ within MPI is an important discernment of this work. A further contribution is a comparative discussion of the design and evolution of three actual object-oriented designs for the Message Passing Interface ( MPI-1SF ) application programmer interface (API), two of which have influenced the standardization of C++ explicit parallel programming with MPI-2, and which strongly indicate the value of a priori object-oriented design and analysis of such APIs. Knowledge of design patterns is assumed herein. Discussion provided here includes systems developed at Mississippi State University (MPI++), the University of Notre Dame (OOMPI), and the merger of these systems that results in a standard binding within the MPI-2 standard. Commentary concerning additional opportunities for further object-oriented analysis and design of message passing systems and APIs, such as MPI-2 and MPI/RT, are mentioned in conclusion. Connection of modern software design and engineering principles to high performance computing programming approaches is a new and important further contribution of this work. Copyright © 2001 John Wiley & Sons, Ltd.     

Relating the implementation techniques of functional and functional logic languages

Functional logic languages are declarative programming languages that integrate the programming paradigms of functional and logic languages within a single framework. They are extensions of functional languages with principles derived from logic programmingNarrowing, the evaluation mechanism of functional logic languages, can be defined as a generalization ofreduction, the evaluation mechanism of purely functional languages. The unidirectional pattern matching, which is used for parameter passing in functional languages, is simply replaced by the bidirectionalunification known from logic programming languages. We show in this paper, how to extend a reduction machine, that has been designed for the evaluation of purely functional programs to a machine that performs narrowing. The necessary extensions concern the realization of unification and backtracking, for which we fall back upon the methods of Warren’s Prolog engine.²¹⁾ The narrowing machine embodies an optimized treatment of deterministic computations. A complete specification of the reduction and the narrowing machine and of the translation of a sample language into abstract machine code is given. Comparative results of a C-implementation of the reduction and the narrowing machine show that the time overhead of the more complex narrowing evaluation is, in general, less than 10% of the reduction evaluation.     

Frequency of postural changes during sitting whilst using a desktop computer – exploring an analytical methodology

Background: Dynamic movement whilst sitting is advocated as a way to reduce musculoskeletal symptoms from seated activities. Conventionally, in ergonomics research, only a ‘snapshot’ of static sitting posture is captured, which does not provide information on the number or type of movements over a period of time. A novel approach to analyse the number of postural changes whist sitting was employed in order to describe the sitting behaviour of adolescents whilst undertaking computing activities. Methods: A repeated-measures observational study was conducted. A total of 12 high school students were randomly selected from a conveniently selected school. Fifteen minutes of 3D posture measurements were recorded to determine the number of postural changes whilst using computers. Results: Data of 11 students were able to be analysed. Large intra-subject variation of the median and IQR was observed, indicating frequent postural changes whilst sitting. Conclusion: Better understanding of usual dynamic postural movements whilst sitting will provide new insights into causes of musculoskeletal symptoms experienced by computer users.     

Animistic design: how to reimagine digital interaction between the human and the nonhuman

ABSTRACT

This article puts forward the notion of animistic design as an uncertainty-driven strategy to reimagine human–machine interaction as a milieu of human and nonhuman. Animistic design is suggested as capable of fostering affects, sensibilities and thoughts that capitalize on the uncertain, the unpredictable and the nonlinear, and their capacity to trigger creative pathways. Informed by post-human philosophies, theories of mediation and materiality, as well as by affect, agency and aesthesia, animistic design eschews the anthropomorphic and the cute playfulness often associated with animism. Instead, it proposes a practical–theoretical framework to articulate the nexus of digital innovation, interaction design practices, technical materialities and affective responses already emerging in the digital cohabitation of the human and the nonhuman. Using a ‘research through making’ approach, the article describes in detail a series of animistic design experiments and prototyping methods that explore ways of rethinking interaction as an open-ended and creative enterprise. Animistic design offers an investigative strategy that exploits degrees of collaboratively curated uncertainty and unpredictability to imagine forms of digital interaction, and to engender creative human–nonhuman relationships within a given digital milieu.     

Artist Space 5 ArCade III

The author describes the development of a series of performance pieces and artworks from 2000 to the present, which use as raw materials ‘silences’ collected in real time from radio broadcasts and improvising musicians. These works employ original software written in the SuperCollider programming language.     

A comparison of connectionist models of music recognition and human performance

Current artificial neural network or connectionist models of music cognition embody feature-extraction and feature-weighting principles. This paper reports two experiments which seek evidence for similar processes mediating recognition of short musical compositions by musically trained and untrained listeners. The experiments are cast within a pattern recognition framework based on the vision-audition analogue wherein music is considered an auditory pattern consisting of local and global features. Local features such as inter-note interval, and global features such as melodic contour, are derived from a two-dimensional matrix in which music is represented as a series of frequencies plotted over time.Manipulation of inter-note interval affected accuracy and reaction time measures in a discrimination task, whereas the same variables were affected by manipulation of melodic contour in a classification task. Musical training is thought of as a form of practice in musical pattern recognition and, as predicted, accuracy and reaction time measures of musically trained subjects were significantly better than those of untrained subjects. Given the evidence for feature-extraction and weighting processes in music recognition tasks, two connectionist models are discussed. The first is a single-layer perceptron which has been trained to discriminate between compositions according to inter-note interval. A second network, using the back-propagation algorithm and sequential input of patterns, is also discussed.     

Towards children's creative museum engagement and collaborative sense-making

Abstract

In this article, we elucidate a socio-culturally framed approach to supporting children's creative museum engagement. Specifically, we focus on social activities and socio-cultural resources that can act as boundary-permeating objects in mediating children's creative engagement and collaborative sense-making regarding cultural content within, across and beyond the spatio-material context of the museum. We contend that designing and organising children's creative engagement and collaborative sense-making in ways that cultivate boundary-crossing broadens opportunities for engagement and leverages children's creative potential and expansive learning. We build our argument by starting with a theoretical introduction to the design principles that constitute the Kids, Museums, and Technology Programme. We will illuminate the design principles of the programme with empirical examples and consider how the design principles and their situated construction can help us re-imagine museum exhibitions as hybrid, boundary-permeating spaces that afford novel transformative interactions, as well as new roles and identities for both children and museums.