Touch as techne: rethinking digitality

Abstract

This article introduces Pulse Project (2011–), a doctoral performance research project that engages in critical discourse between art and science through the creation of digital soundscapes that weave together artistic, medical, technological, ancient and modern methodologies. Pulse ‘reading’, case histories, notations of pulses and programming soundscape compositions are all used together as methods for exploring the cultural encounter between artist, participants and diverse medical practices. Pulse Project seeks to provide an examination of the means with which the temporal materiality of touch can be used together with audio programming to form a translation and synthesis of different ecologies and disciplines, e.g. medicine and art, Eastern and Western practices, touch and digitality, etc. Drawing upon my experience as a clinical acupuncturist (with training in biomedicine), I use Chinese pulse diagnostics together with SuperCollider (an audio synthesis programming language) to inform the composition process of each soundscape. In this way, Pulse Project interrogates the aesthetic and philosophical axioms underpinning contemporary medicine, technology and cognitive embodiment through the exploration of their corollary ‘others’—traditional Chinese medicine and music theory—in order to generate a fresh approach to embodiment and soundscape composition.     

Continuous sonification enhances adequacy of interactions in peripheral process monitoring

As many users who are charged with process monitoring need to focus mainly on other work while performing monitoring as a secondary task, monitoring systems that purely rely on visual means are often not well suited for this purpose. Sonification, the presentation of data as (non-speech) sound, has proven in several studies that it can help in guiding the user's attention, especially in scenarios where process monitoring is performed in parallel with a different, main task. However, there are several aspects that have not been investigated in this area so far, for example if a continuous soundscape can guide the user's attention better than one that is based on auditory cues. We have developed a system that allows reproducible research to answer such questions. In this system, the participants’ performance both for the main task (simulated by simple arithmetic problems) and for the secondary task (a simulation of a production process) can be measured in a more fine-grained manner than has been the case for existing research in this field. In a within-subject study (n=18), we compared three monitoring conditions – visual only, visual + auditory alerts and a condition combining the visual mode with continuous sonification of process events based on a forest soundscape. Participants showed significantly higher process monitoring performances in the continuous sonification condition, compared to the other two modes. The performance in the main task was at the same time not significantly affected by the continuous sonification.     

Music,dimensions and play: composing for autonomous laptop musicians and improvising humans

Abstract

This article discusses three compositions for autonomous laptops performing with improvising musicians. The laptops were programmed with software environments that make narrow-band ‘decisions’ regarding performative soundscapes. They also made ‘decisions’ about visual materials that were presented as onscreen scores for the human improviser to consider. The unfolding aural and visual streams were both functional and creative, and highlighted the here-and-now of indeterminate composition, whilst carefully embedding the aesthetics of the composer within the core of the programming. As such, these laptops were considered as collaborating performers and collaborating composers. The opening section highlights the connecting principles within this collection of works, before discussing each in isolation. This is placed within the context of late twentieth-century and early twenty-first- century composition, whilst offering a framework, for the reading of these, based on Deleuze and Guattari's rhizome philosophy. Three Last Letters is then used as a case study of the concept of wholeness within the computer-generated soundscapes and the algorithmic programming. Section 3 draws together several theories surrounding the mental seeing evoked by sound into dimensionality: a playground for inventive HCI-music improvisation. The values and principles with which the humans and computers interact is developed using creative play theory, followed by some personal responses of the musicians. Finally, an evaluation is offered on the effect on the audience and their perceptual modes when experiencing live and fluid compositions of this nature. The article is concluded with a poetics of experience from the composer's perspective.     

Crossing the line: A field study of inhabited television

This paper reports on an ethnographic field study of 'Out Of This World' (OOTW, Benford et al. 1999) an experiment in 'inhabited television' combining broadcast technologies with a collaborative virtual environment in a live show. The study focuses on the work of producing OOTW and how personnel managed the manifold contingencies of working with complex technology. The use of a specially developed virtual camera control application is discussed together with the methods the director used for live editing views from cameras into a 'broadcast from virtual reality'. The challenges faced by the multiple professions involved (TV personnel, research scientists, actors) are documented and the viability of inhabited TV as a 'new medium' is assessed. Future technological refinements are briefly discussed along with some general implications for CSCW and 'media studies' of the work reported.     

Appropriating an architectural design tool for musical ends

NURBS (non-uniform rational b-spline) modelling has become a ubiquitous tool within architectural design praxis. In this article I examine three projects that utilise NURBS modelling as a means for which a musical system's inherent spatiality is visualised. There are numerous precedents for which architectural form is a derivation of a musical system, or a musical system is proportionally informed by architectonic gesture. I propose in this article three NURBS modelling methodologies: for the spatial analysis of Karlheinz Stockhausen's sound projection geometries in Pole für 2; for a spatial realisation of John Cage's indeterminate work Variations III; and for the generation of a surface manifold informed by musically derived soundscape data from the Japanese garden Kyu Furukawa Teien. Rather than seeking to translate music into inhabitable architecture, or architectonic form into music, I highlight an approach that produces an interstitial territory between discourses on architecture and music analysis.     

Noise disturbance in open-plan study environments: a field study on noise sources,student tasks and room acoustic parameters

Abstract

The aim of this study is to gain more insight in the assessment of noise in open-plan study environments and to reveal correlations between noise disturbance experienced by students and the noise sources they perceive, the tasks they perform and the acoustic parameters of the open-plan study environment they work in. Data were collected in five open-plan study environments at universities in the Netherlands. A questionnaire was used to investigate student tasks, perceived sound sources and their perceived disturbance, and sound measurements were performed to determine the room acoustic parameters. This study shows that 38% of the surveyed students are disturbed by background noise in an open-plan study environment. Students are mostly disturbed by speech when performing complex cognitive tasks like studying for an exam, reading and writing. Significant but weak correlations were found between the room acoustic parameters and noise disturbance of students.

Practitioner Summary: A field study was conducted to gain more insight in the assessment of noise in open-plan study environments at universities in the Netherlands. More than one third of the students was disturbed by noise. An interaction effect was found for task type, source type and room acoustic parameters.     

Reflections on sonic digital unreality

ABSTRACT

This paper explores the possibility of constructing an unreal environment through sound, particularly through the use of contemporary digital tools. The idea of the unreal is approached as a not being-in-the-world experience. That is, an experience which expands the natural embodied physicality. And yet, sound propagation through a physical space, which led to the study of reverberation and spatialization as the main subjects of interest. Consequently, reverb became the main tool to explore the subject in practice. Due to the project’s multidisciplinarity, the subject had different approaches: from the possibility of reverb as a tool for fiction-making, to listener's comprehensibility and readability of the events; exploring spatialization techniques and multichannel reproduction systems to finally conclude with the idea that, in the end, sound relies on the listener’s actual physical experience of the space.     

基于抛物方程法的矢量场分析与仿真

研究海底声纳系统优化识别问题,海洋环境中的声传播是水声学研究领域中的一个经典问题.传统研究只关心声压场而不能实现振速和声压的联合处理,而且对低频声矢量场的研究也比较少.为了提高声纳系统性能,通过尤拉方程给出声矢量场的计算模型,采用抛物方程方法对声场声压进行计算,再利用差分法对声场振速等矢量场参数进行数值求解,并分析了具有弹性海底的浅海波导中的声矢量场.仿真结果表明,采用抛物方程法能够快速准确的计算海洋环境中的声矢量场,并实现振速和声压的联合处理,适用于声矢量场的快速预报,为提高声纳系统性能提供了依据.     

Volterra series for solving weakly non-linear partial differential equations: application to a dissipative Burgers’ equation

A method to solve weakly non-linear partial differential equations with Volterra series is presented in the context of single-input systems. The solution x(z,t) is represented as the output of a z-parameterized Volterra system, where z denotes the space variable, but z could also have a different meaning or be a vector. In place of deriving the kernels from purely algebraic equations as for the standard case of ordinary differential systems, the problem turns into solving linear differential equations. This paper introduces the method on an example: a dissipative Burgers'equation which models the acoustic propagation and accounts for the dominant effects involved in brass musical instruments. The kernels are computed analytically in the Laplace domain. As a new result, writing the Volterra expansion for periodic inputs leads to the analytic resolution of the harmonic balance method which is frequently used in acoustics. Furthermore, the ability of the Volterra system to treat other signals constitutes an improvement for the sound synthesis. It allows the simulation for any regime, including attacks and transients. Numerical simulations are presented and their validity are discussed.     

Blending scenarios of use and informal argumentation to facilitate universal access: Experience with the Universal Access Assessment Workshop method

The accessibility of interactive computer-based products and services has long been an issue of concern to the Assistive Technology (AT) and Human-Computer Interaction (HCI) communities. In recent years, there have been several efforts aiming to document the consolidated wisdom in the form of general guidelines and examples of best practice. Despite their sound human factors content, these guidelines require substantial interpretation by designers, before they can generate practically useful and context-specific recommendations. In this paper, we examine how this task can be aided by blending scenarios of use and informal argumentation in the context of a structured technique referred to as Universal Access Assessment Workshop (UA²W). The paper provides a review of the technique and proposes two instruments, namely scenario screening and growth scenarios, which can be used to structure the conduct of UA²Ws. Finally, the paper summarises recent experiences in the use of these techniques in the context of a case study, which seeks to provide universal access insight to the design of a ward-based service to patient medical data.     

Wireless Acoustic Sensor Networks and Edge Computing for Rapid Acoustic Monitoring

Passive acoustic monitoring is emerging as a promising solution to the urgent, global need for new biodiversity assessment methods. The ecological relevance of the soundscape is increasingly recognised, and the affordability of robust hardware for remote audio recording is stimulating international interest in the potential for acoustic methods for biodiversity monitoring. The scale of the data involved requires automated methods, however, the development of acoustic sensor networks capable of sampling the soundscape across time and space and relaying the data to an accessible storage location remains a significant technical challenge, with power management at its core. Recording and transmitting large quantities of audio data is power intensive, hampering long-term deployment in remote, off-grid locations of key ecological interest. Rather than transmitting heavy audio data, in this paper, we propose a low-cost and energy efficient wireless acoustic sensor network integrated with edge computing structure for remote acoustic monitoring and in situ analysis. Recording and computation of acoustic indices are carried out directly on edge devices built from low noise primo condenser microphones and Teensy microcontrollers, using internal FFT hardware support. Resultant indices are transmitted over a ZigBee-based wireless mesh network to a destination server. Benchmark tests of audio quality, indices computation and power consumption demonstrate acoustic equivalence and significant power savings over current solutions.      

Design space exploration in many-core processors for sound synthesis of plucked string instruments

Recent advances in physics-based sound synthesis have unveiled numerous possibilities for the creation of new musical instruments. Despite the fact that research on physics-based sound synthesis has been going on for three decades, its higher computational complexity compared to that of signal modeling has limited its use in real-time applications. This limitation has motivated research on parallel processing architectures that support the physics-based sound synthesis of musical instruments. In this paper, we present analytical results of the design space exploration of many-core processors for the physics-based sound synthesis of plucked-string instruments including acoustic guitar, classical guitar and the gayageum, which is representative of a Korean plucked-string instrument. We do so by quantitatively evaluating the significance of a sample-per-processing-element (SPE) ratio–i.e., the amount of sample data directly mapped to each processing element, which is equivalent to varying the number of processing elements for a fixed sample size on system performance and efficiency using architectural and workload simulations. The effect of the sample-to-processor ratio is difficult to analyze because it fundamentally affects both hardware and software design when varied. In addition, the optimal SPE ratio is not typically at either extreme of its range–i.e., one sample per processor or one processor per an entire sample. This paper illustrates the correlation between a fixed problem sample size, SPE ratio and processing element (PE) architecture for a target implementation in 130-nm CMOS technology. Experimental results indicate that an SPE in the range of 5513 to 2756, which is equivalent to 48 to 96 PEs for guitars and 96 to 192 PEs for the gayageum, provides the most efficient operation for the synthesis of musical sounds sampled at 44.1 kHz, yielding the highest task throughput per unit area or per unit energy. In addition, the produced synthesized sounds appear to be very similar to the original sounds, and the selected optimal many-core configurations outperform commercial processor architectures including DSPs, FPGAs, and GPUs in terms of area efficiency and energy efficiency.     

JIR-Net：用于光声层析图像重建的联合迭代重建网络

目的 高质量的图像重建是光声层析成像（photoacoustic tomography,PAT）技术的关键,有限角度稀疏测量和组织非均匀的声学特性都会影响重建图像质量。采用迭代重建技术可在一定程度上提高图像质量,但是其结果依赖于有关成像目标的先验假设模型。而且在迭代优化过程中需要反复计算前向成像算子及其伴随算子,因此计算成本较高,需要合理选择正则化方法及其参数。为了解决该问题,提出一种根据不完备光声测量信号联合重建光吸收能量分布图和声速分布图的深度学习方法。方法 设计并搭建基于学习迭代策略的联合迭代重建网络（joint iterative reconstruction network,JIR-Net）,网络由4个结构单元组成,每个单元包括特征提取、特征融合和重建3个模块。网络的输入是探测器在成像平面中采集的不完备光声信号和预设的常数声速,输出是重建的光吸收能量分布图和声速分布图。分别构建仿真、仿体和在体数据集,用于训练、验证和测试网络。在训练网络的过程中,将光吸收能量密度和声速的梯度下降信息整合到网络训练中,并利用反向传播梯度下降法求解非线性最小二乘问题。结果 数值仿真、仿体和在体实验...     

Tuning the Sonic Playing Field: Teaching Ways of Knowing Sound in First Year Writing

In recent years there has been a call within composition to include sound, among other modes, such as word and image in writing. Some of this call relies on a movement to multimodal composition in order to capture both the richness of rhetorical possibility and the reality of communities of practice, and some is in response to a perceived shift in writing due to digital media tools and environments. Regardless of the impetus for including the auditory realm in the composition classroom, it is important for the field of composition and rhetoric to develop further pedagogies of sound so that students are not simply offered the opportunity to produce diverse texts, but instead, are invited to enter “the playing field.” In order to do this I first explore an approach to teaching auditory rhetoric based on ways of knowing sound from an acoustics and musicology perspective, then I consider a phenomenological approach based on listening, and finally I construct a model of “tuning the sonic playing field” that draws on the literal, material practice of tuning as a metaphor for how sound may be taught in composition. The “tuning” approach to teaching sound draws on attention, embodiment, listening, and negotiation. Rather than simply offering students opportunities to use sound in rhetorically sensitive ways, this final approach asks instructors to become “attuned” to how different auditory epistemologies influence students’ ability to design and compose in sound.     

A review of: “Computers,Work and Health”. By TREVOR A. WILLIAMS,Taylor & Francis, 4 John St,London WC1N 2ET (1988), pp. x + 142, £25·00. ISBN 0-85066-453-5

The effects of cabin noise on subjective comfort assessments were systematically investigated in order to reveal optimisation potentials for an improved passenger noise acceptance. Two aircraft simulation studies were conducted. An acoustic laboratory test facility provided with loudspeaker systems for realistic sound presentations and an aircraft cabin simulator (Dornier Do 728) with a high degree of ecological validity were used. Subjects were exposed to nine different noise patterns (three noise levels ranging from 66 to 78 dB(A) combined with three different frequency spectra). Regression analysis demonstrated a significant increase of passengers' acceptance with lower noise levels and significant effects of different frequency spectra determined by seat position in the aircraft cabin (front, middle, rear). Acoustic cabin design should therefore consider measures beyond noise level reduction altering noise characteristics to improve passengers' comfort and well-being in the aircraft cabin.

Practitioner Summary: To improve passenger comfort in the aircraft with respect to cabin noise, passengers' reactions to specific noise conditions were systematically investigated. Two laboratory studies showed significant dose-response relationships between sound pressure level and subjective comfort ratings which differed due to the noise at specific seat positions in the aircraft.     

A Belgian Transect: Field Broadcast in the expanded field of ecology

Abstract

This article sets out the notion of a Field Broadcast from the dual perspective of Rebecca Birch, one of the developers of a bespoke version of Flash Media Live Encoder and Bram Thomas Arnold, an artist who uses a case study from Sideways Festival, Belgium, 2012. Field Broadcast enables an artist to be in a field, suitably equipped, and stream live footage to an audience. It is an experiment in place, site and the notion of a field. It is a new method of making work in the space between site-specific performance and the digital realm: a way of working that enables artists to generate new artworks within the non-place of the Internet. Birch introduces the technology from a number of perspectives before it is fleshed out with evidence and experience from a live project that took place in Belgium in 2012. Sideways was a festival that traversed Belgium over four weeks and 400?km, with artists walking and generating work en route. The possibilities offered by Field Broadcast are explored in relation to the expanded field of ecology amidst Bourriaud's The Radicant (2009), Guatarri's The Three Ecologies (2005) and Morton's Ecology Without Nature (2007 Morton, Timothy. 2007. Ecology Without Nature. Cambridge, MA: Harvard University Press. [Google Scholar]).     

基于生态声学的虚拟现实影视声景指数量化

针对目前虚拟现实影视作品中声音设计过程中缺乏客观量化指标、过于主观的问题,提出一套基于生态声学的虚拟现实影视声景量化方案.依据生态声学研究理论基础,方案将影视声景元素根据其发声动机划分成自然声、人为声和地球声三个部分,在该分类的基础上,通过计算各类别之间的比例来量化影视声景设计中关于地点、时间、天气等因素.实验样本来源...     

炉膛速度场声学检测装置

本文讨论了基于声学检测的炉膛速度场测量装置的可行性，并在此基础上设计了一个声学检测试验平台。利用声学检测技术，得到精确的声波时间，以此重建炉膛速度场。该装置将声波信号的发生和接收、A／D转换、基于三次样条插值的信号处理方法和相关计算等技术相结合，并在实验室条件下对测量装置设计方案进行了试验验证。结果显示，该测量装置对于炉膛速度场测量，结果准确，并且有可操作性好、投入低等优点，有着良好的应用前景。     

A framework for variable content document generation with multiple actors

ContextAdvances in customization have highlighted the need for tools supporting variable content document management and generation in many domains. Current tools allow the generation of highly customized documents that are variable in both content and layout. However, most frameworks are technology-oriented, and their use requires advanced skills in implementation-related tools, which means their use by end users (i.e. document designers) is severely limited.ObjectiveStarting from past and current trends for customized document authoring, our goal is to provide a document generation alternative in which variants are specified at a high level of abstraction and content reuse can be maximized in high variability scenarios.MethodBased on our experience in Document Engineering, we identified areas in the variable content document management and generation field open to further improvement. We first classified the primary sources of variability in document composition processes and then developed a methodology, which we called DPL – based on Software Product Lines principles – to support document generation in high variability scenarios.ResultsIn order to validate the applicability of our methodology we implemented a tool – DPLfw – to carry out DPL processes. After using this in different scenarios, we compared our proposal with other state-of-the-art tools for variable content document management and generation.ConclusionThe DPLfw showed a good capacity for the automatic generation of variable content documents equal to or in some cases surpassing other currently available approaches. To the best of our knowledge, DPLfw is the only framework that combines variable content and document workflow facilities, easing the generation of variable content documents in which multiple actors play different roles.     

聚焦超声在脱气水声场中的多泡分布与声致发光研究

为明确多泡空化在高强度聚焦超声（High intensity focused ultrasound,HIFU）治疗中焦前区、焦后区的副作用,进一步优化HIFU治疗,本文在不同声强下利用高速摄像系统与数字相机拍摄聚焦超声在脱气水声场中的多泡及多泡声致发光（Multi-bubble sonoluminescence,MBSL）的空间分布,通过被动空化检测（Passive cavitation detection,PCD）技术和光电倍增管（Photomultiplier tube,PMT）系统检测多泡空化特征及声场整体发光强度。研究表明：当声强较低时（如3 047 W/cm²）,气泡以驻波场模式分布;随着声强的提高,焦点处出现向焦后方生长的气泡群,3~6 MHz宽带噪声增幅明显,同时MBSL最先出现在焦前区,并向换能器表面扩大,声场整体发光强度与声强成正比;当声强足够高时（如21 328 W/cm²）,焦点处空化泡群剧烈坍缩的同时可见椭球形光斑,嘶嘶声不断,气泡的聚集大大降低了声场的相对发光强度。