Connecting the real world with the digital overlay with smart ambient media—applying Peirce’s categories in the context of ambient media

This article is based on the keynote held at the workshop on Events in Multimedia (EiMM09) that took place in conjunction with the ACM Multimedia conference in Beijing in October 2009. The idea of the keynote was to develop and explain the idea of ambient media in general, it is principles, and additionally relate ambient media to information theoretical approaches such as Peirce’s categories, to the theories of decision making, and to theories discussing how smart objects can be made ‘smart’ by simulating the human mind. This article rounds up with practical examples underlining the presented ideas and theories.     

As Hangs the Flexible Line: Equilibrium of Masonry Arches

In 1675, English scientist Robert Hooke discovered “the true... ...manner of arches for building,” which he summarized with a single phrase: “As hangs the flexible line, so but inverted will stand the rigid arch.” In the centuries that followed, Hooke’s simple idea has been used to understand and design numerous important works. Recent research at MIT on the interactive analysis of structural forces provides new graphical tools for the understanding of arch behavior, which are useful for relating the forces and geometry of masonry structures. The key mathematical principle is the use of graphical analysis to determine possible equilibrium states.     

Intuition Pumps

The award of the 2003 Barwise Prize to Daniel Dennett by the American Philosophical Association signifies Dennett’s importance in the developing area of philosophical inquiry into computing and information. One source of Dennett’s intellectual stature is his command of scientific and engineering ideas, which he effectively applies to philosophical debates over machine intelligence, consciousness, and intentionality. Dennett regards the computer as both a model and a tool that will transform the ways that philosophy is pursued in the 21st century. In order to understand Dennett’s conception of how philosophy changes and fares, if his mechanistic and reductionist conception of the life of the mind succeeds, we turn to an examination of a central idea in Dennett’s thought: the intuition pump.     

A Justified Plan Graph Analysis of the Early Houses (1975-1982) of Glenn Murcutt

The Justified Plan Graph (JPG) technique was developed in the late 1970s and refined in the following two decades as a means of undertaking qualitative and quantitative research into the spatial structure or permeability of buildings. Famously used by Space Syntax researchers to uncover the social logic of architectural types, the technique remains an important, if not widely understood, approach to the analysis of the built environment. This paper uses the JPG method to undertake a three-stage analysis of the early houses of Pritzker Prize winning architect Glenn Murcutt; the stages are visual analysis, mathematical analysis and theoretical analysis. Through this process the paper offers a rare application of the JPG method to multiple works by the same architect and demonstrates the construction of a series of “inequality genotypes”, a partial “statistical genotype” and a partial “statistical archetype” for these houses. Instead of seeking to uncover the social structure of Murcutt’s housing, the paper analyses the architect’s distinctive approach to ordering space within otherwise simple volumes or forms. The ultimate purpose of this analysis is to offer an alternative space-based, rather than form-based, insight into this architect’s work.     

A new problem for rule following

Wittgenstein saw a problem with the idea that ‘rule following’ is a transparent process. Here I present an additional problem, based on recent ideas about non-Turing computing. I show that even the simplest algorithm—Frege’s successor function, i.e. counting—cannot by itself determine the ‘output’. Specification of a computing machine is also required.     

The interior spatial environment: dynamic 0g environments and human places

In outer space, human experiences interject with technology in ways not experienced on Earth. We will examine the human aspects of living both on Earth and in the interior 0g environment, how technology interacts with astronauts, and how the design of the interior environments in outer space considers the interaction of technology and humanity from an astronaut’s perspective. These issues are explored with Marc Garneau, the first Canadian astronaut in outer space, and Travis Baldwin, 0g environment designer, who has worked with astronauts in the USA. Both discuss the complexity of living in space and how the ways that we live on Earth in terms of interior spatial environment characteristics might be interesting to consider. Within interior environments on Earth, function is always affected and mediated by social and dynamic human relationships and people’s experience and perceptions. When these experiences are transferred into the 0g environment, small, confined, and highly technological environments augment and intensify human relations. And yet these are rarely considered in the quest for secure, functional environments in space transport or on stations. These issues are discussed from both theoretical and practical perspectives in conversation with Garneau and Baldwin. In the interview with Marc Garneau, the author validates certain theoretical assumptions about the complex and dynamic interactions that occur in space, understanding the ways that astronauts cope in space and become creative in figuring out how to “mold” the environment to become their own, and what design factors are important to consider. Finally, examples of potential design ideas for integrating and humanizing the interior 0g space environment are explored and ideas about how to integrate human social elements are proposed.     

Cobot in LambdaMOO: An Adaptive Social Statistics Agent

We describe our development of Cobot, a novel software agent who lives in LambdaMOO, a popular virtual world frequented by hundreds of users. Cobot’s goal was to become an actual part of that community. Here, we present a detailed discussion of the functionality that made him one of the objects most frequently interacted with in LambdaMOO, human or artificial. Cobot’s fundamental power is that he has the ability to collect social statistics summarizing the quantity and quality of interpersonal interactions. Initially, Cobot acted as little more than a reporter of this information; however, as he collected more and more data, he was able to use these statistics as models that allowed him to modify his own behavior. In particular, cobot is able to use this data to “self-program,” learning the proper way to respond to the actions of individual users, by observing how others interact with one another. Further, Cobot uses reinforcement learning to proactively take action in this complex social environment, and adapts his behavior based on multiple sources of human reward. Cobot represents a unique experiment in building adaptive agents who must live in and navigate social spaces.     

Computational models of the emotions: from models of the emotions of the individual to modelling the emerging irrational behaviour of crowds

Computational models of emotions have been thriving and increasingly popular since the 1990s. Such models used to be concerned with the emotions of individual agents when they interact with other agents. Out of the array of models for the emotions, we are going to devote special attention to the approach in Adamatzky’s Dynamics of Crowd-Minds. The reason it stands out, is that it considers the crowd, rather than the individual agent. It fits in computational intelligence. It works by mathematical simulation on a crowd of simple artificial agents: by letting the computer program run, the agents evolve, and crowd behaviour emerges. Adamatzky’s purpose is to give an account of the emergence of allegedly “irrational” behaviour. This is not without problem, as the irrational to one person may seem entirely rational to another, and this in turn is an insight that, in the history of crowd psychology, has affected indeed the competition among theories of crowd dynamics. Quite importantly, Adamatzky’s book argues for the transition from individual agencies to a crowd’s or a mob’s coalesced mind as so, and at any rate for coalesced crowd’s agency.     

Crowdsourcing intelligent design

Design intelligence, namely, artificial intelligence to solve creative problems and produce creative ideas, has improved rapidly with the new generation artificial intelligence. However, existing methods are more skillful in learning from data and have limitations in creating original ideas different from the training data. Crowdsourcing offers a promising method to produce creative designs by combining human inspiration and machines’ computational ability. We propose a crowdsourcing intelligent design method called ‘flexible crowdsourcing design’. Design ideas produced through crowdsourcing design can be unreliable and inconsistent because they rely solely on selection among participants’ submissions of ideas. In contrast, the flexible crowdsourcing design method employs a cultivation procedure that integrates the ideas from crowd participants and cultivates these ideas to improve design quality at the same time. We introduce a series of studies to show how flexible crowdsourcing design can produce original design ideas consistently. Specifically, we will describe the typical procedure of flexible crowdsourcing design, the refined crowdsourcing tasks, the factors that affect the idea development process, the method for calculating idea development potential, and two applications of the flexible crowdsourcing design method. Finally, it summarizes the design capabilities enabled by crowdsourcing intelligent design. This method enhances the performance of crowdsourcing design and supports the development of design intelligence.     

Formation conditions of mutual adaptation in human-agent collaborative interaction

When an adaptive agent works with a human user in a collaborative task, in order to enable flexible instructions to be issued by ordinary people, it is believed that a mutual adaptation phenomenon can enable the agent to handle flexible mapping relations between the human user’s instructions and the agent’s actions. To elucidate the conditions required to induce the mutual adaptation phenomenon, we designed an appropriate experimental environment called “WAITER” (Waiter Agent Interactive Training Experimental Restaurant) and conducted two experiments in this environment. The experimental results suggest that the proposed conditions can induce the mutual adaptation phenomenon.     

Multi-user mixed reality system ‘Gulliver’s World’: a case study on collaborative edutainment at the intersection of material and virtual worlds

This case study is about ‘Gulliver’s World’, a multi-user mixed reality environment that functions simultaneously as interactive edutainment platform and learning environment as well as flexible infrastructure for the expansion of mixed reality environments via innovations in technology and media art. As an exhibition project, the installation is characterized by a nonlinear exhibition concept that activates interaction between individual users and different modes of virtual reality as well as collaboration among the users themselves. At seven workstations, people of all age groups range along the Reality–Virtuality continuum while collaboratively creating 3D worlds. Results of these creation activities are interactive worlds at the nexus of theatre, digital film production and game environment. As a research project, ‘Gulliver’s World’ features multilevel infrastructure with exemplary content in which the latest insights and models to emerge from HCI research as well as concepts of mixed reality and virtual environments and their supporting technology are brought together and developed further.     

Mathematical Knowledge of Architecture in the Works of Kâshânî

Ghiyath al-D?n Jamsh?d Kashan? (al-Kash?) is the prominent mathematician and astronomer of ninth/fifteenth century of Islamic civilization who founded the scientific methodology in practical and theoretical knowledge of mathematics in architecture. He played a significant role in the interaction and concurrence among mathematicians and architects, by addressing the relation between architecture and mathematics in two areas of knowledge, theoretical and practical, the latter itself including theoretical practice and practical practice. This paper addresses the question of how Kashan?’s research has led to the foundation of the first theoretical basis for the application of mathematics in architecture in the form of easily practicable solutions for drawing, and measuring different types of ceilings, arches, vaults, domes, and ornaments, as well as estimating required materials. Further, we will discuss how his knowledge in mathematics and astronomy were utilized in management, design and construction of Samarkand observatory and its astronomical instruments. By exploring how the mathematical knowledge of a Persian scientist was utilized in architecture and craftsmanship, we will shed some light on the hidden layers of Kashan?’s architectural life, who until now has been considered only a mathematician and astronomer.     

Hegel and the extended mind

We examine the theory of the extended mind, and especially the concept of the “parity principle” (Clark and Chalmers in Analysis 58.1:7–19, 1998), in light of Hegel’s notion of objective spirit. This unusual combination of theories raises the question of how far one can extend the notion of extended mind and whether cognitive processing can supervene on the operations of social practices and institutions. We raise some questions about putting this research to critical use.     

Modeling appraisal in theory of mind reasoning

Cognitive appraisal theories, which link human emotional experience to their interpretations of events happening in the environment, are leading approaches to model emotions. Cognitive appraisal theories have often been used both for simulating “real emotions” in virtual characters and for predicting the human user’s emotional experience to facilitate human–computer interaction. In this work, we investigate the computational modeling of appraisal in a multi-agent decision-theoretic framework using Partially Observable Markov Decision Process-based (POMDP) agents. Domain-independent approaches are developed for five key appraisal dimensions (motivational relevance, motivation congruence, accountability, control and novelty). We also discuss how the modeling of theory of mind (recursive beliefs about self and others) is realized in the agents and is critical for simulating social emotions. Our model of appraisal is applied to three different scenarios to illustrate its usages. This work not only provides a solution for computationally modeling emotion in POMDP-based agents, but also illustrates the tight relationship between emotion and cognition—the appraisal dimensions are derived from the processes and information required for the agent’s decision-making and belief maintenance processes, which suggests a uniform cognitive structure for emotion and cognition.     

Is ripple effect intuitive? A pilot study

The computation of ripple effect is based on the effect that a change to a single variable will have on the rest of a program; it determines the scope of the change and provides a measure of the program’s complexity. The original algorithm used to compute ripple effect has been reformulated mainly to provide clarity in the operations involved. The reformulation involved some approximation which was shown not to affect the measures produced. The reformulated, approximated algorithm has been implemented as the software tool: Ripple Effect and Stability Tool (REST). This paper uses a software development project as a case study to look at the relationship between the approximated ripple effect and a programmer’s intuitive idea of ripple effect. Four versions of a mutation testing software tool were written in C over a period of several months. After the completion of each version the programmer was asked to detail his predicted/intuitive ripple effect for each module of code. The predictions are compared with the approximated ripple effect measures for each module and some surprising conclusions drawn.     

An American Vision of Harmony

Thomas Jefferson dedicated his later years to establishing the University of Virginia, believing that the availability of a public liberal education was essential to national prosperity and individual happiness. His design for the University stands as one of his greatest accomplishments and has been called “the proudest achievement of American architecture.” Taking Jefferson’s design drawings as a basis for study, this paper explores the possibility that he incorporated incommensurable geometric proportions in his designs for the Rotunda. Without actual drawings to illustrate specific geometric constructions, it cannot be said definitively that Jefferson utilized such proportions. But a comparative analysis between Jefferson’s plans and Palladio’s renderings of the Pantheon (Jefferson’s primary design source) suggests that both designs developed from similar geometric techniques.