Taxonomy,technology and applications of smart objects

Deployment of embedded technologies is increasingly being examined in industrial supply chains as a means for improving efficiency through greater control over purchase orders, inventory and product related information. Central to this development has been the advent of technologies such as bar codes, Radio Frequency Identification (RFID) systems, and wireless sensors which when attached to a product, form part of the product’s embedded systems infrastructure. The increasing integration of these technologies dramatically contributes to the evolving notion of a “smart product”, a product which is capable of incorporating itself into both physical and information environments. The future of this revolution in objects equipped with smart embedded technologies is one in which objects can not only identify themselves, but can also sense and store their condition, communicate with other objects and distributed infrastructures, and take decisions related to managing their life cycle. The object can essentially “plug” itself into a compatible systems infrastructure owned by different partners in a supply chain. However, as in any development process that will involve more than one end user, the establishment of a common foundation and understanding is essential for interoperability, efficient communication among involved parties and for developing novel applications. In this paper, we contribute to creating that common ground by providing a characterization to aid the specification and construction of “smart objects” and their underlying technologies. Furthermore, our work provides an extensive set of examples and potential applications of different categories of smart objects.     

A middleware for a tabletop procedure-aware information display

In this article, we propose a middleware that supports a developer to build tabletop information display systems. We focus on an application that projects information close to a particular object on a table. To let a user focus on his/her primary task, the information is presented based on a current phase in a procedure, which we call procedure-awareness. Here, a current phase is recognized through the utilization of objects. The proposed middleware separates information from acquisition and presentation mechanisms: sensors for operational context recognition, sensors for object identification/tracking for “nearby presentation”, and projecting information near an object. A developer just needs to write a set of rules, i.e. application logic, that represent contents for particular operational contexts. Also, by implementing appropriate handlers, any content can be provided. We present the design and the implementation of the middleware and validate the expressiveness of application logics through prototype development.     

An expressive language and interface for image querying

One key component in providing effective image data management support is an expressive query language/interface. In this paper, we describe the EXQUISI system that we have developed. A main contribution of EXQUISI is its ability to allow a user to express subtle differences that may exist between images to be retrieved and other images that are similar. In particular, it allows the user to incorporate ambiguities and imprecisions in specifying his/her query. Another important aspect of EXQUISI is the provision of a reformulation language by which the user can ask “like this in what” queries, by specifying which parts of a returned image the user wants to include and exclude.     

Designing manipulative technologies for children with different abilities

We present a design approach for manipulative technologies that consider “user diversity” as a main lever for design. Different dimensions of “diversity” are considered, e.g., the users' age, abilities, culture, cultural background, and alphabetization. These dimensions drive the development of a user-centered design process for manipulative technologies for learning and play environments. In particular, we explore the possibility of allowing young children to develop and interact with virtual/physical worlds by manipulating physical objects in different contexts, like the classroom, the hospital, or the playground. In our scenarios, we consider children with different abilities (fully able, physically impaired, or with cognitive delays), in different cultures (Denmark, Tanzania, and Italy), and with a different level of alphabetization. The needs and expectations of such heterogeneous user-groups are taken into account through a user-centered design process to define a concept of tangible media for collaborative and distributed edutainment environments. The concept is implemented as a set of building blocks called I-Blocks with individual processing and communication power. Using the I-Blocks system, children can do “programming by building,” and thereby construct interacting artefacts in an intuitive manner without the need to learn and use traditional programming languages. Here, we describe in detail the technology of I-Blocks and discuss lessons learned from “designing for diversity.”     

A query expansion and user profile enrichment approach to improve the performance of recommender systems operating on a folksonomy

In this paper we propose a query expansion and user profile enrichment approach to improve the performance of recommender systems operating on a folksonomy, storing and classifying the tags used to label a set of available resources. Our approach builds and maintains a profile for each user. When he submits a query (consisting of a set of tags) on this folksonomy to retrieve a set of resources of his interest, it automatically finds further “authoritative” tags to enrich his query and proposes them to him. All “authoritative” tags considered interesting by the user are exploited to refine his query and, along with those tags directly specified by him, are stored in his profile in such a way to enrich it. The expansion of user queries and the enrichment of user profiles allow any content-based recommender system operating on the folksonomy to retrieve and suggest a high number of resources matching with user needs and desires. Moreover, enriched user profiles can guide any collaborative filtering recommender system to proactively discover and suggest to a user many resources relevant to him, even if he has not explicitly searched for them.     

Assisted Navigation for Persons with Reduced Mobility: Path Recognition Through Particle Filtering (Condensation Algorithm)

In the few past decades, several international researchers have worked to develop intelligent wheelchairs for the people with reduced mobility. For many of these projects, the structured set of commands is based on a sensor-based command. Many types of commands are available but the final decision is to be made by the user. A former work established a behaviour-based multi-agent form of control ensuring that the user selects the best option for him/her in relation to his/her preferences or requirements. This type of command aims at “merging” this user and his/her machine—a kind of symbiotic relationship making the machine more amenable and the command more effective. In this contribution, the approach is based on a curve matching procedure to provide comprehensive assistance to the user. This new agent, using a modelization of the paths that are most frequently used, assists the user during navigation by proposing the direction to be taken when the path has been recognized. This approach will spare the user the effort of determining a new direction—which might be a major benefit in the case of severe disabilities. The approach considered uses particle filtering to implement the recognition of the most frequent paths according to a topological map of the environment.     

Hyperlinking reality via camera phones

A novel user interface concept for camera phones, called “Hyperlinking Reality via Camera Phones”, that we present in this article, provides a solution to one of the main challenges facing mobile user interfaces, that is, the problem of selection and visualization of actions that are relevant to the user in her current context. Instead of typing keywords on a small and inconvenient keypad of a mobile device, a user of our system just snaps a photo of her surroundings and objects in the image become hyperlinks to information. Our method commences by matching a query image to reference panoramas depicting the same scene that were collected and annotated with information beforehand. Once the query image is related to the reference panoramas, we transfer the relevant information from the reference panoramas to the query image. By visualizing the information on the query image and displaying it on the camera phone’s (multi-)touch screen, the query image augmented with hyperlinks allows the user intuitive access to information.     

How Helen Keller used syntactic semantics to escape from a Chinese Room

A computer can come to understand natural language the same way Helen Keller did: by using “syntactic semantics”—a theory of how syntax can suffice for semantics, i.e., how semantics for natural language can be provided by means of computational symbol manipulation. This essay considers real-life approximations of Chinese Rooms, focusing on Helen Keller’s experiences growing up deaf and blind, locked in a sort of Chinese Room yet learning how to communicate with the outside world. Using the SNePS computational knowledge-representation system, the essay analyzes Keller’s belief that learning that “everything has a name” was the key to her success, enabling her to “partition” her mental concepts into mental representations of: words, objects, and the naming relations between them. It next looks at Herbert Terrace’s theory of naming, which is akin to Keller’s, and which only humans are supposed to be capable of. The essay suggests that computers at least, and perhaps non-human primates, are also capable of this kind of naming.

Understanding user commands by evaluating fuzzy linguistic information based on visual attention

This article proposes a method for understanding user commands based on visual attention. Normally, fuzzy linguistic terms such as “very little” are commonly included in voice commands. Therefore, a robot’s capacity to understand such information is vital for effective human-robot interaction. However, the quantitative meaning of such information strongly depends on the spatial arrangement of the surrounding environment. Therefore, a visual attention system (VAS) is introduced to evaluate fuzzy linguistic information based on the environmental conditions. It is assumed that the corresponding distance value for a particular fuzzy linguistic command depends on the spatial arrangement of the surrounding objects. Therefore, a fuzzy-logic-based voice command evaluation system (VCES) is proposed to assess the uncertain information in user commands based on the average distance to the surrounding objects. A situation of object manipulation to rearrange the user’s working space is simulated to illustrate the system. This is demonstrated with a PA-10 robot manipulator.     

A framework for modeling positive class expansion with single snapshot

In many real-world data mining tasks, the connotation of the target concept may change as time goes by. For example, the connotation of “learned knowledge” of a student today may be different from his/her “learned knowledge” tomorrow, since the “learned knowledge” of the student is expanding everyday. In order to learn a model capable of making accurate predictions, the evolution of the concept must be considered, and thus, a series of data sets collected at different time is needed. In many tasks, however, there is only a single data set instead of a series of data sets. In other words, only a single snapshot of the data along the time axis is available. In this paper, we formulate the Positive Class Expansion with single Snapshot (PCES) problem and discuss its difference with existing problem settings. To show that this new problem is addressable, we propose a framework which involves the incorporation of desirable biases based on user preferences. The resulting optimization problem is solved by the Stochastic Gradient Boosting with Double Target approach, which achieves encouraging performance on PCES problems in experiments.     

Effects of etiquette strategy on human–robot interaction in a simulated medicine delivery task

The objective of this study was to examine the extent to which a model of linguistic etiquette in human–human interaction could be applied to human–robot interaction (HRI) domain, and how different etiquette strategies proposed through the model might influence performance of humans and robots as mediated by manipulations of robot physical features, in a simulated medicine delivery task. A “wizard of Oz” experiment was conducted in which either a humanoid robot or a mechanical-looking robot was used to present medicine reminding utterances (following different etiquette strategies) to participants, who were engaged in a primary cognitive task (a Sudoku puzzle). Results revealed the etiquette model to partially extend to the HRI domain. Participants were not sensitive to positive language from robots (e.g., appreciation of human values/wants) and such a strategy did not succeed in supporting or enhancing the “positive face” of human users. Both “bald” (no linguistic courtesy) and mixed strategies (positive and “negative face” (minimizing user imposition) saving) resulted in moderate user perceived etiquette scores (PE). However, individual differences suggested such robot linguistic strategies should be applied with caution. Opposite to this, a negative face saving strategy (supporting user freedom of choice) promoted user task and robot performance (in terms of user response time to robot requests), and resulted in the highest PE score. There was also evidence that humanoid robot features provide additional social cues that may be used by patients and support human and robot performance, but not PE. These results provide a basis for determining appropriate etiquette strategies and robot appearance to promote better collaborative task performances for future health care delivery applications of service robots.     

Knowledge based manufacturing system (KBMS)

Production management, in batch type manufacturing environment, is regarded by the current research community as a very complex task. This paper claims that the complexity is a result of the system approach where management performance relies on decisions made at a too early stage in the manufacturing process. Decisions are made and stored in company databases by engineers who are neither economists nor production planner’s experts. This paper presents a new method where engineer’s task is not to make decisions but rather to prepare a knowledge-based “road map”. The road map method does introduce flexibility and dynamics in the manufacturing process and thus simplifies the decision making process in production planning. Each user will generate a routine that meets his/her needs at the time of needs by using KBMS CAPP. Thereby this method increases dramatically manufacturing efficiency.     

Browsing the environment with the SNAP&TELL wearable computer system

This paper provides an overview of a multi-modal wearable computer system, SNAP&TELL. The system performs real-time gesture tracking, combined with audio-based control commands, in order to recognize objects in an environment, including outdoor landmarks. The system uses a single camera to capture images, which are then processed to perform color segmentation, fingertip shape analysis, robust tracking, and invariant object recognition, in order to quickly identify the objects encircled and SNAPped by the user’s pointing gesture. In addition, the system returns an audio narration, TELLing the user information concerning the object’s classification, historical facts, usage, etc. This system provides enabling technology for the design of intelligent assistants to support “Web-On-The-World” applications, with potential uses such as travel assistance, business advertisement, the design of smart living and working spaces, and pervasive wireless services and internet vehicles. An erratum to this article can be found at An erratum to this article can be found at     

NeTra: A toolbox for navigating large image databases

We present here an implementation of NeTra, a prototype image retrieval system that uses color, texture, shape and spatial location information in segmented image regions to search and retrieve similar regions from the database. A distinguishing aspect of this system is its incorporation of a robust automated image segmentation algorithm that allows object- or region-based search. Image segmentation significantly improves the quality of image retrieval when images contain multiple complex objects. Images are segmented into homogeneous regions at the time of ingest into the database, and image attributes that represent each of these regions are computed. In addition to image segmentation, other important components of the system include an efficient color representation, and indexing of color, texture, and shape features for fast search and retrieval. This representation allows the user to compose interesting queries such as “retrieve all images that contain regions that have the color of object A, texture of object B, shape of object C, and lie in the upper of the image”, where the individual objects could be regions belonging to different images. A Java-based web implementation of NeTra is available at http://vivaldi.ece.ucsb.edu/Netra.     

Living with Database Conflicts: A Temporal Branching Technique

Heterogeneous as well as distributed databases make data conflicts inevitable. Both tolerate the entry of conflicting values in data objects, e.g., the value “female” may be entered by one user whereas another may put “male” in the same data object. Such inconsistencies in databases are common and are resolved routinely by built in structured mechanisms. Unresolved conflicts are typically quarantined till they can be resolved. But, there are situations for which there are no structured mechanisms to resolve conflicts while having to maintain application flow. Moreover, there are situations, i.e., medical or financial environments, in which conflicts must be stored so that their effects be analyzed for decision making. The current research proposes a model and a technique for living with database conflicts. The technique, named Temporal Branching, integrates and extends temporal oriented databases, temporal versioning, and log-file approaches and offers a solution and flexible structure readily accessible to retrieval and audits by standard DBMS software. A case study in health care delivery system is given to illustrate the problem and proposed solution. recommend Ahmed Elmagarmid     

The genesis of “The Visual Computer”

A brief history and the prospects of “The Visual Computer”and “The Visual Computer: An International Journal” are presented solely to foster future research on the visual computer. It is still in its infancy, and the author’s view is based on his own limited experiences, and hence is prone to mistakes.     

Exploiting linked data to create rich human digital memories

Memories are an important aspect of a person’s life and experiences. The area of human digital memories focuses on encapsulating this phenomenon, in a digital format, over a lifetime. Through the proliferation of ubiquitous devices, both people and the surrounding environment are generating a phenomenal amount of data. With all of this disjointed information available, successfully searching it and bringing it together, to form a human digital memory, is a challenge. This is especially true when a lifetime of data is being examined. Linked Data provides an ideal, and novel, solution for overcoming this challenge, where a variety of data sources can be drawn upon to capture detailed information surrounding a given event. Memories, created in this way, contain vivid structures and varied data sources, which emerge through the semantic clustering of content and other memories. This paper presents DigMem, a platform for creating human digital memories, based on device-specific services and the user’s current environment. In this way, information is semantically structured to create temporal “memory boxes” for human experiences. A working prototype has been successfully developed, which demonstrates the approach. In order to evaluate the applicability of the system a number of experiments have been undertaken. These have been successful in creating human digital memories and illustrating how a user can be monitored in both indoor and outdoor environments. Furthermore, the user’s heartbeat information is analysed to determine his or her heart rate. This has been achieved with the development of a QRS Complex detection algorithm and heart rate calculation method. These methods process collected electrocardiography (ECG) information to discern the heart rate of the user. This information is essential in illustrating how certain situations can make the user feel.     

SpaceGlue: linking spaces for adaptive service location

We describe a mechanism called SpaceGlue for adaptively locating services based on the preferences and locations of users in a distributed and dynamic network environment. In SpaceGlue, services are bound to physical locations, and a mobile user accesses local services depending on the current space he/she is visiting. SpaceGlue dynamically identifies the relationships between different spaces and links or “glues” spaces together depending on how previous users moved among them and used those services. Once spaces have been glued, users receive a recommendation of remote services (i.e., services provided in a remote space) reflecting the preferences of the crowd of users visiting the area. The strengths of bonds are implicitly evaluated by users and adjusted by the system on the basis of their evaluation. SpaceGlue is an alternative to existing schemes such as data mining and recommendation systems and it is suitable for distributed and dynamic environments. The bonding algorithm for SpaceGlue incrementally computes the relationships or “bonds” between different spaces in a distributed way. We implemented SpaceGlue using a distributed network application platform Ja-Net and evaluated it by simulation to show that it adaptively locates services reflecting trends in user preferences. By using “Mutual Information (MI)” and “F-measure” as measures to indicate the level of such trends and the accuracy of service recommendation, the simulation results showed that (1) in SpaceGlue, the F-measure increases depending on the level of MI (i.e., the more significant the trends, the greater the F-measure values), (2) SpaceGlue achives better precision and F-measure than “Flooding case (i.e., every service information is broadcast to everybody)” and “No glue case” by narrowing appropriate partners to send recommendations based on bonds, and (3) SpaceGlue achieves better F-measure with large number of spaces and users than other cases (i.e., “flooding” and “no glue”). Tomoko Itao is an alumna of NTT Network Innovation Laboratories     

Shared virtual worlds for education: the ExploreNet experiment

ExploreNet is an experimental environment for creating and delivering networked “virtual worlds.” This system's style of user interaction was inspired by the concept of a “habitat” as first articulated in the LucasFilm's Habitat system. Players enter and interact in a habitat via their animated alter egos, called “avatars.” Habitats may be created for many purposes, including social interaction, entertainment and education. Our focus has been to facilitate the creation of habitats in which virtual communities of learners and mentors interact. This paper presents details of the current ExploreNet system, including its user interface, the means it provides for creating complex behaviors, details of its implementation, the outcomes of several experiments using this system, and our plans for its natural migration to a World Wide Web-based system.     

Helen Keller Was Never in a Chinese Room

William Rapaport, in “How Helen Keller used syntactic semantics to escape from a Chinese Room,” (Rapaport 2006), argues that Helen Keller was in a sort of Chinese Room, and that her subsequent development of natural language fluency illustrates the flaws in Searle’s famous Chinese Room Argument and provides a method for developing computers that have genuine semantics (and intentionality). I contend that his argument fails. In setting the problem, Rapaport uses his own preferred definitions of semantics and syntax, but he does not translate Searle’s Chinese Room argument into that idiom before attacking it. Once the Chinese Room is translated into Rapaport’s idiom (in a manner that preserves the distinction between meaningful representations and uninterpreted symbols), I demonstrate how Rapaport’s argument fails to defeat the CRA. This failure brings a crucial element of the Chinese Room Argument to the fore: the person in the Chinese Room is prevented from connecting the Chinese symbols to his/her own meaningful experiences and memories. This issue must be addressed before any victory over the CRA is announced.