Remembering through lifelogging: A survey of human memory augmentation

Human memory is unquestionably a vital cognitive ability but one that can often be unreliable. External memory aids such as diaries, photos, alarms and calendars are often employed to assist in remembering important events in our past and future. The recent trend for lifelogging, continuously documenting ones life through wearable sensors and cameras, presents a clear opportunity to augment human memory beyond simple reminders and actually improve its capacity to remember. This article surveys work from the fields of computer science and psychology to understand the potential for such augmentation, the technologies necessary for realising this opportunity and to investigate what the possible benefits and ethical pitfalls of using such technology might be.     

Connectionism,classical cognitive science and experimental psychology

Classical symbolic computational models of cognition are at variance with the empirical findings in the cognitive psychology of memory and inference. Standard symbolic computers are well suited to remembering arbitrary lists of symbols and performing logical inferences. In contrast, human performance on such tasks is extremely limited. Standard models donot easily capture content addressable memory or context sensitive defeasible inference, which are natural and effortless for people. We argue that Connectionism provides a more natural framework in which to model this behaviour. In addition to capturing the gross human performance profile, Connectionist systems seem well suited to accounting for the systematic patterns of errors observed in the human data. We take these arguments to counter Fodor and Pylyshyn's (1988) recent claim that Connectionism is, in principle, irrelevant to psychology.     

Facebook communities about nostalgic photos of Turkey: creative practices of remembering and representing the past

ABSTRACT

This article focuses on Facebook communities about nostalgic photos of Turkey to explore how citizenship is enacted through the participatory and collaborative use of social media to remember and represent the past. By sharing their personal photos, knowledge, testimonies, narratives and life stories, members of these communities actively and creatively use social media to generate new ways of remembering and representing the past, as well as improving its accessibility and visibility. Furthermore, through exchanging affectively and politically charged photos and conversations about the past, participants fashion nostalgia as a public feeling that becomes a source for affective political criticism of the present. This article addresses the participatory and collaborative creation of knowledge and memory of the past to discuss everyday creative citizenship practices facilitated by social media.     

The end of the information frontier

The possibility now exists of capturing a cradle-to-grave record of everything a person says or does. No longer must a personal history be a partial picture. Technology has made it possible to record, process, store, and retrieve all the text, sounds, and images that are required to paint a complete picture of an individual’s life. The efforts of future historians will be directed more to forgetting than to remembering. By default, society will forget nothing. For almost all of human history, remembering has meant the judicious selection and organization of observations about events and people. There used to be an information frontier beyond which the past was a tabula rasa. That information frontier has gone the way of the dodo. The social memory of events in an individual’s life is not only detailed but permanent. Although physical storage is fallible and changes in technology may make some devices effectively unreadable, these limitations are more than made up for by the negligible cost of duplication and distribution in a network. The record of one’s triumphs and tragedies will haunt one forever. Gone is personal privacy since facts buried in the past can be uncovered at any moment. Gone is personal memory since it is easier to rely on the external social memory of cyberspace. In what follows, we explain these observations and trace their consequences.     

Memory-based cognitive modeling for robust object extraction and tracking

Inspired by the way humans perceive the environment, in this paper, we propose a memory-based cognitive model for visual information processing which can imitate some cognitive functions of the human brain such as remembering, recall, forgetting, learning, classification, and recognition, etc. The proposed model includes five components: information granule, memory spaces, cognitive behaviors, rules for manipulating information among memory spaces, and decision-making processes. Three memory spaces are defined for separately storing the current, temporal and permanent information acquired, i.e. ultra short-term memory space (USTMS), short-term memory space (STMS) and long-term memory space (LTMS). The proposed memory-based model can remember or forget what the scene has ever been which helps the model adapt to the variation of the scene more quickly. We apply the model to address two hot issues in computer vision: background modeling and object tracking. A memory-based Gaussian mixture model (MGMM) for object segmentation and a memory-based template updating (MTU) model for object tracking with particle filter (PF) are exhibited respectively. Experimental results show that the proposed model can deal with scenes with sudden background and foreground changes more robustly when segmenting and tracking moving objects under complex background.     

Towards self-protecting ubiquitous systems: monitoring trust-based interactions

The requirement for spontaneous interaction in ubiquitous computing creates security issues over and above those present in other areas of computing, deeming traditional approaches ineffective. As a result, to support secure collaborations entities must implement self-protective measures. Trust management is a solution well suited to this task as reasoning about future interactions is based on the outcome of past ones. This requires monitoring of interactions as they take place. Such monitoring also allows us to take corrective action when interactions are proceeding unsatisfactorily. In this vein, we first present a trust-based model of interaction based on event structures. We then describe our ongoing work in the development of a monitor architecture which enables self-protective actions to be carried out at critical points during principal interaction. Finally, we discuss some potential directions for future work.     

The Compatibility of Complex Systems and Reduction: A Case Analysis of Memory Research

Some theorists who emphasize the complexity of biological and cognitive systems and who advocate the employment of the tools of dynamical systems theory in explaining them construe complexity and reduction as exclusive alternatives. This paper argues that reduction, an approach to explanation that decomposes complex activities and localizes the components within the complex system, is not only compatible with an emphasis on complexity, but provides the foundation for dynamical analysis. Explanation via decomposition and localization is nonetheless extremely challenging, and an analysis of recent cognitive neuroscience research on memory is used to illustrate what is involved. Memory researchers split between advocating memory systems and advocating memory processes, and I argue that it is the latter approach that provides the critical sort of decomposition and localization for explaining memory. The challenges of linking distinguishable functions with brain processes is illustrated by two examples: competing hypotheses about the contribution of the hippocampus and competing attempts to link areas in frontal cortex with memory processing.     

Cognitive presence in asynchronous online learning: a comparison of four discussion strategies

Some scholars argue that students do not achieve higher level learning, or cognitive presence, in online courses. Online discussion has been proposed to bridge this gap between online and face‐to‐face learning environments. However, the literature indicates that the conventional approach to online discussion – asking probing questions – does not necessarily advance the discussion through the phases of cognitive presence: triggering events, exploration, integration and resolution, which are crucial for deep knowledge construction. Using mixed methods, we examined the contribution of four scenario‐based online discussion strategies – structured, scaffolded, debate and role play – to the learners' cognitive presence, the outcome of the discussion. Learners' discussion postings within each strategy were segmented and categorized according to the four phases. The discussion strategies, each using the same authentic scenario, were then compared in terms of the number of segments representing these phases. We found that the structured strategy, while highly associated with triggering events, produced no discussion pertaining to the resolution phase. The scaffolded strategy, on the other hand, showed a strong association with the resolution phase. The debate and role‐play strategies were highly associated with exploration and integration phases. We concluded that discussion strategies requiring learners to take a perspective in an authentic scenario facilitate cognitive presence, and thus critical thinking and higher levels of learning. We suggest a heuristic for sequencing a series of discussion forums and recommend areas for further related research.     

A high-level architecture for believable social agents

The creation of virtual humans capable of behaving and interacting realistically with each other requires the development of autonomous believable social agents. Standard goal-oriented approaches are not well suited to it because they don't take into account important characteristics identified by the social sciences. This paper tackles the issue of a general social reasoning mechanism, discussing its basic functional requirements using a sociological perspective, and proposing a high-level architecture based on Roles, Norms, Values and Types.     

Epistemology's search for significance

Epistemology is supposed to guide our reason. This means that epistemology is practically important because our beliefs so often play a decisive role in how we decide to act. We argue that in order to be effectively action-guiding, traditional epistemological theories must be supplemented with an account of significance—of what makes one reasoning problem significant and another reasoning problem insignificant. We sketch the beginning of an account of significance that employs Signal Detection Theory. We then argue that any internalist theory of justification is not going to fit smoothly with a plausible account of significance. This calls into question the power of internalist theories of justification to effectively guide reason and action.     

Informing augmented memory system design through autobiographical memory theory

Autobiographical memory (AM) is the “memory for the events in one’s life” [1]. Often it is assumed that in order to remember all those events, you just need to record everything and when you replay these recordings you will remember those events. You can compare this with a library metaphor that has been used to explain AM according to the record-keeping approach. However, after many years of AM-research it was concluded that AM is stored in a different manner, namely according to the constructionist approach, which often is initiated by memory cues. This paper explains these AM theories, surveys literature on existing augmented memory systems and describes our own work in this area. All this input is combined into eight design recommendations for future augmented memory systems.

The architecture of an event correlation service for adaptive middleware-based applications

Loosely coupled component communication driven by events is a key mechanism for building middleware-based applications that must achieve reliable qualities of service in an adaptive manner. In such a system, events that encapsulate state snapshots of a running system are generated by monitoring components. Hence, an event correlation service is necessary for correlating monitored events from multiple sources. The requirements for the event correlation raise two challenges: to seamlessly integrate event correlation services with other services and applications; and to provide reliable event management with minimal delay. This paper describes our experience in the design and implementation of an event correlation service. The design encompasses an event correlator and an event proxy that are integrated with an architecture for adaptive middleware components. The implementation utilizes the common-based event (CBE) specification and stateful Web service technologies to support the deployment of the event correlation service in a distributed architecture. We evaluate the performance of the overall solution in a test bed and present the results in terms of the trade-off between the flexibility and the performance overhead of the architecture.     

Specifying and enforcing high-level semantic obligation policies

Obligation policies specify management actions that must be performed when a particular kind of event occurs and certain conditions are satisfied. Large scale distributed systems often produce event streams containing large volumes of low-level events. In many cases, these streams also contain multimedia data (consisting of text, audio or video). Hence, a key challenge is to allow policy writers to specify obligation policies based on high-level events, that may be derived after performing appropriate processing on raw, low-level events. In this paper, we propose a semantic obligation policy specification language called Eagle, which is based on patterns of high-level events, represented as RDF graph patterns. Our policy enforcement architecture uses a compiler that builds a workflow for producing a stream of events, which match the high-level event pattern specified in a policy. This workflow consists of a number of event sources and event processing components, which are described semantically. We present the policy language and enforcement architecture in this paper.     

Long-term knowledge acquisition using contextual information in a memory-inspired robot architecture

In this paper, we present a novel cognitive framework allowing a robot to form memories of relevant traits of its perceptions and to recall them when necessary. The framework is based on two main principles: on the one hand, we propose an architecture inspired by current knowledge in human memory organisation; on the other hand, we integrate such an architecture with the notion of context, which is used to modulate the knowledge acquisition process when consolidating memories and forming new ones, as well as with the notion of familiarity, which is employed to retrieve proper memories given relevant cues. Although much research has been carried out, which exploits Machine Learning approaches to provide robots with internal models of their environment (including objects and occurring events therein), we argue that such approaches may not be the right direction to follow if a long-term, continuous knowledge acquisition is to be achieved. As a case study scenario, we focus on both robot–environment and human–robot interaction processes. In case of robot–environment interaction, a robot performs pick and place movements using the objects in the workspace, at the same time observing their displacement on a table in front of it, and progressively forms memories defined as relevant cues (e.g. colour, shape or relative position) in a context-aware fashion. As far as human–robot interaction is concerned, the robot can recall specific snapshots representing past events using both sensory information and contextual cues upon request by humans.     

A Cognitive Memory-Augmented Network for Visual Anomaly Detection

With the rapid development of automated visual analysis, visual analysis systems have become a popular research topic in the field of computer vision and automated analysis. Visual analysis systems can assist humans to detect anomalous events (e.g., fighting, walking alone on the grass, etc). In general, the existing methods for visual anomaly detection are usually based on an autoencoder architecture, i.e., reconstructing the current frame or predicting the future frame. Then, the reconstruction error is adopted as the evaluation metric to identify whether an input is abnormal or not. The flaws of the existing methods are that abnormal samples can also be reconstructed well. In this paper, inspired by the human memory ability, we propose a novel deep neural network (DNN) based model termed cognitive memory-augmented network (CMAN) for the visual anomaly detection problem. The proposed CMAN model assumes that the visual analysis system imitates humans to remember normal samples and then distinguishes abnormal events from the collected videos. Specifically, in the proposed CMAN model, we introduce a memory module that is able to simulate the memory capacity of humans and a density estimation network that can learn the data distribution. The reconstruction errors and the novelty scores are used to distinguish abnormal events from videos. In addition, we develop a two-step scheme to train the proposed model so that the proposed memory module and the density estimation network can cooperate to improve performance. Comprehensive experiments evaluated on various popular benchmarks show the superiority and effectiveness of the proposed CMAN model for visual anomaly detection comparing with the state-of-the-arts methods. The implementation code of our CMAN method can be accessed at https://github.com/CMAN-code/CMAN_pytorch.      

Prospective memory in the ICU: the effect of visual cues on task execution in a representative simulation

Despite the potential dangers of clinical tasks being forgotten, few researchers have investigated prospective memory (PM) – the ability to remember to execute future tasks – in health-care contexts. Visual cues help people remember to execute intentions at the appropriate moment. Using an intensive care unit simulator, we investigated whether nurses' memory for future tasks improves when visual cues are present, and how nurses manage PM demands. Twenty-four nurses participated in a 40-minute scenario simulating the start of a morning shift. The scenario included eight PM tasks. The presence or absence of a visually conspicuous cue for each task was manipulated. The presence of a visual cue improved recall compared to no cue (64% vs. 50%, p = 0.03 one-tailed, η _p ² = 0.15). Nurses used deliberate reminders to manage their PM demands. PM in critical care might be supported by increasing the visibility of cues related to tasks.

Practitioner summary: Nurses must remember to execute multiple future tasks to ensure patient safety. We investigated the effect of visual cues on nurses' ability to remember future tasks. Experimental manipulation of cues in a representative intensive care unit simulation indicated that visual cues increase the likelihood that future tasks are executed.     

Equipping the ACT-R cognitive architecture with a temporal ratio model of memory and using it in a new intelligent adaptive interface

ACT-R, as a useful and well-known cognitive architecture, is a theory for simulating and understanding human cognition. However, the standard version of this architecture uses a deprecated forgetting model. So, we equipped it with a temporal ratio model of memory that has been named as SIMPLE (Scale-Independent Memory, Perception, and Learning). On the other hand, one of the usages of cognitive architectures is to model the user in an Intelligent Adaptive Interface (IAI) implementation. Thus, our motivation for this effort is to use this equipped ACT-R in an IAI to deliver the right information at the right time to users based on their cognitive needs. So, to test our proposed equipped ACT-R, we designed and implemented a new IAI to control a swarm of Unmanned Aerial Vehicles (UAVs). This IAI uses the equipped ACT-R for user cognitive modeling, to deliver the right information to the users based on their forgetting model. Thus, our contributions are: equipping the ACT-R cognitive architecture with the SIMPLE memory model and using this equipped version of ACT-R for user modeling in a new IAI to control a group of UAVs. Simulation results, which have been obtained using different subjective and objective measures, show that we significantly improved situation awareness of the users using the IAI empowered by our equipped ACT-R.

     

Behavioral task processing for cognitive robots using artificial emotions

This paper presents an artificial emotional-cognitive system-based autonomous robot control architecture for a four-wheel driven and four-wheel steered mobile robot. Discrete stochastic state-space mathematical model is considered for behavioral and emotional transition processes of the autonomous mobile robot in the dynamic realistic environment. The term of cognitive mechanism system which is composed from rule base and reinforcement self-learning algorithm explain all of the deliberative events such as learning, reasoning and memory (rule spaces) of the autonomous mobile robot. The artificial cognitive model of autonomous robot control architecture has a dynamic associative memory including behavioral transition rules which are able to be learned for achieving multi-objective robot tasks. Motivation module of architecture has been considered as behavioral gain effect generator for achieving multi-objective robot tasks. According to emotional and behavioral state transition probabilities, artificial emotions determine sequences of behaviors for long-term action planning. Also reinforcement self-learning and reasoning ability of artificial cognitive model and motivational gain effects of proposed architecture can be observed on the executing behavioral sequences during simulation. The posture and speed of the robot and the configurations, speeds and torques of the wheels and all deliberative and cognitive events can be observed from the simulation plant and virtual reality viewer. This study constitutes basis for the multi-goal robot tasks and artificial emotions and cognitive mechanism-based behavior generation experiments on a real mobile robot.     

Event recognition hardware based on neuron MOS soft-computing circuits

Neuron MOS transistor (neuMOS or νMOS) mimicking the fundamental behavior of neurons at a primitive device level allows the implementation of intelligent functions directly on the integrated circuit hardware. Based on the νMOS technology a real-time event recognition system has been developed. A neuron MOS association processor searches for the event in the past memory having the maximum similarity to the current event presented to the system. This is based on Manhattan-distance calculation and the minimum distance search by a winner-take-all (WTA) circuitry. The computation is carried out directly on the hardware in a fully parallel architecture. A unique floating-gate analog EEPROM technology has been developed to build a vast memory system storing past events in multi-valued vectors. Test circuits of key subsystems were fabricated by a double-polysilicon CMOS process and their operation was verified by measurements as well as by simulation. The νMOS circuit operation is characterized by analog computation directly conducted on an electrically floating node which is immediately followed by the thresholding action of a transistor to yield a binary decision. νMOS circuits would provide an opportunity for a very flexible soft computing scheme, while preserving the rigorous nature of digital processing.