Supporting real-time supply chain decisions based on RFID data streams

While RFID technology has been widely praised for its ability to streamline supply chain processes, little attention has been given to its unique data capturing characteristics to support real-time decision making. Being able to efficiently perform complex real-time analysis on top of RFID event streams is a key challenge for modern applications. This provides management with a novel data analysis mechanism to allow better, tactical, on time, well-informed decisions. The two main issues in RFID data management (RFDM) concern expressibility (how to simply and concisely express stream queries) and performance (how to efficiently evaluate stream queries). In this paper we claim that a spreadsheet-like query model, where formulation is done in a column-wise fashion, can express intuitively a large class of useful and practical RFDM queries. We propose a simple SQL extension to do that and show how these queries can be evaluated efficiently. We finally discuss a prototype called COSTES (COntinuous SpreadsheeT-likE computations), which implements our SQL extensions and evaluation algorithms. Presentation takes place within the context of two representative RFID applications, namely shelf availability and in-store sales promotions.     

Complex RFID event processing

Advances of sensor and radio frequency identification (RFID) technology provide significant new power for humans to sense, understand and manage the world. RFID provides fast data collection with precise identification of objects with unique IDs without line of sight, thus it can be used for identifying, locating, tracking and monitoring physical objects. Despite these benefits, RFID poses many challenges for data processing and management: (i) RFID observations have implicit meanings, which have to be transformed and aggregated into semantic data represented in their data models; and (ii) RFID data are temporal, streaming, and in high volume, and have to be processed on the fly. Thus, a general RFID data processing framework is needed to automate the transformation of physical RFID observations into the virtual counterparts in the virtual world linked to business applications. In this paper, we take an event-oriented approach to process RFID data, by devising RFID application logic into complex events. We then formalize the specification and semantics of RFID events and rules. We discover that RFID events are highly temporal constrained, and include non-spontaneous events, and develop an RFID event detection engine that can effectively process complex RFID events. The declarative event-based approach greatly simplifies the work of RFID data processing, and can significantly reduce the cost of RFID data integration. This work was done by F. Wang while working at Siemens Corporate Research. This work was done by S. Liu while visiting Siemens Corporate Research.     

Processing count queries over event streams at multiple time granularities

Management and analysis of streaming data has become crucial with its applications to web, sensor data, network traffic data, and stock market. Data streams consist of mostly numeric data but what is more interesting are the events derived from the numerical data that need to be monitored. The events obtained from streaming data form event streams. Event streams have similar properties to data streams, i.e., they are seen only once in a fixed order as a continuous stream. Events appearing in the event stream have time stamps associated with them at a certain time granularity, such as second, minute, or hour. One type of frequently asked queries over event streams are count queries, i.e., the frequency of an event occurrence over time. Count queries can be answered over event streams easily, however, users may ask queries over different time granularities as well. For example, a broker may ask how many times a stock increased in the same time frame, where the time frames specified could be an hour, day, or both. Such types of queries are challenging especially in the case of event streams where only a window of an event stream is available at a certain time instead of the whole stream. In this paper, we propose a technique for predicting the frequencies of event occurrences in event streams at multiple time granularities. The proposed approximation method efficiently estimates the count of events with a high accuracy in an event stream at any time granularity by examining the distance distributions of event occurrences. The proposed method has been implemented and tested on different real data sets including daily price changes in two different stock exchange markets. The obtained results show its effectiveness.     

在线RFID多复杂事件查询处理技术

在线无线射频识别(radio frequency identification,RFID)数据流上的复杂事件处理技术是一个新的课题。现有研究工作仅是针对单一的复杂事件查询,没有考虑多复杂事件同时查询的处理策略。在复杂事件语言SASE(stream-based and shared event processing)的基础上设计了专门针对多查询的自动机及相关的优化技术,解决了RFID数据流上多复杂事件查询的问题。实验结果表明,算法在查询数量较大时,时间与空间上较传统算法有更好的表现。     

面向RFID数据流冰川查询的世系追踪模型

为了获得RFID数据流中热门元素以及相关起源的信息,需要对RFID数据流进行带起源信息的冰川查询。以RFID数据流中单个数据对象的世系追踪为研究对象,分析在海量RFID数据流基础上返回极少查询结果的冰山查询执行机理,初步建立一个面向RFID数据流冰山查询的数据流世系跟踪原型模型。     

基于连续聚类与组合分类的RFID事件探测

RFID数据流随着时间而不断变化,捕捉其中蕴含的变化可以用于检测有意义事件的发生.提出了一种捕获数据流事件的算法--CECD,通过分析聚类结果分布变化和值域中产生的偏差检测数据流中蕴含的变化,同时采用组合分类技术对变化进行分类,捕获观察到的事件或现象的特性,建立事件与响应的映射关系.实验证明提出的框架可以高效检测数据流上的变化,与不借助变化检测的单纯基于规则的事件检测方法相比可以更准确地捕获事件.     

A multivariate Bayesian scan statistic for early event detection and characterization

We present the multivariate Bayesian scan statistic (MBSS), a general framework for event detection and characterization in multivariate spatial time series data. MBSS integrates prior information and observations from multiple data streams in a principled Bayesian framework, computing the posterior probability of each type of event in each space-time region. MBSS learns a multivariate Gamma-Poisson model from historical data, and models the effects of each event type on each stream using expert knowledge or labeled training examples. We evaluate MBSS on various disease surveillance tasks, detecting and characterizing outbreaks injected into three streams of Pennsylvania medication sales data. We demonstrate that MBSS can be used both as a “general” event detector, with high detection power across a variety of event types, and a “specific” detector that incorporates prior knowledge of an event’s effects to achieve much higher detection power. MBSS has many other advantages over previous event detection approaches, including faster computation and easy interpretation and visualization of results, and allows faster and more accurate event detection by integrating information from the multiple streams. Most importantly, MBSS can model and differentiate between multiple event types, thus distinguishing between events requiring urgent responses and other, less relevant patterns in the data.     

Pattern-based event detection in sensor networks

Many applications of wireless sensor networks monitor the physical world and report events of interest. To facilitate event detection in these applications, in this paper we propose a pattern-based event detection approach and integrate the approach into an in-network sensor query processing framework. Different from existing threshold-based event detection, we abstract events into patterns in sensory data and convert the problem of event detection into a pattern matching problem. We focus on applying single-node temporal patterns, and define the general patterns as well as five types of basic patterns for event specification. Considering the limited storage on sensor nodes, we design an on-node cache manager to maintain the historical data required for pattern matching and develop event-driven processing techniques for queries in our framework. We have conducted experiments using patterns for events that are extracted from real-world datasets. The results demonstrate the effectiveness and efficiency of our approach.     

ArchIS: an XML-based approach to transaction-time temporal database systems

Effective support for temporal applications by database systems represents an important technical objective that is difficult to achieve since it requires an integrated solution for several problems, including (i) expressive temporal representations and data models, (ii) powerful languages for temporal queries and snapshot queries, (iii) indexing, clustering and query optimization techniques for managing temporal information efficiently, and (iv) architectures that bring together the different pieces of enabling technology into a robust system. In this paper, we present the ArchIS system that achieves these objectives by supporting a temporally grouped data model on top of RDBMS. ArchIS’ architecture uses (a) XML to support temporally grouped (virtual) representations of the database history, (b) XQuery to express powerful temporal queries on such views, (c) temporal clustering and indexing techniques for managing the actual historical data in a relational database, and (d) SQL/XML for executing the queries on the XML views as equivalent queries on the relational database. The performance studies presented in the paper show that ArchIS is quite effective at storing and retrieving under complex query conditions the transaction-time history of relational databases, and can also assure excellent storage efficiency by providing compression as an option. This approach achieves full-functionality transaction-time databases without requiring temporal extensions in XML or database standards, and provides critical support to emerging application areas such as RFID.     

JetStream: Enabling high throughput live event streaming on multi-site clouds

Scientific and commercial applications operate nowadays on tens of cloud datacenters around the globe, following similar patterns: they aggregate monitoring or sensor data, assess the QoS or run global data mining queries based on inter-site event stream processing. Enabling fast data transfers across geographically distributed sites allows such applications to manage the continuous streams of events in real time and quickly react to changes. However, traditional event processing engines often consider data resources as second-class citizens and support access to data only as a side-effect of computation (i.e. they are not concerned by the transfer of events from their source to the processing site). This is an efficient approach as long as the processing is executed in a single cluster where nodes are interconnected by low latency networks. In a distributed environment, consisting of multiple datacenters, with orders of magnitude differences in capabilities and connected by a WAN, this will undoubtedly lead to significant latency and performance variations. This is namely the challenge we address in this paper, by proposing JetStream, a high performance batch-based streaming middleware for efficient transfers of events between cloud datacenters. JetStream is able to self-adapt to the streaming conditions by modeling and monitoring a set of context parameters. It further aggregates the available bandwidth by enabling multi-route streaming across cloud sites, while at the same time optimizing resource utilization and increasing cost efficiency. The prototype was validated on tens of nodes from US and Europe datacenters of the Windows Azure cloud with synthetic benchmarks and a real-life application monitoring the ALICE experiment at CERN. The results show a 3× increase of the transfer rate using the adaptive multi-route streaming, compared to state of the art solutions.     

面向RFID数据处理的复杂事件模式匹配方法

RFID数据具有不确定性,复杂事件处理技术将RFID数据看作不同类型的事件,从事件流中检测符合特定匹配模式的复杂事件。概率事件流分为多项概率事件流和单项概率事件流;针对多项概率事件流,提出NFA-MMG模式匹配方法,亦即使用多个有向无环图结合自动机实现模式匹配。针对单项概率事件流,提出NFA-Tree模式匹配方法,亦即使用匹配树结合自动机实现模式匹配;并提出改进的NFA-Tree方法,即基于概率阈值进行过滤,提高结果过滤效率。实验结果验证了上述模式匹配方法的性能优势。     

RFID复杂事件处理关键技术的研究与改进

RFID复杂事件处理是一个新兴的技术领域,它用来处理大量的简单事件,并从中整理出有价值的事件。RFID事件和传统的事件相比较具有海量性、空间性和时间性、数据不准确性等特征。文中在分析RFID数据特点的基础上,对RFID复杂事件处理的关键技术进行研究和改进,主要介绍RFID数据的清洗和事件检测技术。对于RFID数据清洗部分,提出了多层次过滤的方法使得到的数据更接近真实情况,而事件检测方面则提出了局部检测和全局检测相结合的方法对相关数据进行检测以得到更有意义的数据供上层应用使用。最后,对RFID复杂事件处理的发展趋势做出展望。     

Exploiting Temporal Relations in Mining Hepatitis Data

Various data mining methods have been developed last few years for hepatitis study using a large temporal and relational database given to the research community. In this work we introduce a novel temporal abstraction method to this study by detecting and exploiting temporal patterns and relations between events in viral hepatitis such as “event A slightly happened before event B and B simultaneously ended with event C”. We developed algorithms to first detect significant temporal patterns in temporal sequences and then to identify temporal relations between these temporal patterns. Many findings by data mining methods applied to transactions/graphs of temporal relations shown to be significant by physician evaluation and matching with published in Medline.     

在线-离线数据流上复杂事件检测

随着数据采集和处理技术的发展,在物联网对象跟踪、网络监控、金融预测、电信消费模式等领域中进行事件检测显得越发重要.事件检测在一次扫描数据流的假设下完成,数据流在被处理完后丢弃.事实上,很多应用场景中,历史数据流因含有丰富的信息而不能简单丢弃,且一些事件检测查询需要同时在实时和历史数据流上进行.鉴于已有复杂事件检测很少考虑同时在实时-历史数据流上进行模式匹配,作者研究了在线-离线数据流上复杂事件检测的关键问题.主要工作如下:(1)针对滑动窗口内产生的大量模式匹配中间结果,提出利用时态关系和时空关系管理中间结果的方法 TPM和STPM.STPM以中间结果的时态和状态信息为权值对中间结果进行管理,将最近的、最有可能更新状态的中间结果置于内存,极大地减少了中间结果的读取操作代价.(2)给出了基于选择度的在线-离线复杂事件检测优化算法;(3)给出了算法的复杂性分析和代价模型;(4)在基于时空关系的中间结果管理模型下,在一个在线-离线复杂事件检测原型系统中进行实验,对多个参数(子窗口大小,选择度,匹配率,命中率)进行了算法对比分析.实验结果充分验证了所提出的算法的可行性和高效性.     

Rule-based composite event queries: the language XChangeEQ and its semantics

Web systems, Web services, and Web-based publish/subscribe systems communicate events as XML messages and in many cases, require composite event detection: it is not sufficient to react to single event messages, but events have to be considered in relation to other events that are received over time. This entails a need for expressive, high-level languages for querying composite events. Emphasizing language design and formal semantics, we describe the rule-based composite event query language XChange^EQ. XChange^EQ is designed to completely cover and integrate the four complementary querying dimensions: event data, event composition, temporal relationships, and event accumulation. Semantics are provided as a model theory with accompanying fixpoint theory, an approach that is established for rule languages but has not been applied to event queries so far. Because they are highly declarative, thus easy to understand and well suited for query optimization, such semantics are desirable for event queries.     

Foundations for Streaming Model Transformations by Complex Event Processing

Streaming model transformations represent a novel class of transformations to manipulate models whose elements are continuously produced or modified in high volume and with rapid rate of change. Executing streaming transformations requires efficient techniques to recognize activated transformation rules over a live model and a potentially infinite stream of events. In this paper, we propose foundations of streaming model transformations by innovatively integrating incremental model query, complex event processing (CEP) and reactive (event-driven) transformation techniques. Complex event processing allows to identify relevant patterns and sequences of events over an event stream. Our approach enables event streams to include model change events which are automatically and continuously populated by incremental model queries. Furthermore, a reactive rule engine carries out transformations on identified complex event patterns. We provide an integrated domain-specific language with precise semantics for capturing complex event patterns and streaming transformations together with an execution engine, all of which is now part of the Viatra reactive transformation framework. We demonstrate the feasibility of our approach with two case studies: one in an advanced model engineering workflow; and one in the context of on-the-fly gesture recognition.     

原始RFID数据流上复杂事件处理研究

一般的RFID复杂事件检测是建立在经过数据清洗的数据模型上,但RFID数据清洗往往代价较高且目的单一,更为影响效率的是其数据清洗步骤和复杂事件处理步骤需要扫描数据流两次.针对这些问题,提出直接在原始RFID数据流上进行复杂事件处理,将数据清洗步骤与复杂事件处理步骤相结合的方法,并设计出了集成此方法的复杂事件处理引擎架构,最后编程实现了上述架构的处理引擎.通过大量对比实验分析验证了该方法的正确性与高效性.     

RFID供应链系统中的在线复杂事件检测方法

在实际的供应链系统中,物品通常会被包装起来流通,检测最低包装层级物品的标签代价高昂。现有的在线和离线的无线射频识别(radio frequency identification,RFID)复杂事件检测方法中都假定可以检测到每一个最低包装层级的标签,不支持含有多种包装层级数据的RFID数据流上的复杂事件检测。根据部署有RFID的供应链系统产生的RFID数据流的特点,提出了一种新的复杂事件检测方法。采用区间编码离线保存物品的包装关系,通过在线数据和离线数据结合来完成复杂事件检测,对不同类型的复杂事件采用不同的检测策略以提高复杂事件检测效率。实验证明该方法能够有效地支持供应链系统中的复杂事件检测,并具有较好的性能。     

Temporal event searches based on event maps and relationships

To satisfy a user’s need to find and understand the whole picture of an event effectively and efficiently, in this paper we formalize the problem of temporal event searches and propose a framework of event relationship analysis for search events based on user queries. We define three kinds of event relationships: temporal, content dependence, and event reference, that can be used to identify to what extent a component event is dependent on another in the evolution of a target event (i.e., the query event). The search results are organized as a temporal event map (TEM) that serves as the whole picture about an event’s evolution or development by showing the dependence relationships among events. Based on the event relationships in the TEM, we further propose a method to measure the degrees of importance of events, so as to discover the important component events for a query, as well as the several algebraic operators involved in the TEM, that allow users to view the target event. Experiments conducted on a real data set show that our method outperforms the baseline method Event Evolution Graph (EEG), and it can help discover certain new relationships missed by previous methods and even by human annotators.     

Discovering multi-label temporal patterns in sequence databases

Sequential pattern mining is one of the most important data mining techniques. Previous research on mining sequential patterns discovered patterns from point-based event data, interval-based event data, and hybrid event data. In many real life applications, however, an event may involve many statuses; it might not occur only at one certain point in time or over a period of time. In this work, we propose a generalized representation of temporal events. We treat events as multi-label events with many statuses, and introduce an algorithm called MLTPM to discover multi-label temporal patterns from temporal databases. The experimental results show that the efficiency and scalability of the MLTPM algorithm are satisfactory. We also discuss interesting multi-label temporal patterns discovered when MLTPM was applied to historical Nasdaq data.