共查询到20条相似文献,搜索用时 0 毫秒
1.
Moustafa A. Hammad Walid G. Aref Ahmed K. Elmagarmid 《The VLDB Journal The International Journal on Very Large Data Bases》2008,17(3):469-488
This paper introduces a class of join algorithms, termed W-join, for joining multiple infinite data streams. W-join addresses
the infinite nature of the data streams by joining stream data items that lie within a sliding window and that match a certain
join condition. In addition to its general applicability in stream query processing, W-join can be used to track the motion
of a moving object or detect the propagation of clouds of hazardous material or pollution spills over time in a sensor network
environment. We describe two new algorithms for W-join and address variations and local/global optimizations related to specifying
the nature of the window constraints to fulfill the posed queries. The performance of the proposed algorithms is studied experimentally
in a prototype stream database system, using synthetic data streams and real time-series data. Tradeoffs of the proposed algorithms
and their advantages and disadvantages are highlighted, given variations in the aggregate arrival rates of the input data
streams and the desired response times per query.
This is an extended version of the paper published in the Proceedings of the 15th International Conference on Scientific and
Statistical Database Management, SSDBM 2003, Boston, U.S.A., pp. 75–84. 相似文献
2.
We present an adaptive load shedding approach for windowed stream joins. In contrast to the conventional approach of dropping
tuples from the input streams, we explore the concept ofselective processing for load shedding. We allow stream tuples to be stored in the windows and shed excessive CPU load by performing the join
operations, not on the entire set of tuples within the windows, but on a dynamically changing subset of tuples that are learned
to be highly beneficial. We support such dynamic selective processing through three forms of runtimeadaptations: adaptation to input stream rates, adaptation to time correlation between the streams and adaptation to join directions.
Our load shedding approach enables us to integrateutility-based load shedding withtime correlation-based load shedding. Indexes are used to further speed up the execution of stream joins. Experiments are conducted to evaluate
our adaptive load shedding in terms of output rate and utility. The results show that our selective processing approach to
load shedding is very effective and significantly outperforms the approach that drops tuples from the input streams.
Bugra Gedik received the B.S. degree in C.S. from the Bilkent University, Ankara, Turkey, and the Ph.D. degree in C.S. from the College
of Computing at the Georgia Institute of Technology, Atlanta, GA, USA. He is with the IBM Thomas J. Watson Research Center,
currently a member of the Software Tools and Techniques Group. Dr. Gedik's research interests lie in data intensive distributed
computing systems, spanning data-centric peer-to-peer overlay networks, mobile and sensor-based distributed data management
systems, and distributed data stream processing systems. His research focus is on developing system-level architectures and
techniques to address scalability problems in distributed continual query systems and applications. He is the recipient of
the ICDCS 2003 best paper award. He has served in the program committees of several international conferences, such as ICDE,
MDM, and CollaborateCom.
Kun-Lung Wu received the B.S. degree in E.E. from the National Taiwan University, Taipei, Taiwan, the M.S. and Ph.D. degrees in C.S.
both from the University of Illinois at Urbana-Champaign. He is with the IBM Thomas J. Watson Research Center, currently a
member of the Software Tools and Techniques Group. His recent research interests include data streams, continual queries,
mobile computing, Internet technologies and applications, database systems and distributed computing. He has published extensively
and holds many patents in these areas.
Dr. Wu is a Senior Member of the IEEE Computer Society and a member of the ACM. He is the Program Co-Chair for the IEEE Joint
Conference on e-Commerce Technology (CEC 2007) and Enterprise Computing, e-Commerce and e-Services (EEE 2007). He was an Associate
Editor for the IEEE Trans. on Knowledge and Data Engineering, 2000–2004. He was the general chair for the 3rd International
Workshop on E-Commerce and Web-Based Information Systems (WECWIS 2001). He has served as an organizing and program committee
member on various conferences. He has received various IBM awards, including IBM Corporate Environmental Affair Excellence
Award, Research Division Award, and several Invention Achievement Awards. He received a best paper award from IEEE EEE 2004.
He is an IBM Master Inventor.
Philip S. Yu received the B.S. Degree in E.E. from National Taiwan University, the M.S. and Ph.D. degrees in E.E. from Stanford University,
and the M.B.A. degree from New York University. He is with the IBM Thomas J. Watson Research Center and currently manager
of the Software Tools and Techniques group. His research interests include data mining, Internet applications and technologies,
database systems, multimedia systems, parallel and distributed processing, and performance modeling. Dr. Yu has published
more than 430 papers in refereed journals and conferences. He holds or has applied for more than 250 US patents.
Dr. Yu is a Fellow of the ACM and a Fellow of the IEEE. He is associate editors of ACM Transactions on the Internet Technology
and ACM Transactions on Knowledge Discovery in Data. He is a member of the IEEE Data Engineering steering committee and is
also on the steering committee of IEEE Conference on Data Mining. He was the Editor-in-Chief of IEEE Transactions on Knowledge
and Data Engineering (2001–2004), an editor, advisory board member and also a guest co-editor of the special issue on mining
of databases. He had also served as an associate editor of Knowledge and Information Systems. In addition to serving as program
committee member on various conferences, he will be serving as the general chair of 2006 ACM Conference on Information and
Knowledge Management and the program chair of the 2006 joint conferences of the 8th IEEE Conference on E-Commerce Technology
(CEC' 06) and the 3rd IEEE Conference on Enterprise Computing, E-Commerce and E-Services (EEE' 06). He was the program chair
or co-chairs of the 11th IEEE Intl. Conference on Data Engineering, the 6th Pacific Area Conference on Knowledge Discovery
and Data Mining, the 9th ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery, the 2nd IEEE Intl.
Workshop on Research Issues on Data Engineering: Transaction and Query Processing, the PAKDD Workshop on Knowledge Discovery
from Advanced Databases, and the 2nd IEEE Intl. Workshop on Advanced Issues of E-Commerce and Web-based Information Systems.
He served as the general chair of the 14th IEEE Intl. Conference on Data Engineering and the general co-chair of the 2nd IEEE
Intl. Conference on Data Mining. He has received several IBM honors including 2 IBM Outstanding Innovation Awards, an Outstanding
Technical Achievement Award, 2 Research Division Awards and the 84th plateau of Invention Achievement Awards. He received
an Outstanding Contributions Award from IEEE Intl. Conference on Data Mining in 2003 and also an IEEE Region 1 Award for “promoting
and perpetuating numerous new electrical engineering concepts” in 1999. Dr. Yu is an IBM Master Inventor.
Ling Liu is an associate professor at the College of Computing at Georgia Tech. There, she directs the research programs in Distributed
Data Intensive Systems Lab (DiSL), examining research issues and technical challenges in building large scale distributed
computing systems that can grow without limits. Dr. Liu and the DiSL research group have been working on various aspects of
distributed data intensive systems, ranging from decentralized overlay networks, exemplified by peer to peer computing, data
grid computing, to mobile computing systems and location based services, sensor network computing, and enterprise computing
systems. She has published over 150 international journal and conference articles. Her research group has produced a number
of software systems that are either open sources or directly accessible online, among which the most popular ones are WebCQ
and XWRAPElite. Dr. Liu is currently on the editorial board of several international journals, including IEEE Transactions
on Knowledge and Data Engineering, International Journal of Very large Database systems (VLDBJ), International Journal of
Web Services Research, and has chaired a number of conferences as a PC chair, a vice PC chair, or a general chair, including
IEEE International Conference on Data Engineering (ICDE 2004, ICDE 2006, ICDE 2007), IEEE International Conference on Distributed
Computing (ICDCS 2006), IEEE International Conference on Web Services (ICWS 2004). She is a recipient of IBM Faculty Award
(2003, 2006). Dr. Liu's current research is partly sponsored by grants from NSF CISE CSR, ITR, CyberTrust, a grant from AFOSR,
an IBM SUR grant, and an IBM faculty award. 相似文献
3.
4.
Recent advances in networking and information technology boost the development of new and advanced services offered over communication systems that integrate a widely heterogeneous mix of applications and computer devices. Without careful traffic control and resource management, the implied dramatic increase in the demand for networking resources and remote application services may lead to substantial degradation of the Quality of Service as experienced by the end users. In this paper, we consider the problem of joint admission control and dynamic resource allocation in a stream processing network so as to optimize the overall system utility. With a primal-dual-based optimization approach, we show that the resource allocation problem and the admission control problem can be decomposed. We then present a distributed algorithm which incorporates a push-and-pull-based admission control mechanism, and a pressure-based cμ rule for resource allocation. We show that the algorithm guarantees the stability of the network and converges to the optimal solution. Various numerical experiments are then presented to demonstrate the quality of the solution and the speed of convergence. 相似文献
5.
以纺织企业中一类企业面临的实际问题为背景,为帮助企业提升管理水平和提高核心竞争力,设计了一个以订单流为主线的进销存系统。此系统具有平台无关、延展性性好、柔韧性强等特点。从系统体系结构、功能和关键技术几个方面详细介绍了此系统的设计方案,并给出了以模版开发的实现方法。应用结果数据显示此系统有着较好的应用效果。 相似文献
6.
7.
字符串相似连接是指在字符串集合中找出相似的字符串对,是许多应用的关键操作,寻找高效的字符串相似连接算法已成为研究热点。基于划分的过滤-验证方法(Pass-Join)与其他方法相比具有较高的效率。它按照字符串长度递增的顺序访问字符串集合,通过查找一个字符串的划分块是否存在于另一个字符串中,快速筛选出可能相似的字符串对(候选集),然后利用编辑距离进行相似性验证。研究发现,按照字符串长度递减的顺序进行过滤(长度递减过滤)的效果优于按照长度递增的顺序过滤(长度递增过滤)的效果,基于此,提出双向过滤-验证机制:在过滤阶段对长度递减过滤的结果再进行一次长度递增过滤,进一步减小候选集大小;在验证阶段利用双向过滤产生的两对划分块和其匹配子串分隔字符串对,从而减小需要验证的字符串的长度,加速验证过程。实验证明,双向过滤-验证算法在真实数据集上优于原算法。 相似文献
8.
9.
滑动窗口聚集查询在数据流管理系统中应用广泛,数据流到达高峰期,必须考虑滑动窗口聚集查询中出现的降载问题。分析了子集模型的特点和已有降载策略的不足,给出了数据流滑动窗口聚集查询降载问题的约束条件,提出了能保证子集结果产生的基于丢弃窗口更新策略的降载算法。理论分析和实验结果表明,该算法对数据流滑动窗口聚集查询降载问题的处理具有较高的有效性和实用性。 相似文献
10.
针对间歇过程批次与批次之间,操作条件缓慢变化的特性,提出一种基于自适应多向独立成分分析(MICA)的监控算法。该方法首先用MICA法建模,然后在历史数据集中加入新的正常批次并剔除最早批次,逐渐更新模型,同时引入遗忘因子,提高对新过程特性的适应性。青霉素发酵过程的仿真结果表明,自适应MICA比MICA更准确地描述过程行为,并有效减少检测故障时的误报。 相似文献
11.
12.
13.
由于发酵过程中,系统的非线性特性与发酵阶段密切相关,而应用常规MPLS时的静态单一模型、模型失配、不能充分有效压缩和抽取非线性信息、估计未来测量变量引入模型误差等问题,提出1种多阶段MPLS法.先采用ISODATA动态分类算法聚类分析过程数据,划分子阶段,针对各批次子阶段不等长的特点,再用DTW算法同步阶段轨迹,然后在... 相似文献
14.
15.
基于时变向量场的多无人机编队集结控制方法 总被引:1,自引:0,他引:1
编队集结是无人机编队飞行的必要条件.固定翼无人机由于其速度可变范围有限,难以通过单一的速度调整的方式构成编队.本文提出了一种以路径调节为主、速度调节为辅的无人机编队集结策略,根据各无人机到达集结位置的最短预估时间确定集结时间.并提出一种时变向量场的构造方式,将水平机动的思想与时变向量场相结合,根据无人机的飞行状况通过时变向量场实时调整路径长度,同时调整自身速度,使各无人机都能在集结时间到达集结位置,且速度和航向角保持一致,从而实现编队集结.随后证明了系统的收敛性,并分析了实际情况中避撞问题的处理.论文用某型无人机的大偏差非线性数学模型对所提的编队集结方法进行了验证,证明了所提方法具有较好的实用性. 相似文献
16.
17.
以线性等距无线传感器阵列为例,提出一种有效的到达方向检测算法.列堆栈两个平移不变子阵的相关矩阵,给出一种奇异值分解和特征值分解相结合的两步算法,估计传感器阵列的导向矢量矩阵,通过分析导向矢量矩阵的结构化信息,构造估计导向矢量和理想导向矢量的相关函数,进而求解相关函数的驻点,搜索有限个驻点中使相关函数最大的驻点对应的角度估计到达方向,避免了穷尽搜索.仿真结果表明:所提算法在相同信噪比下分辨成功率高于著名的ESPRIT算法、同一分辨成功率下要求的信噪比更低.在信噪比、快拍数、阵元个数变化下对目标定位的均方根误差均优于ESPRIT算法,更接近于理论最优值. 相似文献
18.
《Engineering Applications of Artificial Intelligence》2005,18(5):633-642
The concept of Receding Horizon Control (RHC) is introduced into Genetic Algorithm (GA) in this paper to solve the problem of arrival scheduling and sequencing (ASS) at a busy hub airport. A GA-based method is proposed for solving the dynamic ASS problem, and the focus is put on the methodology of integrating the RHC strategy into the GA for real-time implementations in a dynamic environment of air traffic control. Receding horizon and terminal penalty are investigated in depth as two key techniques of this novel RHC-based GA. Simulation results show that the new method proposed in this paper is effective and efficient to solve the ASS problem in a dynamic environment. 相似文献
19.
基于视频具有数据量大、实时性要求高等特点,在一维耦合映像格子模型的基础上,对耦合格点结构和动力学模型这两个方面进行了改进,在系统结构上采用一维链加二维网格再加上一维链的组合模型来构造驱动和响应系统及多级加密的方案.实验结果表明,系统在增强密钥序列复杂度的同时也提高了加密效率,使得系统的抗破译能力大大加强. 相似文献
20.
基于MSP430单片机的远程束流诊断系统编程 总被引:5,自引:5,他引:0
通过RS-485总线和Intranet网络,来实现对柬流的远程测量与控制,并给出软件的C程序实现和部分硬件功能模块的程序流程图。系统功能模块VAC401采用TI公司的MSP430F169混合信号处理器,具有超低功耗和高集成度等优点。利用它构建的控制网络功能强大,结构简单,可靠性高,抗干扰能力强,一般不需扩展外围器件。本来流诊断系统能够完成对加速器中柬流的远距离测量与控制。 相似文献