一种分布式Twitter数据处理方案及应用

针对社交媒体数据的特点及其分析的挑战性,提出了一种基于实时计算框架Storm、批处理框架Hadoop和高效可水平扩展的NoSQL数据库MongoDB的分布式社交媒体数据处理方案,并依此指导实现基于Twitter流式数据的流感疫情可视化分析系统.实验证明,该分布式方案能较好支持Twitter流式数据的高效处理和储存,使之满足系统的性能需求.     

The Weka4WS framework for distributed data mining in service‐oriented Grids

The service‐oriented architecture paradigm can be exploited for the implementation of data and knowledge‐based applications in distributed environments. The Web services resource framework (WSRF) has recently emerged as the standard for the implementation of Grid services and applications. WSRF can be exploited for developing high‐level services for distributed data mining applications. This paper describes Weka4WS, a framework that extends the widely used open source Weka toolkit to support distributed data mining on WSRF‐enabled Grids. Weka4WS adopts the WSRF technology for running remote data mining algorithms and managing distributed computations. The Weka4WS user interface supports the execution of both local and remote data mining tasks. On every computing node, a WSRF‐compliant Web service is used to expose all the data mining algorithms provided by the Weka library. The paper describes the design and implementation of Weka4WS using the WSRF libraries and services provided by Globus Toolkit 4. A performance analysis of Weka4WS for executing distributed data mining tasks in different network scenarios is presented. Copyright © 2008 John Wiley & Sons, Ltd.     

开源软件自动化评估证据框架

互联网上已形成了规模巨大、种类丰富的开源软件资源。如何准确、快速地判断一个开源项目的各种可信属性是否满足需求是当前软件工程领域研究的热点。深入分析已有开源软件评估模型,总结互联网上软件质量相关的各种信息,提出了面向开源软件的可信评估证据框架,并基于该框架构建了一种开源软件可信证据查询平台。利用该平台能够极大地提高评估效率,用户可以准确、快速、全面地了解相关软件项目的各种信息。最后,以一个知名开源软件证实了该证据框架及证据查询平台的可行性。     

Bounded model checking of C++ programs based on the Qt cross‐platform framework

The software development process for embedded systems is getting faster and faster, which generally incurs an increase in the associated complexity. As a consequence, technology companies tend to invest in fast and automatic verification mechanisms, to create robust systems and reduce product recall rates. In addition, further development‐time reduction and system robustness can be achieved through cross‐platform frameworks, such as Qt, which favor the reliable port of software stacks to different devices. Based on that, the present paper proposes a simplified version of the Qt framework, which is integrated into a checker based on satisfiability modulo theories (SMT), known as the Efficient SMT‐based Context‐Bounded Model Checker, for verifying actual Qt‐based applications, with a success rate of 89%, for the developed benchmark suite. Furthermore, the simplified version of the Qt framework, named as Qt Operational Model, was also evaluated using other state‐of‐the‐art verifiers for C++ programs. In fact, Qt Operational Model was combined with 2 different verification approaches: explicit‐state model checking and also symbolic (bounded) model checking, during the experimental evaluation, which highlights its flexibility. The proposed methodology is the first one to formally verify Qt‐based applications, which has the potential to devise new directions for software verification of portable code.     

Implementing interactive applications in C++

This paper presents an object-oriented approach for the implementation of interactive systems. This approach applies the model-view-controller (MVC) paradigm, which is modified for the C++ environment. The modified paradigm is called MVC++. In this approach the design of interactive applications starts by constructing an object model that represents the key concepts of the problem domain. This object model does not contain any user interface elements. According to the MVC++ approach, the object model is called the model part of the application. Only after the model part has been created, is the user interface designed. A collection of user interface classes is called the view. The classes that connect the model and the view form the controller, which is designed to communicate with both the model and the view. The approach presented in this paper provides a number of benefits. Advantages of the original MVC approach are obtained in a standard C++ environment, the structure of applications becomes clear and stable, and reusability as well as readability of the classes increases. Reusability is achieved by employing a mechanism called ‘abstract partners’. The approach presented in this paper is being applied in the development of a large network management system with over one million lines of C++ code running in the XII environment.     

A framework for creating hybrid-open source software communities

Abstract The open source software (OSS) model is a fundamentally new and revolutionary way to develop software. The success of the OSS model is also setting the stage for a structural change in the software industry; it is beginning to transform software industry from manufacturing to a service industry. Despite the success of the OSS model, for‐profit organizations are having difficulty building a business model around the open source paradigm. Whereas there are some isolated empirical studies, little rigorous research has been done on how traditional organizations can implement and benefit from OSS practices. This research explores how organizations can foster an environment similar to OSS to manage their software development efforts to reap its numerous advantages. Drawing on organizational theory, we develop a framework that guides the creation and management of a hybrid‐OSS community within an organization. We discuss the implications of this framework and suggest areas for future research.     

Methodology evaluation framework for dynamic evolution in composition-based distributed applications

Dynamic evolution can be used to upgrade distributed applications without shutdown and restart as a way of improving service levels while minimising the loss of business revenue caused by the downtime. An evaluation framework assessing the level of support offered by existing methodologies in composition-based application (e.g. component-based and service-oriented) development is proposed. It was developed by an analysis of the literature and existing methodologies together with a refinement based on a survey of experienced practitioners and researchers. The use of the framework is demonstrated by applying it to twelve methodologies to assess their support for dynamic evolution.     

Revisiting actor programming in C++

The actor model of computation has gained significant popularity over the last decade. Its high level of abstraction makes it appealing for concurrent applications in parallel and distributed systems. However, designing a real-world actor framework that subsumes full scalability, strong reliability, and high resource efficiency requires many conceptual and algorithmic additives to the original model.In this paper, we report on designing and building CAF, the C++ Actor Framework. CAF targets at providing a concurrent and distributed native environment for scaling up to very large, high-performance applications, and equally well down to small constrained systems. We present the key specifications and design concepts—in particular a message-transparent architecture, type-safe message interfaces, and pattern matching facilities—that make native actors a viable approach for many robust, elastic, and highly distributed developments. We demonstrate the feasibility of CAF in three scenarios: first for elastic, upscaling environments, second for including heterogeneous hardware like GPUs, and third for distributed runtime systems. Extensive performance evaluations indicate ideal runtime at very low memory footprint for up to 64 CPU cores, or when offloading work to a GPU. In these tests, CAF continuously outperforms the competing actor environments Erlang, Charm++, SalsaLite, Scala, ActorFoundry, and even the raw message passing framework OpenMPI.     

支持现场建档系统的异质数据传输软件平台

普适计算是计算模式的发展趋势。现场建档系统是其中一类重要应用,这通常需要在多台设备间互通性质不同的数据。为实现功能重用,加快建档系统的开发,提出一种支持异质数据通讯的软件平台,采用“订阅/发布”模式支持传输关系的动态调整及系统的易扩展性;针对异质数据设计不同传输结构保证传输的高效;平台的调用接口屏蔽了数据传输的实现细节,简单易用。该平台已成功用于课堂建档系统,验证了它在现场建档系统构建中的功用。     

A framework of auto-adapting distributed object for mobile computing

The low bandwidth hinders the development of mobile computing.Besides providing relatively higher bandwidth on communication layer,constructing adaptable upper application is important.In this paper,a framework of autoadapting distributed object is proposed,and evaluating methods of object performance are given as well.Bistributed objects can abjust their behaviors automatically in the framework and keep in relatively good performance to serve requests of remote applications.It is an efficient way to implement the performance transparency for mobile clients.     

ParSA: High-throughput scientific data analysis framework with distributed file system

Scientific data analysis and visualization have become the key component for nowadays large scale simulations. Due to the rapidly increasing data volume and awkward I/O pattern among high structured files, known serial methods/tools cannot scale well and usually lead to poor performance over traditional architectures. In this paper, we propose a new framework: ParSA (parallel scientific data analysis) for high-throughput and scalable scientific analysis, with distributed file system. ParSA presents the optimization strategies for grouping and splitting logical units to utilize distributed I/O property of distributed file system, scheduling the distribution of block replicas to reduce network reading, as well as to maximize overlapping the data reading, processing, and transferring during computation. Besides, ParSA provides the similar interfaces as the NetCDF Operator (NCO), which is used in most of climate data diagnostic packages, making it easy to use this framework. We utilize ParSA to accelerate well-known analysis methods for climate models on Hadoop Distributed File System (HDFS). Experimental results demonstrate the high efficiency and scalability of ParSA, getting the maximum 1.3 GB/s throughput on a six nodes Hadoop cluster with five disks per node. Yet, it can only get 392 MB/s throughput on a RAID-6 storage node.     

定位数据访问框架的设计与实现

目前定位应用访问定位数据的方法常和特定的技术绑定在一起,这种相关性限制了应用向新技术的移植,也将最终用户绑定在了特定的设备上.通过对现存的定位技术方案的研究,抽象出了通用的定位数据访问模型,根据模型给出了定位数据访问框架的体系结构和实现方法.框架对定位应用屏蔽各种定位数据访问的细节和定位数据接收终端硬件的差异,为定位应用提供个一个与平台、技术体制和设备独立的定位数据访问接口.     

基于CSC的分布式应用对象交互服务模型DAOIS的研究

介绍一种新的分布式计算支撑模型与工具──分布式应用对象交互服务模型ＤＡＯＩＳ。它主要面向松散型的分布或独立应用系统的开发与集成。以ＣＳＣ为工作与构造模型,以平台级的、应用独立的、悬挂式的、网络透明平台透明应用代码透明的“软插件”为基础,辅以“包装”式的构造协议。     

SESAG: an object-oriented application framework for real-time systems

Advancements in hardware and software technologies have made possible the design of real-time systems and applications where stringent timing constraints are imposed on critical tasks. The design of such systems is more complex than that of temporally unrestricted systems because system correctness depends on the satisfaction of functional as well as temporal requirements. To aid users in correctly and efficiently designing systems, object-oriented frameworks provide a useful environment for significant reuse and reduction in design effort. In contrast to other application domains, there has been relatively little work on an application framework for the design of real-time systems. Facing the growing need for real-time applications, we propose a novel application framework called SESAG, which consists of five components, namely Specifier, Extractor, Scheduler, Allocator, and Generator. Within SESAG, several design patterns are proposed and used for the development of real-time applications. A new evaluation metric called relative design effort is proposed for evaluating SESAG. Experiences in using SESAG show a significant increase in design productivity through design reuse and a significant decrease in design time and effort. Two complex application examples have been developed using SESAG and evaluated using the new evaluation metric. The examples demonstrate relative design efforts of at most 18% of the design efforts required by conventional methods. Copyright © 2005 John Wiley & Sons, Ltd.     

A JAVA application framework for scientific software development

This paper presents AIBench (SING group, Ourense, Spain), a JAVA desktop application framework mainly focused on scientific software development, with the goal of improving the productivity of research groups. Following the MVC design pattern, the programmer is able to develop applications using only three types of concepts: operations, data‐typesand views. The framework provides the rest of the functionality present in typical scientific applications, including user parameter requests, logging facilities, multithreading execution, experiment repeatability and graphic user interface generation, among others. The proposed framework is implemented following a plugin‐based architecture, which also allows assembling new applications by the reuse of modules from past development projects. Copyright © 2011 John Wiley & Sons, Ltd.     

基于网格的分布式虚拟环境仿真海量数据管理

分析了分布式虚拟环境仿真的特点,提出了基于网格的分布式虚拟环境仿真的海量数据管理框架.该框架结构采用分层结构,自底向上依次为网格节点、高性能通信系统、数据存储与处理系统和计算系统.给出了一个基于上述体系结构的原型系统.对该原型系统的仿真结果表明,该海量数据管理体系结构设计符合虚拟环境仿真实时性、稳定性和高可靠性的要求.     

基于Qt的Android应用程序C/C++开发方法与实践

Android Java应用程序（APP）运行时需依赖Dalvik虚拟机环境,从而影响APP的性能表现.C/C++程序具有极好的硬件适应能力,但不能直接开发Android APP.针对该问题,利用Dalvik虚拟机支持JNI （Java Native Inter-face）调用的特点,设计了基于Qt的Android APP C/C++开发方法框架体系,给出了框架的详细工作机制和解决方案,系统总结了利用C/C++开发Android APP的主要过程和步骤.最后,以Android版机械CAD导教APP的开发工作为例,验证了方法的有效性和可行性,为直接利用C/C++开发Android APP提供了一种新途径.     

NUAPC: A Parallelizing Compiler for C++

is paper presents a model for automatically parallelizing compiler based on C which consists of compile-time and run-time parallelizing facilities.The paper also describes a method for finding both intra-object and inter-object parallelism.The parallelism detection is completely transparent to users.     

Code quality analysis in open source software development

Abstract Proponents of open source style software development claim that better software is produced using this model compared with the traditional closed model. However, there is little empirical evidence in support of these claims. In this paper, we present the results of a pilot case study aiming: (a) to understand the implications of structural quality; and (b) to figure out the benefits of structural quality analysis of the code delivered by open source style development. To this end, we have measured quality characteristics of 100 applications written for Linux, using a software measurement tool, and compared the results with the industrial standard that is proposed by the tool. Another target of this case study was to investigate the issue of modularity in open source as this characteristic is being considered crucial by the proponents of open source for this type of software development. We have empirically assessed the relationship between the size of the application components and the delivered quality measured through user satisfaction. We have determined that, up to a certain extent, the average component size of an application is negatively related to the user satisfaction for this application.