基于高并发的分布式系统幂等设计

随着数字生活不断发展,分布式系统被广泛用来解决高并发等一系列问题。为了保证数据的准确性和一致性,分布式系统需要做幂等控制。通过分析对比各服务端幂等设计方法在高并发场景下的性能表现,提出一种改进的分布式锁设计方法。该方法主要解决由于集群服务主节点宕机可能造成分布式锁失效的问题,通过探讨RedLock算法方案及争论,提出对高并发请求二次拦截的理论,并将失效的锁通过消息队列服务进行通知,实现锁失效问题的排查以及服务监测治理。经实验测试表明,该方法实现了对失效锁的拦截,在高并发场景下有良好的性能表现,为分布式系统幂等设计提供了可行性方案。     

Labeled Network Stack: A High-Concurrency and Low-Tail Latency Cloud Server Framework for Massive IoT Devices

Internet of Things(IoT)applications have massive client connections to cloud servers,and the number of networked IoT devices is remarkably increasing.IoT services require both low-tail latency and high concurrency in datacenters.This study aims to determine whether an order of magnitude improvement is possible in tail latency and concurrency in mainstream systems by proposing a hardware-software codesigned labeled network stack(LNS)for future datacenters.The key innovation is a cross-layered payload labeling mechanism that distinguishes different requests by payload across the full network stack,including application,TCP/IP,and Ethernet layers.This type of design enables prioritized data packet processing and forwarding along the full datapath,such that latency-insensitive requests cannot significantly interfere with high-priority requests.We build a prototype datacenter server to evaluate the LNS design against a commercial X86 server and the mTCP research,using a cloud-supported IoT application scenario.Experimental results show that the LNS design can provide an order of magnitude improvement in tail latency and concurrency.A single datacenter server node can support over 2 million concurrent long-living connections for IoT devices as a 99-percentile tail latency of 50 ms is maintained.In addition,the hardware-software codesign approach remarkably reduces the labeling and prioritization overhead and constrains the interference of high-priority requests to low-priority requests.     

Combining replication and checkpointing redundancies for reducing resiliency overhead

We herein propose a heuristic redundancy selection algorithm that combines resubmission, replication, and checkpointing redundancies to reduce the resiliency overhead in fault‐tolerant workflow scheduling. The appropriate combination of these redundancies for workflow tasks is obtained in two consecutive phases. First, to compute the replication vector (number of task replicas), we apportion the set of provisioned resources among concurrently executing tasks according to their needs. Subsequently, we obtain the optimal checkpointing interval for each task as a function of the number of replicas and characteristics of tasks and computational environment. We formulate the problem of obtaining the optimal checkpointing interval for replicated tasks in situations where checkpoint files can be exchanged among computational resources. The results of our simulation experiments, on both randomly generated workflow graphs and real‐world applications, demonstrated that both the proposed replication vector computation algorithm and the proposed checkpointing scheme reduced the resiliency overhead.     

面向大数据的分布式缓存设计

大数据平台的底层存储系统往往无法匹配上层计算应用的读写性能,而一个设计良好的分布式缓存系统将缩小CPU密集型应用和IO密集型应用之间不匹配的性能差距。设计的面向大数据应用的分布式缓存系统,在读写流程、I/O事件驱动并发模型及元数据模型等方面进行了合理设计与优化,并使用fio工具测试了顺序写、随机写、顺序读及随机读场景下的吞吐率与IOPS等性能指标,验证了该分布式缓存系统的高性能优势和应对高并发场景的扩展能力。     

一种利它锁协议及其操作原语

针对长事务在较长一段时间占有数据库资料,明显地延迟了短事务和普通事务的执行这一问题,对两步锁进行改进和扩充,给出一种利它锁协议及其操作原理,让长事务按一定的协议较早地释放其占用的锁,使需要存取这些被锁数据的事务在长事务处于唤醒状态时,遵循一定的锁规则同时运行。     

分布式环境下书目数据库的安全性问题

基于MicrosoftSQLServer 6 .5分布式数据库的完整性、安全性、并发性和故障恢复等机制 ,对分布式环境下书目的数据库安全性提出了一套有效的策略 ,并通过身份验证、利用视图等 7个方面来实现该系统的数据库安全性问题     

Common Knowledge and Update in Finite Environments

Logics of knowledge have been shown to provide a useful approach to the high level specification and analysis of distributed systems. It has been proposed that such systems can be developed using knowledge- based protocols, in which agents' actions have preconditions that test their state of knowledge. Both computer-assisted analysis of the knowledge properties of systems and automated compilation of knowledge-based protocols require the development of algorithms for the computation of states of knowledge. This paper studies one of the computational problems of interest, the model checking problem for knowledge formulae in the S5_nKripke structures generated by finite state environments in which states determine an observation for each agent. Agents are assumed to have perfect recall and may operate synchronously or asynchronously. It is shown that, in this setting, model checking of common knowledge formulae is intractable, but efficient incremental algorithms are developed for formulae containing only knowledge operators. Connections to knowledge updates and compilation of knowledge-based protocols are discussed.     

分布式事务的并发控制机制的研究

分析了在分布式系统中事务的并发控制和死锁问题,对相应的解决方法进行了 和研究,并在此基础上提出了实际解决方法。     

多查询共享技术研究综述

传统的数据库系统围绕单次查询的模型构建,独立地执行并发查询.由于该模型的限制,传统数据库无法一次对多个查询进行优化.多查询共享技术旨在共享查询之间的公共部分,从而达到提高系统整体响应时间和吞吐量的目的.将多查询执行模式分为两类,介绍了各自的原型系统——基于全局查询计划的多查询原型系统和以运算符为中心的多查询原型系统,并且讨论了两种系统的优势以及所适用场景.在之后的内容中,将多查询共享技术按照查询的各个阶段分为查询编译阶段中的多查询共享技术以及查询执行阶段中的多查询共享技术两大类.以这两个方向为线索,梳理了多查询计划的表示方法、多查询表达式合并、多查询共享算法、多查询优化等各种方向的研究成果.在此基础上,还介绍了共享查询技术在关系数据库和非关系数据库中的应用.最后,分析了共享查询技术面临的机遇和挑战.     

操作系统内核并发错误检测研究进展

并发错误是程序设计语言和软件工程领域的研究热点之一.近年来,针对应用程序并发错误检测的研究已取得了很大进展.但由于操作系统内核的并发和同步机制复杂、代码规模庞大,与应用程序级并发错误检测相比,操作系统内核的并发错误检测研究仍面临巨大挑战.对此,国内外学者提出了各种用于操作系统内核并发错误检测的方法.首先介绍了并发错误的基本类型、检测方法和评价指标,讨论了现有的并发错误检测方法和工具的局限性;接着,从形式化验证、静态分析、动态分析和静态动态相结合四个方面,对现有的操作系统内核并发错误检测的研究工作进行了分类阐述,并作了系统总结和对比分析;最后,探讨了操作系统内核并发错误检测研究面临的挑战,并对该领域未来的研究趋势进行了展望.