一种MapReduce实时调度算法设计及实现

MapReduce是云计算中重要的批数据处理框架,多任务共享MapReduce机群并满足任务实时性要求是调度算法急需解决的问题。提出两阶段实时调度算法,将调度划分为任务间调度和任务内调度。对于任务间调度,使用抽样法和经验值法确定子任务执行时间,利用该参数建立资源分配模型,动态确定任务优先级进行调度;对于子任务使用延迟调度策略进行调度,保证计算的本地性。实验结果显示,两阶段实时调度算法相比公平调度算法和FIFO算法,在保证吞吐量的同时能够满足任务实时性要求。     

A genetic approach for downlink packet scheduling in HSDPA system

High speed downlink packet access (HSDPA) system currently under development in 3rd generation partnership project (3GPP) [1] employs number of parallel shared channels and adaptive modulation and coding scheme (MCS) to enable a high rate packet data transfer from base station (Node-B) to user equipment (UE). To decide how to share the available radio channel capacity amongst the active UEs, a packet scheduling and channel assignment algorithms are employed at Node B. Packet scheduling techniques such as max carrier to interference ratio (C/I) or round robin (RR) fails to take into account all the aspects of the quality of service (QoS) provisioning. In this paper, the fundamentals genetic algorithms and conventional wireless scheduling techniques are combined and the weaknesses of the existing known techniques are exploited to propose a novel hybrid genetic packet scheduler (HGPS) for the HSDPA system. A combination of random and intelligent diversity of population and comparative nature of the selection process of genetic engine contribute to its robustness. The proposed HGPS outperforms Max C/I packet scheduler in terms of total delivered throughput within low delay thresholds. Unlike conventional packet scheduling technique HGPS does not rely only on the current or past status of the scheduling process. By treating the possible solutions as points in a search space, the proposed HGPS through a genetically guided search visits and examines the possible solutions and estimates the impact of each these solution on overall performance of system in terms of fairness, throughput or QoS without actually performing a transmission. Subsequently, the solution that achieves the best estimated overall performance is chosen for the actual transmission. By means of computer simulation, performance of the HGPS algorithm is characterized in Rayleigh fading and shadowing radio channel conditions. The impact of imperfect reporting on the performance of HSDPA system is evaluated. We examine the joint impact of reporting latency, imperfect channel estimation and the corruption of reports in the feedback channel on the performance. It is shown that the proposed class of intelligent parallel random schedulers is highly robust against the imperfect reports from UEs. Acknowledgment The authors would like to acknowledge the valuable assistance provided by Mr Pierre Coulon of Fujitsu Laboratories of Europe Ltd.This work was presented in part at IEEE WPMC’2002, and will be presented in part at IEEE IST’2003.     

Optimal adaptive channel scheduling for scalable video broadcasting over MIMO wireless networks

Video broadcasting is an efficient way to deliver video content to multiple receivers. However, due to heterogeneous channel conditions in MIMO wireless networks, it is challenging for video broadcasting to map scalable video layers to proper MIMO transmit antennas to minimize the average overall video transmission distortion. In this paper, we investigate the channel scheduling problem for broadcasting scalable video content over MIMO wireless networks. An adaptive channel scheduling based unequal error protection (UEP) video broadcasting scheme is proposed. In the scheme, video layers are protected unequally by being mapped to appropriate antennas, and the average overall distortion of all receivers is minimized. We formulate this scheme into a non-linear combinatorial optimization problem. It is not practical to solve the problem by an exhaustive search method with heavy computational complexity. Instead, an efficient branch-and-bound based channel scheduling algorithm, named TBCS, is developed. TBCS finds the global optimal solution with much lower complexity. The complexity is further reduced by relaxing the termination condition of TBCS, which produces a (1 − ε)-optimal solution. Experimental results demonstrate both the effectiveness and efficiency of our proposed scheme and algorithm. As compared with some existing channel scheduling methods, TBCS improves the quality of video broadcasting across all receivers significantly.     

Optimal admission control algorithms for scheduling burst data in CDMA multimedia systems

The third generation mobile communication systems are widely envisioned to be based on wideband code division multiple access (CDMA) technologies to support high data rate (HDR) packet data services. To effectively harness the precious bandwidth while satisfying the HDR requests from users, it is crucial to use a judicious burst admission control algorithm. In this paper, we propose and evaluate the performance of a novel jointly adaptive burst admission algorithm, called the synergistic burst admission control algorithm to allocate valuable resources (i.e., channels) in wideband CDMA systems to burst HDR requests. We consider the spatial dimension only, and by that we mean the algorithm performs scheduling and admission control, for the current frame only, based solely on the selection diversity in the geographical and mobility aspects. The scheduler does not exploit the temporal dimension in that it does not make allocation decisions about future frames (i.e., requests that do not get allocation are simply ignored and such requests will be treated as new request in future frames). In the physical layer, we use a variable rate channel-adaptive modulation and coding system which offers variable throughput depending on the instantaneous channel condition. In the MAC layer, we use the proposed optimal multiple-burst admission algorithm, induced by our novel integer programming formulation of the admission control and scheduling problem. We demonstrate that synergy could be attained by interactions between the adaptive physical layer and the burst admission layer. Both the forward link and the reverse link burst requests are considered and the system is evaluated by dynamic simulations which takes into account of the user mobility, power control and soft handoff. We found that significant performance improvement, in terms of average packet delay, data user capacity and coverage, could be achieved by our scheme compared to the existing burst assignment algorithms.     

A novel multicast mechanism for optical local area networks

In this paper, we propose a simple and efficient multicast scheduling algorithm, the Lookback Queue Access (LBQA) protocol for the employment in an asynchronous WDM optical star network. Network nodes use ALOHA based random access schemes to send their reservation packets over a common packet reservation control channel. A central traffic scheduler is allocated to coordinate message transmissions. The central scheduler carries out the LBQA scheme in real time in two phases. The first phase is to search (through input queues) for a candidate multicast message that can be sent, without partition, to all of its intended recipients. If phase 1 fails, the second phase is then activated to efficiently partition a multicast message into multiple transmissions. Performance results conducted via simulations have revealed that networks employing such a scheduling mechanism can exhibit superior network throughput levels and delay performances. To obtain sustainable high performances, we further propose an interconnected dual-star structure employed with the enhanced scheduling mechanism, the DS_LBQA algorithm. By using a simple heuristic approach, the DS_LBQA scheme is able to successfully exploit the inter data channels and properly utilize the wavelength reuse property of the intra data channels. Performance results have demonstrated the merits of deploying the DS_LBQA multicast algorithm in such a dual-star structure.     

A multi-hop resource scheduling algorithm for IEEE 802.16j relay networks

The IEEE 802.16j standard defines both transparent and non-transparent relay transmission mode. The present study formulates and optimizes the relay resource scheduling problem for the case of a non-transparent relay network. It is shown that the resource scheduling problem is NP-Complete. A method is proposed for optimizing the position of the zone boundary adaptively during the resource scheduling process in order to maximize the system throughput. In addition, a low time complexity algorithm designated as MRRS (multi-hop relay resource scheduling) is proposed to obtain an approximate solution for the NP-Complete scheduling problem. In the proposed algorithm, the zone boundary is adjusted adaptively in accordance with the user distribution and the channel state information in such a way as to improve the utilization of the available slots. The simulation results show that MRRS achieves a higher throughput than existing relay resource scheduling algorithms (GenArgMax and Eliminate-Repeat) with no significant loss in fairness. In addition, it is shown that the performance improvement provided by MRRS increases as the hop-count is increased.     

基于公平性的D2D时隙调度算法

针对设备到设备（D2D）通信资源分配中的时隙调度时延以及信道增益变化导致吞吐率下降的问题,提出了一种公平性时隙调度（FTDS）算法。首先,基于频谱复用模式建立系统模型,并归纳为一组合优化问题;然后,在模型的次优求解中,FTDS算法将调度周期划分为多个等长的时隙,根据优先级策略将D2D用户分配至不同时隙调度,从而适应D2D用户多于蜂窝用户的应用场景;同时,为了权衡服务质量（QoS）与系统吞吐率的关系,构造一满足性权值与传输速率相互制约,共同决定用户调度优先级。仿真实验中,FTDS算法相比TDS、RANDOM算法,吞吐率平均增幅分别达到11.09%和40.64%,且FTDS算法下D2D用户被调度频次累积分布更为集中;同时,相比TDS算法调度时延最大降低31.22%。仿真实验表明,FTDS算法拥有更优的吞吐率性能、更公平的调度机制、更小的调度时延。     

Distributed fair DRAM scheduling in network-on-chips architecture

Memory access scheduling is an effective manner to improve performance of Chip Multi-Processors (CMPs) by taking advantage of the timing characteristics of a DRAM. A memory access scheduler can subdivide resources utilization (banks and rows) to increase throughput by accessing different DRAM banks in parallel. However, different threads running on different cores may exhibit different performance. One thread may experience starvation while the others are serviced normally. Therefore, designing a scheduler which reduces the unfairness in the DRAM system, while also improving system throughput on a variety of workloads and systems, is necessary. In this paper, a distributed fair DRAM scheduling for two-dimensional mesh network-on-chips (NoCs), called DFDS, is presented. The key design points in DFDS are: (i) assessing the total waiting cycles of a memory request in NoC and considering it as a metric in arbitration. For this purpose waiting cycles of a memory request are put in an additional flit in a packet and are updated while traversing the NoC, and (ii) proposing a semi-dynamic virtual channel allocation to provide in-order memory requests to memory controllers (MCs). Consequently, we use a simple scheduling algorithm in MCs, instead of complex algorithms. To validate our approach, we apply synthetic and real workload from Parsec benchmark suite. The results show effectiveness of our approach, as we reduce the waiting time of memory requests by up to 15%.     

Performance analysis of scheduling and interference coordination policies for OFDMA networks

In orthogonal frequency division multiple access systems there is an intimate relationship between the packet scheduler and the inter-cell interference coordination (ICIC) functionalities: they determine the set of frequency channels (sub-carriers) that are used to carry the packets of in-progress sessions. In this paper we build on previous work – in which we compared the so called random and coordinated ICIC policies – and analyze three packet scheduling methods. The performance measures of interest are the session blocking probabilities and the overall throughput. We find that the performance of the so-called Fifty-Fifty and What-It-Wants scheduling policies is somewhat improved by coordinated sub-carrier allocation, especially in poor signal-to-noise-and-interference situations and at medium traffic load values. The performance of the All-Or-Nothing scheduler is practically insensitive to the choice of the sub-carrier allocation policy.     

A software-based dynamic-warp scheduling approach for load-balancing the Viola–Jones face detection algorithm on GPUs

Face detection is a key component in applications such as security surveillance and human–computer interaction systems, and real-time recognition is essential in many scenarios. The Viola–Jones algorithm is an attractive means of meeting the real time requirement, and has been widely implemented on custom hardware, FPGAs and GPUs. We demonstrate a GPU implementation that achieves competitive performance, but with low development costs. Our solution treats the irregularity inherent to the algorithm using a novel dynamic warp scheduling approach that eliminates thread divergence. This new scheme also employs a thread pool mechanism, which significantly alleviates the cost of creating, switching, and terminating threads. Compared to static thread scheduling, our dynamic warp scheduling approach reduces the execution time by a factor of 3. To maximize detection throughput, we also run on multiple GPUs, realizing 95.6 FPS on 5 Fermi GPUs.     

Multi-criteria and satisfaction oriented scheduling for hybrid distributed computing infrastructures

Assembling and simultaneously using different types of distributed computing infrastructures (DCI) like Grids and Clouds is an increasingly common situation. Because infrastructures are characterized by different attributes such as price, performance, trust, and greenness, the task scheduling problem becomes more complex and challenging. In this paper we present the design for a fault-tolerant and trust-aware scheduler, which allows to execute Bag-of-Tasks applications on elastic and hybrid DCI, following user-defined scheduling strategies. Our approach, named Promethee scheduler, combines a pull-based scheduler with multi-criteria Promethee decision making algorithm. Because multi-criteria scheduling leads to the multiplication of the possible scheduling strategies, we propose SOFT, a methodology that allows to find the optimal scheduling strategies given a set of application requirements. The validation of this method is performed with a simulator that fully implements the Promethee scheduler and recreates an hybrid DCI environment including Internet Desktop Grid, Cloud and Best Effort Grid based on real failure traces. A set of experiments shows that the Promethee scheduler is able to maximize user satisfaction expressed accordingly to three distinct criteria: price, expected completion time and trust, while maximizing the infrastructure useful employment from the resources owner point of view. Finally, we present an optimization which bounds the computation time of the Promethee algorithm, making realistic the possible integration of the scheduler to a wide range of resource management software.     

Analysis of temporal and throughput fair scheduling in multirate WLANs

The IEEE 802.11e Wireless Local Area Network (WLAN) provides controlled access features that can be used in conjunction with scheduling algorithms to provide guaranteed per-session services. However, the multirate operation of the WLAN complicates the design of scheduling and Quality of Service (QoS) provisioning algorithms. We propose a new solution based on Controlled Access Phase Scheduling (CAPS) framework, introduced for fixed rate WLANs in our earlier works, and employ a new fair central scheduler to achieve guaranteed fair services in a WLAN. We examine the fairness issues involved in designing such an algorithm and study several fair scheduling algorithms that can be used with CAPS. We present a modified start time fair queuing based scheduler as our choice and analyze its performance under dynamic and static multirate operation. The algorithm is then evaluated through several simulation experiments. We show that the enhanced CAPS is able to adapt to multirate environments and provide both temporal and throughput fair services in 802.11e WLANs.     

An 802.11e HCCA scheduler with an end-to-end quality aware territory method

In this paper we present a solution for the IEEE 802.11e HCCA (Hybrid coordination function Controlled Channel Access) mechanism which aims both at supporting strict real-time traffic requirements and, simultaneously, at handling TCP applications efficiently. Our proposal combines a packet scheduler and a dynamic resource allocation algorithm. The scheduling discipline is based on the Monolithic Shaper-Scheduler, which is a modification of a General Processor Sharing (GPS) related scheduler. It supports minimum-bandwidth and delay guarantees and, at the same time, it achieves the optimal latency of all the GPS-related schedulers. In addition, our innovative resource allocation procedure, called the territory method, aims at prioritizing real time services and at improving the performance of TCP applications. For this purpose, it splits the wireless channel capacity (in terms of transmission opportunities) into different territories for the different types of traffic, taking into account the end-to-end network dynamics. In order to give support to the desired applications, we consider the following traffic classes: conversational, streaming, interactive and best-effort. The so called territories shrink or expand depending on the current quality experienced by the corresponding traffic class. We evaluated the performance of our solution through extensive simulations in a heterogeneous wired-cum-wireless scenario under different traffic conditions. Additionally, we compare our proposal to other HCCA scheduling algorithms, the HCCA reference scheduler and Fair Hybrid Coordination Function (FHCF). The results show that the combination of the MSS and the territory method obtains higher system capacity for VoIP traffic (up to 32 users) in the simulated scenario, compared to FHCF and the HCCA reference scheduler (13 users). In addition, the MSS with the territory method also improves the throughput of TCP sources (one FTP application achieves between 6.1 Mbps without VoIP traffic and 2.1 Mbps with 20 VoIP users) compared to the reference scheduler (at most 388 kbps) and FHCF (with a maximum FTP throughput of 4.8 Mbps).     

Channel switching control policy for wireless mesh networks

Dynamic channel assignment algorithms allow wireless nodes to switch channels when their traffic loads exceed certain thresholds. These thresholds represent estimations of their throughput capacities. Unfortunately, the threshold estimation may not be accurate due to co-channel interference (CCI) and adjacent-channel interference (ACI), especially with high traffic loads in dense networks. When the link capacity is over-estimated, these channel assignment algorithms are not effective. This is because the channel switch is not triggered even with overloaded data traffic and the link quality decreases significantly as the channel is overloaded. When the link capacity is under-estimated, the link is under-utilized. Moreover, when link traffic load increases from time to time, channel switch occurs frequently. Such frequent channel switches increase latency and degrade throughput, and can even cause network wide channel oscillations. In this paper, we propose a novel threshold-based control system, called balanced control system (BCS). The proposed threshold-based control policy consists of deciding, according to the real time traffic load and interference, whether to switch to another channel, which channel should be switched to and how to perform the switch. Our control model is based on a fuzzy logic control. The threshold which assists to make the channel switch decisions, could be deduced dynamically according to the real-time traffic of each node. We also design a novel dynamic channel assignment scheme, which is used for the selection of the new channel. The channel switch scheduler is provided to perform channel-switch processing for sender and receiver over enhanced routing protocols. We implement our system in NS2, and the simulation results show that with our proposed system, the performance improves by 12.3%–72.8% in throughput and reduces 23.2%–52.3% in latency.     

时变信道正交频分复用系统绿色资源分配算法

传统的基于终端节能的绿色资源分配算法大都忽略了无线信道的时间选择性对能耗与系统性能的影响,为此提出一种结合多用户分集的绿色资源分配算法。在保证用户公平性的基础上引入多用户分集,动态调整多用户分集子模块大小以满足信道时变特性要求;分集子模块内有多个频带可供使用,用户资源尽可能集中在比较少的时隙内传输以降低总的终端接收能耗,通过顶点搜索快速寻找最优解。仿真结果表明,系统吞吐量可以提高13%左右,其稳定性得到改善;同时算法具有计算复杂度低、收敛速度快等特点,保持了良好的终端节能增益。     

Anchored opportunity queueing: a low-latency scheduler for fair arbitration among virtual channels

Many wormhole interconnection networks for parallel systems, and more recently system area networks, implement virtual channels to provide a number of services including improved link utilization and lower latencies. The forwarding of flits from the virtual channels on to the physical channel is typically accomplished using flit-based round-robin (FBRR) scheduling. This paper presents a novel scheduling strategy, anchored opportunity queueing (AOQ), which preserves the throughput and fairness characteristics of FBRR while significantly reducing the average delay experienced by packets. The AOQ scheduler achieves lower average latencies by trying, as far as possible, to complete the transmission of a complete packet before beginning the transmission of flits from another packet. The AOQ scheduler achieves provable fairness in the number of opportunities it offers to each of the virtual channels for transmissions of flits over the physical channel. We prove this by showing that the relative fairness bound, a popular measure of fairness, is a small finite constant in the case of the AOQ scheduler. Finally, we present simulation results comparing the delay characteristics of AOQ with other schedulers for virtual channels. The AOQ scheduler is simple to implement in hardware, and also offers a practical solution in other contexts such as in scheduling ATM cells in Internet backbone switches.     

RRR: recursive round robin scheduler

Scheduling has been an interesting problem since its inception. In the context of real-time networks, a scheduling algorithm is concerned with dispatching streams of packets sharing the same bandwidth such that certain guaranteed performance for each stream like rate and delay bound is provided. This function has a wide range of applications in network elements such as host adaptors, routers and switches. This paper proposes and describes a new scheduling algorithm named as recursive round robin (RRR) scheduler. It is based on the concept of the construction of a scheduling tree in which distinct cell streams are scheduled recursively. Special emphasis is placed on the design and analysis of the scheduler. A delay bound is analytically derived for the scheduler and verified using simulation. The scheduler can work in either a work-conserving mode or non-work-conserving mode. It is shown that the work conserving scheduler is fair. Fairness indexes for the work conserving scheduler are analytically derived. The simple nature of this algorithm makes it possible to implement it at very high speeds, while considerably reducing the implementation cost.     

基于延迟调度策略的Reduce调度优化算法

在大规模的Hadoop集群中,良好的任务调度策略对提高数据本地性、减小网络传输开销、减少作业执行时间以及提高集群的作业吞吐量都有着重要的影响。本文针对Hadoop架构中Reduce任务的数据本地性较低问题,提出了一种基于延迟调度策略的Reduce任务调度优化算法,通过提高Reduce任务的数据本地性来减少作业执行时间以及提高作业吞吐量,该算法在Hadoop架构的Early Shuffle阶段,使用多级延迟调度策略来提高Reduce任务的数据本地性。最后重写原生公平调度器代码实现了该调度算法,并与原生公平调度器进行了对比实验分析,实验结果表明该算法明显减少了作业执行时间,提高了集群的作业吞吐量。     

A hybrid TDMA/Random-access scheme for multiple-access communication networks

A hybrid TDMA/Random-Access multiple-access (HTRAMA) scheme is introduced for providing an access-control coordination for a multi-access communication channel. Such a scheme is applicable to a large spectrum of computer communication network applications. Under this hybrid scheme, the system sources are divided into groups. Sources in different groups are allocated disjoint time slots for their transmissions. Sources within a group share their allocated time slots by transmitting according to a tree random-access policy. The number of groups (and their sizes) is dynamically adjusted to properly (and optimally) match the underlying channel traffic characteristics. In this fashion the hybrid scheme dynamically adapts to a random-access structure at lower traffic throughput levels and to a TDMA structure at higher throughput levels. We carry out detailed delay throughput analysis for these hybrid schemes under both limited and unlimited source buffer capacities. The hybrid scheme is demonstrated to yield very good delay-throughput performance curves under wide ranges of network traffic statistical fluctuations and spans.     

An ideal multi-secret sharing scheme based on MSP

A multi-secret sharing scheme is a protocol to share m arbitrarily related secrets s₁, … , s_m among a set of n participants. In this paper, we propose an ideal linear multi-secret sharing scheme, based on monotone span programs, where each subset of the set of participants may have the associated secret. Our scheme can be used to meet the security requirement in practical applications, such as secure group communication and privacy preserving data mining etc. We also prove that our proposed scheme satisfies the definition of a perfect multi-secret sharing scheme.