Direct push–pull or assisted push–pull? Toward optimal video content delivery using shared storage-based cloud CDN (SS-CCDN)

Video content delivery networks face many challenges such as scalability, quality of service and flexibility. Video suppliers address them through CDN. Cloud computing and Video content Delivery as a Service (VDaaS) plays a key role in improving the content delivery standard and makes the work of content providers, easier. By hosting video contents in the cloud, the content delivery costs are minimized and the overall content delivery performance enhanced by optimization of cloud CDN. Cost optimization of the cloud-based content delivery network requires a focus on delay or throughput, the overall performance and content delivery. The content placement and content access, the QoS and the QoE in CDN can be improved by enhancing the video content delivery performance. In this paper, a unique model for video content delivery, cloud-based is developed, titled as shared storage-based cloud CDN (SS-CCDN) to achieve the objective. This design optimizes through algorithms, the effective placement of video data and dynamic update of video data. For analysis, GA, PSO, and ACO algorithms are used. The proposed model uses direct and assisted push–pull content delivery schemes for cost-efficient content delivery. The low-cost VDaaS model reduces the storage cost, keeps the latency and the traffic cost. Experimental results validate that this model, with regard to storage, traffic, and latency generate higher performance with lower price and satisfy the QoS and QoE aspects in content delivery.

     

Analysis of YouTube’s traffic adaptation to dynamic environments

The popular Internet service, YouTube, has adopted by default the HyperText Markup Language version 5 (HTML5). With this adoption, YouTube has moved to Dynamic Adaptive Streaming over HTTP (DASH) as Adaptive BitRate (ABR) video streaming technology. Furthermore, rate adaptation in DASH is solely receiver-driven. This issue motivates this work to make a deep analysis of YouTube’s particular DASH implementation. Firstly, this article provides a state of the art about DASH and adaptive streaming technology, and also YouTube traffic characterization related work. Secondly, this paper describes a new methodology and test-bed for YouTube’s DASH implementation traffic characterization and performance measurement. This methodology and test-bed do not make use of proxies and, moreover, they are able to cope with YouTube traffic redirections. Finally, a set of experimental results are provided, involving a dataset of 310 YouTube’s videos. The depicted results show a YouTube’s traffic pattern characterization and a discussion about allowed download bandwidth, YouTube’s consumed bitrate and quality of the video. Moreover, the obtained results are cross-validated with the analysis of HTTP requests performed by YouTube’s video player. The outcomes of this article are applicable in the field of Quality of Service (QoS) and Quality of Experience (QoE) management. This is valuable information for Internet Service Providers (ISPs), because QoS management based on assured download bandwidth can be used in order to provide a target end-user’s QoE when YouTube service is being consumed.     

Video super-resolution network using detail component extraction and optical flow enhancement algorithm

The video super-resolution (SR) task refers to the use of corresponding low-resolution (LR) frames and multiple neighboring frames to generate high-resolution (HR) frames. Existing deep learning-based approaches usually utilize LR optical flow for video SR tasks. However, the accuracy of LR optical flow is not enough to recover the fine detail part. In this paper, we propose a video SR network that uses optical flow SR and optical flow enhancement algorithms to provide accurate temporal dependency. And extract the detail component of LR adjacent frames as supplementary information for accurate feature extraction. Firstly, the network infers HR optical flow from LR optical flow, and uses the optical flow enhancement algorithm to enhance HR optical flow. Then the processed HR optical flows are used as the input of the motion compensation network. Secondly, we extract detail component to reduce the error caused by motion compensation based on optical flow. Finally, the SR results are generated through the SR network. We perform comprehensive comparative experiments on two datasets: Vid4 and DAVIS. The results show that, compared with other state-of-the-art methods, the proposed video SR method achieves the better performance.

     

实现网络视频流多分类的迁移学习算法

在现实世界中,可用的训练数据通常较少,且很容易过时,所以需要不断采集和标记大量新的数据集;针对此问题,提出一种基于SAMME和TrAdaBoost算法的迁移学习分类方法。该方法的核心思想是：从老视频流数据集中筛选出有用的样本来帮助模型识别新的未知视频流集样本,这里新老视频流数据集的样本特征分布是不相同的。同时该方法结合SAMME算法将TrAdaBoost算法从只可实现两分类扩展至多分类。实验结果表明,与现有方法比较,该方法能更好地实现对六种类型视频流的精细分类,并减少大量已标注老数据集的浪费。     

Traffic Engineering of Management Flows by Link Augmentations on Confluent Trees

Service providers rely on the management systems housed in their Network Operations Centers (NOCs) to remotely operate, monitor and provision their data networks. Lately there has been a tremendous increase in management traffic due to the growing complexity and size of the data networks and the services provisioned on them. Traffic engineering for management flows is essential for the smooth functioning of these networks to avoid congestion, which can result in loss of critical data such as billing records, network alarms, etc. As is the case with most intra-domain routing protocols, the management flows in many of these networks are routed on shortest paths connecting the NOC with the service provider’s POPs (points of presence). This collection of paths thus forms a “confluent” tree rooted at the gateway router connected to the NOC. The links close to the gateway router may form a bottleneck in this tree resulting in congestion. Typically this congestion is alleviated by adding layer two tunnels (virtual links) that offload the traffic from some links of this tree by routing it directly to the gateway router. The traffic engineering problem is then to minimize the number of virtual links needed for alleviating congestion. In this paper we formulate a traffic engineering problem motivated by the above mentioned applications. We show that the general versions of this problem are hard to solve. However, for some simpler cases in which the underlying network is a tree, we design efficient algorithms. In particular, we design fully polynomial-time approximate schemes (FPTAS) for different variants of this problem on trees. We use these algorithms as the basis for designing efficient heuristics for alleviating congestion in general (non-tree) service provider network topologies.     

FlowSpectrum: a concrete characterization scheme of network traffic behavior for anomaly detection

As the 5G rolls out around the world, many edge applications will be deployed by app vendors and accessed by massive end-users. Efficient detection of malicious network behavior is paid more and more attention. The current traffic detection work is still stuck on the analysis of high-dimensional data. It will restrict the improvement of threat monitoring and network governance when facing massive network flows. Characterization of network flows within simple domains is required to simplify the process of network analysis. Traffic characterization is a key task that allows service providers to detect and intercept anomalous traffic, such that high QoS (Quality of Service) and service availability are maintained and spread of malicious content is prevented. Unfortunately, there is still a lack of research on the concrete characterization of network data. Analogous to spectrum, in this paper, we proposed the concept of FlowSpectrum for the first time in order to represent the network flow, concretely. In the FlowSpectrum, network flow is represented as a spectral line rather than the raw data or a feature vector of the network flow. All flows are able to be mapped as spectral lines, and traffic identification is achieved by analyzing the positions of spectral lines. FlowSpectrum can significantly reduce the complexity of network traffic behavior analysis while enhancing the interpretability of detection and facilitating cyberspace behavior management. We designed a neural network structure based on semi-supervised AutoEncoder for decomposition and dimensionality reduction of network flows in FlowSpectrum. The characterization capability of FlowSpectrum is proved by thorough experiments. Moreover, we realized the correspondence between network behaviors and intervals of spectral lines, preliminarily. Generally speaking, FlowSpectrum can provide new ideas for the field of network traffic analysis.

     

A generalizable dynamic flow pairing method for traffic classification

The goal of network traffic classification is to identify the protocols or types of protocols in the network traffic. In particular, the identification of network traffic with high resource consumption, such as peer-to-peer (P2P) traffic, represents a great concern for Internet Service Providers (ISP) and network managers. Most current flow-based classification approaches report high accuracy without paying attention to the generalization ability of the classifier. However, without this ability, a classifier may not be suitable for on-line classification. In this paper, a number of experiments on real traffic help to elucidate the reason for this lack of generalization. It is also shown that one way to attain the generalization ability is by using dynamic classifiers. From these results, a dynamic classification approach based on the pairing of flows according to a similarity criterion is proposed. The pairing method is not a classifier by itself. Rather, its goal is to determine in a fast way that two given flows are similar enough to conclude they correspond to the same protocol. Combining this method with a classifier, most of the flows do not need to be explicitly evaluated by the later, so that the computational overhead is reduced without a significant reduction in accuracy. In this paper, as a case study, we explore complementing the pairing method with payload inspection. In the experiments performed, the pairing approach generalizes well to traffic obtained in different conditions and scenarios than that used for calibration. Moreover, a high portion of the traffic unclassified by payload inspection is categorized with the pairing method.     

Optimizing an artificial immune system algorithm in support of flow-Based internet traffic classification

The problem of classifying traffic flows in networks has become more and more important in recent times, and much research has been dedicated to it. In recent years, there has been a lot of interest in classifying traffic flows by application, based on the statistical features of each flow. Information about the applications that are being used on a network is very useful in network design, accounting, management, and security. In our previous work we proposed a classification algorithm for Internet traffic flow classification based on Artificial Immune Systems (AIS). We also applied the algorithm on an available data set, and found that the algorithm performed as well as other algorithms, and was insensitive to input parameters, which makes it valuable for embedded systems. It is also very simple to implement, and generalizes well from small training data sets. In this research, we expanded on the previous research by introducing several optimizations in the training and classification phases of the algorithm. We improved the design of the original algorithm in order to make it more predictable. We also give the asymptotic complexity of the optimized algorithm as well as draw a bound on the generalization error of the algorithm. Lastly, we also experimented with several different distance formulas to improve the classification performance. In this paper we have shown how the changes and optimizations applied to the original algorithm do not functionally change the original algorithm, while making its execution 50–60% faster. We also show that the classification accuracy of the Euclidian distance is superseded by the Manhattan distance for this application, giving 1–2% higher accuracy, making the accuracy of the algorithm comparable to that of a Naïve Bayes classifier in previous research that uses the same data set.     

Fast Filtered Sampling

Traffic sampled from the network backbone using uniform packet sampling is commonly utilized to detect heavy hitters, estimate flow level statistics, as well as identify anomalies like DDoS attacks and worm scans. Previous work has shown however that this technique introduces flow bias and truncation which yields inaccurate flow statistics and “drowns out” information from small flows, leading to large false positives in anomaly detection.In this paper, we present a new sampling design: Fast Filtered Sampling (FFS), which is comprised of an independent low-complexity filter, concatenated with any sampling scheme at choice. FFS ensures the integrity of small flows for anomaly detection, while still providing acceptable identification of heavy hitters. This is achieved through a filter design which suppresses packets from flows as a function of their size, “boosting” small flows relative to medium and large flows. FFS design requires only one update operation per packet, has two simple control parameters and can work in conjunction with existing sampling mechanisms without any additional changes. Therefore, it accomplishes a lightweight online implementation of the “flow-size dependent” sampling method. Through extensive evaluation on traffic traces, we show the efficacy of FFS for applications such as portscan detection and traffic estimation.     

An RL-based scheduling algorithm for video traffic in high-rate wireless personal area networks

The emerging high-rate wireless personal area network (WPAN) technology is capable of supporting high-speed and high-quality real-time multimedia applications. In particular, video streams are deemed to be a dominant traffic type, and require quality of service (QoS) support. However, in the current IEEE 802.15.3 standard for MAC (media access control) of high-rate WPANs, the implementation details of some key issues such as scheduling and QoS provisioning have not been addressed. In this paper, we first propose a Markov decision process (MDP) model for optimal scheduling for video flows in high-rate WPANs. Using this model, we also propose a scheduler that incorporates compact state space representation, function approximation, and reinforcement learning (RL). Simulation results show that our proposed RL scheduler achieves nearly optimal performance and performs better than F-SRPT, EDD + SRPT, and PAP scheduling algorithms in terms of a lower decoding failure rate.     

An overview of rate control techniques in HEVC and SHVC video encoding

Video standards are crucial for exchanging video content, enabling a myriad of services and supporting a wide variety of devices ranging from personal devices to clouds and IoT. One of the core requirements in video standards is the rate control that regulates the bit allocation and picture quality. This paper presents an overview of rate control techniques in the HEVC video coding standard. While providing an insight into the rate control mechanism specific to HEVC, it describes the basic operating principle of rate control algorithms, including their essential parameter, outputs, and performance measures. We review rate control in past coding standards and bring out the basic features of HEVC that drive the need for new rate control algorithms. Alongside, we delineate the Rate-Distortion model-based taxonomy of various algorithms, including their classification criteria. The paper gives out another classification of the rate control algorithms based on their basic principle and mechanisms. The article also explains the scalable extension of HEVC, namely SHVC, while highlighting some of the possible SHVC rate control design challenges. Finally, we present some of the unresolved research issues in HEVC rate control and outline possible future research directions.

     

Supporting high-quality video streaming with SDN-based CDNs

Videos and other multimedia contents become increasing popular among users of the Internet nowadays. With the improvement of underlying infrastructure of the Internet, users are allowed to enjoy video contents with much higher quality than last decade. Content delivery networks (CDNs) are a type of content hosting solution that widely used across the Internet. Content providers offload the task of content hosting to CDN providers and redirect users’ requests to CDNs. Video contents, especially high quality videos at real-time has occupying a major part of the Internet traffic. It is challenging to handle such workloads even for a large- scale CDN. Load balancing algorithms are critical to address this issue. However, traditional load balancing algorithms such as round-robin and randomization are unaware of user side requirements. Therefore, it is not uncommon that requests for high-quality videos at real-time are not satisfied. In this paper, we try to fulfill such requests by integrating software-defined networking technology with CDN infrastructure. We also propose revised load balancing algorithms and develop simulations to verify our approaches. The results show that the proposed algorithms achieve much higher user satisfaction in bandwidth-idle environments.     

基于模糊控制的流媒体自适应传输算法

基于HTTP的动态自适应流媒体DASH传输协议可以使用户根据自身的终端显示能力和信道条件选择合适的视频质量,是网络视频服务技术的发展方向。如何根据网络吞吐量的变化自适应地选择视频码率,以获得最佳的用户体验质量QOE,在已有的DASH系统中还没有得到很好的解决。 提出了一种基于模糊控制的自适应传输算法,将缓存的视频余量以及用户申请的视频码率和网络吞吐量的码率失配度作为输入,将预期的缓存变化量作为输出,通过模糊逻辑实现以下控制目的：(1）将缓存稳定在一个安全的区间;(2）使传输视频的平均质量最大化;(3）避免因带宽波动所造成的视频播放中断。最后,分别在两种虚拟网络环境和两种实际网络环境下进行性能测试,实验结果表明,与已有的算法相比较,本文提出的算法可以给用户带来更好的QOE。     

A machine learning approach for feature selection traffic classification using security analysis

Class imbalance has become a big problem that leads to inaccurate traffic classification. Accurate traffic classification of traffic flows helps us in security monitoring, IP management, intrusion detection, etc. To address the traffic classification problem, in literature, machine learning (ML) approaches are widely used. Therefore, in this paper, we also proposed an ML-based hybrid feature selection algorithm named WMI_AUC that make use of two metrics: weighted mutual information (WMI) metric and area under ROC curve (AUC). These metrics select effective features from a traffic flow. However, in order to select robust features from the selected features, we proposed robust features selection algorithm. The proposed approach increases the accuracy of ML classifiers and helps in detecting malicious traffic. We evaluate our work using 11 well-known ML classifiers on the different network environment traces datasets. Experimental results showed that our algorithms achieve more than 95% flow accuracy results.     

Identifying hazardousness of sewer pipeline gas mixture using classification methods: a comparative study

In this work, we formulated a real-world problem related to sewer pipeline gas detection using the classification-based approaches. The primary goal of this work was to identify the hazardousness of sewer pipeline to offer safe and non-hazardous access to sewer pipeline workers so that the human fatalities, which occurs due to the toxic exposure of sewer gas components, can be avoided. The dataset acquired through laboratory tests, experiments, and various literature sources was organized to design a predictive model that was able to identify/classify hazardous and non-hazardous situation of sewer pipeline. To design such prediction model, several classification algorithms were used and their performances were evaluated and compared, both empirically and statistically, over the collected dataset. In addition, the performances of several ensemble methods were analyzed to understand the extent of improvement offered by these methods. The result of this comprehensive study showed that the instance-based learning algorithm performed better than many other algorithms such as multilayer perceptron, radial basis function network, support vector machine, reduced pruning tree. Similarly, it was observed that multi-scheme ensemble approach enhanced the performance of base predictors.

     

The probability relationship between video’s instantaneous and average aggregate rates

The growing demand for video applications and services has contributed substantially to the increase of video traffic on the Internet. Measurement-based admission control was proposed with the primary aim of eliminating or reducing the need of flow state information; also to control overhead for admission decision and maximize utilization at the potential cost of QoS degradation. Some of the admission algorithms depend on the instantaneous rate for its operation. On the other hand, the average aggregate rate has been proposed to better suit variable rate such as video traffic. In this paper, we investigate the probability relationship between the instantaneous and average aggregate rates for video traffic. A mathematical model has been developed to quantify the probability relationship between both rates and validated through extensive simulations using real video sequences. The average rate was found to be lower than instantaneous for a small number of flows, however there was no pronounced difference for a large number of flows. Furthermore, the difference between both rates increases for fast moving content such as sport or longer measurement time window.     

基于网络流量预测的DASH系统优化

近年来基于超文本传输协议（HTTP）的自适应视频流量大幅上升,传统HTTP动态自适应流（DASH）速率算法无法准确预测网络吞吐量,导致网络带宽波动,使传输控制协议慢启动并触发抛弃规则,从而降低视频质量。提出一种基于网络流量预测的改进DASH速率算法。将DASH算法分为视频质量选择阶段、视频下载阶段和请求等待阶段,在视频质量选择阶段引入支持向量回归模型和长短期记忆网络预测网络吞吐量,结合缓冲时长选择更优质量的视频片段,在视频下载阶段通过预测实时吞吐量降低触发抛弃规则的次数。仿真结果表明,该算法可自适应流速率并减少抛弃规则的命中次数,有效提高视频体验质量。     

Joint In-network Video Rate Adaptation and Measurement-Based Admission Control: Algorithm Design and Evaluation

The important new revenue opportunities that multimedia services offer to network and service providers come with important management challenges. For providers, it is important to control the video quality that is offered and perceived by the user, typically known as the quality of experience (QoE). Both admission control and scalable video coding techniques can control the QoE by blocking connections or adapting the video rate but influence each other’s performance. In this article, we propose an in-network video rate adaptation mechanism that enables a provider to define a policy on how the video rate adaptation should be performed to maximize the provider’s objective (e.g., a maximization of revenue or QoE). We discuss the need for a close interaction of the video rate adaptation algorithm with a measurement based admission control system, allowing to effectively orchestrate both algorithms and timely switch from video rate adaptation to the blocking of connections. We propose two different rate adaptation decision algorithms that calculate which videos need to be adapted: an optimal one in terms of the provider’s policy and a heuristic based on the utility of each connection. Through an extensive performance evaluation, we show the impact of both algorithms on the rate adaptation, network utilisation and the stability of the video rate adaptation. We show that both algorithms outperform other configurations with at least 10 %. Moreover, we show that the proposed heuristic is about 500 times faster than the optimal algorithm and experiences only a performance drop of approximately 2 %, given the investigated video delivery scenario.     

Traffic information interpolation method based on traffic flow emergence using swarm intelligence

Traffic congestion has become one of the most pressing social problems in today’s society, and research into appropriate traffic signal control is actively underway. At present, most traffic signal control methods define traffic signal parameters on the basis of traffic information such as the number of passing vehicles. Installing sensors at a vast number of intersections is necessary for more precise and real-time adaptive control, but this is unrealistic from the viewpoint of cost. As an alternative, we propose a swarm intelligence-based methodology that creates routes with a similar traffic volume using the traffic information from intersections already equipped with sensors and interpolates this information in the intersections without sensors in real time. Our simulation results show that the proposed methodology can effectively create similar traffic routes for main traffic flows with high traffic volumes. The results also show that it has an excellent interpolation performance for heavy traffic flows and can adapt and interpolate to situations where traffic flow changes suddenly. Moreover, the interpolation results are highly accurate at a road link where traffic flows confluence. We also developed an interpolation algorithm that is adaptable to traffic patterns with confluence traffic flows. Experiments were conducted with a simulation of merging traffic flows and the proposed method showed good results.

     

Optimal volume anomaly detection and isolation in large-scale IP networks using coarse-grained measurements

Recent studies from major network technology vendors forecast the advent of the Exabyte era, a massive increase in network traffic driven by high-definition video and high-speed access technology penetration. One of the most formidable difficulties that this forthcoming scenario poses for the Internet is congestion problems due to traffic volume anomalies at the core network. In the light of this challenging near future, we develop in this work different network-wide anomaly detection and isolation algorithms to deal with volume anomalies in large-scale network traffic flows, using coarse-grained measurements as a practical constraint. These algorithms present well-established optimality properties in terms of false alarm and miss detection rate, or in terms of detection/isolation delay and false detection/isolation rate, a feature absent in previous works. This represents a paramount advantage with respect to current in-house methods, as it allows to generalize results independently of particular evaluations. The detection and isolation algorithms are based on a novel linear, parsimonious, and non-data-driven spatial model for a large-scale network traffic matrix. This model allows detecting and isolating anomalies in the Origin-Destination traffic flows from aggregated measurements, reducing the overhead and avoiding the challenges of direct flow measurement. Our proposals are analyzed and validated using real traffic and network topologies from three different large-scale IP backbone networks.