音乐情感自动分析研究

音乐情感自动分析在音乐检索和音乐推荐等方面具有广泛的应用.对3种音乐情感模型进行了对比分析,介绍了音乐情感分类方法,并指出已有研究存在的不足.音乐分割与摘要是高效音乐浏览与推荐的基础,在对音乐分割与摘要方法进行分析的基础上,指出了定长分割策略的不足;借助音乐相似性与情感可视化实现音乐推荐,对音乐相似性度量与可视化方法进行了概述.最后,展望了对音乐情感自动分析的研究方向.     

An approach of genetic programming for music emotion classification

In this paper, we suggest a new approach of genetic programming for music emotion classification. Our approach is based on Thayer’s arousal-valence plane which is one of representative human emotion models. Thayer’s plane which says human emotions is determined by the psychological arousal and valence. We map music pieces onto the arousal-valence plane, and classify the music emotion in that space. We extract 85 acoustic features from music signals, rank those by the information gain and choose the top k best features in the feature selection process. In order to map music pieces in the feature space onto the arousal-valence space, we apply genetic programming. The genetic programming is designed for finding an optimal formula which maps given music pieces to the arousal-valence space so that music emotions are effectively classified. k-NN and SVM methods which are widely used in classification are used for the classification of music emotions in the arousal-valence space. For verifying our method, we compare with other six existing methods on the same music data set. With this experiment, we confirm the proposed method is superior to others.     

Affective video content representation and modeling

This paper looks into a new direction in video content analysis - the representation and modeling of affective video content . The affective content of a given video clip can be defined as the intensity and type of feeling or emotion (both are referred to as affect) that are expected to arise in the user while watching that clip. The availability of methodologies for automatically extracting this type of video content will extend the current scope of possibilities for video indexing and retrieval. For instance, we will be able to search for the funniest or the most thrilling parts of a movie, or the most exciting events of a sport program. Furthermore, as the user may want to select a movie not only based on its genre, cast, director and story content, but also on its prevailing mood, the affective content analysis is also likely to contribute to enhancing the quality of personalizing the video delivery to the user. We propose in this paper a computational framework for affective video content representation and modeling. This framework is based on the dimensional approach to affect that is known from the field of psychophysiology. According to this approach, the affective video content can be represented as a set of points in the two-dimensional (2-D) emotion space that is characterized by the dimensions of arousal (intensity of affect) and valence (type of affect). We map the affective video content onto the 2-D emotion space by using the models that link the arousal and valence dimensions to low-level features extracted from video data. This results in the arousal and valence time curves that, either considered separately or combined into the so-called affect curve, are introduced as reliable representations of expected transitions from one feeling to another along a video, as perceived by a viewer.     

基于内容的音乐检索研究

对音乐的特征表示进行了详细的阐述,详细介绍了乐曲相似性度量方法,并字符串编码方法对音乐特征进行编码,然后使用蛋白质序列局部比对方法对2239首中国乐曲和3960首西方乐曲的音乐库进行大量的检索实验。获取了大量的实验数据,对实验结果进行分析,并取得了较好的检索性能。最后针对音乐检索过程中的每次比对都相互无关的特点,具有可以很好地进行并行化特性,对算法进行并行化,实现在4台高档微机构成的群集上进行了并行检索实验,其检索时间约为串行检索时间的1／4,有很高的加速比。     

End-to-end music emotion variation detection using iteratively reconstructed deep features

Multimedia Tools and Applications - Automatic music emotion recognition (MER) has received increased attention in areas of music information retrieval and user interface development. Music emotion...     

EEG based emotion recognition using fusion feature extraction method

Multimedia Tools and Applications - As a high-level function of the human brain, emotion is the external manifestation of people’s psychological characteristics. The emotion has a great...     

Stereo music visualization through manifold harmonics

Music visualizations are nowadays included with virtually any media player. They usually rely on harmonic analysis of each sound channel, which automatically generate parameters for procedural image generation. However, only few music visualizations make use of 3d shapes. This paper proposes to use spectral mesh processing techniques, here manifold harmonics, to produce 3d stereo music visualization. The images are generated from 3d models by deforming an initial shape, mapping the sound frequencies to the mesh harmonics. A symmetry criterion is introduced to enhance the stereo effects on the deformed shape. A concise representation of the frequency mapping is proposed to allow for an animated gallery interface with genetic reproduction. Such galleries let the user quickly navigate between visual effects. Rendering such animated galleries in real time is a challenging task, since it requires computing and rendering the deformed shapes at a very high rate. This paper introduces a direct GPU implementation of manifold harmonics filters, which allows the displaying of the animated galleries.     

Personalized smart home audio system with automatic music selection based on emotion

Multimedia Tools and Applications - In this paper, we introduce a personalized home audio system that uses IoT technologies to recommend and play music remotely based on a user’s estimated...     

Object oriented music analysis

An object oriented approach to programming of music analysis is described. Different kinds of structural analysis are shown, ranging from simple melodic analysis, through harmonic analysis to cluster analysis used for analysis of modern music.     

Collaborative service system design for music content creation

This article presents a novel collaborative service system (CSS) design methodology supporting the work of multiple participating users and collaborative services processes. The CSS application of music content creation paradigm called DesignStorming as it pertains to CSS modeling and developing. CSS is an interactive service system that can be systematic service innovation and automatic semi-automated value co-creation between service providers and customers to enhance service productivity. The objectives of this research are threefold─to propose a novel design methodology for CSS, to incorporate simple service machine (SSM) and intelligent service machine (ISM) for CSS, and to demonstrate a CSS application of music content creation. SSM is a road map that can be used to define the salient attribute elements for CSS design. According to the predefined SSM, ISM conducts modeling the cognitive process of service exchanges and service provision, their knowledge representations, and value co-production process for a computerized CSS. DesignStorming demonstrates how a CSS can be automated the interactive communications and problem solving processes during the music content creation work. The CSS application involves the three system components: 1) Ontology Developer, 2) SFGA Partnership Matcher, and 3) Co-created Value Appraiser. In addition, a service evaluation model is described and evaluated, so that some of their important characteristics can be identified. Such a comprehensive design methodology can provide the foundation for building future more diversified and innovative collaborative service system.     

乐谱识别中音符结构分析方法

将物理学中的作用场概念引入到音符基元的关系表达,采用作用场的势函数描述音符基元闻的位置关系,在表达方式上实现了知识性、鲁棒性和精确性三者的统一.对音符结构进行细分,定义了6个音符子结构,并对各个子结构的关键程度进行分析和排序,设计了关键结构优先定位的音符结构分析流程和算法.该方法模拟了人工识谱时突出重点,从整体到细节的思维习惯.实验结果表明,该方法对识别各种复杂音符有良好的正确性、适应性和容错性.     

Modeling emotional content of music using system identification.

Research was conducted to develop a methodology to model the emotional content of music as a function of time and musical features. Emotion is quantified using the dimensions valence and arousal, and system-identification techniques are used to create the models. Results demonstrate that system identification provides a means to generalize the emotional content for a genre of music. The average R2 statistic of a valid linear model structure is 21.9% for valence and 78.4% for arousal. The proposed method of constructing models of emotional content generalizes previous time-series models and removes ambiguity from classifiers of emotion.     

Criminal psychological emotion recognition based on deep learning and EEG signals

Neural Computing and Applications - The difficulty of criminal psychological recognition is that it is difficult to classify emotions, and the accuracy of traditional recognition methods is...     

A novel DRM framework for peer-to-peer music content delivery

Due to rapid advances in the network communications field in recent years, the distribution of large-scale music contents has become easier and more efficient than ever before. However, the unauthorized distribution of copyright-protected content has emerged as a major concern. Accordingly, this paper presents a content distribution framework with a DRM capability for P2P networks. The robustness of the content distribution is ensured by using a network coding approach based on the Lagrange polynomial interpolation method. When the downloading peer within the network receives sufficient coded pieces, it not only reconstructs the associated blocks using a finite field Gaussian elimination method, but also creates its own copies of the coded pieces within these blocks and shares these copies amongst the other peers in the network. As a result, the distribution overhead imposed on the music provider is substantially reduced and the number of coded pieces within the network is significantly increased, thereby overcoming the “last piece problem” inherent in existing P2P schemes. In the DRM module of the framework, the RSA public-key cryptosystem is used to generate a unique digital fingerprint for every user within the network. The fingerprint is embedded within the music file in a protected form such that the music provider can establish the identification of any user performing an unauthorized distribution of the file. The experimental results confirm that the proposed framework provides an efficient and secure means of distributing large-scale copyright-protected music contents with no discernible degradation in the audio quality.     

Machine learning model for mapping of music mood and human emotion based on physiological signals

Emotion is considered a physiological state that appears whenever a transformation is observed by an individual in their environment or body. While studying the literature, it has been observed that combining the electrical activity of the brain, along with other physiological signals for the accurate analysis of human emotions is yet to be explored in greater depth. On the basis of physiological signals, this work has proposed a model using machine learning approaches for the calibration of music mood and human emotion. The proposed model consists of three phases (a) prediction of the mood of the song based on audio signals, (b) prediction of the emotion of the human-based on physiological signals using EEG, GSR, ECG, Pulse Detector, and finally, (c) the mapping has been done between the music mood and the human emotion and classifies them in real-time. Extensive experimentations have been conducted on the different music mood datasets and human emotion for influential feature extraction, training, testing and performance evaluation. An effort has been made to observe and measure the human emotions up to a certain degree of accuracy and efficiency by recording a person’s bio- signals in response to music. Further, to test the applicability of the proposed work, playlists are generated based on the user’s real-time emotion determined using features generated from different physiological sensors and mood depicted by musical excerpts. This work could prove to be helpful for improving mental and physical health by scientifically analyzing the physiological signals.