Scaling up stochastic gradient descent for non-convex optimisation

Stochastic gradient descent (SGD) is a widely adopted iterative method for optimizing differentiable objective functions. In this paper, we propose and discuss a novel approach to scale up SGD in applications involving non-convex functions and large datasets. We address the bottleneck problem arising when using both shared and distributed memory. Typically, the former is bounded by limited computation resources and bandwidth whereas the latter suffers from communication overheads. We propose a unified distributed and parallel implementation of SGD (named DPSGD) that relies on both asynchronous distribution and lock-free parallelism. By combining two strategies into a unified framework, DPSGD is able to strike a better trade-off between local computation and communication. The convergence properties of DPSGD are studied for non-convex problems such as those arising in statistical modelling and machine learning. Our theoretical analysis shows that DPSGD leads to speed-up with respect to the number of cores and number of workers while guaranteeing an asymptotic convergence rate of \(O(1/\sqrt{T})\) given that the number of cores is bounded by \(T^{1/4}\) and the number of workers is bounded by \(T^{1/2}\) where T is the number of iterations. The potential gains that can be achieved by DPSGD are demonstrated empirically on a stochastic variational inference problem (Latent Dirichlet Allocation) and on a deep reinforcement learning (DRL) problem (advantage actor critic - A2C) resulting in two algorithms: DPSVI and HSA2C. Empirical results validate our theoretical findings. Comparative studies are conducted to show the performance of the proposed DPSGD against the state-of-the-art DRL algorithms.

     

Adaptive control: theory and practical aspects

An a overview of discrete adaptive contollers is given. A unified presentation of the underlying control designs is made and the fundamental properties of the parameter estimations to be used are given. Particular emphasis is put on the design assumptions and several adaptive control applications are reported.     

Term rewriting and its application to recognizing handwritten Hindu numerals

In this paper the theoretical basis is presented and the implementation of a term rewriting system based on algebraic specifications is described. The input to this system is represented by an algebraic specification language, which forms not only the set of axioms but also the sorts, variables, operators and terms of a specific simulated theory or application. Rewriting and matching mechanisms provide the formal methodology for evaluating terms and proving assertions in an algebraic theory. Specifications are evaluated by interpreting terms by means of rewrite rules. The rules are described by the axioms of the specifications where the finite termination and congruence properties are assumed. A term rewriting system to recognize handwritten Hindu numerals is introduced as a case study. Besides rewriting, a robust algorithm is proposed to segment the numeral's image into strokes based on feature points and to identify cavity features. A syntactic representation (term) of the input image is matched and rewritten against a set of rules. Experimental results proved that the proposed system is tolerant to recognize a variety of numeral shapes with 96% successful recognition rate.     

Robust formation control without velocity measurement of the leader robot

In this paper a control problem of leader–follower motion coordination of multiple nonholonomic mobile robots is addressed and subsequently in the proposed scheme, a reference trajectory generated based on the information from the leader is tracked by the follower robots. To alleviate demanded information on the leader, specifically to eliminate the measurement requirement or estimation of the leader's velocity and dynamics, a virtual vehicle is constructed whereby its trajectory converges to the reference trajectory of the follower. Trajectory tracking controller is then designed to allow the follower robot to track the virtual vehicle using neural network approximation, in combination with the backstepping and Lyapunov direct design technique and finally the performance and effectiveness of the controller is verified throughout the experiments.     

Ion track-based electronic elements

In the past years, fundaments were set for a new type of electronics which is based on tracks in insulators formed by individual or multiple swift heavy ions. Due to the possibility of inserting any (semi)conducting material into these tracks, various active and passive electronic devices can be created. Among them are also transistor-like and Esaki diode-like elements. As many of these structures have sensing properties and the capability to undergo logic decisions, autonomous intelligent sensors appear to be a favourite field for future application. The use of liquid conductors may even expand the range of applicability towards medical implants.     

STOPE‐based approach for e‐readiness assessment case studies

The STOPE‐based approach is used here for practical e‐readiness assessment case studies. The approach integrates and evaluates the various e‐readiness assessment issues, over its well‐structured domains: strategy, technology, organization, people, and environment. The case studies considered concern three Saudi organizations: a government organization, a bank, and a private sector company. The results obtained provide e‐readiness indicators that illustrate the strengths and weaknesses of each organization with regard to e‐readiness strategic development issues, information and communication technology (ICT) issues, management and organization issues, people issues, and environment issues within which the organization operates. The work shows how the approach can be practically used, and it also provides practical results that direct the organizations concerned toward enhancing their e‐readiness in response to the current trend of globalizing e‐business. Copyright © 2007 John Wiley & Sons, Ltd.     

Probiotic potential and sensory properties of coconut flan supplemented with Lactobacillus paracasei and Bifidobacterium lactis

The effect of probiotic cultures on sensory performance of coconut flan during storage at 5 °C and the viability of these micro organisms for up to 28 days were investigated. Sensory analyses of the product were performed after 7, 14 and 21 days of storage. Coconut flans were produced with no addition of cultures (T1, control), or supplemented with Bifidobacterium lactis (T2), Lactobacillus paracasei (T3) and B. lactis  +  L. paracasei (T4). Populations of L. paracasei and B. lactis as single or in co-culture remained above 7 log CFU g⁻¹ during the entire storage period. Viability of L. paracasei was higher for T3. All products were well accepted and no significant differences ( P  > 0.05) were detected between the coconut flans studied. The addition of L. paracasei and B. lactis to coconut flan resulted in its having great potential as a functional food, which has high sensory acceptability.     

Model Free Approach for Non-Isothermal Decomposition of Un-Irradiated and γ-Irradiated Silver Acetate: New Route for Synthesis of Ag(2)O Nanoparticles

Kinetic studies for the non-isothermal decomposition of unirradiated and γ-irradiated silver acetate with 10(3) kGy total γ-ray doses were carried out in air. The results showed that the decomposition proceeds in one major step in the temperature range of (180-270 °C) with the formation of Ag(2)O as solid residue. The non-isothermal data for un-irradiated and γ-irradiated silver acetate were analyzed using Flynn-Wall-Ozawa (FWO) and nonlinear Vyazovkin (VYZ) iso-conversional methods. These free models on the investigated data showed a systematic dependence of Ea on α indicating a simple decomposition process. No significant changes in the thermal decomposition behavior of silver acetate were recorded as a result of γ-irradiation. Calcinations of γ-irradiated silver acetate (CH(3)COOAg) at 200 °C for 2 hours only led to the formation of pure Ag(2)O mono-dispersed nanoparticles. X-ray diffraction, FTIR and SEM techniques were employed for characterization of the synthesized nanoparticles.     

Energy efficiency and renewable energy under extreme conditions: Case studies from Antarctica

This article showcases a range of small and large scale energy efficiency and renewable energy deployments at Antarctic research stations and field camps. Due to the cold and harsh environment, significant amounts of fuel are needed to support humans working and living in Antarctica. The purchase, transportation and storage of large amounts of fossil fuel entail significant economic costs and environmental risks and have motivated developments in energy efficiency and renewable energy deployment. Over the past three decades, improved building design, behavioral change, cogeneration, solar collectors, solar panels and wind turbines have been found to be effective in Antarctica, demonstrating that harsh environmental conditions and technological barriers do not have to limit the deployment of energy efficiency and renewable energy. The ambition to run entire stations or field camps on 100% renewable energy is increasingly common and feasible. While the power requirements of Antarctic research stations are small compared to urban installations on other continents, these case studies clearly demonstrate that if energy efficiency and renewable energy can be deployed widely on the coldest, darkest and most remote continent of the world, their deployment should be more widespread and encouraged on other continents.     

Hybrid Secure Equivalent Computing Model for Distributed Computing Applications

Wireless Personal Communications - Distributed computing applications provide concurrent processing and services executed from different systems through a common cloud platform. However, without...