Actuator fault detection for masonry robot manipulator arm with the interval observer

The main construction method of building wall is artificial masonry, the main problem is that the process is associated with low construction efficiency and poor safety, workers are prone fall from high altitude. The research of automatic masonry robot has become an urgent need. The masonry mechanical arm system is the main executing part of the masonry robot, special attention should be paid to the robot fault. Therefore, it is necessary to establish a suitable model to detect the actuator faults of the manipulator system. In this paper, a dynamic model of manipulator fault is presented and a fault detection scheme of masonry robot manipulator arm is proposed based on the model. The model is simplified by analyzing the state parameters of each joint during robot masonry and the interval observer with more design freedom was designed based on the established mathematical model of actuator faults. In this paper, a joint method for solving S and L matrices is proposed, which avoids the limitation of the traditional method for solving L matrices by two-step. In the presence of external interference, $l_1$/$H_{\infty }$ performance are introduced into the generation process of residual interval, and the interval observer has better disturbance robustness and fault sensitivity. Simulation experiments verify that the scheme can effectively detect the actuator fault of the manipulator, and experiments are carried out on a 6-axis manipulator. The experimental results show that when actuator faults occur at joints 2 and 3, the residual rapidly exceeds the threshold range, which proves the effectiveness of the fault detection scheme designed in this paper.     

Power and energy consumption of skid-steer rovers turning on loose soil

This study highlights two key phenomena affecting power and energy consumption of skid-steer rovers on loose soil that is not present on the hard ground: soil excavation due to wheel counterrotation and impeded turning when dragging a braked wheel. Experiments in the field and in a controlled laboratory sandbox show that, on sand, power peaks by 15%–20% in a newly identified range of turns with radii between half the rover width, $B∕2$, to $R^{\prime }$, the radius at which the inner wheel does not turn. In this range of turns, the inner wheels rotate backwards but are being dragged forward through piles of sand they excavate by counterrotation. At $R\prime$, turns are shown to take much longer, leading to higher total energy consumption over time. Experiments in a controlled laboratory sandbox isolate the high motor torque and the resistance force experienced when a skid-steer rover drags a counterrotating or braked wheel, respectively, through loose soil. Other field experiments also demonstrate that paths combining circular arcs and lines can lead to energy savings of up to 15% relative to common ones consisting of point turns and lines; the experimental results suggest the circular arcs should have radii of approximately $2R\prime$. The quantitative values presented in this paper are specific to the rover and soils tested, but there are reasons to support the overall conclusions generalizing to all skid-steer rovers in loose soil.     

Longitudinal deep truck: Deep longitudinal model with application to sim2real deep reinforcement learning for heavy-duty truck control in the field

To develop cooperative adaptive cruise control (CACC), the choice of control approach often influences and can limit the choice of model structure, and vice versa. For heavy-duty trucks, practical application of CACC in the field is heavily influenced by the accuracy of the used model. Deep learning and deep reinforcement learning (deep-RL) have recently been used to demonstrate improved modeling and control performance for vehicles such as cars and quadrotors compared to state-of-the-art. The literature on the application of deep learning and deep-RL for heavy-duty trucks in the field, which are significantly more complex than cars, is still sparse, however. In this article, we develop a two-layer gray-box deep learning model to capture longitudinal dynamics of heavy-duty trucks while abstracting their complexity and present an approach to properly break the nested feedback loops in the model for training. We compare this model with three other alternative models and show that it achieves $~10x$ better general performance compared to a standard artificial neural network and results in $~4x$ and $~40x$ slower steady-state acceleration and speed error growth rates, respectively. We then present an architecture to utilize these deep learning models within the deep-RL framework and use it to develop baseline CACC controllers that can be zero-shot transferred to the field. To carry out the work, we present a setup of differently configured trucks along with their interface architecture and stochastic driving cycle generators for data collection. Numerical validation of the approach demonstrated stationary and bounded modeling error, and demonstrated transfer of CACC controllers with consistent overshoot bounds and a stable approximately-zero steady-state error. Validation from field experiments demonstrated similarly consistent results. Compared to a state-of-the-art benchmark, the deep-RL controller achieved lower speed and time-gap error variance but higher time-gap error offset.     

Field-based robotic leaf angle detection and characterization of maize plants using stereo vision and deep convolutional neural networks

Maize (Zea mays L.) is one of the three major cereal crops in the world. Leaf angle is an important architectural trait of crops due to its substantial role in light interception by the canopy and hence photosynthetic efficiency. Traditionally, leaf angle has been measured using a protractor, a process that is both slow and laborious. Efficiently measuring leaf angle under field conditions via imaging is challenging due to leaf density in the canopy and the resulting occlusions. However, advances in imaging technologies and machine learning have provided new tools for image acquisition and analysis that could be used to characterize leaf angle using three-dimensional (3D) models of field-grown plants. In this study, PhenoBot 3.0, a robotic vehicle designed to traverse between pairs of agronomically spaced rows of crops, was equipped with multiple tiers of PhenoStereo cameras to capture side-view images of maize plants in the field. PhenoStereo is a customized stereo camera module with integrated strobe lighting for high-speed stereoscopic image acquisition under variable outdoor lighting conditions. An automated image processing pipeline (AngleNet) was developed to measure leaf angles of nonoccluded leaves. In this pipeline, a novel representation form of leaf angle as a triplet of keypoints was proposed. The pipeline employs convolutional neural networks to detect each leaf angle in two-dimensional images and 3D modeling approaches to extract quantitative data from reconstructed models. Satisfactory accuracies in terms of correlation coefficient (r) and mean absolute error (MAE) were achieved for leaf angle ($r>0.87,MAE<5^{°}$) and internode heights ($r>0.99,MAE<3.5\mathrm{cm}$). Our study demonstrates the feasibility of using stereo vision to investigate the distribution of leaf angles in maize under field conditions. The proposed system is an efficient alternative to traditional leaf angle phenotyping and thus could accelerate breeding for improved plant architecture.     

Convergence on iterative learning control of Hilfer fractional impulsive evolution equations

In this paper, impulsive fractional differential equations with Hilfer fractional derivatives of order and type $0\le \nu \le 1$ is considered. Convergence analysis of $P$-type and $P{I}^{\mu }$-type open-loop iterative learning scheme is studied in the sense of $\lambda$-norm. Examples are provided to explain the theory developed.     

NDT-6D for color registration in agri-robotic applications

Registration of point cloud data containing both depth and color information is critical for a variety of applications, including in-field robotic plant manipulation, crop growth modeling, and autonomous navigation. However, current state-of-the-art registration methods often fail in challenging agricultural field conditions due to factors such as occlusions, plant density, and variable illumination. To address these issues, we propose the NDT-6D registration method, which is a color-based variation of the Normal Distribution Transform (NDT) registration approach for point clouds. Our method computes correspondences between pointclouds using both geometric and color information and minimizes the distance between these correspondences using only the three-dimensional (3D) geometric dimensions. We evaluate the method using the GRAPES3D data set collected with a commercial-grade RGB-D sensor mounted on a mobile platform in a vineyard. Results show that registration methods that only rely on depth information fail to provide quality registration for the tested data set. The proposed color-based variation outperforms state-of-the-art methods with a root mean square error (RMSE) of 1.1–1.6 cm for NDT-6D compared with 1.1–2.3 cm for other color-information-based methods and 1.2–13.7 cm for noncolor-information-based methods. The proposed method is shown to be robust against noises using the TUM RGBD data set by artificially adding noise present in an outdoor scenario. The relative pose error (RPE) increased $~$14% for our method compared to an increase of $~$75% for the best-performing registration method. The obtained average accuracy suggests that the NDT-6D registration methods can be used for in-field precision agriculture applications, for example, crop detection, size-based maturity estimation, and growth modeling.     

Distributed dynamic matrix control with constrained optimization for collision and obstacle avoidance of simulated multiple quadcopters

A system of fast moving quadcopters has a high risk of collisions with neighboring quadcopters or obstacles. The objective of this work is to develop a control strategy for collision and obstacle avoidance of multiple quadcopters. In this paper, the problem of distributed dynamic matrix control (DMC) for collision avoidance among a team of multiple quadcopters attempting to reach consensus in the horizontal plane and yaw direction ( $x,y$, and $\psi$) is investigated. Violations of a predetermined safety radius generates output constraints on the DMC optimization function, which has not been dealt with in the literature. Different from past works, the proposed strategy can perform collision avoidance in the $x$, $y$, and $z$-directions. In addition, logarithmic barrier functions are implemented as input rate constraints on the control actions. Extensive simulation studies for a team of quadcopters illustrate promising results of the proposed control strategy and case variations. In addition, DMC parameter effects on the system performance are studied, and a successful study for obstacle avoidance is presented.     

Bifurcation and control of disease spreading networks model with two delays

In this paper, the dynamical behaviors are investigated for a complex network with two independent delays. Instead of taking time delays as bifurcation parameters, we choose probability $p$ and parameter $\mu$ as the control parameters to study their effects on local stability and Hopf bifurcation, respectively. Moreover, the conditions for generating Hopf bifurcation are given. Furthermore, we further discuss the effects of two time delays on the critical values of parameters $p$ and $\mu$. Finally, numerical simulations are used to illustrate the validity of the obtained results.     

Risk-sensitive large-population linear-quadratic-Gaussian Games with major and minor agents

This paper studies large-population dynamic games involving a linear-quadratic-Gaussian (LQG) system with an exponential cost functional. The parameter in the cost functional can describe an investor's risk attitude. In the game, there are a major agent and a population of $N$ minor agents where $N$ is very large. The agents in the games are coupled via both their individual stochastic dynamics and their individual cost functions. The mean field methodology yields a set of decentralized controls, which are shown to be an $ϵ$-Nash equilibrium for a finite $N$ population system where $ϵ=O\left(\frac{1}{\sqrt{N}}\right)$. Numerical results are established to illustrate the impact of the population's collective behaviors and the consistency of the mean field estimation.     

TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge

For decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems, like, disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground for new technology at the limits of the autonomous systems capabilities. This paper outlines the software stack and approach of the TUM Autonomous Motorsport team for their participation in the IAC, which holds two competitions: A single-vehicle competition on the Indianapolis Motor Speedway and a passing competition at the Las Vegas Motor Speedway. Nine university teams used an identical vehicle platform: A modified Indy Lights chassis equipped with sensors, a computing platform, and actuators. All the teams developed different algorithms for object detection, localization, planning, prediction, and control of the race cars. The team from Technical University of Munich (TUM) placed first in Indianapolis and secured second place in Las Vegas. During the final of the passing competition, the TUM team reached speeds and accelerations close to the limit of the vehicle, peaking at around $270\text{km h}{}^{-1}$ and $28ms{}^{-2}$. This paper will present details of the vehicle hardware platform, the developed algorithms, and the workflow to test and enhance the software applied during the 2-year project. We derive deep insights into the autonomous vehicle's behavior at high speed and high acceleration by providing a detailed competition analysis. On the basis of this, we deduce a list of lessons learned and provide insights on promising areas of future work based on the real-world evaluation of the displayed concepts.     

A dual-channel ensembled deep convolutional neural network for facial expression recognition in the wild

Facial expression recognition (FER) in the wild is an active and challenging field of research. A system for automatic FER finds use in a wide range of applications related to advanced human–computer interaction (HCI), human–robot interaction (HRI), human behavioral analysis, gaming and entertainment, etc. Since their inception, convolutional neural networks (CNNs) have attained state-of-the-art accuracy in the facial analysis task. However, recognizing facial expressions in the wild with high confidence running on a low-cost embedded device remains challenging. To this end, this study presents an efficient dual-channel ensembled deep CNN (DCE-DCNN) for FER in the wild. Initially, two DCNNs, namely the ${\text{DCNN}}_G$ and ${\text{DCNN}}_S$, are trained separately on the grayscale and Scharr-convolved vertical gradient facial images, respectively. The proposed network later integrates the two pre-trained DCNNs to obtain the dual-channel integrated DCNN (DCI-DCNN). Finally, all three neural networks, namely the ${\text{DCNN}}_G$, ${\text{DCNN}}_S$, and DCI-DCNN, are jointly fine-tuned to get a single dual-channel-multi-output model. The multi-output model produces three prediction scores for the given input facial image. The prediction scores are thus fused using the max-voting ensemble scheme to obtain the DCE-DCNN with the final classification label. On the FER2013, RAF-DB, NCAER-S, AffectNet, and CKPlus benchmark FER datasets, the proposed DCE-DCNN consistently outperforms the two individual DCNNs and numerous state-of-the-art CNNs. Moreover, the network achieves competitive recognition accuracy on all four FER in the wild datasets with reduced memory storage size and parameters. The proposed DCE-DCNN model with high throughput on resource-limited embedded devices is suitable for applications that seek real-time classification of facial expressions in the wild with high confidence.     

Interval observer-based consensus control of discrete-time multi-agent systems under stealthy attacks

In this paper, interval observer-based consensus control is developed for multi-agent systems with unknown input signals, sensor noises, and stealthy attack signals. Compared with the conventional robust observer, the interval observer has more relaxed preconditions for the systems, so the interval observer is more extensive. First, design an interval observer for each subsystem and linearly transform into a form that is easy to obtain the coefficient matrices of the interval observer. Then, based on the upper bound estimation of the observer, design a consensus control protocol, and the $H_{\infty }$ technique is used to suppress the unknown input signals. Finally, a numerical simulation example is presented to illustrate the feasibility and correctness of the developed method.     

On the regional control problem of the bilinear wave equation

The present research deals with regional optimal control problem of the bilinear wave equation evolving on a spatial domain $\mathrm{\Omega }\subset {\mathrm{ℝ}}^n,n\ge 1$. Such an equation is excited by bounded controls that act on the velocity term. It addresses the tracking of a desired state all over the time interval $[0,T]$ only on a subregion $\omega$ of $\mathrm{\Omega }$ with minimum energy. Then, we prove that an optimal control exists and is characterized as a solution to an optimality system. Algorithm for the computation of such a control is given and successfully illustrated through simulations.     

Lumped uncertainty alleviation in output channels of MIMO nonlinear systems based on robust disturbance observer-based control strategy

Disturbances and uncertainties can produce unsatisfactory responses in many industrial and engineering systems. Besides, the practical systems and processes are multiple-input multiple-output (MIMO). Hence, achieving a good control performance with adequate output responses is not simple. Many different methods were provided for control of industrial processes in some references. However, in this paper, the primary goal is to design an appropriate tracking controller for alleviating the destructive effects of uncertainties in output channels of MIMO nonlinear systems. For this purpose, a robust mechanism has been introduced according to the optimal design of centralized extended proportional-derivative (CEPD) and disturbance observer (DOB). By designing the derivative part $K_d$ based on famous Vandermonde matrix and DOB gain $\mathrm{\Gamma }$, the robust criterion $R={\left(I+CG{K}_d\right)}^{-1}$ is obtained to tackle the undesirable factors such as nonlinear functions and uncertainties in error dynamics. The closed-loop stability is guaranteed by tuning the proportional part $K_p$ under linear matrix inequality. The proposed scheme in this paper can be used for a wide range of MIMO nonlinear systems in practical situations.     

Existence and stability of impulsive stochastic coupled systems in infinite dimensions

This paper considers the exponential stability of a class of infinite-dimensional impulsive stochastic coupled systems. With the help of generalized Itô's formula for the mild solution of infinite-dimensional systems, we avoid limiting the domain of the mild solution. Then we use the combination of the Lyapunov function and graph theory to construct the Lyapunov function of the systems; the criteria of $p$-th moment exponential stability are obtained, which is related to the average impulsive interval $T_a$ and the connectivity of impulsive stochastic systems. In addition, noting that the existence may be affected by impulsive effects and stochastic perturbations, using the graph theory and the principle of contraction mapping, we get the condition that guarantees the existence and uniqueness, which is also related to the structure of the networks. Finally, we consider the stability of impulsive stochastic coupled heat equations and neural networks with reaction diffusion and give some numerical simulations to verify the theoretical results.     

An improved nonlinear extended state observer with adaptive variable gain

This paper proposes an improved nonlinear extended observer with adaptive gain (ANESO), which can be applied to systems with large disturbance amplitude changes without parameter re-tuning. An ANESO is added an adaptive factor $A_i$ to the gain of the original NESO. The expression of $A_i$ is obtained by assuming that nonlinear extended states observer (NESO) and linear extended states observer (LESO) has the same characteristic of steady-state error when the amplitude of disturbance turns. On this basis, an adaptive adjustment algorithm based on tracking error is designed. In addition, the stability of the proposed ANESO is analyzed by using the Routh-Hurwitz criterion. Finally, through simulation experiments, the observation performance of ANESO was compared with NESO and switching extended state observer (SESO). The experimental results demonstrate that the observation performance of ANESO is hardly affected by the disturbance amplitude, and moreover, its observation precision is higher than that of NESO and SESO.     

Observer-based event-triggered H-infinity sliding control of Markovian jump system suffer from actuator attacks

This article investigates the issue of observer-based $H_{\infty }$ sliding mode control for Markovian jump systems suffer from actuator attacks through an adaptive technique. During the communication channel from the plant output to estimator, a dynamic event-triggered generator is employed in enhancing communication efficiency. Taking consideration of malicious attacks on the plant actuator, an adaptive compensator is put forward for security purposes. By designing a state observer, the desired sliding mode dynamics can be derived based on an integral-type sliding surface. Further, maintaining the sliding motion with uncertain mode information is ensured in finite time by proposing a feasible sliding mode control law. In addition, both stochastic stability and $H_{\infty }$ performance conditions are established for closed-loop systems in terms of linear matrix inequalities. Finally, a numerical example is offered to illustrate the validity of the constructed strategy.     

Linear Diophantine equation (LDE) decoder: A training-free decoding algorithm for multifrequency SSVEP with reduced computation cost

Multifrequency steady-state visual evoked potentials (SSVEPs) have been developed to extend the capability of SSVEP-based brain-machine interfaces (BMIs) to complex applications that have large numbers of targets. Even though various multifrequency stimulation methods have been introduced, the decoding algorithms for multifrequency SSVEP are still in early development. The recently developed multifrequency canonical correlation analysis (MFCCA) was shown to be a feasible training-free option to use in decoding multifrequency SSVEPs. However, the time complexity of MFCCA is shown to be $O(n^3)$, which will lead to long computation time as $n$ grows, where $n$ represents the input size in decoding. In this paper, a novel decoding algorithm is proposed with the aim to reduce the time complexity. This algorithm is based on linear Diophantine equation solvers and has a reduced computation cost $O(nlogn)$ while remaining training-free. Our simulation results demonstrated that linear Diophantine equation (LDE) decoder run time is only one fifth of MFCCA run time under respective optimal settings on 5-s single-channel data. This reduced computation cost makes it easier to implement multifrequency SSVEP in real-time systems. The effectiveness of this new decoding algorithm is validated with nine healthy participants when using dry electrode scalp electroencephalography (EEG).