TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge

For decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems, like, disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground for new technology at the limits of the autonomous systems capabilities. This paper outlines the software stack and approach of the TUM Autonomous Motorsport team for their participation in the IAC, which holds two competitions: A single-vehicle competition on the Indianapolis Motor Speedway and a passing competition at the Las Vegas Motor Speedway. Nine university teams used an identical vehicle platform: A modified Indy Lights chassis equipped with sensors, a computing platform, and actuators. All the teams developed different algorithms for object detection, localization, planning, prediction, and control of the race cars. The team from Technical University of Munich (TUM) placed first in Indianapolis and secured second place in Las Vegas. During the final of the passing competition, the TUM team reached speeds and accelerations close to the limit of the vehicle, peaking at around $270\text{km h}{}^{-1}$ and $28ms{}^{-2}$. This paper will present details of the vehicle hardware platform, the developed algorithms, and the workflow to test and enhance the software applied during the 2-year project. We derive deep insights into the autonomous vehicle's behavior at high speed and high acceleration by providing a detailed competition analysis. On the basis of this, we deduce a list of lessons learned and provide insights on promising areas of future work based on the real-world evaluation of the displayed concepts.     

Deep neural networks to predict autonomous ground vehicle behavior on sloping terrain field

Conventional large agricultural machinery or implements are unsafe and unsuitable to operate on slopes $>6^{\circ }$ or 10%. Tractor rollovers are frequent on slopes, precluding farming on arable hills, uneven or highly sloped land. Therefore, a fleet of autonomous ground vehicles (AGV) is proposed to cultivate highly sloped land ($>6^{\circ }$). The fleet aims to expand agricultural land to the slopes and to strengths the human-robot collaboration in an unsafe sloped environment. However, the fleet's success largely depends on vehicle behavior models regarding traction, mobility, and energy consumption on varying slopes. The vehicle intelligent behavior models are essential and would solve multiple objectives ranging from simulations to path planning & navigation. Therefore, this study aimed to build a deep learning-based vehicle behavior models on sloping terrain. A standard drawbar test was performed on a single AGV operating on an actual sloped field at varying speeds and load conditions. The drawbar test quantified the AGV's behavior on slopes in metrics related to traction (traction efficiency), mobility (travel reduction), and energy consumption (power number). Deep learning-based models were developed from the experimental data to predict the AGV's behavior on slopes as a function of vehicle velocity, drawbar, and slope. A special model called the proposed model, which combined multiple deep neural networks with a mixture of Gaussians, was developed and trained with a hybrid training method. The proposed model consistently outperformed the other well-known machine learning models. This study explored the capabilities of machine learning algorithms to simulate the behavior of small-track vehicle or AGV on sloping terrain. The fleet aims to provide safer agriculture keeping human safety in focus, and the developed predictive vehicle behavior models would empower the fleet's operation on currently unsafe sloped terrain by assisting in vehicle path planning, route optimization, and decision making.     

Power and energy consumption of skid-steer rovers turning on loose soil

This study highlights two key phenomena affecting power and energy consumption of skid-steer rovers on loose soil that is not present on the hard ground: soil excavation due to wheel counterrotation and impeded turning when dragging a braked wheel. Experiments in the field and in a controlled laboratory sandbox show that, on sand, power peaks by 15%–20% in a newly identified range of turns with radii between half the rover width, $B∕2$, to $R^{\prime }$, the radius at which the inner wheel does not turn. In this range of turns, the inner wheels rotate backwards but are being dragged forward through piles of sand they excavate by counterrotation. At $R\prime$, turns are shown to take much longer, leading to higher total energy consumption over time. Experiments in a controlled laboratory sandbox isolate the high motor torque and the resistance force experienced when a skid-steer rover drags a counterrotating or braked wheel, respectively, through loose soil. Other field experiments also demonstrate that paths combining circular arcs and lines can lead to energy savings of up to 15% relative to common ones consisting of point turns and lines; the experimental results suggest the circular arcs should have radii of approximately $2R\prime$. The quantitative values presented in this paper are specific to the rover and soils tested, but there are reasons to support the overall conclusions generalizing to all skid-steer rovers in loose soil.     

Convergence on iterative learning control of Hilfer fractional impulsive evolution equations

In this paper, impulsive fractional differential equations with Hilfer fractional derivatives of order and type $0\le \nu \le 1$ is considered. Convergence analysis of $P$-type and $P{I}^{\mu }$-type open-loop iterative learning scheme is studied in the sense of $\lambda$-norm. Examples are provided to explain the theory developed.     

Longitudinal deep truck: Deep longitudinal model with application to sim2real deep reinforcement learning for heavy-duty truck control in the field

To develop cooperative adaptive cruise control (CACC), the choice of control approach often influences and can limit the choice of model structure, and vice versa. For heavy-duty trucks, practical application of CACC in the field is heavily influenced by the accuracy of the used model. Deep learning and deep reinforcement learning (deep-RL) have recently been used to demonstrate improved modeling and control performance for vehicles such as cars and quadrotors compared to state-of-the-art. The literature on the application of deep learning and deep-RL for heavy-duty trucks in the field, which are significantly more complex than cars, is still sparse, however. In this article, we develop a two-layer gray-box deep learning model to capture longitudinal dynamics of heavy-duty trucks while abstracting their complexity and present an approach to properly break the nested feedback loops in the model for training. We compare this model with three other alternative models and show that it achieves $~10x$ better general performance compared to a standard artificial neural network and results in $~4x$ and $~40x$ slower steady-state acceleration and speed error growth rates, respectively. We then present an architecture to utilize these deep learning models within the deep-RL framework and use it to develop baseline CACC controllers that can be zero-shot transferred to the field. To carry out the work, we present a setup of differently configured trucks along with their interface architecture and stochastic driving cycle generators for data collection. Numerical validation of the approach demonstrated stationary and bounded modeling error, and demonstrated transfer of CACC controllers with consistent overshoot bounds and a stable approximately-zero steady-state error. Validation from field experiments demonstrated similarly consistent results. Compared to a state-of-the-art benchmark, the deep-RL controller achieved lower speed and time-gap error variance but higher time-gap error offset.     

On the regional control problem of the bilinear wave equation

The present research deals with regional optimal control problem of the bilinear wave equation evolving on a spatial domain $\mathrm{\Omega }\subset {\mathrm{ℝ}}^n,n\ge 1$. Such an equation is excited by bounded controls that act on the velocity term. It addresses the tracking of a desired state all over the time interval $[0,T]$ only on a subregion $\omega$ of $\mathrm{\Omega }$ with minimum energy. Then, we prove that an optimal control exists and is characterized as a solution to an optimality system. Algorithm for the computation of such a control is given and successfully illustrated through simulations.     

Field-based robotic leaf angle detection and characterization of maize plants using stereo vision and deep convolutional neural networks

Maize (Zea mays L.) is one of the three major cereal crops in the world. Leaf angle is an important architectural trait of crops due to its substantial role in light interception by the canopy and hence photosynthetic efficiency. Traditionally, leaf angle has been measured using a protractor, a process that is both slow and laborious. Efficiently measuring leaf angle under field conditions via imaging is challenging due to leaf density in the canopy and the resulting occlusions. However, advances in imaging technologies and machine learning have provided new tools for image acquisition and analysis that could be used to characterize leaf angle using three-dimensional (3D) models of field-grown plants. In this study, PhenoBot 3.0, a robotic vehicle designed to traverse between pairs of agronomically spaced rows of crops, was equipped with multiple tiers of PhenoStereo cameras to capture side-view images of maize plants in the field. PhenoStereo is a customized stereo camera module with integrated strobe lighting for high-speed stereoscopic image acquisition under variable outdoor lighting conditions. An automated image processing pipeline (AngleNet) was developed to measure leaf angles of nonoccluded leaves. In this pipeline, a novel representation form of leaf angle as a triplet of keypoints was proposed. The pipeline employs convolutional neural networks to detect each leaf angle in two-dimensional images and 3D modeling approaches to extract quantitative data from reconstructed models. Satisfactory accuracies in terms of correlation coefficient (r) and mean absolute error (MAE) were achieved for leaf angle ($r>0.87,MAE<5^{°}$) and internode heights ($r>0.99,MAE<3.5\mathrm{cm}$). Our study demonstrates the feasibility of using stereo vision to investigate the distribution of leaf angles in maize under field conditions. The proposed system is an efficient alternative to traditional leaf angle phenotyping and thus could accelerate breeding for improved plant architecture.     

Lumped uncertainty alleviation in output channels of MIMO nonlinear systems based on robust disturbance observer-based control strategy

Disturbances and uncertainties can produce unsatisfactory responses in many industrial and engineering systems. Besides, the practical systems and processes are multiple-input multiple-output (MIMO). Hence, achieving a good control performance with adequate output responses is not simple. Many different methods were provided for control of industrial processes in some references. However, in this paper, the primary goal is to design an appropriate tracking controller for alleviating the destructive effects of uncertainties in output channels of MIMO nonlinear systems. For this purpose, a robust mechanism has been introduced according to the optimal design of centralized extended proportional-derivative (CEPD) and disturbance observer (DOB). By designing the derivative part $K_d$ based on famous Vandermonde matrix and DOB gain $\mathrm{\Gamma }$, the robust criterion $R={\left(I+CG{K}_d\right)}^{-1}$ is obtained to tackle the undesirable factors such as nonlinear functions and uncertainties in error dynamics. The closed-loop stability is guaranteed by tuning the proportional part $K_p$ under linear matrix inequality. The proposed scheme in this paper can be used for a wide range of MIMO nonlinear systems in practical situations.     

NDT-6D for color registration in agri-robotic applications

Registration of point cloud data containing both depth and color information is critical for a variety of applications, including in-field robotic plant manipulation, crop growth modeling, and autonomous navigation. However, current state-of-the-art registration methods often fail in challenging agricultural field conditions due to factors such as occlusions, plant density, and variable illumination. To address these issues, we propose the NDT-6D registration method, which is a color-based variation of the Normal Distribution Transform (NDT) registration approach for point clouds. Our method computes correspondences between pointclouds using both geometric and color information and minimizes the distance between these correspondences using only the three-dimensional (3D) geometric dimensions. We evaluate the method using the GRAPES3D data set collected with a commercial-grade RGB-D sensor mounted on a mobile platform in a vineyard. Results show that registration methods that only rely on depth information fail to provide quality registration for the tested data set. The proposed color-based variation outperforms state-of-the-art methods with a root mean square error (RMSE) of 1.1–1.6 cm for NDT-6D compared with 1.1–2.3 cm for other color-information-based methods and 1.2–13.7 cm for noncolor-information-based methods. The proposed method is shown to be robust against noises using the TUM RGBD data set by artificially adding noise present in an outdoor scenario. The relative pose error (RPE) increased $~$14% for our method compared to an increase of $~$75% for the best-performing registration method. The obtained average accuracy suggests that the NDT-6D registration methods can be used for in-field precision agriculture applications, for example, crop detection, size-based maturity estimation, and growth modeling.     

Linear Diophantine equation (LDE) decoder: A training-free decoding algorithm for multifrequency SSVEP with reduced computation cost

Multifrequency steady-state visual evoked potentials (SSVEPs) have been developed to extend the capability of SSVEP-based brain-machine interfaces (BMIs) to complex applications that have large numbers of targets. Even though various multifrequency stimulation methods have been introduced, the decoding algorithms for multifrequency SSVEP are still in early development. The recently developed multifrequency canonical correlation analysis (MFCCA) was shown to be a feasible training-free option to use in decoding multifrequency SSVEPs. However, the time complexity of MFCCA is shown to be $O(n^3)$, which will lead to long computation time as $n$ grows, where $n$ represents the input size in decoding. In this paper, a novel decoding algorithm is proposed with the aim to reduce the time complexity. This algorithm is based on linear Diophantine equation solvers and has a reduced computation cost $O(nlogn)$ while remaining training-free. Our simulation results demonstrated that linear Diophantine equation (LDE) decoder run time is only one fifth of MFCCA run time under respective optimal settings on 5-s single-channel data. This reduced computation cost makes it easier to implement multifrequency SSVEP in real-time systems. The effectiveness of this new decoding algorithm is validated with nine healthy participants when using dry electrode scalp electroencephalography (EEG).     

Iterative learning algorithms for boundary tracing of fractional diffusion equations

We study P-type and PI ${}^{\theta }$-type iterative learning control (ILC) schemes for boundary tracing problem of nonhomogeneous fractional diffusion equations. Based on Sobolev imbedding theorem, we derive sufficient conditions for the convergence of four ILC schemes in the sense of $\lambda$-norm. Numerical examples are presented to illustrate effectiveness of the proposed control methods. The results show that closed-loop ILC scheme converges faster than open-loop ILC scheme; moreover, PI ${}^{\theta }$-type $(0.5<\theta <1)$ ILC scheme outperforms P-type and PI-type $(\theta =1)$ ILC schemes in terms of the convergence speed.     

Observer-based path following control for autonomous vehicles with localization errors and tire slip effects

This paper investigates the problem of path following control for an autonomous vehicle subject to the localization errors and the tire slip effects. First, by analyzing the effects of localization errors, a loss-of-effectiveness actuator model is formulated, and a new chain form model is constructed for path following system of the vehicle. Then, the polytopic model is proposed to characterize the nonlinearity of the system, and an observer-based path following controller is designed by satisfying both the $H_{\infty }$ criterion and $L_1$ criterion. Finally, the path following controller design problem is converted into an optimization problem, which can be solved readily through convex optimization techniques. The effectiveness of the proposed control strategy is verified by simulation results.     

An optimal robust design method for fractional-order reset controller

In this paper, a novel design method for the reset controller structure (i.e., fractional-order proportional and integral plus Clegg integrator (PI ${}^{\alpha }$ + CI ${}^{\alpha }$)), is proposed for a second-order plus time delay plant. To this end, the designer can get an optimal fractional reset controller that gives the control system more phase margin over the base linear PI controller and robust to loop gain variation. The describing function method is used to investigate the capability of phase lead and the frequency domain properties of PI ${}^{\alpha }$ + CI ${}^{\alpha }$. The gain crossover frequency and phase margin specifications ensure the stability of the control system, and the flat phase constraint makes the control system robust to loop gain variations. Meanwhile, the integral of time and absolute error (ITAE) value is applied to achieve the optimal dynamic performance as the cost function. PI ${}^{\alpha }$ + CI ${}^{\alpha }$ is compared with its integer-order counterpart (i.e., proportional and integral plus Clegg integrator (PI + CI) controller) and their base controllers (i.e., integer-order PI and fractional-order PI controllers) in terms of the step response and robustness to loop gain variations. The simulation results illustrate that the PI ${}^{\alpha }$ + CI ${}^{\alpha }$ control system obtains lower overshoot and oscillation and better robustness to loop gain variations than others. The experiments are performed on the speed control of an air bearing stage. Experimental results show that the designed PI ${}^{\alpha }$ + CI ${}^{\alpha }$ control system behaves better than others. The proposed PI ${}^{\alpha }$ + CI ${}^{\alpha }$ design method can be applied to other general control plants easily.     

Gradient-free algorithms for distributed online convex optimization

In this paper, we consider the distributed bandit convex optimization of time-varying objective functions over a network. By introducing perturbations into the objective functions, we design a deterministic difference and a randomized difference to replace the gradient information of the objective functions and propose two classes of gradient-free distributed algorithms. We prove that both the two classes of algorithms achieve regrets of $O(T^{3/4})$ for convex objective functions and $O(T^{2/3})$ for strongly convex objective functions, with respect to the time index $T$ and consensus of the estimates established as well. Simulation examples are given justifying the theoretical results.     

Active fault detection and isolation in linear time-invariant systems: A geometric approach

In this work, a novel approach on active fault detection and isolation for linear time-invariant systems, named forced diagnosability, is proposed. This approach computes a continuous state feedback law to render a fault diagnosable, even when it cannot be diagnosed by using passive diagnosis methods. To do that, this work derives novel geometric relationships between unobservability and $(A,B)$-invariant subspaces that, under certain conditions, guarantee the existence of such state feedback law. The objective of the state feedback law is to force all the faults, except the one required to be diagnosed, named $L_d$, to reside in an unobservability subspace. This effectively decouples the effect of $L_d$ on the system output, from the effect of the other faults, allowing the design of a residual generator to detect and isolate the desired fault. The proposed state feedback law continuously forces diagnosability, and it can be computed in polynomial time. This avoids testing faults only at fixed time intervals and solving complex optimization problems required in other active diagnosis approaches. A numerical example is presented to illustrate the efficiency of the proposed approach.     

Remark on stability margin of Lur'e systems

In a recent work, a definition of stability margin (gain and phase margins) for a class of nonlinear systems (Lur'e systems consisting of a linear time-invariant (LTI) plant and a sector-constrained nonlinearity) is proposed based on the famous circle criterion. This definition is indeed interesting because the concept of gain and phase margins has been largely limited to linear systems with a single input and a single output (SISO), but it was further established for Lur'e systems with a particular type of sector constraints ( $k_1=0$, e.g., saturation). In this paper, the previously established concept is extended to cover Lur'e systems with a general type of sector constraints ( $k_1\ge 0$). It is, however, pointed out that in case of $k_1=0$, the definitions of phase margin based on the circle criterion can be misleading and need to be modified or replaced perhaps by a time-delay margin for Lur'e systems including an integrator, for which the phase margin can be trivially zero.     

On Hurwitz stability for families of polynomials

The robustness of a linear system in the view of parametric variations requires a stability analysis of a family of polynomials. If the parameters vary in a compact set $A$, then obtaining necessary and sufficient conditions to determine stability of the family ${\mathfrak{F}}_A$ is one of the most important tasks in the field of robust control. Three interesting classes of families arise when $A$ is a diamond, a box or a ball of dimension $n+1$. These families will be denoted by ${\mathfrak{F}}_{D_n}$, ${\mathfrak{F}}_{B_n}$, and ${\mathfrak{F}}_{S_n}$, respectively. In this article, a study is presented to contribute to the understanding of Hurwitz stability of families of polynomials ${\mathfrak{F}}_A$. As a result of this study and the use of classical results found in the literature, it is shown the existence of an extremal polynomial $f({\alpha }^{\ast },x)$ whose stability determines the stability of the entire family ${\mathfrak{F}}_A$. In this case $f({\alpha }^{\ast },x)$ comes from minimizing determinants and in some cases $f({\alpha }^{\ast },x)$ coincides with a Kharitonov's polynomial. Thus another extremal property of Kharitonov's polynomials has been found. To illustrate our approach, it is applied to families such as ${\mathfrak{F}}_{D_n}$, ${\mathfrak{F}}_{B_n}$, and ${\mathfrak{F}}_{S_n}$ with $n\le 5$. The study is also used to obtain the maximum robustness of the parameters of a polynomial. To exemplify the proposed results, first, a family ${\mathfrak{F}}_{D_n}$ is taken from the literature to compare and corroborate the effectiveness and the advantage of our perspective. Followed by two examples where the maximum robustness of the parameters of polynomials of degree 3 and 4 are obtained. Lastly, a family ${\mathfrak{F}}_{B_5}$ is proposed whose extreme polynomial is not necessarily a Kharitonov's polynomial. Finally, a family ${\mathfrak{F}}_{S_3}$ is used to exemplify that if the boundary of $A$ is given by a polynomial equation in several variables, the number of candidates to be an extremal polynomial is finite.     

MARCEL: Mobile active rover chassis for enhanced locomotion

To extend planetary exploration beyond the current limitations of wheeled vehicles while preserving reliability, simplicity, and efficiency, actuation can be judiciously incorporated into the locomotion system. Based on a static analysis, we propose a new four-wheeled chassis concept for planetary rovers that can traverse more challenging terrain with the help of two internal active joints. These joints are arranged as follows: a vertical pivot articulates the chassis around its center while a bogie allows the rear wheels to rotate around the longitudinal axis of the vehicle. We also introduce a control method that uses a two-stage procedure to produce an interpretable controller based on a policy devised by reinforcement learning. This way, we eliminate the black box made of a neural network and facilitate the transfer from simulation to reality. The resulting controller efficiently harnesses the internal mobility of the chassis to climb over obstacles in a sequenced manner while relying only on proprioceptive data provided by the chassis. A rover prototype named MARCEL has been built and tested experimentally. Contrary to any state-of-the-art six-wheeled passive chassis, the proposed locomotion system and its associated control has proven to be able to overcome solid step obstacles as tall as the diameter of the wheels with a $90^{\circ }$ edge and a friction coefficient as low as 0.5. This simple but capable design will enable future missions to explore more challenging areas while providing better guarantees in the face of unforeseen difficulties that could arise.     

A generalized reaching-law-based discrete-time integral sliding-mode controller with matched/mismatched disturbance attenuation

A generalized reaching-law-based (RL-based) discrete-time integral sliding-mode controller, which is versatile for either matched or mismatched disturbances, is designed in this paper to obtain high output tracking accuracy and avoid tremendous control efforts. Specifically, a disturbance decomposition-based discrete-time integral sliding surface is designed, and a generalized discrete-time reaching law is established. Different from the existing integral sliding surfaces, the proposed sliding surface synthesizes a disturbance-related integral term that is defined based on disturbance decomposition; this is crucial to the disturbance attenuation. Moreover, different from the available discrete-time reaching laws, the proposed reaching law introduces an adaptive exponential term into the control gains, and hence, the conventional RL-based discrete-time integral sliding-mode control (DISMC) and the equivalent-control-based (EC-based) DISMC can be integrated. Rigorous analysis shows that the closed-loop system is stable, the control effort can be satisfactory, and the steady-state output tracking accuracy is of order $O(T^2)$ for both matched and mismatched disturbances. The proposed method is proven effective through numerical simulations.