Power and energy consumption of skid-steer rovers turning on loose soil

This study highlights two key phenomena affecting power and energy consumption of skid-steer rovers on loose soil that is not present on the hard ground: soil excavation due to wheel counterrotation and impeded turning when dragging a braked wheel. Experiments in the field and in a controlled laboratory sandbox show that, on sand, power peaks by 15%–20% in a newly identified range of turns with radii between half the rover width, $B∕2$, to $R^{\prime }$, the radius at which the inner wheel does not turn. In this range of turns, the inner wheels rotate backwards but are being dragged forward through piles of sand they excavate by counterrotation. At $R\prime$, turns are shown to take much longer, leading to higher total energy consumption over time. Experiments in a controlled laboratory sandbox isolate the high motor torque and the resistance force experienced when a skid-steer rover drags a counterrotating or braked wheel, respectively, through loose soil. Other field experiments also demonstrate that paths combining circular arcs and lines can lead to energy savings of up to 15% relative to common ones consisting of point turns and lines; the experimental results suggest the circular arcs should have radii of approximately $2R\prime$. The quantitative values presented in this paper are specific to the rover and soils tested, but there are reasons to support the overall conclusions generalizing to all skid-steer rovers in loose soil.     

TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge

For decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems, like, disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground for new technology at the limits of the autonomous systems capabilities. This paper outlines the software stack and approach of the TUM Autonomous Motorsport team for their participation in the IAC, which holds two competitions: A single-vehicle competition on the Indianapolis Motor Speedway and a passing competition at the Las Vegas Motor Speedway. Nine university teams used an identical vehicle platform: A modified Indy Lights chassis equipped with sensors, a computing platform, and actuators. All the teams developed different algorithms for object detection, localization, planning, prediction, and control of the race cars. The team from Technical University of Munich (TUM) placed first in Indianapolis and secured second place in Las Vegas. During the final of the passing competition, the TUM team reached speeds and accelerations close to the limit of the vehicle, peaking at around $270\text{km h}{}^{-1}$ and $28ms{}^{-2}$. This paper will present details of the vehicle hardware platform, the developed algorithms, and the workflow to test and enhance the software applied during the 2-year project. We derive deep insights into the autonomous vehicle's behavior at high speed and high acceleration by providing a detailed competition analysis. On the basis of this, we deduce a list of lessons learned and provide insights on promising areas of future work based on the real-world evaluation of the displayed concepts.     

Longitudinal deep truck: Deep longitudinal model with application to sim2real deep reinforcement learning for heavy-duty truck control in the field

To develop cooperative adaptive cruise control (CACC), the choice of control approach often influences and can limit the choice of model structure, and vice versa. For heavy-duty trucks, practical application of CACC in the field is heavily influenced by the accuracy of the used model. Deep learning and deep reinforcement learning (deep-RL) have recently been used to demonstrate improved modeling and control performance for vehicles such as cars and quadrotors compared to state-of-the-art. The literature on the application of deep learning and deep-RL for heavy-duty trucks in the field, which are significantly more complex than cars, is still sparse, however. In this article, we develop a two-layer gray-box deep learning model to capture longitudinal dynamics of heavy-duty trucks while abstracting their complexity and present an approach to properly break the nested feedback loops in the model for training. We compare this model with three other alternative models and show that it achieves $~10x$ better general performance compared to a standard artificial neural network and results in $~4x$ and $~40x$ slower steady-state acceleration and speed error growth rates, respectively. We then present an architecture to utilize these deep learning models within the deep-RL framework and use it to develop baseline CACC controllers that can be zero-shot transferred to the field. To carry out the work, we present a setup of differently configured trucks along with their interface architecture and stochastic driving cycle generators for data collection. Numerical validation of the approach demonstrated stationary and bounded modeling error, and demonstrated transfer of CACC controllers with consistent overshoot bounds and a stable approximately-zero steady-state error. Validation from field experiments demonstrated similarly consistent results. Compared to a state-of-the-art benchmark, the deep-RL controller achieved lower speed and time-gap error variance but higher time-gap error offset.     

MARCEL: Mobile active rover chassis for enhanced locomotion

To extend planetary exploration beyond the current limitations of wheeled vehicles while preserving reliability, simplicity, and efficiency, actuation can be judiciously incorporated into the locomotion system. Based on a static analysis, we propose a new four-wheeled chassis concept for planetary rovers that can traverse more challenging terrain with the help of two internal active joints. These joints are arranged as follows: a vertical pivot articulates the chassis around its center while a bogie allows the rear wheels to rotate around the longitudinal axis of the vehicle. We also introduce a control method that uses a two-stage procedure to produce an interpretable controller based on a policy devised by reinforcement learning. This way, we eliminate the black box made of a neural network and facilitate the transfer from simulation to reality. The resulting controller efficiently harnesses the internal mobility of the chassis to climb over obstacles in a sequenced manner while relying only on proprioceptive data provided by the chassis. A rover prototype named MARCEL has been built and tested experimentally. Contrary to any state-of-the-art six-wheeled passive chassis, the proposed locomotion system and its associated control has proven to be able to overcome solid step obstacles as tall as the diameter of the wheels with a $90^{\circ }$ edge and a friction coefficient as low as 0.5. This simple but capable design will enable future missions to explore more challenging areas while providing better guarantees in the face of unforeseen difficulties that could arise.     

Field-based robotic leaf angle detection and characterization of maize plants using stereo vision and deep convolutional neural networks

Maize (Zea mays L.) is one of the three major cereal crops in the world. Leaf angle is an important architectural trait of crops due to its substantial role in light interception by the canopy and hence photosynthetic efficiency. Traditionally, leaf angle has been measured using a protractor, a process that is both slow and laborious. Efficiently measuring leaf angle under field conditions via imaging is challenging due to leaf density in the canopy and the resulting occlusions. However, advances in imaging technologies and machine learning have provided new tools for image acquisition and analysis that could be used to characterize leaf angle using three-dimensional (3D) models of field-grown plants. In this study, PhenoBot 3.0, a robotic vehicle designed to traverse between pairs of agronomically spaced rows of crops, was equipped with multiple tiers of PhenoStereo cameras to capture side-view images of maize plants in the field. PhenoStereo is a customized stereo camera module with integrated strobe lighting for high-speed stereoscopic image acquisition under variable outdoor lighting conditions. An automated image processing pipeline (AngleNet) was developed to measure leaf angles of nonoccluded leaves. In this pipeline, a novel representation form of leaf angle as a triplet of keypoints was proposed. The pipeline employs convolutional neural networks to detect each leaf angle in two-dimensional images and 3D modeling approaches to extract quantitative data from reconstructed models. Satisfactory accuracies in terms of correlation coefficient (r) and mean absolute error (MAE) were achieved for leaf angle ($r>0.87,MAE<5^{°}$) and internode heights ($r>0.99,MAE<3.5\mathrm{cm}$). Our study demonstrates the feasibility of using stereo vision to investigate the distribution of leaf angles in maize under field conditions. The proposed system is an efficient alternative to traditional leaf angle phenotyping and thus could accelerate breeding for improved plant architecture.     

Roboat III: An autonomous surface vessel for urban transportation

In this paper, we present our novel autonomous surface vessel platform, a full-scale Roboat for urban transportation. This 4-m-long Roboat is designed with six seats and can carry a payload of up to 1000 kg. Roboat has two main thrusters for cruising and two tunnel thrusters to accommodate docking and interconnectivity between Roboats. We build an adaptive nonlinear model predictive controller for trajectory tracking to account for payload changes while transporting passengers. We use a sparse directed graph to represent the canal topological map and then find the most time-efficient global path in a city-scale environment using the $A^{*}$ algorithm. We then employ a multiobjective algorithm's lexicographic search to generate an obstacle-free path using a point-cloud projected two-dimensional occupancy grid map. We also develop a docking mechanism to allow Roboat to “grasp” the docking station. Extensive experiments in Amsterdam waterways demonstrate that Roboat can (1) successfully track the optimal trajectories generated by the planner with varying numbers of passengers on board; (2) autonomously dock to the station without human intervention; (3) execute an autonomous water taxi task where it docks to pick up passengers, drive passengers to the destination while planning its path to avoid obstacles, and finally dock to drop off passengers.     

Gradient-free algorithms for distributed online convex optimization

In this paper, we consider the distributed bandit convex optimization of time-varying objective functions over a network. By introducing perturbations into the objective functions, we design a deterministic difference and a randomized difference to replace the gradient information of the objective functions and propose two classes of gradient-free distributed algorithms. We prove that both the two classes of algorithms achieve regrets of $O(T^{3/4})$ for convex objective functions and $O(T^{2/3})$ for strongly convex objective functions, with respect to the time index $T$ and consensus of the estimates established as well. Simulation examples are given justifying the theoretical results.     

NDT-6D for color registration in agri-robotic applications

Registration of point cloud data containing both depth and color information is critical for a variety of applications, including in-field robotic plant manipulation, crop growth modeling, and autonomous navigation. However, current state-of-the-art registration methods often fail in challenging agricultural field conditions due to factors such as occlusions, plant density, and variable illumination. To address these issues, we propose the NDT-6D registration method, which is a color-based variation of the Normal Distribution Transform (NDT) registration approach for point clouds. Our method computes correspondences between pointclouds using both geometric and color information and minimizes the distance between these correspondences using only the three-dimensional (3D) geometric dimensions. We evaluate the method using the GRAPES3D data set collected with a commercial-grade RGB-D sensor mounted on a mobile platform in a vineyard. Results show that registration methods that only rely on depth information fail to provide quality registration for the tested data set. The proposed color-based variation outperforms state-of-the-art methods with a root mean square error (RMSE) of 1.1–1.6 cm for NDT-6D compared with 1.1–2.3 cm for other color-information-based methods and 1.2–13.7 cm for noncolor-information-based methods. The proposed method is shown to be robust against noises using the TUM RGBD data set by artificially adding noise present in an outdoor scenario. The relative pose error (RPE) increased $~$14% for our method compared to an increase of $~$75% for the best-performing registration method. The obtained average accuracy suggests that the NDT-6D registration methods can be used for in-field precision agriculture applications, for example, crop detection, size-based maturity estimation, and growth modeling.     

Convergence on iterative learning control of Hilfer fractional impulsive evolution equations

In this paper, impulsive fractional differential equations with Hilfer fractional derivatives of order and type $0\le \nu \le 1$ is considered. Convergence analysis of $P$-type and $P{I}^{\mu }$-type open-loop iterative learning scheme is studied in the sense of $\lambda$-norm. Examples are provided to explain the theory developed.     

Actuator fault detection for masonry robot manipulator arm with the interval observer

The main construction method of building wall is artificial masonry, the main problem is that the process is associated with low construction efficiency and poor safety, workers are prone fall from high altitude. The research of automatic masonry robot has become an urgent need. The masonry mechanical arm system is the main executing part of the masonry robot, special attention should be paid to the robot fault. Therefore, it is necessary to establish a suitable model to detect the actuator faults of the manipulator system. In this paper, a dynamic model of manipulator fault is presented and a fault detection scheme of masonry robot manipulator arm is proposed based on the model. The model is simplified by analyzing the state parameters of each joint during robot masonry and the interval observer with more design freedom was designed based on the established mathematical model of actuator faults. In this paper, a joint method for solving S and L matrices is proposed, which avoids the limitation of the traditional method for solving L matrices by two-step. In the presence of external interference, $l_1$/$H_{\infty }$ performance are introduced into the generation process of residual interval, and the interval observer has better disturbance robustness and fault sensitivity. Simulation experiments verify that the scheme can effectively detect the actuator fault of the manipulator, and experiments are carried out on a 6-axis manipulator. The experimental results show that when actuator faults occur at joints 2 and 3, the residual rapidly exceeds the threshold range, which proves the effectiveness of the fault detection scheme designed in this paper.     

On the regional control problem of the bilinear wave equation

The present research deals with regional optimal control problem of the bilinear wave equation evolving on a spatial domain $\mathrm{\Omega }\subset {\mathrm{ℝ}}^n,n\ge 1$. Such an equation is excited by bounded controls that act on the velocity term. It addresses the tracking of a desired state all over the time interval $[0,T]$ only on a subregion $\omega$ of $\mathrm{\Omega }$ with minimum energy. Then, we prove that an optimal control exists and is characterized as a solution to an optimality system. Algorithm for the computation of such a control is given and successfully illustrated through simulations.     

Iterative learning algorithms for boundary tracing of fractional diffusion equations

We study P-type and PI ${}^{\theta }$-type iterative learning control (ILC) schemes for boundary tracing problem of nonhomogeneous fractional diffusion equations. Based on Sobolev imbedding theorem, we derive sufficient conditions for the convergence of four ILC schemes in the sense of $\lambda$-norm. Numerical examples are presented to illustrate effectiveness of the proposed control methods. The results show that closed-loop ILC scheme converges faster than open-loop ILC scheme; moreover, PI ${}^{\theta }$-type $(0.5<\theta <1)$ ILC scheme outperforms P-type and PI-type $(\theta =1)$ ILC schemes in terms of the convergence speed.     

Design,development, and field evaluation of a rubber tapping robot

Modern robotics has made great strides in the field of agricultural harvesting. In the field of rubber tapping, however, there are still significant challenges to be faced. As one of the few agricultural harvesting areas primarily manual, rubber tapping requires precise control of the knife in the forest's unstructured environment at night. This paper presents the design of a rubber cutting robot for natural rubber plantations. An upgraded spatial spiral trajectory is established by analyzing the trunk and manual rubber cutting trajectories for the operation of a six-axis tandem robot arm. A compact binocular stereo vision system achieved the initial perception of the parameters required for the cutting trajectory. Besides, an integrated end-effector was developed to enable further precise perception of the cutting trajectory and cutting operations. The system was tested in the field at a rubber plantation in Hainan Province. The upgraded trajectory's adaptability and the accuracy and efficiency of the starting point positioning algorithm were verified. The operational performance of the complete system was evaluated to help guide future improvements. The error of the start point locating is 1.0 $\pm$ 0.1 mm, and the execution time is 17.01 $\pm$ 3.65 s. The bark consumption—cutting depth setting of (2.0 and 5.0 mm) is more appropriate for the rubber tapping robot. Then the accuracy of chip thickness and chip width is about 1.73 $\pm$ 0.28 mm, and 5.07 $\pm$ 0.13 mm, respectively. Furthermore, the accuracy of chip weight is 1.99 $\pm$ 0.24 mm. The average operating time of the whole tapping operation is 80 $\pm$ 5 s. Future work will include improvements in system stiffness and robustness. This study demonstrates the potential of applying industrial robotics to the field of latex harvesting. Robotics may be a future development towards mechanized rubber tapping.     

Fast vision‐based autonomous detection of moving cooperative target for unmanned aerial vehicle landing

We propose a fast and effective method, fast target detection (FTD), to detect the moving cooperative target for the unmanned aerial vehicle landing, and the target is composed of double circles and a cross. The purpose of our strategy is to land on the target. The FTD method needs to detect the target at the high and low heights. At the high height, the target appears completely and stably in the camera field. The FTD method can detect the circle and cross to rapidly reach the target center, named cross and circle–FTD (). To detect the cross, we propose a slope distance equation to obtain the distance between two slopes. The proposed slopes cluster method, based on the distance equation and K‐means, is used to determine the cross center. At the low height, the target appears incompletely and unstably. Therefore, FTD methods detect only the cross, named cross–FTD (). We extract the cross features ( CFs) based on line segments. Then, four CFs are combined based on graph theory. Experiments on our four datasets show that FTD has rapid speed and good performance. (Our method is implemented in C++ and is available at https://github.com/Li-Zhaoxi/UAV-Vision-Servo .) On the Mohamed Bin Zayed International Robotics Challenge datasets made we constructed, detects the target from a image approximately per pipeline with F‐measure and tracks target approximately per pipeline with F‐measure. detects centers from a image at approximately per image with F‐measure.     

Lumped uncertainty alleviation in output channels of MIMO nonlinear systems based on robust disturbance observer-based control strategy

Disturbances and uncertainties can produce unsatisfactory responses in many industrial and engineering systems. Besides, the practical systems and processes are multiple-input multiple-output (MIMO). Hence, achieving a good control performance with adequate output responses is not simple. Many different methods were provided for control of industrial processes in some references. However, in this paper, the primary goal is to design an appropriate tracking controller for alleviating the destructive effects of uncertainties in output channels of MIMO nonlinear systems. For this purpose, a robust mechanism has been introduced according to the optimal design of centralized extended proportional-derivative (CEPD) and disturbance observer (DOB). By designing the derivative part $K_d$ based on famous Vandermonde matrix and DOB gain $\mathrm{\Gamma }$, the robust criterion $R={\left(I+CG{K}_d\right)}^{-1}$ is obtained to tackle the undesirable factors such as nonlinear functions and uncertainties in error dynamics. The closed-loop stability is guaranteed by tuning the proportional part $K_p$ under linear matrix inequality. The proposed scheme in this paper can be used for a wide range of MIMO nonlinear systems in practical situations.     

Remark on stability margin of Lur'e systems

In a recent work, a definition of stability margin (gain and phase margins) for a class of nonlinear systems (Lur'e systems consisting of a linear time-invariant (LTI) plant and a sector-constrained nonlinearity) is proposed based on the famous circle criterion. This definition is indeed interesting because the concept of gain and phase margins has been largely limited to linear systems with a single input and a single output (SISO), but it was further established for Lur'e systems with a particular type of sector constraints ( $k_1=0$, e.g., saturation). In this paper, the previously established concept is extended to cover Lur'e systems with a general type of sector constraints ( $k_1\ge 0$). It is, however, pointed out that in case of $k_1=0$, the definitions of phase margin based on the circle criterion can be misleading and need to be modified or replaced perhaps by a time-delay margin for Lur'e systems including an integrator, for which the phase margin can be trivially zero.     

Linear Diophantine equation (LDE) decoder: A training-free decoding algorithm for multifrequency SSVEP with reduced computation cost

Multifrequency steady-state visual evoked potentials (SSVEPs) have been developed to extend the capability of SSVEP-based brain-machine interfaces (BMIs) to complex applications that have large numbers of targets. Even though various multifrequency stimulation methods have been introduced, the decoding algorithms for multifrequency SSVEP are still in early development. The recently developed multifrequency canonical correlation analysis (MFCCA) was shown to be a feasible training-free option to use in decoding multifrequency SSVEPs. However, the time complexity of MFCCA is shown to be $O(n^3)$, which will lead to long computation time as $n$ grows, where $n$ represents the input size in decoding. In this paper, a novel decoding algorithm is proposed with the aim to reduce the time complexity. This algorithm is based on linear Diophantine equation solvers and has a reduced computation cost $O(nlogn)$ while remaining training-free. Our simulation results demonstrated that linear Diophantine equation (LDE) decoder run time is only one fifth of MFCCA run time under respective optimal settings on 5-s single-channel data. This reduced computation cost makes it easier to implement multifrequency SSVEP in real-time systems. The effectiveness of this new decoding algorithm is validated with nine healthy participants when using dry electrode scalp electroencephalography (EEG).     

ARDEA—An MAV with skills for future planetary missions

We introduce a prototype flying platform for planetary exploration: autonomous robot design for extraterrestrial applications (ARDEA). Communication with unmanned missions beyond Earth orbit suffers from time delay, thus a key criterion for robotic exploration is a robot's ability to perform tasks without human intervention. For autonomous operation, all computations should be done on‐board and Global Navigation Satellite System (GNSS) should not be relied on for navigation purposes. Given these objectives ARDEA is equipped with two pairs of wide‐angle stereo cameras and an inertial measurement unit (IMU) for robust visual‐inertial navigation and time‐efficient, omni‐directional 3D mapping. The four cameras cover a $240^{°}$ vertical field of view, enabling the system to operate in confined environments such as caves formed by lava tubes. The captured images are split into several pinhole cameras, which are used for simultaneously running visual odometries. The stereo output is used for simultaneous localization and mapping, 3D map generation and collision‐free motion planning. To operate the vehicle efficiently for a variety of missions, ARDEA's capabilities have been modularized into skills which can be assembled to fulfill a mission's objectives. These skills are defined generically so that they are independent of the robot configuration, making the approach suitable for different heterogeneous robotic teams. The diverse skill set also makes the micro aerial vehicle (MAV) useful for any task where autonomous exploration is needed. For example terrestrial search and rescue missions where visual navigation in GNSS‐denied indoor environments is crucial, such as partially collapsed man‐made structures like buildings or tunnels. We have demonstrated the robustness of our system in indoor and outdoor field tests.     

Dynamic event-triggered distributed interval functional observers for interconnected systems

In this paper, we design dynamic event-triggered interval functional observers (FOs) for interconnected systems comprising $M$ $(M\ge 2)$ subsystems where each subsystem is subject to nonlinearities and output disturbances. Our design method consists of two main steps. First, we design decentralized dynamic event-triggered mechanisms (ETMs) which use only locally measured output information. We then consider the design of distributed interval FOs by using the newly proposed ETMs. Their existence conditions are established and formulated in terms of linear programming. We also derive a bound on the estimated error vector and show that this bound is the smallest. Thus, this ensures that the unknown linear functional state vector can be estimated within an upper and lower bound of its true value by the designed interval observers. Finally, we apply the obtained results to design dynamic event-triggered interval observers for linear functions of the state vectors of an $N$-machine power system.