Achievable sum‐rate maximization using relay/antenna selection for MIMO two‐way AF relaying with channel uncertainties

This paper focuses on the layered relay‐and‐antenna selection (LRAS) for achievable sum‐rate (ASR) maximization while considering the impacts of residual self‐interference due to channel estimation errors in multiple‐input multiple‐output two‐way amplify‐and‐forward relaying systems. Two LRAS algorithms, namely, the Gram–Schmidt and the adaptive discrete stochastic approximation selection techniques, are investigated based on the ASR maximization under an equal power allocation. To alleviate the complexity burden of the LRAS strategies, the optimal relay and the subset of transmit‐and‐receive antenna pairs are determined by a two‐stage selection mechanism. By taking two LRAS strategies and correlated channel uncertainties into account, the development of a two‐way multiple‐input multiple‐output multi‐amplify‐and‐forward‐relay system is able to provide improved robustness against the channel state information mismatch and the residual self‐interference. Copyright © 2016 John Wiley & Sons, Ltd.