Adaptive control strategy for regulation of blood glucose levels in patients with type 1 diabetes

Current insulin therapy for patients with type 1 diabetes often results in high variability in blood glucose concentrations and may cause hyperglycemic/hypoglycemic episodes. Closing the glucose control loop with a fully automated electro-mechanical pancreas will improve the quality of life for insulin-dependent patients. An adaptive control algorithm is proposed to keep glucose concentrations within normoglycemic range and dynamically respond to glycemic challenges. A model-based control strategy is used to calculate the required insulin infusion rate, while the model parameters are recursively tuned. The algorithm handles delays associated with insulin absorption, time-lag between subcutaneous and blood glucose concentrations, and variations in inter/intra-subject glucose–insulin dynamics. Simulation results for simultaneous meal and physiological disturbances are demonstrated for subcutaneous insulin infusion.     

A Learning Strategy for the Autonomous Control of Type 1 Diabetes

This article proposes a learning strategy for the control of the blood glucose in type 1 diabetes based on continuous subcutaneous glucose measurement and subcutaneous insulin administration. The method relies on an Iterative Learning Control strategy that exploits the approximated repetitiveness of the daily feeding habits of a patient. The administration strategy for the insulin is based on a mixed feedback and feedforward law whose parameters are tuned through a learning process based on the day-by-day analysis of the glucose response to the infusion of exogenous insulin. The proposed scheme is fully autonomous in the sense that it does not require any a priori information on the insulin/glucose response of the patient, on the amount of ingested carbohydrates, and on the announcement of the mealtimes. A novel filtering strategy of the subcutaneous glucose signal is proposed to provide a robust detection of the meal occurrence despite the significant noise introduced by the subcutaneous glucose sensor. A specific module is proposed to detect and prevent possible hypoglycemia events. Considering a prototype diabetic virtual patient it was showed that, thanks to the learning mechanism, the scheme in a few days is able to bring and to maintain the blood glucose in the normoglycemia region and that the control performance can improve over time. Long-run simulation studies have also shown the robustness of the learning scheme in the presence of realistic uncertainties and interpatient variability.     

Extended distributed learning automata

In this paper, a new structure for cooperative learning automata called extended learning automata (eDLA) is introduced. Based on the new structure, an iterative randomized heuristic algorithm using sampling is proposed for finding an optimal subgraph in a stochastic edge-weighted graph. Stochastic graphs are graphs in which the weights of edges have an unknown probability distribution. The proposed algorithm uses an eDLA to find a policy that leads to a subgraph that satisfy some restrictions such as minimum or maximum weight (length). At each stage of the proposed algorithm, the eDLA determines which edges should be sampled. The proposed eDLA-based sampling method may reduce unnecessary samples and hence decrease the time required for finding an optimal subgraph. It is shown that the proposed method converges to an optimal solution, the probability of which can be made arbitrarily close to 1 by using a sufficiently small learning parameter. A new variance-aware threshold value is also proposed that can significantly improve the convergence rate of the proposed eDLA-based algorithm. It is further shown that our algorithm is competitive in terms of the quality of the solution.     

A new real-coded Bayesian optimization algorithm based on a team of learning automata for continuous optimization

Estimation of distribution algorithms have evolved as a technique for estimating population distribution in evolutionary algorithms. They estimate the distribution of the candidate solutions and then sample the next generation from the estimated distribution. Bayesian optimization algorithm is an estimation of distribution algorithm, which uses a Bayesian network to estimate the distribution of candidate solutions and then generates the next generation by sampling from the constructed network. The experimental results show that the Bayesian optimization algorithms are capable of identifying correct linkage between the variables of optimization problems. Since the problem of finding the optimal Bayesian network belongs to the class of NP-hard problems, typically Bayesian optimization algorithms use greedy algorithms to build the Bayesian network. This paper proposes a new real-coded Bayesian optimization algorithm for solving continuous optimization problems that uses a team of learning automata to build the Bayesian network. This team of learning automata tries to learn the optimal Bayesian network structure during the execution of the algorithm. The use of learning automaton leads to an algorithm with lower computation time for building the Bayesian network. The experimental results reported here show the preference of the proposed algorithm on both uni-modal and multi-modal optimization problems.     

面向智能血糖管理的餐前胰岛素剂量贝叶斯学习优化方法

餐前胰岛素剂量精准决策是改善糖尿病患者血糖管理的关键. 临床治疗中胰岛素剂量调整一般在较短时间内完成, 具有典型的小样本特征; 数据驱动建模在该情形下无法准确学习患者餐后血糖代谢规律, 难以确保胰岛素剂量的安全和有效决策. 针对这一问题, 设计一种临床经验辅助的餐前胰岛素剂量自适应优化决策框架, 构建高斯过程血糖预测模型和模型有效性在线评估机制, 提出基于历史剂量和临床经验决策约束的贝叶斯优化方法, 实现小样本下餐后血糖轨迹的安全预测和餐前胰岛素注射剂量的优化决策. 该方法的安全性和有效性通过美国食品药品监督管理局接受的UVA/Padova T1DM平台测试结果和1型糖尿病患者实际临床数据决策结果充分验证. 可为餐前胰岛素剂量智能决策及临床试验提供方法基础和技术支持, 也为中国糖尿病患者血糖管理水平的有效改善, 提供了精准医学治疗手段.     

A consultation system for insulin therapy

This paper describes a computer system to advice on insulin therapy for diabetic in-patients. A mathematical model was developed to describe the effect of insulin on blood glucose (BG) level. The system uses an adaptive approach to analyse the response to an applied insulin dosage. It learns the patient's individual parameters. All conventional injection and insulin pump regimens are supported. The individualised model is used to predict BG level of the proposed insulin dosage. The system uses a generate-reject strategy to output optimum insulin therapy in terms of optimum BG. The predictive capability of the system was tested and it is able to predict BG with a precision of 2.5 mmol/l after 3 days and 6 days of insulin pump treatment and conventional injection therapy, respectively.     

A relative reward-strength algorithm for the hierarchical structure learning automata operating in the general nonstationary multiteacher environment.

A new learning algorithm for the hierarchical structure learning automata (HSLA) operating in the nonstationary multiteacher environment (NME) is proposed. The proposed algorithm is derived by extending the original relative reward-strength algorithm to be utilized in the HSLA operating in the general NME. It is shown that the proposed algorithm ensures convergence with probability 1 to the optimal path under a certain type of the NME. Several computer-simulation results, which have been carried out in order to compare the relative performance of the proposed algorithm in some NMEs against those of the two of the fastest algorithms today, confirm the effectiveness of the proposed algorithm.     

A hierarchical system of learning automata that can learn die globally optimal path

Systems of learning automata have been studied by various researchers to evolve useful strategies for decision making under uncertainity. Considered in this paper are a class of hierarchical systems of learning automata where the system gets responses from its environment at each level of the hierarchy. A classification of such sequential learning tasks based on the complexity of the learning problem is presented. It is shown that none of the existing algorithms can perform in the most general type of hierarchical problem. An algorithm for learning the globally optimal path in this general setting is presented, and its convergence is established. This algorithm needs information transfer from the lower levels to the higher levels. Using the methodology of estimator algorithms, this model can be generalized to accommodate other kinds of hierarchical learning tasks.     

Asymptotic behavior of a hierarchical system of learning automata

Learning automata arranged in a two-level hierarchy are considered. The automata operate in a stationary random environment and update their action probabilities according to the linear-reward-ε-penalty algorithm at each level. Unlike some hierarchical systems previously proposed, no information transfer exists from one level to another, and yet the hierarchy possesses good convergence properties. Using weak-convergence concepts it is shown that for large time and small values of parameters in the algorithm, the evolution of the optimal path probability can be represented by a diffusion whose parameters can be computed explicitly.     

Assignment of cells to switches in cellular mobile network: a learning automata-based memetic algorithm

Handoff and cabling costs management plays an important role in the design of cellular mobile networks. Efficient assigning of cells to switches can have a significant impact on handoff and cabling cost. Assignment of cells to switches problem (ACTSP) in cellular mobile network is NP-hard problem and consequently cannot be solved by exact methods. In this paper a new memetic algorithm which is obtained from the combination of learning automata (LA) and local search is proposed for solving the ACTSP in which the learning automata keeps the history of the local search process and manages the problem’s constraints. The proposed algorithm represents chromosome as object migration automata (OMAs), whose states represent the history of the local search process. Each state in an OMA has two attributes: the value of the gene (allele), and the degree of association with those values. The local search changes the degree of association between genes and their values. To show the superiority of the proposed algorithm several computer experiments have been conducted. The obtained results confirm the efficiency of proposed algorithm in comparison with the existing algorithms such as genetic algorithm, memetic algorithm, and a hybrid Hopfield network-genetic algorithm.     

Speeding up learning automata based multi agent systems using the concepts of stigmergy and entropy

Learning automata (LA) were recently shown to be valuable tools for designing Multi-Agent Reinforcement Learning algorithms and are able to control the stochastic games. In this paper, the concepts of stigmergy and entropy are imported into learning automata based multi-agent systems with the purpose of providing a simple framework for interaction and coordination in multi-agent systems and speeding up the learning process. The multi-agent system considered in this paper is designed to find optimal policies in Markov games. We consider several dummy agents that walk around in the states of the environment, make local learning automaton active, and bring information so that the involved learning automaton can update their local state. The entropy of the probability vector for the learning automata of the next state is used to determine reward or penalty for the actions of learning automata. The experimental results have shown that in terms of the speed of reaching the optimal policy, the proposed algorithm has better learning performance than other learning algorithms.     

CLA-DE: a hybrid model based on cellular learning automata for numerical optimization

This paper presents a hybrid model named: CLA-DE for global numerical optimization. This model is based on cellular learning automata (CLA) and differential evolution algorithm. The main idea is to learn the most promising regions of the search space using cellular learning automata. Learning automata in the CLA iteratively partition the search dimensions of a problem and learn the most admissible partitions. In order to facilitate incorporation among the CLA cells and improve their impact on each other, differential evolution algorithm is incorporated, by which communication and information exchange among neighboring cells are speeded up. The proposed model is compared with some evolutionary algorithms to demonstrate its effectiveness. Experiments are conducted on a group of benchmark functions which are commonly used in the literature. The results show that the proposed algorithm can achieve near optimal solutions in all cases which are highly competitive with the ones from the compared algorithms.     

BNC-VLA: bayesian network structure learning using a team of variable-action set learning automata

Bayesian Network (BN) is a probabilistic graphical model which describes the joint probability distribution over a set of random variables. One of the most important challenges in the field of BNs is to find an optimal network structure based on an available training dataset. Since the problem of searching the optimal BN structure belongs to the class of NP-hard problems, typically greedy algorithms are used to solve it. In this paper a learning automata-based algorithm has been proposed to solve the BNs structure learning problem. There is a learning automaton corresponding with each random variable and at each stage of the proposed algorithm, named BNC-VLA, a set of learning automata is randomly activated and determined the graph edges that must be appeared in that stage. Finally, the constructed network is evaluated using a scoring function. As BNC-VLA algorithm proceeds, the learning process focuses on the BN structure with higher scores. The convergence of this algorithm is theoretically proved; and also some experiments are designed to evaluate the performance of it. Experimental results show that BNC-VLA is capable of finding the optimal structure of BN in an acceptable execution time; and comparing against other search-based methods, it outperforms them.     

Continuous and discretized pursuit learning schemes: variousalgorithms and their comparison

A learning automaton (LA) is an automaton that interacts with a random environment, having as its goal the task of learning the optimal action based on its acquired experience. Many learning automata (LAs) have been proposed, with the class of estimator algorithms being among the fastest ones, Thathachar and Sastry, through the pursuit algorithm, introduced the concept of learning algorithms that pursue the current optimal action, following a reward-penalty learning philosophy. Later, Oommen and Lanctot extended the pursuit algorithm into the discretized world by presenting the discretized pursuit algorithm, based on a reward-inaction learning philosophy. In this paper we argue that the reward-penalty and reward-inaction learning paradigms in conjunction with the continuous and discrete models of computation, lead to four versions of pursuit learning automata. We contend that a scheme that merges the pursuit concept with the most recent response of the environment, permits the algorithm to utilize the LAs long-term and short-term perspectives of the environment. In this paper, we present all four resultant pursuit algorithms, prove the E-optimality of the newly introduced algorithms, and present a quantitative comparison between them.     

Learning automata based dynamic guard channel algorithms

In this paper, we first propose two learning automata based decentralized dynamic guard channel algorithms for cellular mobile networks. These algorithms use learning automata to adjust the number of guard channels to be assigned to cells of network. Then, we introduce a new model for nonstationary environments under which the proposed algorithms work and study their steady state behavior when they use L_R-I learning algorithm. It is also shown that a learning automaton operating under the proposed nonstationary environment equalizes its penalty strengths. Computer simulations have been conducted to show the effectiveness of the proposed algorithms. The simulation results show that the performances of the proposed algorithms are close to the performance of guard channel algorithm that knows all the traffic parameters.     

基于自动学习机的社会网络链路预测算法

针对社会网络中新关系出现的预测,提出一种基于自动学习机的社会网络链路预测算法.将自动学习机与三元组转化相结合,将不同类型三元组的转化作为预测的重要依据并构造学习函数,提出六种三元组内节点相似性指标.实验结果表明,该算法所提出的六个预测指标的预测准确度和稳定性要好于六种常用的链路预测指标,对于社会网络分析具有实际应用价值...     

A habituating blood glucose control strategy for the critically ill

In the intensive care unit patients benefit from being fed and from having well controlled glucose levels. Insulin and glucose infusion serves as manipulated inputs to regulate blood glucose, while glucose infusion serves as a sole nutritional input. In this paper, a model predictive control strategy, based on simultaneously manipulating glucose and insulin infusion, is developed to improve blood glucose regulation in intensive care unit patients. In the short term, glucose infusion is used for tighter glucose control, particularly for disturbance rejection, while, in the long-term (24 h period), glucose infusion is used to meet nutritional needs. The “habituating control” algorithm is proposed and tested against a model predictive control (MPC) strategy that only manipulates insulin. The simulation results indicate that the Habituating MPC strategy outperforms the single input–single output MPC by providing faster setpoint tracking and tighter glucose control for a patient population, and producing less glucose variability while rejecting disturbances in insulin infusion and insulin sensitivity.     

具有学习特点的控制器设计算法

对于普遍存在的具有未知参数的随机最优控制问题,本文提出了一种具有学习特点的控制器设计算法.该算法用Kalman滤波估计系统的未知参数,在滚动优化机制下用动态规划获取控制增益,为了赋予控制器的学习特点,在LQG控制律中附加使下一时刻估计方差最小的学习控制分量.仿真结果表明了算法的有效性.     

Dosage planning for type 2 diabetes mellitus patients using Indexing HDMR

Diabetes mellitus type 2 (T2DM), is also named as non-insulin dependent diabetes mellitus (NIDDM) or adult-onset diabetes, is a metabolic disease characterized by high blood glucose. Deficient insulin production or insulin resistance of the body are the causes of T2DM. Drug treatment has a very important role as well as exercise and diet. To keep the body sugar level stable within the accepted range values, drug dosage planning is a part of combinational treatment. In this study, Indexing HDMR method which is a multivariate data partitioning method is used to produce a polynomial based rule structure to manage the drug dosage planning process. For this purpose, 142 diabetic assays, 96 of which as training data and 46 of which as testing data, were used in this study. The Indexing HDMR method worked well in modeling drug dosages and it is obvious that the method is reliable for the purpose.     

A Truncated Sum of Processing‐Times–Based Learning Model for a Two‐Machine Flowshop Scheduling Problem

Scheduling with learning effects has gained increasing attention in recent years. A well‐known learning model is called “sum‐of‐processing‐times‐based learning” in which the actual processing time of a job is a nonincreasing function of the jobs already processed. However, the actual processing time of a given job drops to zero precipitously when the normal job processing times are large. Moreover, the concept of learning process is relatively unexplored in a flowshop environment. Motivated by these observations, this article addresses a two‐machine flowshop problem with a truncated learning effect. The objective is to find an optimal schedule to minimize the total completion time. First, a branch‐and‐bound algorithm incorporating with a dominance property and four lower bounds is developed to derive the optimal solution. Then three simulated annealing algorithms are also proposed for near‐optimal solution. The experimental results indicated that the branch‐and‐bound algorithm can solve instances up to 18 jobs, and the proposed simulated annealing algorithm performs well in item of CPU time and error percentage. © 2011 Wiley Periodicals, Inc.