Asynchronous action-reward learning for nonstationary serial supply chain inventory control

Action-reward learning is a reinforcement learning method. In this machine learning approach, an agent interacts with non-deterministic control domain. The agent selects actions at decision epochs and the control domain gives rise to rewards with which the performance measures of the actions are updated. The objective of the agent is to select the future best actions based on the updated performance measures. In this paper, we develop an asynchronous action-reward learning model which updates the performance measures of actions faster than conventional action-reward learning. This learning model is suitable to apply to nonstationary control domain where the rewards for actions vary over time. Based on the asynchronous action-reward learning, two situation reactive inventory control models (centralized and decentralized models) are proposed for a two-stage serial supply chain with nonstationary customer demand. A simulation based experiment was performed to evaluate the performance of the proposed two models. Chang Ouk Kim received his Ph.D. in industrial engineering from Purdue University in 1996 and his B.S. and M.S. degrees from Korea University, Republic of Korea in 1988 and 1990, respectively. From 1998--2001, he was an assistant professor in the Department of Industrial Systems Engineering at Myongji University, Republic of Korea. In 2002, he joined the Department of Information and Industrial Engineering at Yonsei University, Republic of Korea and is now an associate professor. He has published more than 30 articles at international journals. He is currently working on applications of artificial intelligence and adaptive control theory in supply chain management, RFID based logistics information system design, and advanced process control in semiconductor manufacturing. Ick-Hyun Kwon is a postdoctoral researcher in the Department of Civil and Environmental Engineering at University of Illinois at Urbana-Champaign. Previous to this position, Dr. Kwon was a research assistant professor in the Research Institute for Information and Communication Technology at Korea University, Seoul, Republic of Korea. He received his B.S., M.S., and Ph.D. degrees in Industrial Engineering from Korea University, in 1998, 2000, and 2006, respectively. His current research interests are supply chain management, inventory control, production planning and scheduling. Jun-Geol Baek is an assistant professor in the Department of Business Administration at Kwangwoon University, Seoul, Korea. He received his B.S., M.S., and Ph.D. degrees in Industrial Engineering from Korea University, Seoul, Korea, in 1993, 1995, and 2001 respectively. From March 2002 to February 2007, he was an assistant professor in the Department of Industrial Systems Engineering at Induk Institute of Technology, Seoul, Korea. His research interests include machine learning, data mining, intelligent machine diagnosis, and ubiquitous logistics information systems. An erratum to this article can be found at     

Performance of the MAP/G/1 Queue Under the Dyadic Control of Workload and Server Idleness

This paper studies the steady-state queue length process of the MAP/G/1 queue under the dyadic control of the D-policy and multiple server vacations. We derive the probability generating function of the queue length and the mean queue length. We then present computational experiences and compare the MAP queue with the Poisson queue.

Diagnosability Analysis of a Class of Hierarchical State Machines

This paper addresses the problem of fault detection and isolation for a particular class of discrete event dynamical systems called hierarchical finite state machines (HFSMs). A new version of the property of diagnosability for discrete event systems tailored to HFSMs is introduced. This notion, called L₁-diagnosability, captures the possibility of detecting an unobservable fault event using only high level observations of the behavior of an HFSM. Algorithms for testing L₁-diagnosability are presented. In addition, new methodologies are presented for studying the diagnosability properties of HFSMs that are not L₁-diagnosable. These methodologies avoid the complete expansion of an HFSM into its corresponding flat automaton by focusing the expansion on problematic indeterminate cycles only in the associated extended diagnoser.

The Strong Property of Morphological Connected Alternated Filters

This paper studies connectivity aspects that arise in image operators that process connected components. The focus is on morphological image analysis (i.e., on increasing image operators) and, in particular, on a robustness property satisfied by certain morphological filters that is denominated the strong property. The behavior of alternated compositions of openings and closings is investigated under certain assumptions, particularly connectedness and a connected component preserving condition. It is shown that these conditions cannot in general guarantee the strong property of certain connected alternated filters because of issues related to the locality of the filters. As treated in the paper, there have been a series of misunderstandings in the literature concerning this topic, and it is important to clarify them. The root cause of those problems is discussed, and a solution is indicated. The class of connected openings and closings used to build connected alternated filters should therefore be defined to avoid such situations, since the strong property of alternated filters should be a distinctive characteristic of this class.

Topological and Variational Properties of a Model for the Reconstruction of Three-Dimensional Transparent Images with Self-Occlusions

We introduce and study a two-dimensional variational model for the reconstruction of a smooth generic solid shape E, which may handle the self-occlusions and that can be considered as an improvement of the 2.1D sketch of Nitzberg and Mumford (Proceedings of the Third International Conference on Computer Vision, Osaka, 1990). We characterize from the topological viewpoint the apparent contour of E, namely, we characterize those planar graphs that are apparent contours of some shape E. This is the classical problem of recovering a three-dimensional layered shape from its apparent contour, which is of interest in theoretical computer vision. We make use of the so-called Huffman labeling (Machine Intelligence, vol. 6, Am. Elsevier, New York, 1971), see also the papers of Williams (Ph.D. Dissertation, 1994 and Int. J. Comput. Vis. 23:93–108, 1997) and the paper of Karpenko and Hughes (Preprint, 2006) for related results. Moreover, we show that if E and F are two shapes having the same apparent contour, then E and F differ by a global homeomorphism which is strictly increasing on each fiber along the direction of the eye of the observer. These two topological theorems allow to find the domain of the functional ℱ describing the model. Compactness, semicontinuity and relaxation properties of ℱ are then studied, as well as connections of our model with the problem of completion of hidden contours.

Artificial institutions: a model of institutional reality for open multiagent systems

Software agents’ ability to interact within different open systems, designed by different groups, presupposes an agreement on an unambiguous definition of a set of concepts, used to describe the context of the interaction and the communication language the agents can use. Agents’ interactions ought to allow for reliable expectations on the possible evolution of the system; however, in open systems interacting agents may not conform to predefined specifications. A possible solution is to define interaction environments including a normative component, with suitable rules to regulate the behaviour of agents. To tackle this problem we propose an application-independent metamodel of artificial institutions that can be used to define open multiagent systems. In our view an artificial institution is made up by an ontology that models the social context of the interaction, a set of authorizations to act on the institutional context, a set of linguistic conventions for the performance of institutional actions and a system of norms that are necessary to constrain the agents’ actions.     

A multi-agent legal recommender system

Infonorma is a multi-agent system that provides its users with recommendations of legal normative instruments they might be interested in. The Filter agent of Infonorma classifies normative instruments represented as Semantic Web documents into legal branches and performs content-based similarity analysis. This agent, as well as the entire Infonorma system, was modeled under the guidelines of MAAEM, a software development methodology for multi-agent application engineering. This article describes the Infonorma requirements specification, the architectural design solution for those requirements, the detailed design of the Filter agent and the implementation model of Infonorma, according to the guidelines of the MAAEM methodology.     

Topological Repairing of 3D Digital Images

We present here a new randomized algorithm for repairing the topology of objects represented by 3D binary digital images. By “repairing the topology”, we mean a systematic way of modifying a given binary image in order to produce a similar binary image which is guaranteed to be well-composed. A 3D binary digital image is said to be well-composed if, and only if, the square faces shared by background and foreground voxels form a 2D manifold. Well-composed images enjoy some special properties which can make such images very desirable in practical applications. For instance, well-known algorithms for extracting surfaces from and thinning binary images can be simplified and optimized for speed if the input image is assumed to be well-composed. Furthermore, some algorithms for computing surface curvature and extracting adaptive triangulated surfaces, directly from the binary data, can only be applied to well-composed images. Finally, we introduce an extension of the aforementioned algorithm to repairing 3D digital multivalued images. Such an algorithm finds application in repairing segmented images resulting from multi-object segmentations of other 3D digital multivalued images.

Optimal infinite scheduling for multi-priced timed automata

This paper is concerned with the derivation of infinite schedules for timed automata that are in some sense optimal. To cover a wide class of optimality criteria we start out by introducing an extension of the (priced) timed automata model that includes both costs and rewards as separate modelling features. A precise definition is then given of what constitutes optimal infinite behaviours for this class of models. We subsequently show that the derivation of optimal non-terminating schedules for such double-priced timed automata is computable. This is done by a reduction of the problem to the determination of optimal mean-cycles in finite graphs with weighted edges. This reduction is obtained by introducing the so-called corner-point abstraction, a powerful abstraction technique of which we show that it preserves optimal schedules. This work has been mostly done while visiting CISS at Aalborg University in Denmark and has been supported by CISS and by ACI Cortos, a program of the French Ministry of Research.     

Verification of evolving software via component substitutability analysis

This paper presents an automated and compositional procedure to solve the substitutability problem in the context of evolving software systems. Our solution contributes two techniques for checking correctness of software upgrades: (1) a technique based on simultaneous use of over-and under-approximations obtained via existential and universal abstractions; (2) a dynamic assume-guarantee reasoning algorithm—previously generated component assumptions are reused and altered on-the-fly to prove or disprove the global safety properties on the updated system. When upgrades are found to be non-substitutable, our solution generates constructive feedback to developers showing how to improve the components. The substitutability approach has been implemented and validated in the ComFoRT reasoning framework, and we report encouraging results on an industrial benchmark. This is an extended version of a paper, Dynamic Component Substitutability Analysis, published in the Proceedings of the Formal Methods 2005 Conference, Lecture Notes in Computer Science, vol. 3582, by the same authors. This research was sponsored by the National Science Foundation under grant nos. CNS-0411152, CCF-0429120, CCR-0121547, and CCR-0098072, the Semiconductor Research Corporation under grant no. TJ-1366, the US Army Research Office under grant no. DAAD19-01-1-0485, the Office of Naval Research under grant no. N00014-01-1-0796, the ICAST project and the Predictable Assembly from Certifiable Components (PACC) initiative at the Software Engineering Institute, Carnegie Mellon University. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the US government or any other entity.