Study of load balancing technology for EAST data management

With the continuous renewal and increasing number of diagnostics, the EAST tokamak routinely generates ∼3 GB of raw data per pulse of the experiment, which is transferred to a centralized data management system. In order to strengthen international cooperation, all the acquired data has been converted and stored in the MDSplus servers. During the data system operation, there are some problems when a lot of client machines connect to a single MDSplus data server. Because the server process keeps the connection until the client closes its connection, a lot of server processes use a lot of network ports and consume a large amount of memory, so that the speed of access to data is very slow, but the CPU resource is not fully utilized. To improve data management system performance, many MDSplus servers will be installed on the blade server and form a server cluster to realize load balancing and high availability by using LVS and heartbeat technology. This paper will describe the details of the design and the test results.     

New developments of the EAST data system

As the discharge time becomes longer, the old EAST data system will not meet the requirements of EAST discharge experiment gradually. The problems mainly focus on continuous data acquisition and real-time data access.To meet the requirements of EAST 1000s discharges, the data acquisition mode, data storage structure and data access interface must be improved or reconstructed. Time slice mechanism is one of the popular solutions for continuous data acquisition and real-time data transfer. Data access of large signal files has been successfully solved via a hierarchical storage management system. Moreover, conventional “remote indexing” is replaced with “local indexing” in data access, which greatly enhanced data access speed. These improvements have been implemented and applied to EAST long-pulse plasma experiments which are described in this paper in detail.     

MDSplus evolution continues

The MDSplus data system has been in operation on several fusion machines since 1991 and it is currently in use at over 30 sites spread over 5 continents. A consequence is the extensive feedback provided by the MDSplus user community for bug fixes and improvements and therefore the evolution of MDSplus is keeping pace with the evolution in data acquisition and management techniques. In particular, the recent evolution of MDSplus has been driven by the change in the paradigm for data acquisition in long lasting plasma discharges, where a sustained data stream is transferred from the acquisition devices into the database. Several new features are currently available or are being implemented in MDSplus. The features already implemented include a comprehensive Object-Oriented interface to the system, the python support for data acquisition devices and the full integration in EPICS. Work is in progress for the integration of multiple protocols and security systems in remote data access, a new high level data view layer and a new version of the jScope tool for online visualization and the optimized visualization of very large signals.     

Input/output plugin architecture for MDSplus

The first version of MDSplus was released in 1991 for VAX/VMS. Since that time the underlying file formats have remained constant. The software however has evolved, it was ported to unix, linux, Windows, and Macintosh. In 1997 a TCP based protocol, mdsip, was added to provide network access to MDSplus data. In 2011 a mechanism was added to allow protocol plugins to permit the use of other transport mechanisms such as ssh to access data users. This paper describes a similar design which permits the insertion of plugins to handle the reading and writing of MDSplus data at the data storage level. Tree paths become URIs which specify the protocol, host, and protocol specific information. The protocol is provided by a dynamically activated shared library that can provide any consistent subset of the data store access API, treeshr. The existing low level network protocol called mdsip, is activated by defining tree paths like “host::/directory”. Using the new plugin mechanism this is re-implemented as an instance of the general plugin that replaces the low level treeshr input/output routines. It is specified by using a path like “mdsip://host/directory”.This architecture will make it possible to adapt the MDSplus data organization and analysis tools to other underlying data storage. The first new application of this, after the existing network protocol is implemented, will be a plugin based on a key value store. Key value stores, can provide inexpensive scalable, redundant data storage. An example of this might be an Amazon G3 plugin which would let you specify a tree path such as “AG3://container” to access MDSplus data stored in the cloud.     

Upgraded data service system for HT-7 Tokamak

A data service system plays an indispensable role in HT-7 Tokamak experiment. Since the former system doesn＇t provide the function of timely data procession and analysis, and all client software are based on Windows, it can＇t fulfill virtual fusion laboratory for remote researchers. Therefore, a new system which is simplified by three kinds of data servers and one data analysis and visualization software tool has been developed. The data servers include a data acquisition server based on file system, an MDSplus server used as the central repository for analysis data, and a web server. Users who prefer the convenience of application that can be run in a Web Browser can easily access the experiment data without knowing X-Windows. In order to adjust instruments to control experiment the operators need to plot data duly as soon as they are gathered. To satisfy their requirement, an upgraded data analysis and visualization software GT-7 is developed. It not only makes 2D data visualization more efficient, but also it can be capable of processing, analyzing and displaying interactive 2D and 3D graph of raw. analyzed data by the format of ASCII, LZO and MDSplus.     

Future directions of MDSplus

The first version of MDSplus was released in 1991 for VAX/VMS. Since then MDSplus has been progressively adopted in an increasing number of fusion experiments and its original implementation has been extended during these years to cover new requirements and toward a multi-platform implementation. Currently MDSplus is in use at more than 30 laboratories and is being used both for pulsed applications as well as for continuous data streaming for long lasting experiments. Thanks to its large user base, it has been possible to collect requirements driving the evolution of the system toward improved usability and better performance. An important recent feature of the MDSplus is its ability of handling a continuous stream of data, which is readily available as soon at it has been stored in the pulse files. Current development is oriented toward an improved modularity of MDSplus and the integration of new functionality.Improved modularity is achieved by moving away from monolithic implementation toward a plug-ins approach. This has already been achieved recently for the management of remote data access, where the original TCP/IP implementation can now be integrated with new user-provided network protocols. Following a similar approach, work is in progress to let new back-ends be integrated in the MDSplus data access layer. By decoupling the MDSplus data management from the disk data file format it is possible to integrate new solutions such as data cloud without affecting the user Application Programming Interface.     

H1DS: A new web-based data access system

A new data access system, H1DS, has been developed and deployed for the H-1 Heliac at the Australian Plasma Fusion Research Facility. The data system provides access to fusion data via a RESTful web service. With the URL acting as the API to the data system, H1DS provides a scalable and extensible framework which is intuitive to new users, and allows access from any internet connected device. The H1DS framework, originally designed to work with MDSplus, has a modular design which can be extended to provide access to alternative data storage systems.     

The upgrade of the J-TEXT experimental data access and management system

The experimental data of J-TEXT tokamak are stored in the MDSplus database. The old J-TEXT data access system is based on the tools provided by MDSplus. Since the number of signals is huge, the data retrieval for an experiment is difficult. To solve this problem, the J-TEXT experimental data access and management system (DAMS) based on MDSplus has been developed. The DAMS left the old MDSplus system unchanged providing new tools, which can help users to handle all signals as well as to retrieve signals they need thanks to the user information requirements. The DAMS also offers users a way to create their jScope configuration files which can be downloaded to the local computer. In addition, the DAMS provides a JWeb-Scope tool to visualize the signal in a browser. JWeb-Scope adopts segment strategy to read massive data efficiently. Users can plot one or more signals on their own choice and zoom-in, zoom-out smoothly. The whole system is based on B/S model, so that the users only need of the browsers to access the DAMS. The DAMS has been tested and it has a better user experience. It will be integrated into the J-TEXT remote participation system later.     

基于PXI Express的托卡马克同步数据采集系统设计

随着J-TEXT装置的发展,原有的数据采集系统在稳定性、模块化、采样率等方面已不能满足装置运行的需要,所以需建立一套新的数据采集系统来满足实验需求。本文介绍了基于PXI Express的托卡马克分布式高速同步数据采集系统的设计与实现。系统的采集单元由PXIe机箱NI PXIe-1062Q、PXIe控制器NI PXIe-8133和高速同步数据采集卡NI PXIe-6368组成,兼容ITER CODAC最新标准,具有良好的机械封装性、模块化程度高和高采样率等优点。系统采用同步差分采集方式采集实验数据,并将数据存储于核聚变领域通用的MDSplus数据库中。测试和使用结果表明,系统能在2 MSps采样率下连续稳定工作,可较好地满足装置运行的需要。     

Tokamak TCABR: Acquisition system,data analysis,and remote participation using MDSplus

Each plasma physics laboratory has a proprietary scheme to control and data acquisition system. Usually, it is different from one laboratory to another. It means that each laboratory has its own way to control the experiment and retrieving data from the database. Fusion research relies to a great extent on international collaboration and this private system makes it difficult to follow the work remotely. The TCABR data analysis and acquisition system has been upgraded to support a joint research programme using remote participation technologies. The choice of MDSplus (Model Driven System plus) is proved by the fact that it is widely utilized, and the scientists from different institutions may use the same system in different experiments in different tokamaks without the need to know how each system treats its acquisition system and data analysis. Another important point is the fact that the MDSplus has a library system that allows communication between different types of language (JAVA, Fortran, C, C++, Python) and programs such as MATLAB, IDL, OCTAVE. In the case of tokamak TCABR interfaces (object of this paper) between the system already in use and MDSplus were developed, instead of using the MDSplus at all stages, from the control, and data acquisition to the data analysis. This was done in the way to preserve a complex system already in operation and otherwise it would take a long time to migrate. This implementation also allows add new components using the MDSplus fully at all stages.     

Construction and Implementation of the Online Data Analysis System on EAST

In order to obtain diagnostic data with physical meaning,the acquired raw data must be processed through a series of physical formulas or processing algorithms.Some diagnostic data are acquired and processed by the diagnostic systems themselves.The data processing programs are specific and usually run manually,and the processed results of the analytical data are stored in their local disk,which is unshared and unsafe.Thus,it is necessary to integrate all the specific process programs and build an automatic and unified data analysis system with shareable data storage.This paper introduces the design and implementation of the online analysis system.Based on the MDSplus event mechanism,this system deploys synchronous operations for different processing programs.According to the computational complexity and real-time requirements,combined with the programmability of parallel algorithms and hardware costs,the OpenMP parallel processing technology is applied to the EAST analysis system,and significantly enhances the processing efficiency.     

基于MDSplus分段技术的Web数据显示系统

先进托卡马克装置需要长脉冲放电实验运行,针对装置长脉冲放电实验的数据存储和交互技术是重要的研究内容之一。本工作设计了一种Web数据显示系统,系统采用ASP.NET架构和数据分段技术从MDSplus数据库读取分段数据,利用NI Measurement Studio控件库将数据显示在Web页面中。数据分段技术将长脉冲实验数据划分为多个较小的数据单元--数据段,用户可按需读取长脉冲实验数据中的部分数据段。同时系统制定了高效的分段读取策略,可准确、快速地显示用户所需的数据波形。Web数据显示系统在J-TEXT托卡马克上进行了测试,运行性能稳定,达到了系统的设计目标。     

MDSplus automated build and distribution system

Support of the MDSplus data handling system has been enhanced by the addition of an automated build system which does nightly builds of MDSplus for many computer platforms producing software packages which can now be downloaded using a web browser or via package repositories suitable for automatic updating. The build system was implemented using an extensible continuous integration server product called Hudson which schedules software builds on a collection of VMware based virtual machines. New releases are created based on updates via the MDSplus cvs code repository and versioning are managed using cvs tags and branches. Currently stable, beta and alpha releases of MDSplus are maintained for eleven different platforms including Windows, MacOSX, RedHat Enterprise Linux, Fedora, Ubuntu and Solaris. For some of these platforms, MDSplus packaging has been broken into functional modules so users can pick and choose which MDSplus features they want to install. An added feature to the latest Linux based platforms is the use of package dependencies. When installing MDSplus from the package repositories, any additional required packages used by MDSplus will be installed automatically greatly simplifying the installation of MDSplus. This paper will describe the MDSplus package automated build and distribution system.     

The development of data acquisition and processing application system for RF ion source

As the key ion source component of nuclear fusion auxiliary heating devices, the radio frequency(RF) ion source is developed and applied gradually to offer a source plasma with the advantages of ease of control and high reliability. In addition, it easily achieves long-pulse steady-state operation. During the process of the development and testing of the RF ion source, a lot of original experimental data will be generated. Therefore, it is necessary to develop a stable and reliable computer data acquisition and processing application system for realizing the functions of data acquisition, storage, access, and real-time monitoring. In this paper, the development of a data acquisition and processing application system for the RF ion source is presented. The hardware platform is based on the PXI system and the software is programmed on the Lab VIEW development environment. The key technologies that are used for the implementation of this software programming mainly include the long-pulse data acquisition technology, multithreading processing technology, transmission control communication protocol, and the Lempel–Ziv–Oberhumer data compression algorithm. Now, this design has been tested and applied on the RF ion source. The test results show that it can work reliably and steadily. With the help of this design, the stable plasma discharge data of the RF ion source are collected,stored, accessed, and monitored in real-time. It is shown that it has a very practical application significance for the RF experiments.     

MobileCoDaC – A transportable control,data acquisition and communication infrastructure for Wendelstein 7-X

MobileCoDaC is a test bed allowing in situ testing and commissioning the control and data acquisition of components to be operated at Wendelstein 7-X. It is a minimized replica of the functionality of the complete W7-X CoDaC infrastructure and can be operated independently.MobileCoDaC contains a set of W7-X CoDaC servers, network infrastructure, and accessories for remote access. All hardware is mounted in a single transportable rack system. Moreover, it provides the software infrastructure and user applications for experiment preparation, experiment operation, trouble shooting and experiment data access.MobileCoDaC has been operated successfully for test and commissioning of the control and data acquisition of the HEXOS (high efficiency extreme ultraviolet overview spectrometer) diagnostic at Forschungszentrum Jülich.     

合肥光源冷却水温度测量系统改造

合肥光源冷却水温度测量系统采用EPICS作为开发环境，以智能温度巡检仪表作为前端控制设备。IOC通过串口通信获取温度数据，运行ChannelArchiver对数据进行存档和检索。该系统将实现实时监测多路冷却水温度，能够实现历史数据存储和数据网络检索功能。     

A time-sharing multi-channel pulse amplitude analyzer

A conventional multi-channel pulse amplitude analyzer acquires single energy spectrum,but provides no information on its tendency with time.To address the limitation,we propose a scheme of time-sharing multichannel pulse amplitude analyzer (TSMCA).A dual-port random access memory is divided into two storage spaces,one for current energy spectrum data acquisition and another for previous energy spectrum data storage.The two tasks can be performed simultaneously,and the time-related variation tendency of energy spectrum can be obtained.A prototype system of TSMCA is designed.It performs nicely,with maximum channel number of 4096 in capacity of 232/Ch,minimal time-sharing slice of 25 ms,the differential nonlinearity of ＜1.5％,and the integral nonlinearity of ＜0.3％.     

A multi-parameter system for acquisition, monitoring, and analysis of scanning ion microprobe data

The multi-parameter data-acquisition system for the Eindhoven scanning ion microprobe set-up is described. The front-end part of the system is based on an M68030 in VME and handles real-time data acquisition, experiment control and data transport. It is linked to a DEC ALPHA-AXP workstation for data storage, on-line data monitoring and data analysis and off-line data analysis. The system can be used to apply simultaneously the micro-PIXE, NBS and NFS techniques to determine elemental concentration distributions on biomedical samples, but can also be used for coincident ion scattering experiments and time or dose dependent studies of e.g. ion-beam induced radiation damage.