基于数据更新间隔的NAND闪存垃圾回收算法

闪存因具有速度快、体积小等优点而广泛应用于数据存储领域,为提高NAND闪存的垃圾回收效率、延长闪存使用寿命,提出一种基于数据更新间隔的垃圾回收算法UIGC。计算闪存中空闲页的分散度,将其作为垃圾回收触发条件。从垃圾回收效率和磨损均衡效果2个方面出发,综合考虑块中无效页年龄累计和以及块中有效页比例,使用动态回收块选择和静态回收块选择相结合的策略来选择目标回收块,根据回收块中有效页数据更新间隔判断有效页热度,同时提出数据更新稳定性的概念来划分有效页的数据更新状态,将具有不同热度和更新状态的有效页数据分别存储在不同的空闲块中,从而提高块中数据的同步更新概率。实验结果表明,UIGC算法相较于CAT、FaGC等现有垃圾回收算法具有更优的垃圾回收效率和磨损均衡效果,并能有效延长闪存使用寿命。     

基于逻辑页冷热分离的NAND闪存磨损均衡算法

针对现有的NAND闪存垃圾回收算法对磨损均衡考虑不足的问题,提出了一种基于逻辑页冷热分离的NAND闪存磨损均衡算法。算法同时考虑了无效页的年龄、物理块的擦除次数以及物理块更新的频率,采用混合模式选择回收符合条件的物理块。同时,推导了一种新的逻辑页热度计算方法,并将回收块上有效页数据按照逻辑页的热度进行了冷热分离。实验结果表明,与GR算法、CB算法、CAT算法以及FaGC算法相比,该算法不仅在磨损均衡上取得了很好的效果,而且总的擦除次数与拷贝次数也有了明显减少。     

基于逻辑区间热度的NAND闪存垃圾回收算法

针对现有的NAND闪存垃圾回收算法中回收性能不高,磨损均衡效果差,并且算法内存开销大的问题,提出了一种基于逻辑区间热度的垃圾回收算法。该算法重新定义了热度计算公式,把连续逻辑地址的NAND内存定义为一个热度区间,以逻辑区间的热度来代替逻辑页的热度,并将不同热度的数据分开存储到不同擦除次数的闪存块上,有效地实现了数据冷热分离,并且节约了内存空间。同时,算法还构造了一种新的回收代价函数来选择回收块,在考虑回收效率的同时,还兼顾了磨损均衡的问题。实验结果表明,该算法与性能优异的FaGC算法相比,总的擦除次数减少了11%,总的拷贝次数减少了13%,擦次数最大差值减少了42%,内存消耗能减少了75%。因此,该算法有利于增加闪存可用空间,改善闪存系统的读写性能,延长闪存使用寿命。     

FeGC: An efficient garbage collection scheme for flash memory based storage systems

NAND flash memory is a promising storage media that provides low-power consumption, high density, high performance, and shock resistance. Due to these versatile features, NAND flash memory is anticipated to be used as storage in enterprise-scale systems as well as small embedded devices. However, unlike traditional hard disks, flash memory should perform garbage collection that consists of a series of erase operations. The erase operation is time-consuming and it usually degrades the performance of storage systems seriously. Moreover, the number of erase operations allowed to each flash memory block is limited. This paper presents a new garbage collection scheme for flash memory based storage systems that focuses on reducing garbage collection overhead, and improving the endurance of flash memory. The scheme also reduces the energy consumption of storage systems significantly. Trace-driven simulations show that the proposed scheme performs better than various existing garbage collection schemes in terms of the garbage collection time, the number of erase operations, the energy consumption, and the endurance of flash memory.     

利用页面重构与数据温度识别的闪存缓存算法

基于闪存的固态盘(SSD)具有比磁盘更加优越的性能,并且在桌面系统中逐渐替代磁盘.但是,尽管在SSD中嵌入了DRAM作为缓存,闪存在不断写入的过程中也可能产生不稳定的写性能,主要是因为逻辑页写入时会频繁引发非覆盖写和垃圾回收操作.针对此问题,提出了一种叫作PRLRU的新型闪存缓存管理方法,通过页面重构机制以及数据温度识...     

Android存储系统中NAND闪存性能的改进

针对Android存储系统在闪存管理上存在较差的磨损均衡效果和较高的垃圾回收额外开销的缺陷,引入冷热数据分离策略,将文件按照不同热度写入对应热度的物理存储单元,同时改进垃圾回收策略,以达到良好的磨损均衡效果并减少垃圾回收额外开销。基于Android平台的实验结果表明,改进后的策略在有效减少NAND闪存垃圾回收额外开销的同时,还能有效改善其磨损均衡效果。     

全程优化的固态硬盘垃圾回收方法

由于NAND闪存的固有限制,写前擦除和擦除粒度较大,基于NAND Flash的固态硬盘（SSD）需要执行垃圾回收以重用失效页。然而垃圾回收带来的高开销会显著降低SSD的性能,也会直接影响SSD的寿命。特别是对于频繁使用的有数据碎片的SSD,垃圾回收带来的性能下降问题将更为严重,现有的垃圾回收（GC）算法各自侧重垃圾回收操作的某个步骤,并没有给出全面考虑各步骤对整体影响的综合方案。针对该问题,在详细剖析垃圾回收过程的基础上,提出了一种全程优化的垃圾回收方法WPO-GC,在数据初始放置、垃圾回收目标块的选择、有效数据的迁移、触发回收的时间点以及中断处理方式上,尽可能全面地考虑各步骤对SSD正常读写请求和寿命的影响。通过开源模拟器SSDsim上的WPO-GC的有效性验证表明,同典型GC算法相比,WPO-GC可以减少SSD读请求延迟20%~40%和写请求延迟17%~40%,均衡磨损近30%。     

Uniform scheduling of interruptible garbage collection and request IO to improve performance and wear-leveling of SSDs

In the past ten years, solid-state drive (SSD) has become one of the mainstream storage devices due to its performance advantages. However, how to reduce the impact of garbage collection operations on host-side IO is still a challenging issue in SSD firmware design. To solve this problem, this paper proposes a uniform scheduling of interruptible garbage collection and host IO (USIGC). The main contributions of USIGC include: First, USIGC sets up an interruptible garbage collection sub-request queue, and then uniformly schedules this sub-request queue with the host IO queue. In this way, the idle time of each channel is fully utilized to complete valid page migration and erase operation of interrupt garbage collection. Second, USIGC predicts the probability that the erase operation can be completed in the current idle time based on the historical idle interval and then makes the erase operation decision to reduce the probability of blocking the host IO. Third, by converting the amount of data that can be written in future to the present and taking the number of invalid pages of the current block as the selection basis of the victim block, the garbage collection and wear-leveling can be unified. Experimental results show that, compared with existing works, USIGC can reduce the average response time and max wait time and achieve the best wear-leveling performance.

     

Virtual duplication and mapping prefetching for emerging storage primitives in NAND flash memory storage systems

NAND flash memory has become the mainstream storage medium for both enterprise high performance computers and embedded systems. However, over the past several decades, the storage primitives that access secondary storage have remained unchanged, forcing NAND flash memory to serve merely as a block device like hard disk drive. Recently, several emerging storage primitives have been presented to explore the potential value of non-volatile memory devices. Although these primitives can significantly boost the access performance by providing virtual to logical address mappings, they still suffer from large RAM footprint to maintain the address mapping table and require further support for update operations.This paper presents ESP to optimize E merging S torage P rimitives with virtualization for flash memory storage systems. We propose two optimization strategies, virtual duplication and mapping prefetching to solve the critical issues in existing emerging storage primitives. The objective is to reduce unnecessary flash memory accesses and keep RAM footprint of address mapping table well under control. We have evaluated ESP on an embedded development platform. Experimental results show that ESP can significantly improve the write/read performance and reduce over 30% of garbage collection operations.     

Performance of garbage collection algorithms for flash-based solid state drives with hot/cold data

To avoid a poor random write performance, flash-based solid state drives typically rely on an internal log-structure. This log-structure reduces the write amplification and thereby improves the write throughput and extends the drive’s lifespan. In this paper, we analyze the performance of the log-structure combined with the d$d$-choices garbage collection algorithm, which repeatedly selects the block with the fewest number of valid pages out of a set of d$d$ randomly chosen blocks, and consider non-uniform random write workloads. Using a mean field model, we show that the write amplification worsens as the hot data gets hotter.     

即时编译器辅助的垃圾收集中的插桩算法研究

即时编译器辅助的垃圾收集技术结合显式和自动内存管理的优点,在编译阶段由即时编译器分析应用程序并在其中插桩显式释放内存的指令,以便垃圾收集器及时回收死亡对象所占用的内存空间,从而减轻垃圾收集器的负担.提出一种应用于该项技术的插桩算法,它基于控制流中的支配关系并提供不同的插桩策略,保证插桩的正确性和灵活性;它能够主动获得域引用从而释放对象及其域引用的内存空间.实验表明基于该插桩算法的垃圾收集器能够回收大量的内存空间,提高Java程序的执行效率.     

Algorithms for realtime component labelling of images

Connected component labelling is an important part of identifying regions and performing feature extraction. In a realtime industrial environment where online image analysis is required, it is highly desirable to have a labelling algorithm that can be implemented at pixel rate in parallel with a raster scan of the image. Using such algorithms based on a fixed-size local window, relevant feature data are stored online and updated as each pixel is labelled. As the number of labels increases with the complexity and size of the image, the hardware feature memory (FM), fixed a priori, may overflow. Any realtime industrial application such as a visual inspection system must deal with this problem. Two solutions to the problem which involve reusing the online memory are proposed in this paper. The relationship between the FM size and the size of the image is discussed. It is shown that the proposed algorithm makes optimal use of the online feature memory and delays purging the FM and storing data offline as long as possible. Also derived are the maximum number of purging phases needed in terms of the size of the FM, N and M, where N × M is the size of the image.     

O1FS: Flash file system with O(1) crash recovery time

The crash recovery time of NAND flash file systems increases with flash memory capacity. Crash recovery usually takes several minutes for a gigabyte of flash memory and becomes a serious problem for mobile devices. To address this problem, we propose a new flash file system, O1FS. A key concept of our system is that a small number of blocks are modified exclusively until we change the blocks explicitly. To recover from crashes, O1FS only accesses the most recently modified blocks rather than the entire flash memory. Therefore, the crash recovery time is bounded by the size of the blocks. We develop mathematical models of crash recovery techniques and prove that the time complexity of O1FS is O(1), whereas that of other methods is proportional to the number of blocks in the flash memory. Our evaluation shows that the crash recovery of O1FS is about 18.5 times faster than that of a state-of-the-art method.     

磨损均衡算法在NAND Flash管理中的改进

由于Flash具有擦除次数有限、先擦后写的特点,会带来使用寿命有限的缺陷。为延长其预期使用寿命,普遍采用磨损均衡算法对各存储单元进行管理。该算法核心在每次写操作时将新数据写入到最少被使用的物理块中。本文对该算法在垃圾回收策略和对静态文件管理方式做出改进。垃圾回收时在遵照磨损均衡原则的前提下提高写入数据效率,同时增强该算法对不同类型的文件存储单元管理能力,从而达到更加有效的磨损均衡。     

Software paging in an interpretive environment

A simple software paging scheme for the interpretive language BALM is described. BALM, a high level extensible language, is extended to BALMSETL by adding the primitives of SETL, a very high-level set-theoretic language. BALMSETL is then used to execute SETL programs. Since these runs required large amounts of core memory (170K_Band up) the need for a paging system arose. The paging system implemented is completely automatic and hidden from the user to whom it appears that he has 200K_B words of additional core memory. Code blocks of varying size constitute pages which are rolled in on demand. Pages to be rolled out are selected on the basis of whether they are active or not and how long they have not been active. The algorithm for selecting pages to be rolled out is invoked by the garbage collector, which uses bit tables to identify code blocks and uses patterns within code blocks to select the appropriate number of pages.     

Flash as cache extension for online transactional workloads

Considering the current price gap between hard disk and flash memory SSD storages, for applications dealing with large-scale data, it will be economically more sensible to use flash memory drives to supplement disk drives rather than to replace them. This paper presents FaCE, which is a new low-overhead caching strategy that uses flash memory as an extension to the RAM buffer of database systems. FaCE aims at improving the transaction throughput as well as shortening the recovery time from a system failure. To achieve the goals, we propose two novel algorithms for flash cache management, namely multi-version FIFO replacement and group second chance. This was possible due to flash write optimization as well as disk access reduction obtained by the FaCE caching methods. In addition, FaCE takes advantage of the nonvolatility of flash memory to fully support database recovery by extending the scope of a persistent database to include the data pages stored in the flash cache. We have implemented FaCE in the PostgreSQL open-source database server and demonstrated its effectiveness for TPC-C benchmarks in comparison with existing caching methods such as Lazy Cleaning and Linux Bcache.     

高效的闪存空间分配与回收方法

随着闪存容量的不断增长以及企业计算、Web数据管理等新型闪存应用的出现,如何管理大容量闪存的存储空间已成为一个迫切需要解决的问题.针对已有闪存空间管理方法存在的低垃圾回收效率和低空间利用率等问题,提出了一种新的高效的闪存空间分配与回收方法,称为BSFTL.BSFTL将数据块区分为冷热两种类型并采用不同的存储管理方式.实验结果表明,BSFTL方法可以显著降低垃圾回收的代价,同时提供了较高的闪存空间利用率.     

Memory management of safety-critical hard real-time systems designed in SystemJ

SystemJ is a programming language based on the Globally Asynchronous Locally Synchronous (GALS) Model of Computation (MoC) used to design safety critical hard real-time systems. SystemJ uses the Java programming language as the “host” language, for carrying out data computations, because Java provides clearly defined operational semantics, type and memory safety in the form of the Garbage Collector(GC), which help with formal functional verification. The same GC, which helps in functional verification, makes Worst Case Reaction Time (WCRT)¹ analysis challenging. Any WCRT analysis framework for GALS programs needs to consider the operations performed by the host language. It has been shown that the worst case time estimates for garbage collection cycles are in seconds, whereas the program’s WCRT itself is in micro-seconds. These pessimistic estimates render the WCRT analysis framework ineffective. In order to overcome this problem, we develop a compiler assisted memory management technique for applications written in SystemJ. The SystemJ MoC plays the central role in the proposed technique. The SystemJ MoC allows clearly demarcating the state boundaries of the program, which in turn allows us to partition the heap, at compile time, into two distinct areas: (1) the memory area called the permanent heap, which holds objects that are alive throughout the life time of the application, and (2) the memory area used to hold all other objects, called the transient heap. The size of these memory areas are bounded statically. Furthermore, the memory allocation and reclaim procedures are simple load and pointer reset operations, respectively, which are guaranteed to complete within a bounded number of clock-cycles, thereby alleviating the need for large pessimistic WCRT bounds obtained due to the GC. Experimental results also show that the proposed approach is approximately three times faster, in terms of memory allocation times as compared to standard real-time GC approaches.     

Generic and efficient framework for search trees on flash memory storage systems

Tree index structures are crucial components in data management systems. Existing tree index structure are designed with the implicit assumption that the underlying external memory storage is the conventional magnetic hard disk drives. This assumption is going to be invalid soon, as flash memory storage is increasingly adopted as the main storage media in mobile devices, digital cameras, embedded sensors, and notebooks. Though it is direct and simple to port existing tree index structures on the flash memory storage, that direct approach does not consider the unique characteristics of flash memory, i.e., slow write operations, and erase-before-update property, which would result in a sub optimal performance. In this paper, we introduce FAST (i.e., Flash-Aware Search Trees) as a generic framework for flash-aware tree index structures. FAST distinguishes itself from all previous attempts of flash memory indexing in two aspects: (1) FAST is a generic framework that can be applied to a wide class of data partitioning tree structures including R-tree and its variants, and (2) FAST achieves both efficiency and durability of read and write flash operations through memory flushing and crash recovery techniques. Extensive experimental results, based on an actual implementation of FAST inside the GiST index structure in PostgreSQL, show that FAST achieves better performance than its competitors.     

Linked Block-based Multiversion B-Tree index for PCM-based embedded databases

In this paper, by exploring the application characteristics of cyber-physical systems (CPS) and the performance characteristics of PCM, we propose a new B-tree index structure, called Linked Block-based Multi-Version B-Tree (LBMVBT), for indexing multi-version data in an embedded multi-version database for CPS. In LBMVBT, to reduce the number of writes to PCM in maintaining the index and improve the efficiency in serving version-range queries, we introduce the block-based scheme for indexing in which multiple versions of a data item are grouped into a version block to be indexed by a single entry in the multi-version index. To reduce the index re-organization cost (i.e., number of writes to PCM) due to node overflow and underflow, we add external entries in each index leaf node so that a node re-organization can be done by only updating pointers without copying the index entries of the re-organized leaf nodes to the new node. Analytic studies have been performed on LBMVBT and a series of experiments has been conducted to evaluate the efficacy of LBMVBT. The experimental results show that LBMVBT can effectively reduce the number of writes to the index and achieve a good overall performance on serving update transactions while version-range queries can be served with smaller number of reads to the index compared with MVBT.