内存数据库容错关键技术研究

硬件价格下降，性能却大幅提升，使内存数据库凭借卓越的性能在越来越多的关键领域得到深入应用。然而内存数据易失的特性与磁盘、网络IO性能的瓶颈共同给基于持久化的内存数据库容错系统设计带来了巨大挑战。一方面，基于日志恢复的容错机制下，事务处理速度加快，导致产生的日志数量激增，而为了控制日志数量，提高恢复速度，需要更加频繁地生成检查点；另一方面，基于复制协议的容错机制下，强一致性和分区容忍性的制约使内存数据库性能难以发挥。可见，研究能够权衡恢复性能与正常事务性能的内存数据库容错技术至关重要。本课题拟结合新型处理器和存储设备特性，研究日志的高效生成与快速恢复，轻量级一致性检查点，面向延迟优化的一致性复制，以及面向内存数据库容错的测试基准等内容，形成一系列权衡内存数据库数据恢复性能和正常事务处理性能的关键技术。最终，通过代码开源，并与主流开源内存数据库系统集成，体现研究的应用价值。

In recent years, computer hardware performance has increased significantly while prices have fallen a lot. This makes the in-memory databases (IMDB) are accepted by more and more applications in key areas. However, the volatility of data in main memory, and the IO bottleneck of hard disk and network significantly challenge the design of IMDB fault-tolerance sub-system that relies on data persistence. On one hand, for logging-based fault-tolerance scheme, the transaction processing speed of IMDB is much faster than that of disk-based database, which leads more log records generated in equal time. In order to limit the number of log records and increase the speed of recovery, more frequent checkpoints are required. On the other hand, for replication-based fault-tolerance scheme, the strong consistency and partition tolerance constraints make it difficult for IMDB to run in high performance. Therefore, it is essential to research on fault-tolerance technologies that effectively balance the performance of recovery and the performance of regular transaction processing. In this research, the characteristics of new CPU architectures and storage devices will be taken into account, and the main research contents cover: efficient generation of logs and fast recovery from logs; light-weight consistent checkpoints; latency optimization oriented consistent replication; and IMDB fault-tolerance oriented benchmarks. Research results will be a series of key technologies that effectively balance the performance of recovery and the performance of regular transaction processing. Furthermore, to demonstrate the application value of the research, the code will be opensourced and integrated with mainstream IMDB systems.