CSR: Small: Elastic Soft State Cache as an OS Service

为了有效地服务用户请求，云应用程序将软状态数据缓存到内存中。当数据只对效率有帮助并且丢失它不会导致功能问题时，数据被认为是“软状态”。软状态被广泛使用，因为它使应用程序能够权衡内存以获得更好的性能，并且性能增益通常随着所提供的内存量而增长。今天的系统静态地为应用程序提供内存，以防止它们的缓存数据无限制地增长并耗尽服务器的内存。然而，静态内存配置要能够满足每个应用程序的软状态需求，同时最大限度地利用全局内存是非常困难的。动态配置具有挑战性，因为当前系统无法在应用程序之间安全有效地添加/删除内存。这个项目为软状态缓存构建了一个新的虚拟内存抽象（称为“软内存”），以及一个名为Midas的软内存管理系统，使应用程序开发人员能够轻松地将他们的软状态数据注册到Midas API中，而将缓存管理和动态扩展留给操作系统内核。在Midas下，软状态缓存不再是特定于应用程序的；相反，它是由操作系统内核作为服务提供的。现代社会越来越依赖云平台，而云平台的内存严重未得到充分利用。在应用程序端，大多数云应用程序需要缓存中间结果或软状态数据；更多的内存可以显著提高它们的吞吐量并减少它们的延迟。然而，现有的云系统静态地提供内存，导致内存浪费。同时，许多需要大量内存的应用程序的缓存需求没有得到满足。该项目开发了一种新的软状态缓存系统，将缓存作为一种操作系统服务提供给所有应用程序。这样的服务可以显著提高云的内存利用率，从而节省成本。同时，它通过允许运行在云上的单个应用程序使用更多内存来提高性能，从而改善用户体验。该项目还在教育和多样性方面作出努力，将研究纳入课程，并从代表性不足的群体中招募和培训初级研究人员。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

To efficiently serve user requests, cloud applications cache soft-state data in memory. Data is considered “soft state” when it is helpful only for efficiency and losing it would not lead to functional issues. Soft state is ubiquitously used as it enables applications to trade off memory for better performance, and performance gains often grow with the amount of memory provisioned. Today’s systems provision memory statically to applications to prevent their cached data from growing unbounded and draining the server’s memory. It is extremely hard, however, for static memory provisioning to be able to address each application’s soft-state needs while simultaneously maximizing the use of global memory. Dynamic provisioning is challenging as current systems are unable to safely and efficiently add/remove memory between applications. This project builds a new virtual memory abstraction for soft-state caching (referred to as “soft memory’”), as well as a soft memory management system called Midas, enabling application developers to easily register their soft-state data with the Midas API and leave cache management and dynamic scaling to the OS kernel. Under Midas, soft-state caching is no longer application-specific; instead, it is provided by the OS kernel as a service. Modern society is relying increasingly on cloud platforms where memory is heavily underutilized. On the application side, most cloud applications need to cache intermediate results or soft-state data; more memory can significantly boost their throughput and reduce their latency. However, existing cloud systems provision memory statically, leading to memory waste. Simultaneously, the caching needs of many memory-hungry applications are not being satisfied. This project develops a new soft-state caching system, providing caching to all applications as an operating system service. Such a service can significantly improve memory utilization for the cloud, leading to monetary cost savings. In the meantime, it improves the performance of individual applications running on the cloud by letting them opportunistically use more memory, resulting in improved user experiences. The project also makes efforts in education and diversity by incorporating research into courses and recruiting and training junior researchers from underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.