The Columbia Hot Spot Rescue Service

虽然互联网流量通常相当大，但通常有足够的存储、处理和带宽能力来提供可接受的性能。然而，众所周知，对网络资源的需求越来越频繁地在本地迅速发展成热点或数据风暴，在这种情况下，受影响的网站和子网几乎完全崩溃，造成大规模的收入损失和客户不满。我们提出了一种新的协作技术，以阐明这些热点的影响，我们将通过设计和原型设计热点RescueService（HRS）的技术。这项工作将建立在PI过去在网络，操作系统和分布式缓存方面的研究基础上。 该技术建立的一个关键前提在于观察现有的互联网带宽资源足以有效地处理热点。换句话说，对严重超载的地点的救援可以从其他地方未充分利用的资源中汇集起来。因此，没有必要专门为此目的保留储备资源，即，不需要过度供应、未充分利用的资源（例如分布式高速缓存）来保护免受热点影响。相反，我们提出了一个范式转变，我们提供的有效机制将使社区的参与网站分享他们的资源，以抑制热点。这项服务将是透明的，部分是自我调节的，并将采取自动流量重定向到可用带宽的网站的形式。 拟议的哥伦比亚HRS将是主动和被动的。我们将收集热点数据，这些数据将被建模和分析，目的是设计热点守护程序或插件，通过热点监视器和监控器监控流量行为和信号早期热点的软件设备，以及相关的统计数据沿着。我们将实施该技术的两种互补方法：基于服务器的方法和基于客户端的点对点（P2P）方法。 在基于服务器的方法中，服务器监视自己的负载，在其他服务器过载的情况下，它们将服务的一小部分服务器的负载，并且通过服务器之间的探测来监视网络条件。当服务器或网络组件被识别为可能过载时，系统激活复制机制以复制热内容。然后，客户端可以从充当副本的服务器站点检索内容，从而减轻原始过载资源上的负载。 在P2P方法中，客户端将插件安装到其浏览器中，该插件与安装在其他客户端浏览器上的类似插件以及分布式目录服务进行通信。客户端缓存它们最近的下载，并通过插件通知目录服务缓存的对象。然后，目录服务可以识别最流行的内容以及缓存的位置，并将这些替代位置通知给其他客户端以供下载。通过使用客户端机器来存储和传递热内容，并且如果对给定内容的请求可以被重定向到客户端机器，则可以消除服务器处的热点。通过这种方式，热点响应变得自组织和自我调节。 有几个问题需要解决，因为我们发展这项救援服务。首先，我们将使用实验和分析收集的数据（包括通过公司合作获得的数据集）来开发网络热点活动的原因和影响模型。接下来，我们将分析评估缓存、重定向和迁移等技术对网络热点的服务器和网络负载的影响。最后，我们将实施和评估原型系统，以验证其有效性，无论是在一个测试平台内的模拟热点活动，或如果可能的话，在实际的热点通过与内容提供商的协议。 HRS的开发将为哥伦比亚的多名学生提供研究机会，其部署将提高学术机构和非营利公司提供的对象的Web性能，这些对象的内容迄今为止尚未由专业的第三方内容交付公司托管。

Although Internet traffic is routinely quite heavy, there has usually been more than enough storage,processing and bandwidth capacity to provide acceptable performance. However, it is well known that, moreand more frequently, demands for network resources are mushrooming locally into hotspots or data storms,and in such cases the affected web sites and subnetworks founder almost completely, creating revenuelosses and client dissatisfaction on a large scale. We propose a novel collaborative technology to alleviatethe effects of these hotspots, a technology that we will apply by designing and prototyping a Hotspot RescueService (HRS). This work will build on the past research of the PI's in networking, operating systems, anddistributed caching. A key premise on which the technology is founded lies in the observation that existing Internet band-widthresources are sufficient to deal effectively with hotspots. In other words, rescues of heavily-overloadedsites can be assembled from underutilized resources lying elsewhere. It follows that there is no inherent needfor resources held in reserve uniquely for this purpose, i.e., there is no need for over-provisioned, under-utilized resources such as distributed caches to protect against hotspots. We propose instead a paradigm shift in which efficient mechanisms that we provide will enable communities of participating sites to share their resources to suppress hotspots. The service in action will be transparent, in part self-regulating and will take the form of automated traffic redirection to sites with available bandwidth. The proposed Columbia HRS will be proactive as well as reactive. We will amass hotspot data thatwill be modeled and analyzed with the aim of designing hotspot daemons or plugins, software devices formonitoring traffic behavior and signalling incipient hotspots via hotspot watches and advisories, along withrelevant statistics. We will implement two complementary approaches to the technology, a server-basedapproach and a client-based, peer-to-peer (P2P) approach. In the server-based approach, servers monitor their own loads, the loads of a small set of servers thatthey would service in the event that the other server overloads, and, via probes between servers, networkconditions. When a server or network component is identified as going into possible overload, the systemactivates a replication mechanism to duplicate the hot content. Clients can then retrieve the content fromthe server sites acting as replicas, alleviating the load on the original overloaded resource. In the P2P approach, clients install a plugin into their browser that communicates with similar pluginsinstalled on other clients' browsers, as well as with a distributed directory service. Clients cache their recent downloads, and, through the plugin, inform the directory service of the objects that are cached. The directory service can then identify the most popular content, as well as cached locations and notify additional clients of these alternative locations for download. By using client machines to store and deliver the hot content and if requests for the given content can be redirected to the client machines, the hotspot at the server can be eliminated. In this way, hotspot response becomes self-organizing and self-regulatory. There are several issues that need to be addressed as we develop this rescue service. First, we will useexperimentation and analysis of collected data (including data sets obtained via a company partnerships)to develop models of causes and effects of network hotspot activity. Next, we will analytically evaluatethe effect on server and network load that techniques such as caching, redirecting, and migrating haveupon hotspots within the network. Last, we will implement and evaluate prototype systems to validatetheir effectiveness, either upon simulated hotspot activity within a testbed, or if possible, on actual hotspotsthrough agreements with content providers. Development of the HRS will provide research opportunities to multiple students at Columbia, andits deployment will improve web performance of objects served from academic institutions and non-profitcompanies whose content is to date not hosted by commerical third-party content delivery companies.