MapReduce Design Patterns: Building Effective Algorithms and Analytics for Hadoop and Other Systems
MapReduce Design Patterns: Building Effective Algorithms and Analytics for Hadoop and Other Systems
复制标题
DOI：
--
复制
发表时间：
2012-11
期刊：
影响因子：
--
通讯作者：
Donald. Miner;Adam Shook
中科院分区：
文献类型：
--
作者：
Donald. Miner;Adam Shook
到目前为止，MapReduce框架的设计模式已经分散在各种研究论文、博客和书籍中。这本方便的指南汇集了一组独特的有价值的MapReduce模式，无论您使用的是哪个领域、语言或开发框架，这些模式都将节省您的时间和精力。每个模式都在上下文中进行了解释，并明确指出了陷阱和注意事项，以帮助您在对大数据架构建模时避免常见的设计错误。本书还提供了MapReduce的完整概述，解释了它的起源和实现，以及为什么设计模式如此重要。所有的代码示例都是为Hadoop编写的。汇总模式：通过对数据进行汇总和分组，获得一个顶层视图。过滤模式：查看数据子集，例如从一个用户生成的记录。数据组织模式：重新组织数据，以便与其他系统协同工作，或者使MapReduce分析更容易。连接模式：将不同的数据集一起分析，发现有趣的关系。拼凑几个模式来解决多阶段的问题，或者在同一个工作中执行几个分析输入和输出模式：自定义使用Hadoop加载或存储数据的方式”“对MapReduce程序的常见数据处理模式的清晰阐述，这本书对于任何使用Hadoop的人都是不可缺少的。”——Tom White，《Hadoop：权威指南》的作者
Until now, design patterns for the MapReduce framework have been scattered among various research papers, blogs, and books. This handy guide brings together a unique collection of valuable MapReduce patterns that will save you time and effort regardless of the domain, language, or development framework youre using. Each pattern is explained in context, with pitfalls and caveats clearly identified to help you avoid common design mistakes when modeling your big data architecture. This book also provides a complete overview of MapReduce that explains its origins and implementations, and why design patterns are so important. All code examples are written for Hadoop. Summarization patterns: get a top-level view by summarizing and grouping data Filtering patterns: view data subsets such as records generated from one user Data organization patterns: reorganize data to work with other systems, or to make MapReduce analysis easier Join patterns: analyze different datasets together to discover interesting relationships Metapatterns: piece together several patterns to solve multi-stage problems, or to perform several analytics in the same job Input and output patterns: customize the way you use Hadoop to load or store data ""A clear exposition of MapReduce programs for common data processing patternsthis book is indespensible for anyone using Hadoop."" --Tom White, author of Hadoop: The Definitive Guide