无穷扩展性的秘密:NetFlix开源武器库清单
近日MaxCDN的首席执行官Chris Ueland转发了一篇关于NetFlix高扩展性背后的技术架构揭秘文章。
作者收集整理了互联网上的大量公开资料,为我们描画出了NetFlix的核心技术架构,并列出了开源“武器库”清单。
首先我们来看下NetFlix的总体技术架构,分为应用与数据、业务工具、基础服务、DevOps和Open Connect CDN五大块(下图)。
NetFlix技术架构高可扩展性的秘密在于六点:
1.可扩展性的企业文化:信任员工、不搞企业政治
2.开发自己的CDN和多重CDN能力
3.倚重Amazon的多种云服务:所有的存储都放在亚马逊S3云,以及大量的计算载荷…(参考阅读:全球最大的云计算实例:Netflix如何架构IT)
4.大量开发应用开源项目,从安全到测试、大数据,到亚马逊工具等
5.支持大量设备类型,超过50种
6.保持可扩展性算法的先进性,通过奖金高昂的NetFlix Prize竞赛不断推出更好的可扩展性算法
NetFlix高可扩展性背后的企业文化
NetFlix的开源“武器库”清单
NetFlix的工程师最近发布了一篇名为Netflix的开源进化的文章,读者还可以在这里(https://netflix.github.io/)找到NetFlix的开源工具清单,下面我们分门别类简单介绍一下这些开源工具:
大数据
- Genie – 一个NetFlix开发的,强大的基于REST的任务处理引擎,可运行包括Hadoop、Pig、Hive在内的各种数据处理任务。
- Inviso – Hadoop任务或集群性能的监控工具
- Lipstick – 简洁直观地展示Pig任务工作流
- Aegisthus – 从Cassandra中大规模抽取数据提供给下游的分析处理工作。
开发与交付工具
- Nebula – Effort at Netflix to share its internal build infrastructure.
- Aminator – A tool for creating EBS AMIs.
- Asgard – Web interface for application deployments and cloud management in Amazon Web Services (AWS).
通用Runtime服务与代码库
- Eureka – Service discovery for the Netflix cloud platform.
- Archaius – Distributed configuration.
- Ribbon – Resilent and intelligent inter-process and service communication.
- Hystrix – Provides reliability beyond single service calls. Isolates latency and fault tolerance at runtime.
- Karyon and Governator – JVM container services.
- Prana sidecar – Prana provides proxy capabilities within an instance.
- Zuul – Provides dyamically scriptable proxying at the edge of the cloud deployment.
- Fenzo – Provides advanced scheduling and resource management for cloud native frameworks.
数据持久性
- EVCache and Dynomite – For using Memcached and Redis at scale.
- Astyanax and Dyno – Client libraries to better consume datastores in the Cloud.
分析、可靠性与性能
- Atlas – Time-series telemetry platform
- Edda – Service to track changes in your cloud
- Spectator – Easy integration of Java application code with Atlas
- Vector – Exposes high-resolution host-level metrics with minimal overhead.
- Ice – Exposes ongoing cost and and cloud utilization trends.
- Simian Army – Tests Netflix instances for random failures.
安全工具
- Security Monkey – Helps monitor and secure large AWS-based environments.
- Scumblr – Leverages Internet-wide targeted searches to surface specific security issues for investigation.
- MSL – An extensible and flexible secure messaging protocol that addresses a number of secure communications use cases and requirements.
- Falcor – Represent remote data sources as a single domain model via a virtual JSON graph.
- Restify – node.js REST framework specifically meant for web service APIs
- RxJS – A reactive programming library for JavaScript
第一时间获取面向IT决策者的独家深度资讯,敬请关注IT经理网微信号:ctociocom
除非注明,本站文章均为原创或编译,未经许可严禁转载。
相关文章: