Apache Hadoop

The Apache™ Hadoop® project develops open-source software for reliable, scalable, distributed computing. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.

User Stories
Kunpeng BoostKit for Big Data
Kunpeng BoostKit for Big Data addresses issues such as low query efficiency and difficult component performance tuning. It provides open source enablement and tuning guides for major big data components, basic acceleration software packages for smart I/O prefetch and Chinese cryptographic encryption and decryption, application acceleration software packages for machine learning and graph analysis algorithms, and open the openLooKeng cross-source and cross-domain query engine. This improves the big data analysis efficiency and maximizes the computing performance.
Jiangsu Telecom run BigData on Kunpeng - (Chinese)
Jiangsu Telecom big data platform carries the operation data, storage and analysis of all production systems of Jiangsu Telecom. It is one of the core business systems and has high requirements for computing performance, concurrent processing capacity and operation stability. After many scheme demonstrations and performance test evaluations, Jiangsu Telecom finally chose Huawei Taishan server based on Kunpeng(Aarch64) processor and open source Hadoop software to build a big data platform. After the platform was launched, it operated stably and significantly improved the business efficiency of Jiangsu Telecom.

News and Events
[Online Session] Linaro Connect 21: Boosting Application Performance on Arm Data Centers (English)
[Online Session] Linaro Connect 20: State of Big Data and Data Science on Arm (English)

Work Items
[DONE] [HADOOP-16614] Missing leveldbjni package of aarch64 platform
[DONE] [YARN-10042] Upgrade grpc-xxx depdencies to 1.26.0
[DONE] [YARN-9898] Dependency netty-all-4.1.27.Final doesn't support ARM platform