Apache Hadoop

Thursday August 09, 2018

[ANNOUNCE] Apache Hadoop 3.1.1 release

It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 3.1.1.

Hadoop 3.1.1 is the first stable maintenance release for the year 2018 in the Hadoop-3.1 line, and brings a number of enhancements.

IMPORTANT NOTES

3.1.1 is the first stable release of 3.1 line which is prod-ready.

The Hadoop community fixed 435 JIRAs [1] in total as part of the 3.1.1 release. Of these fixes:

  • 60 in Hadoop Common
  • 139 in HDFS
  • 223 in YARN
  • 13 in MapReduce
  • Apache Hadoop 3.1.1 contains a number of significant features and enhancements. A few of them are noted below.

  • ENTRY_POINT support for Docker containers.
  • Restart policy support for YARN native services.
  • Capacity Scheduler: Intra-queue preemption for fairness ordering policy.
  • Stabilization works for scheduler / YARN service / docker support, etc.
  • Please see the Hadoop 3.1.1 CHANGES) for the detailed list of issues resolved. The release news is posted on the Apache Hadoop website too, you can go to the downloads section.

    Many thanks to everyone who contributed to the release, and everyone in the Apache Hadoop community! The release is a result of direct and indirect efforts from many contributors, listed below are the those who contributed directly by submitting patches and reporting issues.

    Abhishek Modi, Ajay Kumar, Akhil PB, Akira Ajisaka, Allen Wittenauer, Anbang Hu, Andrew Wang, Arpit Agarwal, Atul Sikaria, BELUGA BEHR, Bharat Viswanadham, Bibin A Chundatt, Billie Rinaldi, Bilwa S T, Botong Huang, Brahma Reddy Battula, Brook Zhou, CR Hota, Chandni Singh, Chao Sun, Charan Hebri, Chen Liang, Chetna Chaudhari, Chun Chen, Daniel Templeton, Davide Vergari, Dennis Huo, Dibyendu Karmakar, Ekanth Sethuramalingam, Eric Badger, Eric Yang, Erik Krogen, Esfandiar Manii, Ewan Higgs, Gabor Bota, Gang Li, Gang Xie, Genmao Yu, Gergely Novák, Gergo Repas, Giovanni Matteo Fumarola, Gour Saha, Greg Senia, Haibo Yan, Hanisha Koneru, Hsin-Liang Huang, Hu Ziqian, Istvan Fajth, Jack Bearden, Jason Lowe, Jeff Zhang, Jian He, Jianchao Jia, Jiandan Yang , Jim Brennan, Jinglun, John Zhuge, Joseph Fourny, K G Bakthavachalam, Karthik Palanisamy, Kihwal Lee, Kitti Nanasi, Konstantin Shvachko, Lei (Eddy) Xu, LiXin Ge, Lokesh Jain, Lukas Majercak, Miklos Szegedi, Mukul Kumar Singh, Namit Maheshwari, Nanda kumar, Nilotpal Nandi, Pavel Avgustinov, Prabhu Joseph, Prasanth Jayachandran, Robert Kanter, Rohith Sharma K S, Rushabh S Shah, Sailesh Patel, Sammi Chen, Sean Mackrory, Sergey Shelukhin, Shane Kumpf, Shashikant Banerjee, Siyao Meng, Sreenath Somarajapuram, Steve Loughran, Suma Shivaprasad, Sumana Sathish, Sunil Govindan, Surendra Singh Lilhore, Szilard Nemeth, Takanobu Asanuma, Tao Jie, Tao Yang, Ted Yu, Thomas Graves, Thomas Marquardt, Todd Lipcon, Vinod Kumar Vavilapalli, Wangda Tan, Wei Yan, Wei-Chiu Chuang, Weiwei Yang, Wilfred Spiegelenburg, Xiao Chen, Xiao Liang, Xintong Song, Xuan Gong, Yang Wang, Yesha Vora, Yiqun Lin, Yiran Wu, Yongjun Zhang, Yuanbo Liu, Zian Chen, Zoltan Haindrich, Zsolt Venczel, Zuoming Zhang, fang zhenyi, john lilley, jwhitter, kyungwan nam, liaoyuxiangqin, liuhongtong, lujie, skrho, yanghuafeng, yimeng, Íñigo Goiri.

    [1] JIRA query: project in (YARN, HADOOP, MAPREDUCE, HDFS) AND resolution = Fixed AND fixVersion = 3.1.1 ORDER BY key ASC, updated ASC, created DESC, priority DESC

    Friday April 06, 2018

    [ANNOUNCE] Apache Hadoop 3.1.0 release

    It gives us great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 3.1.0.

    Hadoop 3.1.0 is the first minor release for the year 2018 in the Hadoop-3.x line, and brings a number of enhancements.

    IMPORTANT NOTES

    This release is *not* yet ready for production use. Critical issues are being ironed out via testing and downstream adoption. Production users should wait for a 3.1.1/3.1.2 release.

    The Hadoop community fixed 768 JIRAs (https://s.apache.org/apache-hadoop-3.1.0-all-tickets) in total as part of the 3.1.0 release. Of these fixes:

    • 141 in Hadoop Common
    • 266 in HDFS
    • 329 in YARN
    • 32 in MapReduce

    Apache Hadoop 3.1.0 contains a number of significant features and enhancements. A few of them are noted below.

    Hadoop Common

    • HADOOP-14831 / HADOOP-14531 / HADOOP-14825 / HADOOP-14325. S3/S3A/S3Guard related improvements.

    Hadoop HDFS

    • HDFS-9806 - HDFS block replicas to be provided by an external storage system

    Hadoop YARN

    • YARN-6223. First class GPU support on YARN
    • YARN-5983. First class FPGA support on YARN
    • YARN-5079 / YARN-4793 / YARN-4757 / YARN-6419. YARN native service support
    • YARN-6592. Rich placement constraints in YARN
    • YARN-5881. Enable configuration of queue capacity in terms of absolute resources for Capacity Scheduler.
    • YARN-7117. Capacity Scheduler: Support auto-creation of leaf queues while doing queue mapping

    Please see the Hadoop 3.1.0 CHANGES for the detailed list of issues resolved. The release news is posted on the Apache Hadoop website too, you can go to the downloads section directly


    Many thanks to everyone who contributed to the release, and everyone in the Apache Hadoop community! The release is a result of direct and indirect efforts from many contributors, listed below are the those who contributed directly by submitting patches.

    Aaron Fabbri, Aaron Gresch, Abdullah Yousufi, Ajay Kumar, Ajith S, Akhil PB, Aki Tanaka, Akira Ajisaka, Alessandro Andrioni, Allen Wittenauer, Andras Bokor, Andrew Wang, Anis Elleuch, Anu Engineer, Arpit Agarwal, Arun Suresh, Atul Sikaria, BELUGA BEHR, Bharat Viswanadham, Bibin A Chunda, Billie Rinaldi, Botong Huang, Brahma Reddy Battula, Carlo Curino, Chandni Singh, Chang Li, Chao Sun, Charan Hebri, Chen Hongfei, Chen Liang, Chetna Chaudhari, Chris Douglas, Da Ding, Daniel Templeton, Daryn Sharp, Devaraj K, Dmitry Chuyko, Ekanth S, Elek, Marton, Ellen Hui, Eric Badger, Eric Payne, Eric Yang, Erik Krogen, Esther Kundin, Ewan Higgs, Gabor Bota, Genmao Yu, Gera Shegalov, Gergely Novák, Gergo Repas, Gour Saha, Greg Phillips, Grigori Rybkine, Haibo Chen, Haibo Yan, Hanisha Koneru, He Xiaoqiao, Igor Dvorzhak, Jack Bearden, Jason Lowe, Jian He, Jiandan Yang, Jianfei Jiang, Jim Brennan, Jinjiang Ling, Joe McDonnell, Johan Gustavsson, Johannes Alberti, John Zhuge, Jonathan Eagles, Jonathan Hung, Jongyoul Lee, Juan Rodríguez Hortalá, Junping Du, Kai Sasaki, Kannapiran Srinivasan, Karthik Kambatla, Karthik Palaniappan, Karthik Palanisamy, Keqiu Hu, Kihwal Lee, Konstantin Shvachko, Konstantinos Karanasos, Kuhu Shukla, Lei (Eddy) Xu, LiXin Ge, Lokesh Jain, Lukas Majercak, Manikandan R, Manoj Govindassamy, Masahiro Tanaka, Mikhail Erofeev, Miklos Szegedi, Ming Ma, Mukul Kumar Singh, Nanda kumar, Nandor Kollar, Nishant Bangarwa, Oleg Danilov, Panagiotis Garefalakis, PandaMonkey, Peter Bacsko, Ping Liu, Prabhu Joseph, Rahul Pathak, Rajesh Balamohan, Ray Chiang, Remi Catherino, Robert Kanter, Rohith Sharma K S, Rui Li, Rushabh S Shah, SammiChen, Sampada Dehankar, Santhosh G Nayak, Sean Mackrory, Sean Po, Sen Zhao, Shane Kumpf, Sharad Sonker, Shashikant Banerjee, Shawna Martell, Sivaguru Sankaridurg, Soumabrata Chakraborty, Stephen O'Donnell, Steve Loughran, Steven Rand, Subru Krishnan, Suma Shivaprasad, Sunil G, Surendra Singh Lilhore, Szilard Nemeth, Takanobu Asanuma, Tanuj Nayak, Tao Yang, Tarun Parimi, Thejas M Nair, Thomas Marquard, Todd Lipcon, Tsz Wo Nicholas Sze, Varada Hemeswari, Varun Saxena, Varun Vasudev, Vasudevan Skm, Vinayakumar B, Vipin Rathor, Virajith Jalaparti, Vishwajeet Dusane, Wangda Tan, Wei Yan, Weiwei Wu, Weiwei Yang, Wilfred Spiegelenburg, Xiao Chen, Xiao Liang, Xiaoyu Yao, Xuan Gong, Yang Wang, Yeliang Cang, Yesha Vora, Yiqun Lin, Yonger, Yongjun Zhang, Young Chen, Yu-Tang Lin, Yufei Gu, Zhankun Tang, Zian Chen, Zsolt Venczel, chencan, fang zhenyi, hu xiaodong, kartheek muthyala, liaoyuxiangqin, ligongyi, liuhongtong, lovekesh bansal, lufei, lujie, maobaolong, shanyu zhao, tartarus, usharani, wangzhiyuan, wujinhu, Íñigo Goiri.

    Best,

    Wangda Tan and Vinod Kumar Vavilapalli

    Tuesday December 12, 2017

    Welcome to Apache™ Hadoop®!

    The Apache™ Hadoop® project develops open-source software for reliable, scalable, distributed computing.

    Calendar

    Search

    Hot Blogs (today's hits)

    Tag Cloud

    Categories

    Feeds

    Links

    Navigation