The Apache Software Foundation Blog

Monday Apr 02, 2012

The Apache Software Foundation Announces Apache Sqoop as a Top-Level Project

Open Source big data tool used for efficient bulk transfer between Apache Hadoop and structured datastores.

Forest Hill, MD --The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of nearly 150 Open Source projects and initiatives, today announced that Apache Sqoop has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the Project’s community and products have been well-governed under the ASF's meritocratic process and principles.

Designed to efficiently transfer bulk data between Apache Hadoop and structured datastores such as relational databases, Apache Sqoop allows the import of data from external datastores and enterprise data warehouses into Hadoop Distributed File System or related systems like Apache Hive and HBase.

"The Sqoop Project has demonstrated its maturity by graduating from the Apache Incubator," explained Arvind Prabhakar, Vice President of Apache Sqoop. "With jobs transferring data on the order of billions of rows, Sqoop is proving its value as a critical component of production environments."

Building on the Hadoop infrastructure, Sqoop parallelizes data transfer for fast performance and best utilization of system and network resources. In addition, Sqoop allows fast copying of data from external systems to Hadoop to make data analysis more efficient and mitigates the risk of excessive load to external systems. 

"Connectivity to other databases and warehouses is a critical component for the evolution of Hadoop as an enterprise solution, and that's where Sqoop plays a very important role" said Deepak Reddy, Hadoop Manager at Coupons.com. "We use Sqoop extensively to store and exchange data between Hadoop and other warehouses like Netezza. The power of Sqoop also comes in the ability to write free-form queries against structured databases and pull that data into Hadoop."

"Sqoop has been an integral part of our production data pipeline" said Bohan Chen, Director of the Hadoop Development and Operations team at Apollo Group. "It provides a reliable and scalable way to import data from relational databases and export the aggregation results to relational databases."

Since entering the Apache Incubator in June 2011, Sqoop was quickly embraced as an ideal SQL-to-Hadoop data transfer solution. The Project provides connectors for popular systems such as MySQL, PostgreSQL, Oracle, SQL Server and DB2, and also allows for the development of drop-in connectors that provide high speed connectivity with specialized systems like enterprise data warehouses.

Craig Ling, Director of Business Systems at Tsavo Media, said "We adopted the use of Sqoop to transfer data into and out of Hadoop with our other systems over a year ago. It is straight forward and easy to use, which has opened the door to allow team members to start consuming data autonomously, maximizing the analytical value of our data repositories."

Availability and Oversight

Apache Sqoop software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. Apache Sqoop source code, documentation, mailing lists, and related resources are available at http://sqoop.apache.org/. A timeline of the project's history through graduation from the Apache Incubator is also available.

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees nearly one hundred fifty leading Open Source projects, including Apache HTTP Server — the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 350 individual Members and 3,000 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(3)(c) not-for-profit charity, funded by individual donations and corporate sponsors including AMD, Basis Technology, Cloudera, Facebook, Google, IBM, HP, Hortonworks, Matt Mullenweg, Microsoft, PSW Group, SpringSource/VMware, and Yahoo!. For more information, visit http://www.apache.org/.

"Apache", "Apache Sqoop", and "ApacheCon" are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

#  #  #

Comments:

Post a Comment:
Comments are closed for this entry.

Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation