Entries tagged [tlp]

Monday November 21, 2016

The Apache Software Foundation Announces Apache® Geode™ as a Top-Level Project

Open Source Big Data in-memory data grid used by hundreds of enterprises to power mission-critical low latency, high concurrency transactional applications at extreme scale.

Forest Hill, MD —21 November 2016— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache® Geode™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Geode is an Open Source in-memory data grid that provides transactional data management for scale-out applications needing low latency response times during high concurrent processing.

"Graduating as a Top-Level Project marks an important milestone for Apache Geode," said Mark Bretl, Vice President of Apache Geode. "Our community is proud to champion a diverse group of developers and users whose support has helped Geode reach a sustainable level of maturity."

The Geode codebase was originally developed by Gemstone Systems in 2002. GemFire, the original commercial distribution of Geode, was first widely adopted by the financial sector as the transactional, low-latency data engine used in Wall Street trading platforms. Pivotal®, which owns the GemFire technology, submitted the Geode code to the Apache Incubator in April 2015.

"We are excited to see Geode graduate from the Apache Incubator to a Top-Level Project. It's quite a feat to transform a mature commercial product into a widely adopted open source project," said Elisabeth Hendrickson, VP of Big Data R&D at Pivotal. "The committers in Geode have worked hard at building community and making the project accessible to newcomers, paving the way for developers everywhere to benefit from a proven in memory data grid technology."

Since entering the Apache Incubator, the project has had significant increases in the number of independent developers contributing to the code, as well as organizations incorporating Apache Geode in their deployments and solutions. Today, over 600 enterprises use the technology behind Apache Geode for high-scale business applications that must meet low latency and 24x7 availability requirements, such as financial risk analysis systems, high volume eCommerce Websites, and transportation & logistics management.

"zData has been deploying big solutions with the technology of Apache Geode well before it became open source software. We look forward to helping more of our customers enjoy the speed, reliability, and scale that Apache Geode brings to any application architecture."
-- Dillon Woods, CTO, zData Inc.

"Apache Geode is an important component of Capgemini's Business Data Lake and fast reacting business scale out analytics solutions. Capgemini congratulates the Apache Geode community on becoming a top level project in The Apache Software Foundation." 
-- Steve Jones, Global Vice President, Big Data, Capgemini

"Apache Apex provides direct support for Apache Geode. Geode helps Apex deployments by providing fast, fault-tolerant storage and query support for stream processing data. Data Torrent welcomes Apache Geode as a peer project of Apache Apex".
--Amol Kekre, CTO at Data Torrent

"Apache Geode is an important component of Ampool Active Data Store. It provides scale-out in-memory processing with transactional consistency. We've been enthusiastic users of Apache Geode since its beginning, and look forward to this next phase".
-- Milind Bhandarkar, CEO at Ampool

"Through the incubation process we have worked to create an open and collaborative community for developers and users to work together, and look forward to seeing new contributions, feedback, bug reports, and subscribers to the Geode email lists," added Bretl.

The Apache Geode project welcomes contributions and community participation through mailing lists, face-to-face MeetUps, Geode Clubhouse online, and other events such as the Apache: Big Data conference series.

Availability and Oversight
Apache Geode software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For project updates, downloads, documentation, and ways to become involved with Apache Geode, visit http://geode.apache.org/ and @ApacheGeode.

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 620 individual Members and 5,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Alibaba Cloud Computing, ARM, Bloomberg, Budget Direct, Cerner, Cloudera, Comcast, Confluent, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Microsoft, OPDi, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ and https://twitter.com/TheASF

© The Apache Software Foundation. "Apache", "Geode", "Apache Geode", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

Wednesday July 27, 2016

Apache Software Foundation Announces Apache® Twill™ as a Top-Level Project

Open Source abstraction layer over Apache Hadoop® YARN simplifies developing distributed Hadoop applications.

Forest Hill, MD –27 July 2016– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache® Twill™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Twill is an abstraction over Apache Hadoop® YARN that reduces the complexity of developing distributed Hadoop applications, allowing developers to focus more on their application logic.

"The Twill community is excited to graduate from the Apache Incubator to a Top-Level Project," said Terence Yim, Vice President of Apache Twill and Software Engineer at Cask. "We are proud of the innovation, creativity and simplicity Twill demonstrates. We are also very excited to bring a technology so versatile in Hadoop into the hands of every developer in the industry."

Apache Twill provides rich built-in features for common distributed applications for development, deployment, and management, greatly easing Hadoop cluster operation and administration.

"Enterprises use big data technologies - and specifically Hadoop - to drive more value," said Patrick Hunt, member of the Apache Software Foundation and Senior Software Engineer at Cloudera. "Apache Twill helps streamline and reduce complexity of developing distributed applications and its graduation to an Apache Top-Level Project means more people will be able to take advantage of Apache Hadoop YARN more easily."

"This is an exciting and major milestone for Apache Twill," said Keith Turner, member of the Apache Fluo (incubating) Project Management Committee, which used Twill in the development of Fluo, an Open Source project that makes it possible to update the results of a large-scale computation, index, or analytic as new data is discovered. "Early in development, we knew we needed a standard way to launch Fluo across a cluster, and we found Twill. With Twill, we quickly and easily had Fluo running across many nodes on a cluster." 

Apache Twill is in production by several organizations across various industries, easing distributed Hadoop application development and deployment.

Twill originated at Cask in early 2013. After 7 major releases, the project was submitted to the Apache Incubator in November of 2013.

"Apache Twill has come a long way through The Apache Software Foundation, and we're thrilled it has become an ASF Top-Level Project," said Nitin Motgi, CTO of Cask. "Apache Twill has become a key component behind the Cask Data Application Platform (CDAP), using YARN containers and Java threads as the processing abstraction. CDAP is an Open Source integration and application platform that makes it easy for developers and organizations to quickly build, deploy and manage data applications on Apache Hadoop and Apache Spark."

"The Apache Twill community worked extremely well within the incubator environment, developing and collaborating openly to follow The Apache Way," said Henry Saputra, ASF Member and member of the Apache Twill Project Management Committee. "There is a tremendous demand for effective APIs and virtualization for developing big data applications and Apache Twill fills that need perfectly. We’re looking forward to continuing the journey with Apache Twill as a Top-Level Project."

Catch Apache Twill in action at:
  • JavaOne, 18-22 September 2016 in San Francisco
  • Strata+Hadoop World, 27-29 September 2016 in New York City
Availability and Oversight
Apache Twill software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Twill, visit http://twill.apache.org/ and follow @ApacheTwill

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 5,300 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Alibaba Cloud Computing, ARM, Bloomberg, Budget Direct, Cerner, Cloudera, Comcast, Confluent, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Microsoft, OPDi, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ and https://twitter.com/TheASF

©The Apache Software Foundation. "Apache", "Twill", "Apache Twill", "Hadoop", "Apache Hadoop", "Apache Hadoop YARN", "Spark", "Apache Spark", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Tuesday July 26, 2016

The Apache Software Foundation Announces Apache® Kudu™ as a Top-Level Project

Open Source columnar storage engine enables fast analytics across the Internet of Things, time series, cybersecurity, and other Big Data applications in the Apache Hadoop ecosystem

Forest Hill, MD –25 July 2016– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache® Kudu™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Kudu is an Open Source columnar storage engine built for the Apache Hadoop ecosystem designed to enable flexible, high-performance analytic pipelines.

"Under the Apache Incubator, the Kudu community has grown to more than 45 developers and hundreds of users," said Todd Lipcon, Vice President of Apache Kudu and Software Engineer at Cloudera. "Recognizing the strong Open Source community is a testament to the power of collaboration and the upcoming 1.0 release promises to give users an even better storage layer that complements Apache HBase and HDFS."

Optimized for lightning-fast scans, Kudu is particularly well suited to hosting time-series data and various types of operational data. In addition to its impressive scan speed, Kudu supports many operations available in traditional databases, including real-time insert, update, and delete operations. Kudu enables a "bring your own SQL" philosophy, and supports being accessed by multiple different query engines including such other Apache projects as Drill, Spark, and Impala (incubating).

Apache Kudu is in use at diverse companies and organizations across many industries, including retail, online service delivery, risk management, and digital advertising.

"Using Apache Kudu alongside interactive SQL tools like Apache Impala (incubating) has allowed us to deploy a next-generation platform for real-time analytics and online reporting," said Baoqiu Cui, Chief Architect at Xiaomi. "Apache Kudu has been deployed in production at Xiaomi for more than six months and has enabled us to improve key reliability and performance metrics for our customers. Kudu's graduation to a Top-Level Project allows companies like ours to operate a hybrid architecture without complexity. We look forward to continuing to contribute to its success."

"We are already seeing the many benefits of Apache Kudu. In fact we're using its combination of fast scans and fast updates for upcoming releases of our risk solutions," said Cory Isaacson, CTO at Risk Management Solutions, Inc. "Kudu is performing well, and RMS is proud to have contributed to the project’s integration with Apache Spark."

"The Internet of Things, cybersecurity and other fast data drivers highlight the demands that real-time analytics place on Big Data platforms," said Arvind Prabhakar, Apache Software Foundation member and CTO of StreamSets. "Apache Kudu fills a key architectural gap by providing an elegant solution spanning both traditional analytics and fast data access. StreamSets provides native support for Apache Kudu to help build real-time ingestion and analytics for our users."

"Graduation to a Top-Level Project marks an important milestone in the Apache Kudu community, but we are really just beginning to achieve our vision of a hybrid storage engine for analytics and real-time processing," added Lipcon. "As our community continues to grow, we welcome feedback, use cases, bug reports, patch submissions, documentation, new integrations, and all other contributions."

The Apache Kudu project welcomes contributions and community participation through mailing lists, a Slack channel, face-to-face MeetUps, and other events. Catch Apache Kudu in action at Strata + Hadoop World, 26-29 September 2016 in New York. 

Availability and Oversight
Apache Kudu software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For project updates, downloads, documentation, and ways to become involved with Apache Kudu, visit http://kudu.apache.org/ , @ApacheKudu, and http://kudu.apache.org/blog/.

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 5,300 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Alibaba Cloud Computing, ARM, Bloomberg, Budget Direct, Cerner, Cloudera, Comcast, Confluent, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Microsoft, OPDi, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ and https://twitter.com/TheASF

© The Apache Software Foundation. "Apache", "Kudu", "Apache Kudu", "Drill", "Apache Drill", "Hadoop", "Apache Hadoop", "Apache Impala (incubating)", "Spark", "Apache Spark", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Wednesday February 17, 2016

The Apache® Software Foundation Announces Apache Arrow™ as a Top-Level Project

Open source Big Data in-memory columnar layer accelerates analytical processing and interchange by more than 100x. 

Forest Hill, MD --17 Feb 2016-- The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache Arrow as a new Top-Level Project. 

A high-performance cross-system data layer for columnar in-memory analytics, Apache Arrow provides the following benefits for Big Data workloads:
  • Accelerates the performance of analytical workloads by more than 100x in some cases
  • Enables multi-system workloads by eliminating cross-system communication overhead

Initially seeded by code from the Apache Drill project, Apache Arrow was built on top of a number of Open Source collaborations, and establishes a de-facto standard for columnar in-memory processing and interchange.

"The Open Source community has joined forces on Apache Arrow," said Jacques Nadeau, Vice President of Apache Arrow and Vice President Apache Drill. "Developers from 13 major Open Source Big Data projects are already on board --by introducing a new era of columnar in-memory analytics, we anticipate the majority of the world's data will be processed through Arrow within the next few years."

Code committers to Apache Arrow include developers from Apache Big Data projects Calcite, Cassandra, Drill, Hadoop, HBase, Impala, Kudu (incubating), Parquet, Phoenix, Spark, and Storm as well as established and emerging Open Source projects such as Pandas and Ibis.

"Arrow's cross platform and cross system strengths will enable Python and R to become first-class languages across the entire Big Data stack," said Wes McKinney, creator of Pandas.

Apache Arrow accelerates analytical processing by providing a high performance columnar in-memory representation. A number of processing algorithms benefit greatly from this memory design. 

"A columnar in-memory data layer enables systems and applications to process data at full hardware speeds," said Todd Lipcon, original Apache Kudu creator and member of the Apache Arrow Project Management Committee. "Modern CPUs are designed to exploit data-level parallelism via vectorized operations and SIMD instructions. Arrow facilitates such processing."

In many workloads, 70-80% of CPU cycles are spent serializing and deserializing data. Arrow solves this problem by enabling data to be shared between systems and processes with no serialization, deserialization or memory copies.

"An industry-standard columnar in-memory data layer enables users to combine multiple systems, applications and programming languages in a single workload without the usual overhead," said Ted Dunning, Vice President of the Apache Incubator and member of the Apache Arrow Project Management Committee.

In addition to traditional relational data, Arrow supports complex data with dynamic schemas. For example, Arrow can handle JSON data which is commonly used in IoT workloads, modern applications and log files. Implementations are also available (or underway) for a number of programming languages including Java, C++ and Python to allow greater interoperability among a number of Big Data solutions.

"Real world use cases often include complex combinations of structured and rapidly growing complex-data. Already tested with Apache Drill, the efficient in-memory columnar representation and processing in Arrow will enable users to enjoy the performance of columnar processing with the flexibility of JSON," said Parth Chandra, member of the Apache Drill and Apache Arrow Project Management Committees.

Catch Apache Arrow in action at Strata + Hadoop World (San Jose: 30 March 2016, and London: 1-3 June 2016), as well as upcoming MeetUps and local events http://arrow.apache.org/events

Availability and Oversight
Apache Arrow software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Arrow, visit http://arrow.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 5,300 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Alibaba Cloud Computing, ARM, Bloomberg, Budget Direct, Cerner, Cloudera, Comcast, Confluent, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Matt Mullenweg, Microsoft, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Apache Arrow", "Arrow", "Apache Calcite", "Calcite", "Apache Cassandra", "Cassandra", "Apache Drill", "Drill", "Apache Hadoop", "Hadoop", "Apache HBase", "HBase", "Apache Impala", "Impala", "Apache Kudu (incubating)", "Kudu (incubating)", "Apache Parquet", "Parquet", "Apache Phoenix", "Phoenix", "Apache Spark", "Spark", "Apache Storm", "Storm", "ApacheCon", and their logos are registered trademarks or trademarks of The Apache Software Foundation in the U.S. and/or other countries. All other brands and trademarks are the property of their respective owners.

# # # 

Tuesday December 08, 2015

The Apache Software Foundation Announces Apache™ Kylin™ as a Top-Level Project

Open Source petabyte-scale Big Data Distributed Analytics Engine in use at eBay, Exponential, JD.com, Meituan, MiningLAMP, and NetEase, among others.

Forest Hill, MD –8 December 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache Kylin has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Kylin is an Open Source Distributed Analytics Engine designed to provide a SQL interface and multi-dimensional analysis (OLAP) on Apache Hadoop, supporting extremely large datasets. 

"Apache Kylin's incubation journey has demonstrated the value of Open Source governance at ASF and the power of building an open-source community and ecosystem around the project," said Luke Han, Vice President of Apache Kylin. "Our community is engaging the world's biggest local developer community in alignment with the Apache Way."

A leading OLAP-on-Hadoop solution, Apache Kylin fills the gap between Big Data exploration and human use, enabling interactive analysis on massive datasets with sub-second latency for analysts, end users, developers, and data enthusiasts. With these capabilities, Apache Kylin brings back business intelligence (BI) to Apache Hadoop to unleash the value of Big Data. Kylin originated at eBay, and was submitted to the Apache Incubator in November 2014.

"Apache Kylin brings Big Data to the enterprise and enables petabyte scale analytics on all the existing enterprise BI tools," said Debashis Saha, Vice President of Commerce Platform and Infrastructure at eBay. "We are extremely happy to do this in a community-driven manner and we look forward to continued innovation and collaboration of community members to advance Big Data OLAP."

"Apache Kylin provides a fantastic solution which enabled us to do 'real' interactive analysis on large-scale data without significant query latency anymore," said Yinan Wu, Lead of Data Platform at NetEase (NASDAQ: NTES). "Many thanks to the Apache Kylin team. Apache Kylin will definitely benefit more users who are interested in OLAP for Big Data."

"As one of the mentors for Apache Kylin it was a pleasure to work with the team," said Henry Saputra, ASF Member and Apache Incubator Project Management Committee member. "The team learned the Apache Way very quickly and have been developing in the open as part of decision-making and adding new committers. As a mentor of the project it was just a matter of providing guidance in the right direction to go --the team just executed to deliver high quality releases."

In addition, Kylin has relationships to several other Apache projects. "We have tightly integrated Apache Calcite as our SQL Engine, and we provided Kylin Interpreter to Apache Zeppelin," added Han. "Also, Kylin is big consumer of Hadoop, Spark, Kafka, HBase and Zookeeper, together with these other key members of the Big Data family, ASF is a natural home for Kylin."

Global adoption of Apache Kylin
Apache Kylin is in use at different organizations world wide, with rapid adoption as a critical analytic platform for the fastest growing Big Data market in China.

"It is great to see Apache Kylin graduate to a Top Level Project within a relatively short period of time," said Seshu Adunuthula, Director of ADI at eBay. "It has been an exciting journey seeing the community evolve around Kylin, adopting it and contributing to its newer capabilities. Several companies in addition to eBay are now using Kylin as their Big Data OLAP engines."

"Apache Kylin and its low query latency with ANSI SQL on extreme datasets feature helped us to replace legacy RMDBs of JD.com's JCloud Open API platform, which eliminated the challenges of extreme data growth and smoothly expanding our capacity," said Ling Zhu, Sr. Director of Cloud Platform at JD.com (NASDAQ: JD). "With insight on JOS API statistics data, growing by more than 700 million records every day, Apache Kylin enabled us to do multi-dimensional analysis on tens of billions of reconrds with latency in seconds."

"We have helped many of our customers with Apache Kylin to set up end-to-end business intelligence solutions," said Shicong Feng, CTO at MiningLAMP. "Apache Kylin has proved to be a very useful and powerful tool for multi-dimensional data analysis and reporting in the Big Data area, I would recommend Apache Kylin to anyone interested in making BI solutions on massive amounts of data."

"Apache Kylin is the best Open Source project for Meituan's data warehouse requirements among of other analytics technologies, with its great features including sub-second query latency on a billion records with high scalability and seamless integration with BI products," said Wen Li, Sr. Researcher of Engineering and Technology at Meituan.com. "The excellent support from Apache Kylin community enabled Meituan's Big Data team to respond quickly for a variety of needs from multi-product lines. We are looking forward to Apache Kylin continuing their good work and quickly evolving to bring more value to Big Data industry."

"Apache Kylin is the best OLAP engine on Big Data so far," said Wilson Pang, Senior Director of Data Services and Solutions at eBay. "At eBay, we collect every user behavior on every eBay screen. While other OLAP engines struggle with the data volume, Kylin enables query responses in the milliseconds. Moreover, we are also starting to leverage Kylin for near real time data streaming storage and analytics engine. All together, Kylin serves as a critical backend component for eBay’s product analytics platform."

"Working with the Apache Kylin team to bring Kylin through incubation to top-level status has been really exciting for me," said Ted Dunning, Apache Kylin incubator mentor, Vice President of the Apache Incubator, and Chief Application Architect at MapR Technologies. "The technical aspects of Kylin are exciting, of course, but just as exciting is the way that Kylin represents a growing involvement of Asian countries like China in the Open Source community."

Availability and Oversight
Apache Kylin software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Kylin, visit http://kylin.apache.org/ and https://twitter.com/ApacheKylin

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 4,700 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Matt Mullenweg, Microsoft, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Kylin", "Apache Kylin", "Calcite", "Apache Calcite", "Hadoop", "Apache Hadoop", "HBase", "Apache HBase", "Kafka", "Apache Kafka", "Spark", "Apache Spark", "Zookeeper", "Apache Zookeeper", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

Monday November 23, 2015

The Apache Software Foundation Announces Apache™ Brooklyn™ as a Top-Level Project

Open Source framework for modelling, deploying, monitoring and managing applications in use at Canopy, IBM, SWIFT, and Virtustream, among others.

Forest Hill, MD –23 November 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Brooklyn™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Brooklyn is an application blueprint and management platform used for integrating services across multiple data centers as well as and a wide range of software in the Cloud.

"We're very proud of the work that our community has done to bring us to graduation," said Richard Downer, Vice President of Apache Brooklyn. "Our time in the Apache Incubator has given us the opportunity to grow the project, both its community and its code. Users of Brooklyn can now be confident that this is a project that is going to be around for a long time to come."

With modern applications being composed of many components, and increasing interest in micro-services architecture, the deployment and ongoing evolution of deployed apps is an increasingly difficult problem. Apache Brooklyn’s blueprints provide a clear, concise way to model an application, its components and their configuration, and the relationships between components, before deploying to public Cloud or private infrastructure. Policy-based management, built on the foundation of autonomic computing theory, continually evaluates the running application and makes modifications to it to keep it healthy and optimize for metrics such as cost and responsiveness.

Cloud service providers Canopy and Virtustream both recognize the value of having an application-centered view of services and have created product offerings built on Apache Brooklyn. IBM has also made extensive use of Apache Brooklyn in order to migrate large workloads from AWS to IBM Softlayer.

Apache Brooklyn is in use at SWIFT (Society for Worldwide Interbank Financial Telecommunication), creators of the industry syntax standard for financial messages. "Apache Brooklyn fills a gap in orchestration of service delivery," said Otmane Benali, Manager of Messaging Integration at SWIFT. "Its use of the CAMP standard provides operations a single window to managing heterogeneous platforms, very common in large enterprises."

Brooklyn was created by ASF sponsor Cloudsoft Corporation in 2011, and was submitted to the Apache Incubator in May 2014. The project recently released version 0.8.0, and is continuing to evolve fast, with the aim of making a stable, well-featured 1.0 release in the first half of 2016.

"Congratulations to Brooklyn for becoming an Apache Top Level Project," said Hadrian Zbarcea, Apache Brooklyn Incubator Mentor, ASF Member, and President of Apifocal. "As a standards based, modular, extensible framework for modeling, monitoring and managing Cloud applications through autonomic blueprints, Brooklyn offers a new paradigm for Cloud platforms deployment and has the potential to create new markets --similar to what virtualization meant for the Cloud computing space."

In addition, Brooklyn has relationships to several other Apache projects. "We are big consumers of Apache jclouds, and contributors to it, so that we get strong cross-Cloud portability," added Downer. "This made the Apache Software Foundation a natural home for Brooklyn. In addition, the Brooklyn community offers off-the-shelf blueprints for many well-known Apache projects, from Cassandra and Qpid to Mesos and Hadoop."

Catch Apache Brooklyn in action at Cloud Foundry Summit Asia in Shanghai on 3 December 2015 http://cfasia2015.sched.org/event/4jwB

Availability and Oversight
Apache Brooklyn software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Brooklyn, visit http://brooklyn.apache.org/ and https://twitter.com/ApacheBrooklyn

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 4,700 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Matt Mullenweg, Microsoft, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Brooklyn", "Apache Brooklyn", "Cassandra", "Hadoop", "jclouds", "Mesos", "Qpid", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Wednesday August 26, 2015

The Apache Software Foundation Announces Apache™ Lens™ as a Top-Level Project

Open Source Big Data platform seamlessly enables unified, multi-dimensional queries across multiple data stores.

Forest Hill, MD –26 August 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Lens™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Lens is a Unified Analytics platform. It provides an optimal execution environment for analytical queries in the unified view. Apache Lens aims to cut the Data Analytics silos by providing a single view of data across multiple tiered data stores.

"Incubating Apache Lens has been an amazing experience at the ASF," said Amareshwari Sriramadasu, Vice President of Apache Lens. "Apache Lens solves a very critical problem in Big Data analytics space with respect to end users. It enables business users, analysts, data scientists, developers and other users to do complex analysis with ease, without knowing the underlying data layout."

"Apache Lens is a fantastic project that enables simplified access to Big Data analytics," said Sharad Agarwal, member of the Apache Lens PMC. "I am very proud and thrilled to see it graduate as a Top-Level Apache project, and, being involved with the project since its inception, it's exciting to see its community grow."

By providing an online analytical processing (OLAP) model on top of data, Lens seamlessly integrates Apache Hadoop with traditional data warehouses to appear as one. It also provides query history and statistics for queries running in the system along with query life cycle management.

"The query service for our data platform is built on top of Apache Lens," said Gaurav Bhalotia, Vice President of Data Platform at Flipkart. "Lens gives us a powerful and simple abstraction to query data consistently across tiers and storage stacks. We at Flipkart are very excited to see it added as a Top-Level Apache project."

"I am really thrilled to see Lens graduating so soon after getting incubated in ASF," said Mohit Saxena, Founder and CTO of InMobi. "Lens is really a perfect example how technology can be leveraged to remove complexity of traditional analytical platforms and provide a simple abstraction for end user. Earlier our reporting and data retrieval system were married to a compute and even storage engine and user had to juggle for results, Hence the need of something like Lens arises and I am so proud that lens has solved a big problem where user can simply use one query layer at the top while lens does all heavy lifting below it and I simply hope this is just the beginning and it will only thrive with the help of vibrant Apache community. God speed."

"Apache Lens has been a truly stellar example of what an incubating project should be," said Jakob Homan, ASF Member and Apache Lens Project Mentor. "It's grown very quickly and will be a tremendously useful part of the Apache Big Data ecosystem."

Catch Apache Lens in action at Apache: Big Data Europe on 29 September 2015 http://events.linuxfoundation.org/events/apache-big-data-europe/

Availability and Oversight
Apache Lens software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Lens, visit http://lens.apache.org/ and https://twitter.com/ApacheLens

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 4,700 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Hadoop", "Apache Hadoop", "Lens", "Apache Lens", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Tuesday August 25, 2015

The Apache Software Foundation Announces Apache™ Ignite™ as a Top-Level Project

Fortune 500 enterprises adopt Open Source in-memory "Fast Data" platform to process large-scale data sets in real-time, at orders of magnitude faster than traditional technologies.

Forest Hill, MD –25 August 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Ignite™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Ignite is a high-performance, integrated and distributed In-Memory Data Fabric for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies. It is designed to easily power both existing and new applications in a distributed, massively parallel architecture on affordable, industry-standard hardware.

"Apache Ignite addresses today's Fast Data needs by providing a comprehensive in-memory data fabric, which includes a data grid with SQL and transactional capabilities, in-memory streaming, an in-memory file system, and more," said Dmitriy Setrakyan, Vice President of the Apache Ignite project and co-founder of GridGain Systems. 

Unlike other Big Data processing solutions, Apache Ignite is an in-memory computing (IMC) system, where RAM is treated as a primary storage facility (as opposed to being used exclusively for processing). As such, Ignite's memory-first approach is more efficient and faster: with improved system indexes, reduced data fetch time, and no delays in a stream content processing, among other benefits.

"Apache Ignite leverages and integrates a host of Apache projects to solve real-time business issues, including Spark, Hadoop, YARN and Mesos," said Dr. Konstantin Boudnik, Apache Ignite Project Management Committee Mentor and Vice President of Open Source Development at WANdisco. "It's exciting that it is graduating to a top-level project. We look forward to working further with the Apache Ignite community to make more enhancements that will benefit customers with real-time requirements and the need for highest performance and scale from their applications. "

"As the speed of memory continues to outpace the capabilities of even the highest performing disks, the importance of managing large pools of RAM at scale increases," said Roman Shaposhnik, Apache Ignite Mentor, and Director of Open Source at Pivotal. "It is essential to innovate at the same pace and the Apache Ignite community is certainly innovative. The enthusiasm in the area of in-memory computing is unmistakable, and the ASF is where important advances happen. It is exciting to see the work of Apache communities advancing the state of Fast Data with projects such as Apache Ignite, Spark, Geode and Flink."

Apache Ignite meets the growing trend for many enterprises seeking to adopt in-memory computing and replace hard drives as their primary storage system, where speed, superior caching, and strong consistency are key concerns. Ignite's ability to reduce latencies and increase application performance bridges Big Data with 'Fast Data' –bringing highly consistent computation and transactions on large data sets in real time. Additionally, Ignite's flexible programming model means it can be run from anywhere –whether a laptop, a commodity cluster, or a supercomputer– with APIs available for Java, Scala, C++, and .NET/C#. 

"Apache Ignite is a maverick of distributed computing," said Raul Kripalani, member of the Apache Camel Project Management Committee, and Integration/Messaging/Big Data Consultant and Engineer. "Rather than focusing on a single goal, it harnesses the power of multiple JVMs to offer services that no modern application can do without, such as caching, streaming and workload distribution. The team is talented, the documentation is superb and the technology has lots of potential."

"Having been with the project from its inception, I am very excited to see our community rapidly grow and build one of the most scalable, performant, and battle tested in-memory data processing platforms on the market today," added Setrakyan.

Catch Apache Ignite in action at Apache: Big Data Europe on 28-30 September 2015 http://events.linuxfoundation.org/events/apache-big-data-europe/ In addition, members of the Apache Ignite community will be present at the Big Data Innovation Summit (9-10 September/Boston), Strata Hadoop World (29 September–1 October/New York), and JavaOne (25-29 October/San Francisco).

Availability and Oversight
Apache Ignite software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Ignite, visit http://ignite.apache.org/ and https://twitter.com/ApacheIgnite

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 4,700 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Flink", "Apache Flink", "Geode", "Apache Geode", "Hadoop", "Apache Hadoop", "Hadoop YARN", "Apache Hadoop YARN", "Ignite", "Apache Ignite", "Mesos", "Apache Mesos", "Spark", "Apache Spark", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Monday July 20, 2015

The Apache Software Foundation Announces Apache™ NiFi™ as a Top-Level Project

NSA-originated Big Data automation system acquires and delivers data easily, securely, and reliably across enterprise systems in real time.

Forest Hill, MD --20 July 2015-- The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ NiFi™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache NiFi tackles a very old but increasingly relevant problem, which is how to automate the flow of data between systems. NiFi was built to address critical gaps in traditional systems where other solutions lacked sufficient security, interactivity, scalability, and data lineage.

"We took a project with more than eight years of development in a closed source environment and transitioned it to a very open and collaborative space," said Joe Witt, Vice President of Apache NiFi. "How easy that transition was speaks volumes to the effectiveness of the Incubator process and the community around Apache in general."

Based on the concepts of flow-based programming, NiFi is easy to use, powerful, reliable, and highly configurable. Two important features of NiFi are its powerful user interface and its fine grained data provenance tools. The interface allows users to intuitively understand and interact with the data flow directly in the browser, promoting faster and safer iteration.

The data provenance features allow the user to see how an object flowed through the system, replay it, and visualize what happened to it before and after key stages, thereby simplifying data flows that are often large, complex directed graphs involving transformations, forks, joins, and more.

"NiFi's seamless user interface, robust security features, and powerful data provenance offer a unique set of capabilities for solving the challenges of managing distributed systems," said Rob Bearden, CEO of Hortonworks. "We are proud NiFi participants and congratulate the NiFi community on becoming a top-level Apache project."

"NiFi's well designed, mature API has made our integration process remarkably straightforward," said Mike Bishop, Chief Systems Architect at Prescient Edge. "With it, we're able to track the origin, transformation, and persistence of data throughout our analytic processes."

In addition, NiFi uses a component based extension model to rapidly add capabilities to complex dataflows. Out of the box NiFi has several extensions for dealing with file-based dataflows such as FTP, SFTP, and HTTP integration as well as integration with HDFS. One of NiFi's unique features is a rich, Web-based interface for designing, controlling, and monitoring a dataflow.

"The NiFi user interface and ease of extension have made it extremely easy to get up and running and even customize," said Craig Connel, CTO of Leverege. "It is great that it also easily integrates with other parts of the Apache Big Data world like Spark, Kafka, and Hadoop."

NiFi originated at the National Security Agency (NSA) as Niagarafiles, and was submitted to the Apache Incubator in November 2014 as part of the NSA Technology Transfer Program.

"The contributions we've seen from the community over the past few months are really exciting," added Witt. "It is a good sign that while this project has been around for more than eight years by moving to the ASF we've really only just started." 

Catch Apache NiFi in action at OSCON in Portland, Oregon, on Friday 24 July 2015 http://www.oscon.com/open-source-2015/public/schedule/detail/42463

Availability and Oversight
Apache NiFi software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache NiFi, visit http://nifi.apache.org/ and https://twitter.com/apachenifi

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 4,700 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "NiFi", "Apache NiFi", "Hadoop", "Apache Hadoop", "Kafka", "Apache Kafka", "Spark", "Apache Spark", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners. 

# # #

Monday April 27, 2015

The Apache Software Foundation Announces Apache™ Parquet™ as a Top-Level Project

Open Source storage format for the Apache™ Hadoop® ecosystem in use at Cloudera, NASA, Netflix, Stripe, and Twitter, among other organizations 

Forest Hill, MD --27 April 2015-- The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Parquet™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

"The incubation process at Apache has been fantastic and really the last step of making Parquet a community driven standard fully integrated within the greater Hadoop ecosystem," said Julien Le Dem, Vice President of Apache Parquet.

Apache Parquet is an Open Source columnar storage format for the Apache™ Hadoop® ecosystem, built to work across programming languages and much more:
  • processing frameworks (MapReduce, Apache Spark, Scalding, Cascading, Crunch, Kite)
  • data models (Apache Avro, Apache Thrift, Protocol Buffers, POJOs)
  • query engines (Apache Hive, Impala, HAWQ, Apache Drill, Apache Tajo, Apache Pig, Presto, Apache Spark SQL)

"At Twitter, Parquet has helped us scale our big data usage by in some cases reducing storage requirements by one third on large datasets as well as scan and deserialization time. This translated into hardware savings as well as reduced latency for accessing the data. Furthermore, Parquet being integrated with so many tools creates opportunities and flexibility regarding query engines," said Chris Aniszczyk, Head of Open Source at Twitter. "Finally, it's just fantastic to see it graduate to a top-level project and we look forward to further collaborating with the Apache Parquet community to continually improve performance."

"Parquet's integration with other object models, like Avro and Thrift, has been a key feature for our customers," said Ryan Blue, Software Engineer at Cloudera. "They can take advantage of columnar storage without changing the classes they already use in their production applications."

"At Netflix, Parquet is the primary storage format for data warehousing. More than 7 petabytes of our 10+ Petabyte warehouse is Parquet formatted data that we query across a wide range of tools including Apache Hive, Apache Pig, Apache Spark, PigPen, Presto, and native MapReduce. The performance benefit of columnar projection and statistics is a game changer for our big data platform," said Daniel Weeks, Software Engineer at Netflix. "We look forward to working with the Apache community to advance the state of big data storage with Parquet and are excited to see the project graduate to full Apache status."

"Stripe's data warehouse has been built on Parquet from the beginning," said Avi Bryant, Engineering Manager at Stripe. "Every aspect of our pipeline, from data import to machine learning to adhoc SQL analysis, uses Apache Parquet as the common interchange format."

"I was extremely happy to see Parquet arrive as an Incubator project," said Chris Mattmann, Apache Parquet Incubator Mentor, and Chief Architect, Instrument and Science Data Systems Section at NASA Jet Propulsion Laboratory. "After talking with some in its community there was a real match with this columnar data format technology and its community with the way that we do things here at the ASF. Parquet has had an exemplar Incubation, and the project has big things ahead of it. I am encouraging my Data Science Team at NASA to evaluate it for data representation especially as it relates to our science holdings in Earth, planetary and space sciences, and astrophysics."

Catch Apache Parquet in action at the Hadoop Summit, 9-11 June 2015 in San Jose, California. The Apache Parquet project welcomes contributions and community participation through mailing lists, face-to-face MeetUps, and user events. For more information, visit http://parquet.apache.org/community/

Availability and Oversight
Apache Parquet software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Parquet, visit http://parquet.apache.org/ and https://twitter.com/ApacheParquet

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/.

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Avro", "Apache Avro", "Drill", "Apache Drill", "Hadoop", "Apache Hadoop", "Parquet", "Apache Parquet", "Pig", "Apache Pig", "Spark", "Apache Spark", "Thrift", "Apache Thrift", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Tuesday January 27, 2015

The Apache Software Foundation Announces Apache™ Samza™ as a Top-Level Project

Open Source Big Data distributed stream processing framework used in business intelligence, financial services, healthcare, mobile applications, security, and software development, among other industries.

Forest Hill, MD –27 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Samza™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

"The incubation process at Apache has been great. It has helped us cultivate a strong community, and provided us with the support and infrastructure to make Samza grow," said Chris Riccomini, Vice President of Apache Samza.

Apache Samza is a distributed stream processing framework, designed to handle fault tolerance, stateful processing, message durability, and scalability. Samza helps users to write light-weight processors that consume streams of data from messaging systems such as Apache Kafka. These processors empower organizations to understand and react to their data in real-time. In addition, Samza uses Apache Hadoop YARN to provide fault tolerance, processor isolation, security, and resource management.

Samza represents a different approach to stream processing. It has been purpose-built first and foremost as a production-grade system with operability and scalability in mind. Samza integrates tightly with Apache Kafka, which makes it a natural fit to those already running Kafka in their data pipeline. The framework also introduces the concept of stateful processing and aggregation as a first-class feature. Stateful processing gives Samza developers a completely new paradigm for aggregating stream data. These features help organizations do high performance stream processing at scale.

Created to process tracking data, service log data, and for data ingestion pipelines for realtime services, Samza originated at LinkedIn, and was submitted to the Apache Incubator in July 2013. 

"LinkedIn is thrilled to see Apache Samza experience such strong adoption and now graduate to a Top-Level Project. Samza was developed to help solve some of LinkedIn's  toughest stream processing challenges and has become a central piece of our infrastructure," said Kevin Scott, Senior Vice President of Engineering and Operations at LinkedIn.

Apache Samza is used in an array of industries, applications, and organizations, including:
  • DoubleDutch, developers of mobile apps for events and conferences, uses Samza to power their analytics platform and stream data live into an event dashboard for real-time insights;
  • Forstcales' Big Data security analytics solutions use Samza to processes security events log as part of the data ingestion pipelines and on-line machine learning models creation process;
  • Happy Pancake, Northern Europe's largest internet dating service, uses Samza for all event handlers and data replication;
  • Advertising technology provider Improve Digital uses Samza as the foundation of a realtime processing capability performing data analytics and as the basis for an alerting system;
  • Jack Henry & Associates uses Samza to process user activity data across its Banno suite of products for financial institutions;
  • MobileAware uses Samza as a foundation for two mobile network products: real time analytics and multi channel notification (push, text message and HTML5);
  • Technology startup Project Florida uses Samza for real-time monitoring of data streams from wearable sensors, for preventative healthcare purposes;
  • Quantiply, providers of Cloud-based micro-applications, uses Samza to bring together user event, system performance, and business operational data for real-time visibility and decision support; and
  • Social media business intelligence solution VinTank uses Samza to power their analysis and natural language processing (NLP) pipeline.


"We've had great experiences with Samza at Improve Digital where it has enabled us to  build out our streaming data platform," said Garry Turkington, CTO of Improve Digital. "It's fantastic to see it graduate to a top-level project."

Jay Kreps, CEO of Confluent, said "Samza is a fantastic piece of infrastructure, and a great complement to Apache Kafka. We at Confluent are really excited to see it added as a top-level Apache project."

"Fortscale has been using Apache Samza successfully to build online machine learning algorithms and detect insider threats," said Dotan Patrich, Software Architect at Fortscale. "It's been a great experience building large scale streaming solution and using Samza's and enjoying it's unique state management architecture. It's fantastic to see it graduate to a Top-Level Project."

"I've been involved in Apache Samza's community since its inception. It's been thrilling to watch the community grow, and I'm very proud and excited to see that the project is graduating. Samza has a bright future, and I'm looking forward to what's to come," added Riccomini.

Availability and Oversight
As with all Apache products, Apache Samza software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache Samza, visit http://samza.apache.org/ and @SamzaStream on Twitter

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow https://twitter.com/TheASF.

© The Apache Software Foundation. "Apache", "Apache Samza", "Samza", "Apache Hadoop", "Hadoop", "Hadoop YARN", "Apache Kafka", "Kafka", "ApacheCon", and the Apache Samza logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

The Apache Software Foundation Announces Apache™ BookKeeper™ as a Top-Level Project

Open Source distributed Big Data logging service and publish/subscribe system used to reliably log streams of records

Forest Hill, MD –27 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ BookKeeper™ has graduated to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache BookKeeper was established in 2011 as a sub-project of Apache ZooKeeper™ (Open Source API for highly reliable distributed coordination) to reliably log streams of records. It serves as a building block for reliable system consistency and recovery, and can be used to turn any standalone service into a highly available replicated service.

With disk/server failure rates up to 10% annually, replication is a must in today's always-on Cloud and Big Data services. One way to build a replicated service is to ensure that all write operations to the service are copied to all replicas; Apache BookKeeper's replicated logging service is well suited for this purpose. A database may have two replicas to ensure availability: if one crashes, the other can continue to serve traffic. However, ensuring that the data in these two replicas is consistent is not an easy problem to solve. Unlike naive solutions that run into problems like deadlock and inconsistency when one or both of the replicas fail, BookKeeper uses a combination of quorum writes, fencing, and, when necessary, outsourcing of consensus to ZooKeeper to ensure no state will be lost in the case of a replica failure. BookKeeper can similarly be applied to different classes of systems, such as messaging systems, filesystems and transaction processing systems.

Apache BookKeeper is highly available (no single point of failure), and scales horizontally as more storage nodes are added. BookKeeper is used in production in many web scale companies. At Yahoo, it is used as the persistence layer for its Cloud messaging infrastructure, which delivers tens of billions of messages in a day. BookKeeper is used at Twitter as the replicated persistence backend for different messaging use cases, and is also used by Huawei as a shared storage in their solution for HDFS Namenode High Availability. 

"We're very proud to have BookKeeper become a Top-Level Project. It is a testament to the hard work that my fellow committers have put in over the years that the ASF would give us their stamp of approval," said Ivan Kelly, Vice President of Apache BookKeeper. "We hope that the increased exposure will bring even more contributions and use cases to the community."

Availability and Oversight
As with all Apache products, Apache BookKeeper software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache BookKeeper, visit http://bookkeeper.apache.org and https://twitter.com/asfbookkeeper

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Apache BookKeeper", "BookKeeper", ApacheCon", and the Apache BookKeeper logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

Monday January 19, 2015

The Apache Software Foundation Announces Apache™ Falcon™ as a Top-Level Project

Open Source Big Data processing and management solution for Apache Hadoop™ in use at Hortonworks, InMobi, and Talend, among others.

Forest Hill, MD –19 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Falcon™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Falcon is a data processing and management solution for Apache Hadoop™, designed for data motion, coordination of data pipelines, lifecycle management, and data discovery. Falcon provides enterprises higher quality and predictable outcomes for their data by enabling end consumers to quickly onboard their data and its associated processing and management tasks on Hadoop clusters. The platform is successfully deployed across various industries, including advertising, healthcare, mobile applications, software solutions, and technology.

"Apache Falcon solves a very important and critical problem in the big data space. Graduation to TLP marks an important step in progression of the project," said Srikanth Sundarrajan, Vice President of Apache Falcon. "Falcon has a robust road map to ease the pain of application developers and administrators alike in authoring and managing complex data management and processing applications."

"Graduation of Apache Falcon's is a proud moment for the community who came together to solve a very relevant problem of data processing and management in Hadoop ecosystem," said Mohit Saxena, CTO and co-founder InMobi, one of the largest users of Apache Falcon. "I also want to applaud the efforts of contributors, committers and user community who actively pitched in the development of Falcon and it is only because of their conviction and efforts project has graduated. I am hoping promotion of Falcon to TLP will increase the contribution and adoption across the community and help Falcon achieve newer heights." 

Falcon represents a significant step forward in the Hadoop platform by enabling easy data management. Users of Falcon platform simply define infrastructure endpoints, data sets and processing rules declaratively. These declarative configurations are expressed in such a way that the dependencies between these configured entities are explicitly described. This information about inter-dependencies between various entities allows Falcon to orchestrate and manage various data management functions.

"Falcon has evolved over the last couple of years into a mature data management solution for Apache Hadoop with many production deployments proving it to be very valuable for users to manage their data and associated processing on Hadoop clusters," said Venkatesh Seetharam, Apache Falcon Project Management Committee member. 

"As Hadoop usage patterns have matured, the highest value implementations are based on the data lake concept. Data lakes require prescriptive and reliable pipelines," explained Greg Pavlik, Vice President of Engineering at Hortonworks. "Apache Falcon represents the best and most mature --and therefore essential-- building block for modeling, managing and operating data lakes."

"Falcon has enabled our team to incrementally build up a complex pipeline comprised of over 90 processes and 200 feeds that would have been very challenging with Apache Oozie alone," said programmer Michael Miklavcic.

"I began to work on Falcon in my spare time for fun, but it quickly became interesting in relation to my job at Talend", said Jean-Baptise Onofré, Vice President of Apache Karaf and Software Architect at Talend. "As Talend DataIntegration provides features like CDC (Change Data Capture), and data notification, we are in the process of integrating Apache Falcon in Talend products." 

"Apache Falcon's graduation is a milestone for the project and a credit to its contributors. Its open, collaborative development has effected a robust community around software essential to the Hadoop ecosystem," said Chris Douglas, Falcon incubation mentor at the ASF. "By becoming a Top-Level Project, the ASF recognizes its demonstrated ability to self-govern. Congratulations to Falcon's users, to its contributors, and particularly to its new Project Management Committee on this achievement."

Availability and Oversight
As with all Apache products, Apache Falcon software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache Falcon, visit http://falcon.apache.org/ and @ApacheFalcon on Twitter

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow https://twitter.com/TheASF.

© The Apache Software Foundation. "Apache", "Apache Falcon", "Falcon", "Apache Hadoop", "Hadoop", "Apache Oozie", "Oozie", "ApacheCon", and the Apache Falcon logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

Monday January 12, 2015

The Apache Software Foundation Announces Apache™ Flink™ as a Top-Level Project

Open Source distributed Big Data system for expressive, declarative, and efficient batch and streaming data processing and analysis

Forest Hill, MD –12 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Flink™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Flink is an Open Source distributed data analysis engine for batch and streaming data. It offers programming APIs in Java and Scala, as well as specialized APIs for graph processing, with more libraries in the making.

"I am very happy that the ASF has become the home for Flink," said Stephan Ewen, Vice President of Apache Flink. "For a community-driven effort, I can think of no better umbrella. It is great to see the project is maturing and many new people are joining the community."

Flink uses a unique combination of streaming/pipelining and batch processing techniques to create a platform that covers and unifies a broad set of batch and streaming data analytics use cases. The project has put significant efforts into making a system that runs reliably and fast in a wide variety of scenarios. For that reason, Flink contained its own type serialization, memory management, and cost-based query optimization components from the early days of the project.

Apache Flink has its roots in the Stratosphere research project that started in 2009 at TU Berlin together with the Berlin and later the European data management communities, including HU Berlin, Hasso Plattner Institute, KTH (Stockholm), ELTE (Budapest), and others. Several Flink committers recently started data Artisans, a Berlin-based startup committed to growing Flink both in code and community as 100% Open Source. More than 70 people have by now contributed to Flink.

"Becoming a Top-Level Project in such short time is a great milestone for Flink and reflects the speed with which the community has been growing," said Kostas Tzoumas, co-founder and CEO of data Artisans. "The community is currently working on some exciting new features that make Flink even more powerful and accessible to a wider audience, and several companies around the world are including Flink in their data infrastructure."

"We use Apache Flink as part of our production data infrastructure," said Ijad Madisch, co-founder and CEO of ResearchGate. "We are happy all around and excited that Flink provides us with the opportunity for even better developer productivity and testability, especially for complex data flows. It’s with good reason that Flink is now a top-level Apache project."

"I have been experimenting with Flink, and we are very excited to hear that Flink is becoming a top-level Apache project," said Anders Arpteg, Analytics Machine Learning Manager at Spotify.

Denis Arnaud, Head of Data Science Development of Travel Intelligence at Amadeus said, "At Amadeus, we continually seek for better improvement in our analytic platform and our experiments with Apache Flink for analytics on our travel data show a lot of potential in the system for our production use."

"Flink was a pleasure to mentor as a new Apache project," said Alan Gates, Apache Flink Incubator champion at the ASF, and architect/co-founder at Hortonworks. "The Flink team learned The Apache Way very quickly. They worked hard at being open in their decision making and including new contributors. Those of us mentoring them just needed to point them in the right direction and then let them get to work."

Availability and Oversight
As with all Apache products, Apache Flink software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache Flink, visit http://flink.apache.org/ and @ApacheFlink on Twitter.

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Apache Flink", "Flink", ApacheCon", and the Apache Flink logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

Not A Mirage: The Apache Software Foundation's official number of projects and initiatives "grows overnight" with census adjustment

110 sub-projects now added to the Foundation's official activity count

Over the past 15 years, The Apache Software Foundation has been going from strength to strength. It's truly impressive to see all that our global community has achieved. 

In documenting the ASF's developments, I've always measured our growth across two metrics: people and projects.

The people behind Apache is a volunteer community comprising 588 individual Members and 4,166 Committers collaborating across six continents. We're a true 24/7 global operation.

Up until now, I have stated that there were more than 200 projects and initiatives at the ASF. To me, "projects and initiatives" = software projects (Top-Level Projects (TLPs) + podlings undergoing development at the Apache Incubator + Apache Labs (our innovation "sandbox" to test technical concepts), plus various community initiatives such as ApacheCon. I never counted sub-projects as part of that census, as we always had just a few handfuls scattered amongst a small number of TLPs.

I was stunned when ASF Member Daniel Gruno recently informed me that my "200+" figure was wrong, and provided an updated tally of our initiatives as detailed on http://projects.apache.org

This audit surprised me, as it had shown that we did not have a "few handfuls" of sub-projects as I had projected, but rather *110* sub-projects! They had grown into an entity unto itself that must be recognized in our official count. 

As such, the ASF appears to have grown overnight with the addition of sub-projects, although they were there all along. As of today, we have:

- 160 Top-Level Projects
- 110 sub-projects (sub-projects of existing TLPs)
- 36 podlings undergoing development in the Apache Incubator
- 39 initiatives in the Apache Labs

So there are currently 345 Open Source software projects and initiatives at the ASF. Add to that the ASF's special committees and activities such as Infrastructure, Travel Assistance, Security Team, Legal Affairs, Brand Management, and ApacheCon: we've exceeded 350.

With this knowledge, I stand corrected, even more impressed, and have adjusted our records accordingly. Thanks again, Daniel!

Join me in celebrating our amazing community –three cheers for Apache!

--Sally Khudairi, Vice President Marketing & Publicity


Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation