Entries tagged [opensource]

Monday Aug 22, 2016

Announcing Apache® CloudStack™ v4.9

Mature, easy-to-deploy Open Source Cloud computing software meets the demands of the world's largest Cloud providers

Forest Hill, MD —22 August 2016— The Apache CloudStack project announced today the immediate availability of Apache CloudStack v4.9, the latest version of the turnkey Open Source Cloud computing software platform used for creating private, public, and hybrid Cloud environments.

Apache CloudStack Clouds enable billions of dollars worth of business transactions annually across their Clouds. Its maturity, stability and ease of deployment has led it to become the Open Source platform of choice for many service providers offering on-demand, elastic public Cloud computing services. CloudStack is also leveraged by many enterprises to provide private or hybrid Clouds for use by their employees.

CloudStack includes an intuitive user interface and rich APIs for managing the compute, networking, software, and storage infrastructure resources. It supports a broad range of integration points for both traditional and Cloud-native workloads.

"We're delighted to be releasing Apache CloudStack 4.9," said Will Stevens, Vice President of Apache CloudStack. "This version brings many new and exciting features, with the continued emphasis on making Apache CloudStack the easiest Cloud platform to deploy and most reliable platform for running IaaS environments. This release is the culmination of many months of hard work by our diverse developer community."

CloudStack 4.9 reflects dozens of new features and improvements, including:
  • Out-of-band power management of hosts through IPMI, iLO, DRAC
  • Enhancements to CloudStack networking model including deeper integration with Nuage VSP SDN
  • User defined security roles
  • More than 180 minor enhancements and issues fixed
A complete overview of all new enhancements can be found in the project release notes at http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.9.0/

Apache CloudStack has been used by thousands of organizations worldwide and is in use/production at Alcatel-Lucent, Autodesk, BT Cloud, China Telecom, cloud.ca, Cloudops, Codero Hosting, DataPipe, Exoscale, Globo.com, GreenQloud, Hokkaido University, IDC Frontier, Ikoula, Interoute, KDDI, KT/Korea Telecom, LeaseWeb, NTT, Orange, PCextreme, Schuberg Philis, Shopzilla, Slovak Telekom, Tata, Telia Latvia, Trader Media Group, TomTom, University of Melbourne, University of São Paolo, Verizon, and WebMD among many others. A list of known users of the software can be found at http://cloudstack.apache.org/users.html

"At Codero, we have been running our Public Cloud flawlessly on CloudStack since 2012 having selected it to power our Public and Hybrid Cloud due to its amazing stability, ease of use, and versatility," said Emil Sayegh, CEO and Chairman of Codero Hosting. "We are excited for the release of the next version of Apache CloudStack with its new features and capabilities that cement it as the Open Source platform of choice for leading hosting providers."

"Apache CloudStack has enabled us at Exoscale to build a public Cloud infrastructure offering that targets Cloud native teams directly," said Marc-Aurèle Brothier, Senior Developer at Exoscale. "CloudStack's flexibility lets us choose standard components for most workloads and then swap in our own code where our product and customers' needs demand it. We're excited to have contributed to CloudStack and also for the future of our CloudStack-based business."

"Apache CloudStack has been at the very foundation of Telia CLOUDY SERVICES already for more than 3 years," said Mārtiņš Paurs, CCO of CLOUDY SERVICES by Telia Latvia. "After starting with regular IaaS services, during time it has taken over orchestration of Cloud based video surveillance, CDN and video lifecycle management workloads, all for the benefit of our customers. And CloudStack does it really well! The ease of use, stability, professional and community support, powerful features  and scale we can achieve by using CloudStack has turned upside down the way and pace we innovate together with the very demanding market."

"cloud.ca is an avid supporter of Apache CloudStack not only because we value the stability and robustness of the platform, but because we highly value the power of the CloudStack community to drive innovation," said Mike Gero, Vice President of Product & Business Development at cloud.ca. "We look forward to benefiting from the myriad of improvements and fixes in CloudStack 4.9, as well as beginning to work with new features like the enhanced networking model."

"At University of São Paulo we are proud to be part of the Apache CloudStack community. We see the community playing a very strong role in solving many issues and providing ongoing effort to improve Apache CloudStack through recent versions, including 4.9," said Professor João Eduardo Ferreira, IT Office CIO, and Cyrano Rizzo, Sao Paulo IT Office Director, at University of São Paulo. "We are very happy to have adopted Apache CloudStack since 2012 as our main tool to run our infrastructure private cloud initiative of +500 hosts. Apache CloudStack is a professional-grade and very robust tool: we continue to use and encourage other institutions to use the same architecture."

Availability
CloudStack 4.9 is available immediately as a free download from http://cloudstack.apache.org/downloads.html. Apache CloudStack software is released under the Apache License v2.0.

Governance and Oversight
Apache CloudStack is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases.

Get Involved!
Apache CloudStack welcomes contribution and community participation through mailing lists as well as attending face-to-face MeetUps, developer trainings, and user events. Anybody wishing to get involved in the project can find out more at http://cloudstack.apache.org/contribute.html

About Apache CloudStack
Apache CloudStack is a mature, turnkey integrated Infrastructure-as-a-Service (IaaS) Open Source software platform that allows users to build feature-rich public and private Cloud environments. Hailed by Gartner Group as "a solid product", CloudStack includes an intuitive user interface and rich APIs for managing the compute, networking, software, and storage infrastructure resources. CloudStack entered the Apache Incubator in April 2012 and became an Apache Top-level Project in March 2013. For downloads, documentation, and ways to become involved with Apache CloudStack, visit http://cloudstack.apache.org/ and https://twitter.com/CloudStack

© The Apache Software Foundation. "Apache", "CloudStack", and "Apache CloudStack" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Monday Jun 06, 2016

The Apache® Software Foundation Announces Annual Report for 2015-2016 Fiscal Year

Trusted community recognized for accelerating Open Source innovation and advancing the future of open development.

Forest Hill, MD —6 June 2016— The Apache® Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the availability of its annual report for its 2015-2016 fiscal year, which ended 30 April 2016. Highlights include:

Open Source Leadership -- 

  • 21st anniversary of the Apache HTTP Server Project
  • 17th anniversary of The Apache Software Foundation
  • 175 committees managing 291 Top-level Projects (TLPs) and dozens of sub-projects
  • Millions of people worldwide access the ASF's two dozen servers and 75 distinct hosts each day
  • Apache featured more than 400 times in Gartner Magic Quadrant reports

Innovation --

  • 20 new Apache Top-level Projects (TLPs)
  • Record 55 podlings undergoing development in the Apache Incubator, plus 39 initiatives in the Apache Labs
  • Apache Hadoop® ecosystem continues to dominate the Big Data marketplace
  • 743 repositories managed
  • 33% increase in signed Individual Contributor License Agreements (CLAs)
  • 3,425 ASF Committers and 5,922 Apache code contributors (21% increase) added nearly 20M lines of code; average 18,000 code commits per month
  • 315,533,038 Lines of code changed (65% increase)

Community --

  • 58 new individual ASF Members elected
  • Apache projects overseen by 2,000+ Project Management Committee (PMC) members
  • Launched new "Help Wanted" application to match volunteers with Apache projects and activities seeking assistance
  • Held hundreds of events globally, including ApacheCon Europe and North America, plus countless conferences, workshops, and regional MeetUps
  • Participated in the Google Summer of Code as a Mentoring Organization for 12 consecutive years

Operations --

  • Apache services running 24x7x365 at near 100% uptime on an annual budget of less than US$5,000 per project
  • Launched new home.apache.org and lists.apache.org services
  • Initiated experiment with Github to automate user management and group permissions for 5,000+ Apache users
  • Launched new ASF brand identity
  • Ongoing trademarks, brand management, and legal support for dozens of existing and new projects
  • Bolstered organizational backing, with 37 ASF Sponsors and 11 Infrastructure partners
  • The ASF exited FY 2015 with revenue at $996K, ahead of projected budget

The full report is available online at https://s.apache.org/Ccml

# # #

© The Apache Software Foundation. "Apache" and "ApacheCon", are registered trademarks or trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

Tuesday Mar 08, 2016

The Apache® Software Foundation announces Apache Flink™ v1.0

Advanced distributed stream processing framework performs 50x faster than other real-time computation systems

Forest Hill, MD —8 March 2016— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the availability of Apache® Flink™ v1.0, the advanced Open Source distributed real-time stream processing system.

Apache Flink is an easy to use, yet sophisticated Open Source stream processor, with recent test results clocking in at least 50x faster than other distributed real-time computation systems.

"Releasing Flink 1.0 is the most important milestone in the project since graduation to a top-level Apache project one year ago," said Stephan Ewen, Vice President of Apache Flink and co-founder/CTO of data Artisans. "This is a collective achievement of more than 150 individuals that have contributed code to date."

Under The Hood
Flink uniquely supports a combination of features that include flexible windowing on event time, out-of-order stream handling, high availability, and exactly-once guarantees, together with high event throughput and low processing latency.

V.1.0 furthers Apache Flink's maturity, making it significantly easier to program, deploy, and maintain Flink pipelines at scale by:
  • initiating backwards compatibility of public APIs against all 1.x.y versions;
  • introducing functionality for complex event processing (CEP);
  • supporting large state beyond memory limits;
  • supporting state versioning and savepoints; and 
  • improving the system's monitoring functionality

"Flink v1.0 is indeed a testament to the maturity of the platform, which now enjoys production use at Fortune Global 500, as well as leading tech companies," said Kostas Tzoumas, member of the Apache Flink Project Management Committee, and co-founder/CEO of data Artisans.

"Google congratulates the Apache Flink community for this achievement," said William Vambenepe, Lead Product Manager for Big Data on Google Cloud Platform. "Flink is unlocking the richness of stream processing at scale, and delivering on the promise of the Dataflow Programming Model for all users, anywhere. We look forward to continuing to work with the Flink community, including further unification of APIs as part of Apache Beam (incubating)."

"At King.com we are using Flink to process more than 30 billion events daily, leveraging Flink's stateful streaming abstractions," said Christofer Waldenström, Team Lead for Streaming Platform at King.com. "We find that Flink provides a convenient way to interact with real-time data for complex streaming use-cases involving large state beyond memory."

"Apache Flink proved to be a valuable framework in our day-to-day business. It helps us to process log events, aggregate tracking information, apply filters and decide upon message routing," said Christian Kreutzfeldt, Senior Solution Developer & Architect at Otto Group BI. "We are still excited to see how fast new applications can be implemented and deployed. Even complex requirements do not constitute a significant challenge. For the upcoming version 1.0 we are looking forward to see a stabilized API and the advanced monitoring features. Especially the back pressure monitoring could become a great tool to understand internal processing behavior much better. Furthermore from an enterprise user perspective we are happy to see that Apache Flink finally reached version 1.0 which typically opens the door to the broader enterprise market."

Flink originated at the Stratosphere research project that started in 2009 by the Technical University of Berlin, along with several other European universities. The project was submitted to the Apache Incubator in April 2014 and became an Apache Top-Level Project in December 2014. 

Today, Flink among the ASF's dynamic Big Data projects, with more than 150 contributors to date, a wealth of production deployments, and commercial support by data Artisans, a company founded by the core team that originally developed Flink.

"The two things that have always struck me about Flink has been the excellence of the code and the excellence of the team," said Ted Dunning, Vice President of the Apache Incubator and Chief Application Architect at MapR. "This pattern is continuing with this release."

Get Involved!
Apache Flink welcomes contribution and community participation through mailing lists as well as attending face-to-face MeetUps, developer trainings, and the following events:
  • QCon (London, 7-9 March 2016)
  • Strata/Hadoop World (San Jose, 28-31 March 2016)
  • Hadoop Summit (Dublin, 13-14 April 2016)
  • Kafka Summit (San Francisco, 26 April 2016)
  • Apache: Big Data (Vancouver, 9-12 May 2016)
  • OSCON (Austin, TX, 18-19 May 2016)
  • Strata/Hadoop World (London, 31 May - 3 June 2016)
  • Berlin Buzzwords (Berlin, 5-7 June 2016)
  • Flink Forward (Berlin, 12-14 September 2016)

Availability and Oversight
Apache Flink software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Flink, visit http://flink.apache.org/ and https://twitter.com/ApacheFlink

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 5,300 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Alibaba Cloud Computing, ARM, Bloomberg, Budget Direct, Cerner, Cloudera, Comcast, Confluent, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Microsoft, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Apache Beam (incubating)", "Beam (incubating)", "Apache Cassandra", "Cassandra", "Apache Flink", "Flink", "Apache Hadoop", "Hadoop", "Apache HBase", "HBase", "Apache Kafka", "Kafka", "Apache MapReduce", "MapReduce", "Apache Storm", "Storm", "Apache YARN", "YARN", "ApacheCon", and their logos are registered trademarks or trademarks of The Apache Software Foundation in the U.S. and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Tuesday Dec 08, 2015

The Apache Software Foundation Announces Apache™ Kylin™ as a Top-Level Project

Open Source petabyte-scale Big Data Distributed Analytics Engine in use at eBay, Exponential, JD.com, Meituan, MiningLAMP, and NetEase, among others.

Forest Hill, MD –8 December 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache Kylin has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Kylin is an Open Source Distributed Analytics Engine designed to provide a SQL interface and multi-dimensional analysis (OLAP) on Apache Hadoop, supporting extremely large datasets. 

"Apache Kylin's incubation journey has demonstrated the value of Open Source governance at ASF and the power of building an open-source community and ecosystem around the project," said Luke Han, Vice President of Apache Kylin. "Our community is engaging the world's biggest local developer community in alignment with the Apache Way."

A leading OLAP-on-Hadoop solution, Apache Kylin fills the gap between Big Data exploration and human use, enabling interactive analysis on massive datasets with sub-second latency for analysts, end users, developers, and data enthusiasts. With these capabilities, Apache Kylin brings back business intelligence (BI) to Apache Hadoop to unleash the value of Big Data. Kylin originated at eBay, and was submitted to the Apache Incubator in November 2014.

"Apache Kylin brings Big Data to the enterprise and enables petabyte scale analytics on all the existing enterprise BI tools," said Debashis Saha, Vice President of Commerce Platform and Infrastructure at eBay. "We are extremely happy to do this in a community-driven manner and we look forward to continued innovation and collaboration of community members to advance Big Data OLAP."

"Apache Kylin provides a fantastic solution which enabled us to do 'real' interactive analysis on large-scale data without significant query latency anymore," said Yinan Wu, Lead of Data Platform at NetEase (NASDAQ: NTES). "Many thanks to the Apache Kylin team. Apache Kylin will definitely benefit more users who are interested in OLAP for Big Data."

"As one of the mentors for Apache Kylin it was a pleasure to work with the team," said Henry Saputra, ASF Member and Apache Incubator Project Management Committee member. "The team learned the Apache Way very quickly and have been developing in the open as part of decision-making and adding new committers. As a mentor of the project it was just a matter of providing guidance in the right direction to go --the team just executed to deliver high quality releases."

In addition, Kylin has relationships to several other Apache projects. "We have tightly integrated Apache Calcite as our SQL Engine, and we provided Kylin Interpreter to Apache Zeppelin," added Han. "Also, Kylin is big consumer of Hadoop, Spark, Kafka, HBase and Zookeeper, together with these other key members of the Big Data family, ASF is a natural home for Kylin."

Global adoption of Apache Kylin
Apache Kylin is in use at different organizations world wide, with rapid adoption as a critical analytic platform for the fastest growing Big Data market in China.

"It is great to see Apache Kylin graduate to a Top Level Project within a relatively short period of time," said Seshu Adunuthula, Director of ADI at eBay. "It has been an exciting journey seeing the community evolve around Kylin, adopting it and contributing to its newer capabilities. Several companies in addition to eBay are now using Kylin as their Big Data OLAP engines."

"Apache Kylin and its low query latency with ANSI SQL on extreme datasets feature helped us to replace legacy RMDBs of JD.com's JCloud Open API platform, which eliminated the challenges of extreme data growth and smoothly expanding our capacity," said Ling Zhu, Sr. Director of Cloud Platform at JD.com (NASDAQ: JD). "With insight on JOS API statistics data, growing by more than 700 million records every day, Apache Kylin enabled us to do multi-dimensional analysis on tens of billions of reconrds with latency in seconds."

"We have helped many of our customers with Apache Kylin to set up end-to-end business intelligence solutions," said Shicong Feng, CTO at MiningLAMP. "Apache Kylin has proved to be a very useful and powerful tool for multi-dimensional data analysis and reporting in the Big Data area, I would recommend Apache Kylin to anyone interested in making BI solutions on massive amounts of data."

"Apache Kylin is the best Open Source project for Meituan's data warehouse requirements among of other analytics technologies, with its great features including sub-second query latency on a billion records with high scalability and seamless integration with BI products," said Wen Li, Sr. Researcher of Engineering and Technology at Meituan.com. "The excellent support from Apache Kylin community enabled Meituan's Big Data team to respond quickly for a variety of needs from multi-product lines. We are looking forward to Apache Kylin continuing their good work and quickly evolving to bring more value to Big Data industry."

"Apache Kylin is the best OLAP engine on Big Data so far," said Wilson Pang, Senior Director of Data Services and Solutions at eBay. "At eBay, we collect every user behavior on every eBay screen. While other OLAP engines struggle with the data volume, Kylin enables query responses in the milliseconds. Moreover, we are also starting to leverage Kylin for near real time data streaming storage and analytics engine. All together, Kylin serves as a critical backend component for eBay’s product analytics platform."

"Working with the Apache Kylin team to bring Kylin through incubation to top-level status has been really exciting for me," said Ted Dunning, Apache Kylin incubator mentor, Vice President of the Apache Incubator, and Chief Application Architect at MapR Technologies. "The technical aspects of Kylin are exciting, of course, but just as exciting is the way that Kylin represents a growing involvement of Asian countries like China in the Open Source community."

Availability and Oversight
Apache Kylin software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Kylin, visit http://kylin.apache.org/ and https://twitter.com/ApacheKylin

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 4,700 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Matt Mullenweg, Microsoft, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Kylin", "Apache Kylin", "Calcite", "Apache Calcite", "Hadoop", "Apache Hadoop", "HBase", "Apache HBase", "Kafka", "Apache Kafka", "Spark", "Apache Spark", "Zookeeper", "Apache Zookeeper", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

Monday Nov 23, 2015

The Apache Software Foundation Announces Apache™ Brooklyn™ as a Top-Level Project

Open Source framework for modelling, deploying, monitoring and managing applications in use at Canopy, IBM, SWIFT, and Virtustream, among others.

Forest Hill, MD –23 November 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Brooklyn™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Brooklyn is an application blueprint and management platform used for integrating services across multiple data centers as well as and a wide range of software in the Cloud.

"We're very proud of the work that our community has done to bring us to graduation," said Richard Downer, Vice President of Apache Brooklyn. "Our time in the Apache Incubator has given us the opportunity to grow the project, both its community and its code. Users of Brooklyn can now be confident that this is a project that is going to be around for a long time to come."

With modern applications being composed of many components, and increasing interest in micro-services architecture, the deployment and ongoing evolution of deployed apps is an increasingly difficult problem. Apache Brooklyn’s blueprints provide a clear, concise way to model an application, its components and their configuration, and the relationships between components, before deploying to public Cloud or private infrastructure. Policy-based management, built on the foundation of autonomic computing theory, continually evaluates the running application and makes modifications to it to keep it healthy and optimize for metrics such as cost and responsiveness.

Cloud service providers Canopy and Virtustream both recognize the value of having an application-centered view of services and have created product offerings built on Apache Brooklyn. IBM has also made extensive use of Apache Brooklyn in order to migrate large workloads from AWS to IBM Softlayer.

Apache Brooklyn is in use at SWIFT (Society for Worldwide Interbank Financial Telecommunication), creators of the industry syntax standard for financial messages. "Apache Brooklyn fills a gap in orchestration of service delivery," said Otmane Benali, Manager of Messaging Integration at SWIFT. "Its use of the CAMP standard provides operations a single window to managing heterogeneous platforms, very common in large enterprises."

Brooklyn was created by ASF sponsor Cloudsoft Corporation in 2011, and was submitted to the Apache Incubator in May 2014. The project recently released version 0.8.0, and is continuing to evolve fast, with the aim of making a stable, well-featured 1.0 release in the first half of 2016.

"Congratulations to Brooklyn for becoming an Apache Top Level Project," said Hadrian Zbarcea, Apache Brooklyn Incubator Mentor, ASF Member, and President of Apifocal. "As a standards based, modular, extensible framework for modeling, monitoring and managing Cloud applications through autonomic blueprints, Brooklyn offers a new paradigm for Cloud platforms deployment and has the potential to create new markets --similar to what virtualization meant for the Cloud computing space."

In addition, Brooklyn has relationships to several other Apache projects. "We are big consumers of Apache jclouds, and contributors to it, so that we get strong cross-Cloud portability," added Downer. "This made the Apache Software Foundation a natural home for Brooklyn. In addition, the Brooklyn community offers off-the-shelf blueprints for many well-known Apache projects, from Cassandra and Qpid to Mesos and Hadoop."

Catch Apache Brooklyn in action at Cloud Foundry Summit Asia in Shanghai on 3 December 2015 http://cfasia2015.sched.org/event/4jwB

Availability and Oversight
Apache Brooklyn software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Brooklyn, visit http://brooklyn.apache.org/ and https://twitter.com/ApacheBrooklyn

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 4,700 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, LeaseWeb, Matt Mullenweg, Microsoft, PhoenixNAP, Pivotal, Private Internet Access, Produban, Red Hat, Serenata Flowers, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Brooklyn", "Apache Brooklyn", "Cassandra", "Hadoop", "jclouds", "Mesos", "Qpid", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Monday Jul 20, 2015

The Apache Software Foundation Announces Apache™ NiFi™ as a Top-Level Project

NSA-originated Big Data automation system acquires and delivers data easily, securely, and reliably across enterprise systems in real time.

Forest Hill, MD --20 July 2015-- The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ NiFi™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache NiFi tackles a very old but increasingly relevant problem, which is how to automate the flow of data between systems. NiFi was built to address critical gaps in traditional systems where other solutions lacked sufficient security, interactivity, scalability, and data lineage.

"We took a project with more than eight years of development in a closed source environment and transitioned it to a very open and collaborative space," said Joe Witt, Vice President of Apache NiFi. "How easy that transition was speaks volumes to the effectiveness of the Incubator process and the community around Apache in general."

Based on the concepts of flow-based programming, NiFi is easy to use, powerful, reliable, and highly configurable. Two important features of NiFi are its powerful user interface and its fine grained data provenance tools. The interface allows users to intuitively understand and interact with the data flow directly in the browser, promoting faster and safer iteration.

The data provenance features allow the user to see how an object flowed through the system, replay it, and visualize what happened to it before and after key stages, thereby simplifying data flows that are often large, complex directed graphs involving transformations, forks, joins, and more.

"NiFi's seamless user interface, robust security features, and powerful data provenance offer a unique set of capabilities for solving the challenges of managing distributed systems," said Rob Bearden, CEO of Hortonworks. "We are proud NiFi participants and congratulate the NiFi community on becoming a top-level Apache project."

"NiFi's well designed, mature API has made our integration process remarkably straightforward," said Mike Bishop, Chief Systems Architect at Prescient Edge. "With it, we're able to track the origin, transformation, and persistence of data throughout our analytic processes."

In addition, NiFi uses a component based extension model to rapidly add capabilities to complex dataflows. Out of the box NiFi has several extensions for dealing with file-based dataflows such as FTP, SFTP, and HTTP integration as well as integration with HDFS. One of NiFi's unique features is a rich, Web-based interface for designing, controlling, and monitoring a dataflow.

"The NiFi user interface and ease of extension have made it extremely easy to get up and running and even customize," said Craig Connel, CTO of Leverege. "It is great that it also easily integrates with other parts of the Apache Big Data world like Spark, Kafka, and Hadoop."

NiFi originated at the National Security Agency (NSA) as Niagarafiles, and was submitted to the Apache Incubator in November 2014 as part of the NSA Technology Transfer Program.

"The contributions we've seen from the community over the past few months are really exciting," added Witt. "It is a good sign that while this project has been around for more than eight years by moving to the ASF we've really only just started." 

Catch Apache NiFi in action at OSCON in Portland, Oregon, on Friday 24 July 2015 http://www.oscon.com/open-source-2015/public/schedule/detail/42463

Availability and Oversight
Apache NiFi software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache NiFi, visit http://nifi.apache.org/ and https://twitter.com/apachenifi

About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 550 individual Members and 4,700 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "NiFi", "Apache NiFi", "Hadoop", "Apache Hadoop", "Kafka", "Apache Kafka", "Spark", "Apache Spark", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners. 

# # #

Tuesday Jun 02, 2015

The Apache® Software Foundation Announces Annual Report for 2014-2015 Fiscal Year

Open Source Leadership Grows with Accelerated Innovation and Adoption of Apache Projects

Forest Hill, MD —2 June 2015— The Apache® Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the availability of its annual report for its 2014-2015 fiscal year, which ended 30 April 2015.

Momentum continues to accelerate under the Apache umbrella, including:

Open Source Leadership --

  • 20th anniversary of the Apache HTTP Server Project
  • 15th anniversary of The Apache Software Foundation
  • More than 16,000 code commits per month
  • 831 Individual Contributor License Agreements (CLA), 58 Corporate CLAs, and 41 Software Grants signed 

Innovation --

  • More than 350 Projects in development
  • 17 new Apache Top-level Projects (TLPs)
  • 41 podlings in the Apache Incubator
  • Facilitated contribution with 317 Git and 296 Subversion repositories 

Community --

  • ASF Membership exceeded 600 individuals
  • 163 TLPs overseen by more than 2,000 Project Management Committee (PMC) members
  • Held hundreds of events globally, including ApacheCon Europe and North America, as well as countless conferences, workshops, and regional MeetUps
  • Mentored hundreds of students in "The Apache Way" of software development for the 11th consecutive year through the Google Summer of Code

Operations --

  • Bolstered Infrastructure, with 99.57% total uptime across all Apache services
  • Launched new "Powered by Apache" graphics and revised apache.org look-and-feel
  • Provided trademarks, brand management, and legal support for dozens of existing and new projects
  • Rise in corporate support, with 39 Sponsors
  • The ASF exited FY 2014 with revenue at $996K, ahead of projected budget


The full report is available online at http://s.apache.org/ZVp

# # #

© The Apache Software Foundation. "Apache" and "ApacheCon", are registered trademarks or trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

Tuesday May 19, 2015

The Apache Software Foundation Announces Apache™ Drill™ 1.0

Thousands of users adopt Open Source, enterprise-grade, schema-free SQL query engine for Apache Hadoop®, NoSQL, and Cloud storage.

Forest Hill, MD --19 May 2015-- The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the availability of Apache™ Drill™ 1.0, the schema-free SQL query engine for Apache Hadoop®, NoSQL, and Cloud storage.

"The production-ready 1.0 release represents a significant milestone for the Drill project," said Tomer Shiran, member of the Apache Drill Project Management Committee. "It is the outcome of almost three years of development involving dozens of engineers from numerous companies. Apache Drill's flexibility and ease-of-use have attracted thousands of users, and the enterprise-grade reliability, security and performance in the 1.0 release will further accelerate adoption."

With the exponential growth of data in recent years, and the shift towards rapid application development, new data is increasingly being stored in non-relational, schema-free datastores including Hadoop, NoSQL and Cloud storage. Apache Drill revolutionizes data exploration and analytics by enabling analysts, business users, data scientists and developers to explore and analyze this data without sacrificing the flexibility and agility offered by these datastores. Drill processes the data in-situ without requiring users to define schemas or transform data.

"Drill introduces the JSON document model to the world of SQL-based analytics and BI" said Jacques Nadeau, Vice President of Apache Drill. "This enables users to query fixed-schema, evolving-schema and schema-free data stored in a variety of formats and datastores. The architecture of relational query engines and databases is built on the assumption that all data has a simple and static structure that’s known in advance, and this 40-year-old assumption is simply no longer valid. We designed Drill from the ground up to address the new reality.”

Apache Drill's architecture is unique in many ways. It is the only columnar execution engine that supports complex and schema-free data, and the only execution engine that performs data-driven query compilation (and re-compilation, also known as schema discovery) during query execution. These unique capabilities enable Drill to achieve record-breaking performance with the flexibility offered by the JSON document model.

The business intelligence (BI) partner ecosystem is embracing the power of Apache Drill. Organizations such as Information Builders, JReport (Jinfonet Software), MicroStrategy, Qlik®, Simba, Tableau, and TIBCO, are working closely with the Drill community to interoperate BI tools with Drill through standard ODBC and JDBC connectivity. This collaboration enables end users to explore data by leveraging sophisticated visualization tools and advanced analytics.

"We've been using Apache Drill for the past six months," said Andrew Hamilton, CTO of Cardlytics. "Its ease of deployment and use along with its ability to quickly process trillions of records has made it an invaluable tool inside Cardlytics. Queries that were previously insurmountable are now common occurrence. Congratulations to the Drill community on this momentous occasion." 

"Drill's columnar execution engine and optimizer take full advantage of Apache Parquet's columnar storage to achieve maximum performance," said Julien Le Dem, Technical Lead of Analytics Data Pipeline at Twitter and Vice President of Apache Parquet. "The Drill team has been a key contributor to the Parquet project, including recent enhancements to Parquet types and vectorization. The Drill team’s involvement in the Parquet community is instrumental in driving the standard."

"Apache Drill 1.0 raises the bar for secure, reliable and scalable SQL-on-Hadoop," said Piyush Bhargava, distinguished engineer, IT, Cisco Systems. "Because Drill integrates with existing data virtualization and visualization tools, we expect it will improve adoption of self-service data exploration and large-scale BI queries on our advanced Hadoop platform at Cisco."

"MicroStrategy recognized early on the value of Apache Drill and is one of the first analytic platforms to certify Drill," said Tim Lang, senior executive vice president and chief technology officer at MicroStrategy Incorporated.  "Because Drill is designed to be used with a minimal learning curve, it opens up more complex data sets to the end user who can immediately visualize and analyze new information using MicroStrategy’s advanced capabilities."

"Apache Drill closes a gap around self-service SQL queries in Hadoop, especially on complex, dynamic NoSQL data types," said Mike Foster, Strategic Alliances Technology Officer at Qlik.  "Drill's performance advantages for Hadoop data access, combined with the Qlik associative experience, enables our customers to continue discovering business value from a wide range of data. Congratulations to the Apache Drill community."

"Apache Drill empowers people to access data that is traditionally difficult to work with," said Jeff Feng, product manager, Tableau.  "Direct access within a centralized data repository and without pre-generating metadata definitions encourages data democracy which is essential for data-driven organizations. Additionally, Drill's instant and secure access to complex data formats, such as JSON, opens up extended analytical opportunities."

"Congratulations to the Apache Drill community on the availability of 1.0," said Karl Van den Bergh, Vice President, Products and Cloud at TIBCO. "Drill promises to bring low-latency access to data stored in Hadoop and HBase via standard SQL semantics. This innovation is in line with the value of Fast Data analysis, which TIBCO customers welcome and appreciate."

"The community's accomplishment is a testament to The Apache Software Foundation's ability to bring together diverse companies to work towards a common goal. None of this would have been possible without the contribution of engineers with advanced degrees and experience in relational databases, data warehousing, MPP, query optimization, Hadoop and NoSQL," added Nadeau. "Our community's strength is what will solidify Apache Drill as a key data technology for the next decade. We welcome interested individuals to learn more about Drill by joining the community's mailing lists, attending upcoming talks by Drill code committers at various conferences including Hadoop Summit, NoSQL Now, Hadoop World, or at a local Apache Drill MeetUp."

Availability and Oversight
Apache Drill 1.0 is available immediately as a free download from http://drill.apache.org/download/. Documentation is available at http://drill.apache.org/docs/. As with all Apache products, Apache Drill software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the project's day-to-day operations, including community development and product releases. For ways to become involved with Apache Drill, visit http://drill.apache.org/ and @ApacheDrill on Twitter.

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Bloomberg, Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Apache Drill", "Drill", "Apache Hadoop", "Hadoop", "Apache Parquet", "Parquet", and "ApacheCon", are registered trademarks or trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

Tuesday Feb 03, 2015

Apache™ PDFBox™ named an Open Source Partner Organization of the PDF Association

Liaison helps enterprise users benefit from enhanced PDF technology and serves as a foundation to other software applications.

Forrest Hill, MD —03 February 2015— Apache PDFBox™, a Top-Level Project of The Apache Software Foundation, today announced the project has been named as a Partner Organization of the PDF Association. 

The Apache PDFBox™ library is an Open Source Java tool for working with Portable Document Format (PDF) documents. It allows for the creation of new PDF documents, manipulation, rendering, signing of existing documents and the ability to extract content from documents. Apache PDFBox also includes several command line utilities.

"We are proud to be recognized by the PDF Association as a driver of ISO-­standardized PDF technology for electronic documents," said Andreas Lehmkühler, Vice President of Apache PDFBox. "Our liaison will help further strengthen the development of Apache PDFBox by providing access to knowledge and resources around PDF technology."

PDF was first released by Adobe Systems in 1993, became an ISO International Standard - ISO 32000-1 in 2008.

Founded in 2006, the PDF Association (http://www.pdfa.org/) is an international organization promoting awareness and adoption of open standards in digital document applications using PDF technology. The association facilitates education, networking, communication, and sharing of expertise and experience with interested parties worldwide. It offers its membership of over 150 enterprises and individual subject-matter experts from more than 20 countries direct contact with PDF technology experts and access to documents from ISO working groups, including release candidates for PDF upcoming standards.

The PDF Association's Partner Organizations are international associations concerned with document management, enterprise content management, long-term archiving and accessibility. Apache PDFBox the PDF Association's first Open Source Partner Organization. The PDF Association delivers vital information about implementing PDF technology to software developers and IT decision makers, and helps document management and ECM implementers understand and leverage ISO-standardized PDF technology. In turn, enterprise systems implementers and end-users benefit from enhanced PDF technology.

"With Apache PDFBox, the first Open Source organization is joining our Association," said Thomas Zellmann, Managing Director of the PDF Association. "This is our contribution to support the growth of freely available PDF solutions and their functionality to further expand the market penetration of the PDF standard."

"Being part of the PDF Association recognizes our commitment to making ISO standardized electronic document technology easily available by leveraging Apache PDFBox as a foundation for other software applications," added Lehmkühler.

About Apache PDFBox
The Apache PDFBox™ library is an Open Source Java tool that allows users to create new PDF documents, manipulate existing documents, extract content, digitally sign, print, and validate files against the PDF/A-1b standard. It also includes several command line utilities, including encrypt, decrypt, overlay, debugger, merger, PDFToImage, and TextToPDF. Apache PDFBox software is released under the Apache License v2.0, and is overseen by a self-selected volunteer team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache PDFBox, visit http://pdfbox.apache.org/

# # #


Tuesday Jan 27, 2015

The Apache Software Foundation Announces Apache™ Samza™ as a Top-Level Project

Open Source Big Data distributed stream processing framework used in business intelligence, financial services, healthcare, mobile applications, security, and software development, among other industries.

Forest Hill, MD –27 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Samza™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

"The incubation process at Apache has been great. It has helped us cultivate a strong community, and provided us with the support and infrastructure to make Samza grow," said Chris Riccomini, Vice President of Apache Samza.

Apache Samza is a distributed stream processing framework, designed to handle fault tolerance, stateful processing, message durability, and scalability. Samza helps users to write light-weight processors that consume streams of data from messaging systems such as Apache Kafka. These processors empower organizations to understand and react to their data in real-time. In addition, Samza uses Apache Hadoop YARN to provide fault tolerance, processor isolation, security, and resource management.

Samza represents a different approach to stream processing. It has been purpose-built first and foremost as a production-grade system with operability and scalability in mind. Samza integrates tightly with Apache Kafka, which makes it a natural fit to those already running Kafka in their data pipeline. The framework also introduces the concept of stateful processing and aggregation as a first-class feature. Stateful processing gives Samza developers a completely new paradigm for aggregating stream data. These features help organizations do high performance stream processing at scale.

Created to process tracking data, service log data, and for data ingestion pipelines for realtime services, Samza originated at LinkedIn, and was submitted to the Apache Incubator in July 2013. 

"LinkedIn is thrilled to see Apache Samza experience such strong adoption and now graduate to a Top-Level Project. Samza was developed to help solve some of LinkedIn's  toughest stream processing challenges and has become a central piece of our infrastructure," said Kevin Scott, Senior Vice President of Engineering and Operations at LinkedIn.

Apache Samza is used in an array of industries, applications, and organizations, including:
  • DoubleDutch, developers of mobile apps for events and conferences, uses Samza to power their analytics platform and stream data live into an event dashboard for real-time insights;
  • Forstcales' Big Data security analytics solutions use Samza to processes security events log as part of the data ingestion pipelines and on-line machine learning models creation process;
  • Happy Pancake, Northern Europe's largest internet dating service, uses Samza for all event handlers and data replication;
  • Advertising technology provider Improve Digital uses Samza as the foundation of a realtime processing capability performing data analytics and as the basis for an alerting system;
  • Jack Henry & Associates uses Samza to process user activity data across its Banno suite of products for financial institutions;
  • MobileAware uses Samza as a foundation for two mobile network products: real time analytics and multi channel notification (push, text message and HTML5);
  • Technology startup Project Florida uses Samza for real-time monitoring of data streams from wearable sensors, for preventative healthcare purposes;
  • Quantiply, providers of Cloud-based micro-applications, uses Samza to bring together user event, system performance, and business operational data for real-time visibility and decision support; and
  • Social media business intelligence solution VinTank uses Samza to power their analysis and natural language processing (NLP) pipeline.


"We've had great experiences with Samza at Improve Digital where it has enabled us to  build out our streaming data platform," said Garry Turkington, CTO of Improve Digital. "It's fantastic to see it graduate to a top-level project."

Jay Kreps, CEO of Confluent, said "Samza is a fantastic piece of infrastructure, and a great complement to Apache Kafka. We at Confluent are really excited to see it added as a top-level Apache project."

"Fortscale has been using Apache Samza successfully to build online machine learning algorithms and detect insider threats," said Dotan Patrich, Software Architect at Fortscale. "It's been a great experience building large scale streaming solution and using Samza's and enjoying it's unique state management architecture. It's fantastic to see it graduate to a Top-Level Project."

"I've been involved in Apache Samza's community since its inception. It's been thrilling to watch the community grow, and I'm very proud and excited to see that the project is graduating. Samza has a bright future, and I'm looking forward to what's to come," added Riccomini.

Availability and Oversight
As with all Apache products, Apache Samza software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache Samza, visit http://samza.apache.org/ and @SamzaStream on Twitter

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow https://twitter.com/TheASF.

© The Apache Software Foundation. "Apache", "Apache Samza", "Samza", "Apache Hadoop", "Hadoop", "Hadoop YARN", "Apache Kafka", "Kafka", "ApacheCon", and the Apache Samza logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

The Apache Software Foundation Announces Apache™ BookKeeper™ as a Top-Level Project

Open Source distributed Big Data logging service and publish/subscribe system used to reliably log streams of records

Forest Hill, MD –27 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ BookKeeper™ has graduated to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache BookKeeper was established in 2011 as a sub-project of Apache ZooKeeper™ (Open Source API for highly reliable distributed coordination) to reliably log streams of records. It serves as a building block for reliable system consistency and recovery, and can be used to turn any standalone service into a highly available replicated service.

With disk/server failure rates up to 10% annually, replication is a must in today's always-on Cloud and Big Data services. One way to build a replicated service is to ensure that all write operations to the service are copied to all replicas; Apache BookKeeper's replicated logging service is well suited for this purpose. A database may have two replicas to ensure availability: if one crashes, the other can continue to serve traffic. However, ensuring that the data in these two replicas is consistent is not an easy problem to solve. Unlike naive solutions that run into problems like deadlock and inconsistency when one or both of the replicas fail, BookKeeper uses a combination of quorum writes, fencing, and, when necessary, outsourcing of consensus to ZooKeeper to ensure no state will be lost in the case of a replica failure. BookKeeper can similarly be applied to different classes of systems, such as messaging systems, filesystems and transaction processing systems.

Apache BookKeeper is highly available (no single point of failure), and scales horizontally as more storage nodes are added. BookKeeper is used in production in many web scale companies. At Yahoo, it is used as the persistence layer for its Cloud messaging infrastructure, which delivers tens of billions of messages in a day. BookKeeper is used at Twitter as the replicated persistence backend for different messaging use cases, and is also used by Huawei as a shared storage in their solution for HDFS Namenode High Availability. 

"We're very proud to have BookKeeper become a Top-Level Project. It is a testament to the hard work that my fellow committers have put in over the years that the ASF would give us their stamp of approval," said Ivan Kelly, Vice President of Apache BookKeeper. "We hope that the increased exposure will bring even more contributions and use cases to the community."

Availability and Oversight
As with all Apache products, Apache BookKeeper software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache BookKeeper, visit http://bookkeeper.apache.org and https://twitter.com/asfbookkeeper

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Apache BookKeeper", "BookKeeper", ApacheCon", and the Apache BookKeeper logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

Monday Jan 19, 2015

The Apache Software Foundation Announces Apache™ Falcon™ as a Top-Level Project

Open Source Big Data processing and management solution for Apache Hadoop™ in use at Hortonworks, InMobi, and Talend, among others.

Forest Hill, MD –19 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Falcon™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Falcon is a data processing and management solution for Apache Hadoop™, designed for data motion, coordination of data pipelines, lifecycle management, and data discovery. Falcon provides enterprises higher quality and predictable outcomes for their data by enabling end consumers to quickly onboard their data and its associated processing and management tasks on Hadoop clusters. The platform is successfully deployed across various industries, including advertising, healthcare, mobile applications, software solutions, and technology.

"Apache Falcon solves a very important and critical problem in the big data space. Graduation to TLP marks an important step in progression of the project," said Srikanth Sundarrajan, Vice President of Apache Falcon. "Falcon has a robust road map to ease the pain of application developers and administrators alike in authoring and managing complex data management and processing applications."

"Graduation of Apache Falcon's is a proud moment for the community who came together to solve a very relevant problem of data processing and management in Hadoop ecosystem," said Mohit Saxena, CTO and co-founder InMobi, one of the largest users of Apache Falcon. "I also want to applaud the efforts of contributors, committers and user community who actively pitched in the development of Falcon and it is only because of their conviction and efforts project has graduated. I am hoping promotion of Falcon to TLP will increase the contribution and adoption across the community and help Falcon achieve newer heights." 

Falcon represents a significant step forward in the Hadoop platform by enabling easy data management. Users of Falcon platform simply define infrastructure endpoints, data sets and processing rules declaratively. These declarative configurations are expressed in such a way that the dependencies between these configured entities are explicitly described. This information about inter-dependencies between various entities allows Falcon to orchestrate and manage various data management functions.

"Falcon has evolved over the last couple of years into a mature data management solution for Apache Hadoop with many production deployments proving it to be very valuable for users to manage their data and associated processing on Hadoop clusters," said Venkatesh Seetharam, Apache Falcon Project Management Committee member. 

"As Hadoop usage patterns have matured, the highest value implementations are based on the data lake concept. Data lakes require prescriptive and reliable pipelines," explained Greg Pavlik, Vice President of Engineering at Hortonworks. "Apache Falcon represents the best and most mature --and therefore essential-- building block for modeling, managing and operating data lakes."

"Falcon has enabled our team to incrementally build up a complex pipeline comprised of over 90 processes and 200 feeds that would have been very challenging with Apache Oozie alone," said programmer Michael Miklavcic.

"I began to work on Falcon in my spare time for fun, but it quickly became interesting in relation to my job at Talend", said Jean-Baptise Onofré, Vice President of Apache Karaf and Software Architect at Talend. "As Talend DataIntegration provides features like CDC (Change Data Capture), and data notification, we are in the process of integrating Apache Falcon in Talend products." 

"Apache Falcon's graduation is a milestone for the project and a credit to its contributors. Its open, collaborative development has effected a robust community around software essential to the Hadoop ecosystem," said Chris Douglas, Falcon incubation mentor at the ASF. "By becoming a Top-Level Project, the ASF recognizes its demonstrated ability to self-govern. Congratulations to Falcon's users, to its contributors, and particularly to its new Project Management Committee on this achievement."

Availability and Oversight
As with all Apache products, Apache Falcon software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache Falcon, visit http://falcon.apache.org/ and @ApacheFalcon on Twitter

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow https://twitter.com/TheASF.

© The Apache Software Foundation. "Apache", "Apache Falcon", "Falcon", "Apache Hadoop", "Hadoop", "Apache Oozie", "Oozie", "ApacheCon", and the Apache Falcon logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

Monday Jan 12, 2015

The Apache Software Foundation Announces Apache™ Flink™ as a Top-Level Project

Open Source distributed Big Data system for expressive, declarative, and efficient batch and streaming data processing and analysis

Forest Hill, MD –12 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ Flink™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.

Apache Flink is an Open Source distributed data analysis engine for batch and streaming data. It offers programming APIs in Java and Scala, as well as specialized APIs for graph processing, with more libraries in the making.

"I am very happy that the ASF has become the home for Flink," said Stephan Ewen, Vice President of Apache Flink. "For a community-driven effort, I can think of no better umbrella. It is great to see the project is maturing and many new people are joining the community."

Flink uses a unique combination of streaming/pipelining and batch processing techniques to create a platform that covers and unifies a broad set of batch and streaming data analytics use cases. The project has put significant efforts into making a system that runs reliably and fast in a wide variety of scenarios. For that reason, Flink contained its own type serialization, memory management, and cost-based query optimization components from the early days of the project.

Apache Flink has its roots in the Stratosphere research project that started in 2009 at TU Berlin together with the Berlin and later the European data management communities, including HU Berlin, Hasso Plattner Institute, KTH (Stockholm), ELTE (Budapest), and others. Several Flink committers recently started data Artisans, a Berlin-based startup committed to growing Flink both in code and community as 100% Open Source. More than 70 people have by now contributed to Flink.

"Becoming a Top-Level Project in such short time is a great milestone for Flink and reflects the speed with which the community has been growing," said Kostas Tzoumas, co-founder and CEO of data Artisans. "The community is currently working on some exciting new features that make Flink even more powerful and accessible to a wider audience, and several companies around the world are including Flink in their data infrastructure."

"We use Apache Flink as part of our production data infrastructure," said Ijad Madisch, co-founder and CEO of ResearchGate. "We are happy all around and excited that Flink provides us with the opportunity for even better developer productivity and testability, especially for complex data flows. It’s with good reason that Flink is now a top-level Apache project."

"I have been experimenting with Flink, and we are very excited to hear that Flink is becoming a top-level Apache project," said Anders Arpteg, Analytics Machine Learning Manager at Spotify.

Denis Arnaud, Head of Data Science Development of Travel Intelligence at Amadeus said, "At Amadeus, we continually seek for better improvement in our analytic platform and our experiments with Apache Flink for analytics on our travel data show a lot of potential in the system for our production use."

"Flink was a pleasure to mentor as a new Apache project," said Alan Gates, Apache Flink Incubator champion at the ASF, and architect/co-founder at Hortonworks. "The Flink team learned The Apache Way very quickly. They worked hard at being open in their decision making and including new contributors. Those of us mentoring them just needed to point them in the right direction and then let them get to work."

Availability and Oversight
As with all Apache products, Apache Flink software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache Flink, visit http://flink.apache.org/ and @ApacheFlink on Twitter.

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Cerner, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, iSigma, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter.

© The Apache Software Foundation. "Apache", "Apache Flink", "Flink", ApacheCon", and the Apache Flink logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.

# # #

Friday Dec 19, 2014

ASF publishes long-overdue Code Of Conduct

tl;dr: The ASF has published a Code of Conduct

We pride ourselves at The Apache Software Foundation on our principles of "community over code" and "don't be a jerk". But, alas, we've been slow to codify some of these things in public. Part of this, I'm sure, is that it’s easy to think we all just know how we're supposed to treat people, and so you shouldn't have to say, right?


But, of course, you do have to say. In part because some people don't know. And in part because it’s important that we communicate our values to the people in our community, and to people who might be considering joining our community. There has been a recent push in tech circles to include a Code of Conduct at events, conferences, etc. (Ashe Dryden maintains an introductory resource for learning more about how Codes of Conduct can help.) Increasingly, open source projects are adopting a Code of Conduct too, and we think this is a good idea that could help improve open source as a whole.

At ApacheCon, I was approached by Joan Touzet, an active member of the Apache CouchDB community, who had noted that we referenced a Code of Conduct on the main ASF website, but that no such document actually existed anywhere on our site. CouchDB has devoted a lot of time over the last few months crafting their Code of Conduct. It addresses everything from what's acceptable on the mailing lists, to how to report it if someone isn’t upholding community standards. This seemed like a great starting point, and so the ASF has adopted this as our initial Code of Conduct, with minor edits that remove the CouchDB-specific language. (It is my understanding that the CouchDB community now intends to use the Foundation level Code of Conduct, and will work with us to bring additional improvements to it.) 

No doubt, we'll get criticism for being so slow to do this, and we accept that. But it's never too late to take steps in the right direction, and we feel that this is an important one. Not just for the ASF, but for all open source projects and organisations.

You are encouraged to join the conversation on the Community Development mailing list. Whether you have changes you'd like to see in that document, or whether you'd like to discuss any other aspect of the Apache community. Any sort of community discussion topic is welcome. For example, Noah Slater, also from the CouchDB community, brought up the subject of punitive measures for infractions, which is an important but difficult issue. We'd love to hear your perspective on this, and help us continue to move in the right direction.

--Rich Bowen, Executive Vice President

Tuesday Dec 09, 2014

The Apache Software Foundation Announces Apache™ MetaModel™ as a Top-Level Project

Dynamic, metadata-driven Open Source framework provides uniform data access and code consolidation across various data stores. 

Forest Hill, MD –09 December 2014– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 200 Open Source projects and initiatives, announced today that Apache™ MetaModel™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles. 

"It's a great privilege for us to have MetaModel graduated to a Top Level Project at Apache. It makes us proud and excited about welcoming more people into our community of coders and users," said Kasper Sørensen, Vice President of Apache MetaModel. "We've learned a lot about the Apache Way since entering the Apache Incubator in July 2013." 

Apache MetaModel is a data access framework that provides a common interface for the discovery, exploration, and querying of different types of data sources. Unlike traditional mapping frameworks, MetaModel emphasizes metadata of the data source itself and the ability to add more data sources at runtime. MetaModel's schema model and SQL-like query API is applicable to databases, CSV files, Excel spreadsheets, NoSQL databases, Cloud-based business applications, and even regular Java objects. This level of abstraction makes MetaModel great for dynamic data processing applications, less so for applications modelled strictly around a particular domain. 

MetaModel is so called as it's a model for interacting with data based on metadata, enabling developers to go above the physical data layer and apply their application to just about any data. 

"MetaModel enables you to consolidate code and consolidate data a lot quicker than any other library out there," Sørensen explained. "In these 'Big Data days' there's a lot of focus on performance and scalability, and surely these topics also surround Apache MetaModel. The Big Data challenge is not always about massive loads of data, but instead massive variation and feeding a lot of different sources into a single application. Now to make such an application you both need a lot of connectivity capabilities and a lot of modelling flexibility. Those are the two aspects where Apache MetaModel shines. We make it possible for you to build applications that retain the complexity of your data – even if that complexity may change over time. The trick to achieve this is to model on the metadata and not on your assumptions." 

"The performance and flexibility of Apache MetaModel is a key building block for us to improve the usability and power for the thousands of users of DataCleaner – the leading Open Source data quality solution, supported by Neopost," said Enno Ebels, Executive Vice President of Customer Information Management at Neopost. 

"It's been a joy to follow the growth in the community and in functionality," added Sørensen. "Over the last year we've introduced connectivity for Apache HBase, JSON files, ElasticSearch, Apache Cassandra and a whole lot more. It's always a great pleasure to see the excitement in people's eyes when they realize that you can develop for these data sources using the same API." 

"Apache MetaModel is the core technology used underneath our MDM offering at Human Inference, providing us an abstraction layer above the different database schemes we currently support, including Postgres, DB2, Oracle, SQL Server, and ElasticSearch," said Ankit Kumar, Technical Lead at Human Inference and Member of the Apache MetaModel Project Management Committee.

"The MetaModel query language helps us write code agnostic of the underlying database. Within our MDM offering we have even implemented some virtual data stores using MetaModel," said Winfried van Holland, CTO of Neopost Customer Information Management. "These expose our data model in a custom view for our consultants - stripping away the technical complexities and exposing the business value in a data model that is natural for the business people to consume."

"Apache MetaModel is a key technology in Stratio Datavis, allowing us to manage metadata and create SQL-based connectors for a bunch of data stores," said David Morales, Big Data Architect at Stratio. "Thanks to Apache MetaModel, Datavis users can create beautiful dashboards using their SQL skills, instead of knowing several query languages. That's why we are proud to be contributors of MetaModel and we will continue to collaborate with this great project." 

Availability and Oversight 
As with all Apache products, Apache MetaModel software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache MetaModel, visit http://metamodel.apache.org and https://twitter.com/ApacheMetaModel

About The Apache Software Foundation (ASF) 
Established in 1999, the all-volunteer Foundation oversees more than two hundred leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ or follow @TheASF on Twitter

© The Apache Software Foundation. "Apache," "Apache MetaModel," "MetaModel," ApacheCon," and the Apache MetaModel logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners. 

# # # 

Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation