The Apache Software Foundation Blog

Friday January 29, 2021

The Apache News Round-up: week ending 29 January 2021

Farewell, January --both the week and month have flown by. Let's review what the Apache community has been up to:

The Apache Way to Sustainable Open Source Success  – Apache is for Everyone. Every developer has their personal motivations for building software. We celebrate their right to choose when and how they build their software, including their right to use a non-open license. https://s.apache.org/GhnI

ASF Security Report 2020 – the annual state of security across all Apache projects https://s.apache.org/SecurityReport2020

Inside Infra – the interview series featuring members of the ASF Infrastructure team.
 - Chris Lambertus --Part II https://s.apache.org/InsideInfra-ChrisL2

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - The Apache Software Foundation Operations Summary: Q2 FY2021 (August - October 2020) https://s.apache.org/Q2FY2021
 - Next Board Meeting: 17 February 2021. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - videos from ApacheCon@Home presentations are available at https://www.youtube.com/c/TheApacheFoundation/  

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.90%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 368 Apache Committers changed 2,919,651 lines of code over 3,273 commits. Top 5 contributors, in order, are: Mark Thomas, Leonid Frolov, Andrea Cosentino, Andi Huber, and Christofer Dutz.  

Apache Project Announcements – the latest updates by category.

Big Data --
 - Apache Arrow 3.0.0 released https://arrow.apache.org/
 - Apache Hadoop CVE-2020-9492: Potential privilege escalation https://s.apache.org/d9h7j

IoT --
 - Apache IoTDB 0.11.2 released https://iotdb.apache.org/

Messaging --
 - Apache ActiveMQ CVE-2021-26117: LDAP-Authentication does not verify passwords on servers with anonymous bind https://s.apache.org/xvpov , and
   CVE-2021-26118: Flaw in ActiveMQ Artemis OpenWire support https://s.apache.org/bpp38

Libraries --
 - The Apache Software Foundation Announces Apache® ECharts™ as a Top-Level Project https://s.apache.org/txmmr
 - Apache Commons Daemon 1.2.4 released https://commons.apache.org/proper/commons-daemon/

Servers --
 - Apache Traffic Server 9.0.0 released https://trafficserver.apache.org/

Testing --
 - Apache JMeter 5.4.1 released https://jmeter.apache.org/

Web Crawler --
 - Apache Nutch 1.18 released https://nutch.apache.org/
 - Apache Nutch CVE-2021-23901: An XML external entity (XXE) injection vulnerability exists in the Nutch DmozParser https://s.apache.org/y0pir


Did You Know?

- Did you know that the Apache Kafka PMC has published a trademark disclaimer for naming non-java clients and connectors to help those building the Apache Kafka ecosystem? https://kafka.apache.org/trademark

- Did you know that video presentations from the 2020 Virtual Druid Summit are available online? http://ow.ly/HLQq50Df7rI

- Did you know that the 2021 Joint Apache Software Foundation – Open Geospatial Consortium – Open Source Geospatial Foundation Code Sprint will be taking place online and free-of-charge 17-19 February? All are welcome to participate https://s.apache.org/ilzbf


Apache Community Notices

- Apache in 2020 - By The Digits https://s.apache.org/Apache2020Digits + Video highlights https://s.apache.org/Apache2020Digits-vid

- The Apache Software Foundation Operations Summary: 1 August - 31 October 2020 https://s.apache.org/Q2FY2021

- Apache Month In Review: December 2020 https://s.apache.org/Dec2020 

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet 
    Chris Thistlethwaite https://s.apache.org/InsideInfra-Chris
    Drew Foulks https://s.apache.org/InsideInfra-Drew
    Greg Stein Part I https://s.apache.org/InsideInfra-Greg
      ...Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3
    Daniel Gruno Part I https://s.apache.org/InsideInfra-Daniel1 and Part II https://s.apache.org/InsideInfra-Daniel2
    Gavin McDonald Part I https://s.apache.org/InsideInfra-Gavin and Part II https://s.apache.org/InsideInfra-Gavin2
    Andrew Wetmore Part I https://s.apache.org/InsideInfra-Andrew and Part II https://s.apache.org/InsideInfra-Andrew2
    Chris Lambertus Part I https://s.apache.org/InsideInfra-ChrisL and Part II https://s.apache.org/InsideInfra-ChrisL2

 - ASF Targeted Sponsor Manning Publications is offering special deals on the latest books on Apache Airflow, Pulsar, Spark, and Thrift, among other titles and eBooks https://deals.manning.com/the-latest-apache-innovations/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Tuesday January 26, 2021

The Apache Software Foundation Announces Apache® ECharts™ as a Top-Level Project

Adaptable, interactive, responsive Open Source charting and data visualization software in use at Alibaba, Amazon, Baidu, GitLab, Intel, and Tencent, among others.


Wilmington, DE —26 January 2021— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® ECharts™ as a Top-Level Project (TLP).

Apache ECharts is an intuitive, interactive, and powerful charting and visualization library ideally suited for commercial-grade presentations. The project originated in 2013 at Baidu and entered the Apache Incubator in January 2018.

"Our decision to incubate ECharts at The Apache Software Foundation was a wise one," said Ovilia Zhang, Vice President of Apache ECharts. "Through the Apache Way, our community is healthier and more diverse, which has improved ECharts to become a more attractive, competitive choice for visualization professionals and enthusiasts."

Written in JavaScript and based on the ZRender rendering engine supporting both Canvas and SVG, Apache ECharts provides an array of dynamic, highly-customizable chart types that include line, column, scatter, pie, radar, candlestick, gauge, funnel, heatmap, and more. Features include:

  • Customized and amalgamated chart styles with more than 20 chart types

  • Multi-dimensional data analysis and coding

  • Interactive components available out-of-the-box

  • Cross-device responsiveness

  • Optimized dynamic scaling

  • Server side rendering

  • Immediate UI response on millions of streaming data through progressive rendering

  • Extensions for:

    • 3-D visualization and other rich special effects

    • Python, R, Julia, and other languages

    • Platforms that include Wechat App and Baidu Smart Program


Examples of ECharts' many data visualization options are available at https://echarts.apache.org/examples/ 

The project has recently released ECharts 5, which provides rendering ability for tens of millions of data points, and supports accessibility requirements in compliance with W3C’s Web Accessibility Initiative Accessible Rich Internet Applications Suite (WAI-ARIA) standards.


Building on EChart’s core features, ECharts 5 makes it even easier for developers to tell the story behind the data through 15 new features and improvements in story-telling and data expression, optimized visualization and responsive design, interaction and performance enhancement, developer experience, internationalization, and more.


Apache ECharts is in use at Alibaba, Amazon, Baidu, GitLab, Intel, and Tencent, among others, as well as solutions such as Apache Superset data visualization software. The project continues to grow in popularity, with more than 44,000 stars on GitHub and 25,000 weekly downloads on npm to date. 


"The world we live in today is powered by software and data," said Erica Brescia, COO of GitHub. "With Apache ECharts, developers around the world have access to a powerful, free and open source library for data visualization. It is great to see the project flourishing on GitHub. Congrats to the Apache ECharts on their graduation to a top level project at the Apache Software Foundation."


"Apache ECharts helps visualization experts and data analysts easily create a wide variety of visualizations that are very helpful for us to analyze and explore the story behind the data," said visualization academia pioneer Professor Wei Chen of Zhejiang University.


"We are glad to witness ECharts’ pleasant process in the Apache Incubator," said Ming Zu, Senior Manager at Baidu. "Our community grew with individuals from many countries and organizations, who contributed to bug fixing, issue resolving, and new feature implementation."


"When the Apache Superset community looked into visualization libraries to rebuild the core visualization plugins, ECharts stood out as the absolute best fit," said Maxime Beauchemin, original creator of both Apache Airflow and Superset, and serves as Vice President of Apache Superset. "It has an unparalleled variety of visualizations, a rich and composable visual grammar, an intuitive and well designed API, a flexible and performant rendering engine, a very lean tree of dependencies, and the important set of guarantees that the ASF provides when committing long term to using an Open Source project."


"It was a pleasure guiding the ECharts community through the Apache Incubator," said Dave Fisher, ASF Member and Apache ECharts Incubating Mentor. "They have embraced the Apache Way of community-led development, encouraging those interested in helping improve ECharts to contribute and become part of its growing community.”


"This is an exciting time for the ECharts community," added Zhang. "We are enjoying continued growth, and invite those interested in contributing to the project to join us on our developer and user lists."


See the range of options available with ECharts in "Apache ECharts in 5 minutes", a new video created by members of the Apache ECharts community (in Mandarin Chinese with English subtitles) https://youtu.be/nKKK0orjSq8 


Availability and Oversight

Apache ECharts software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache ECharts, visit http://echarts.apache.org and https://twitter.com/ApacheECharts


About the Apache Incubator

The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects enter the ASF through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/ 


About The Apache Software Foundation (ASF)

Established in 1999, The Apache Software Foundation (ASF) is the world’s largest Open Source foundation, stewarding 227M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 813 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with nearly 8,000 Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Budget Direct, Capital One, Cloudera, Comcast, Didi Chuxing, Facebook, Google, Handshake, Huawei, IBM, Microsoft, Pineapple Fund, Red Hat, Reprise Software, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF 


© The Apache Software Foundation. "Apache", "ECharts", "Apache ECharts", "Airflow", "Apache Airflow", "Superset", "Apache Superset", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.


# # #

Monday January 25, 2021

Apache Software Foundation Security Report: 2020

Synopsis: This report explores the state of security across all Apache Software Foundation projects for the calendar year 2020. We review key metrics, specific vulnerabilities, and the most common ways users of ASF projects were affected by security issues.


Released: January 2021


Author: Mark Cox, Vice President Security, Apache Software Foundation

Background

The security committee of the Apache Software Foundation (ASF) oversees and coordinates the handling of vulnerabilities across all of the 340+ Apache projects.  Established in 2002 and composed of all volunteers, we have a consistent process for how issues are handled, and this process includes how our projects must disclose security issues.


Anyone finding security issues in any Apache project can report them to security@apache.org where they are recorded and passed on to the relevant dedicated security teams or private project management committees (PMC) to handle.  The security committee monitors all the issues reported across all the addresses and keeps track of the issues throughout the vulnerability lifecycle.


The security committee is responsible for ensuring that issues are dealt with properly and will actively remind projects of their outstanding issues and responsibilities.  As a board committee, we have the ability to take action including blocking their future releases or, worst case, archiving a project if such projects are unresponsive to handling their security issues.  This, along with the Apache Software License, are key parts of the ASF’s general oversight function around official releases, allowing the ASF to protect individual developers and giving users confidence to deploy and rely on ASF software.


The oversight into all security reports, along with tools we have developed, gives us the ability to easily create metrics on the issues.  Our last report covered the metrics for 2019.

Statistics for 2020

In 2020 our security email addresses received in total 18,000 emails. After spam filtering and thread grouping this was 946 (2019: 620) non-spam threads.  Unfortunately many security reports do look like spam and so the security team are careful to review all messages to ensure real reports are not missed for too long.

Diagram 1: Breakdown of ASF security email threads for calendar year 2020


Diagram 1 gives the breakdown of those 946 threads.  257 threads (27%) were people confused by the Apache License.  As many projects use the Apache License, not just those under the ASF umbrella, people can get confused when they see the Apache License and they don't understand what it is.  This is most common for example on mobile phones where the licenses are displayed in the settings menu, usually due to the inclusion of software by Google released under the Apache License.  We no longer reply to these emails. This is nearly double the number we saw in 2019.


The next 220 of the 946 (23%) are email threads with people asking non-security (usually support-type) questions.


The next 93 of those reports were researchers reporting issues in an Apache web site.  These are almost always false negatives; where a researcher reports us having directory listings enabled, source code visible, or the lack of various domain headers.  These reports are generally the unfiltered output of some publicly available scanning tool, and often where the reporter asks us for some sort of monetary reward (bounty) for their report.


That left 376 (2019: 320) reports of new vulnerabilities in 2020, which spanned across 101 of the top level projects.  These 376 reports are a mix of both external reporters and internal; for example where a project has found an issue themselves and followed the ASF process to assign it a CVE name and address it we’d still count it here.  We don’t keep metrics that would give the breakdown of internal vs external reports.


The next step is that the appropriate project triages the report to see if it's really an issue or not.  Invalid reports and reports of things that are not actually vulnerabilities get rejected back to the reporter.  Of the remaining issues that are accepted they are assigned appropriate CVE names and eventually fixes are released.


As of January 1st 2021, 35 of those 376 reports were still under triage (i.e. the project had not yet determined if the report is accepted or rejected).  


The remaining closed 341 (2019: 301) reports led to us assigning 151 (2019: 122) CVE names.  Some vulnerability reports may include multiple issues, some reports are across multiple projects, and some reports are duplicates where the same issue is found by different reporters, so there isn't an exact one-to-one mapping of accepted reports to CVE names.  The Apache Security committee handles CVE name allocation and is a Mitre Candidate Naming Authority (CNA), so all requests for CVE names in any ASF project are routed through us, even if the reporter is unaware and contacts Mitre directly or goes public with an issue before contacting us.

Noteworthy events

During 2020 there were a few events worth discussion; either because they were severe and high risk, they had readily available exploits, or otherwise due to media attention. These included:

  • February: An issue in Tomcat CVE-2020-1938 gained press interest when it was given branding and a name (“Ghostcat”) and was disclosed by a third-party coordination centre before Tomcat released an advisory (although after the issue was fixed in new releases of Tomcat). Although serious if exploited, it only affected Tomcat installations which exposed an unprotected AJP Connector to untrusted networks (which is already not a good thing to do even without this issue). That limits the number of affected installations.  Various proof-of-concept exploits are public for this issue, including a Metasploit exploit.

  • July: Versions of Apache Guacamole 1.1.0 and earlier were vulnerable to issues in RDP, CVE-2020-9497 and CVE-2020-9498.  If a user connects to a malicious or compromised RDP server it could lead to memory disclosure and possible remote code execution. 

  • August: A vulnerability in Apache Struts (CVE-2019-0230) could lead to arbitrary code execution. In order to exploit the vulnerability, an attacker would need to inject malicious Object-Graph Navigation Language (OGNL) expressions into an attribute that is used within an OGNL expression. Although Struts has mitigations to address potential injected expressions, versions before 2.5.22 left an attack vector open which was fixed in updates for this issue.  A metasploit exploit exists for this issue.

  • November: Previously each ASF project was responsible for writing up their own CVE entries and submitting them to Mitre. This leads to many delays in the CVE database being updated with Apache issues as entries are often rejected as the legacy format causes issues. We released an internal tool providing projects dealing with security issues a way to edit, validate, and submit their entries to Mitre.  We aim to have the CVE database updated within a day of an issue being published.

  • December: The CVE project released a new automation API and the ASF became the first organisation to get a live CVE name using it. Instead of the security team holding a pool of names requested in advance we now allocate them on demand, with the service taking care of emails to the PMC and other previously manual parts of the process. We expect more automation available during 2021 allowing us to streamline the CVE process for projects even further.

Timescales

Our security teams and project management teams are all volunteers and so we do not give any formal SLA on handling of issues.  However we can break down our aims and goals for each part of the process:


Triage: Our aim is to handle incoming mails to the security@apache.org alias within three working days.  We do not measure or report on this because we assess the severity of each incoming issue and apply the limited resources we have appropriately.  The alias is staffed by a very small number of volunteers taken from the different project PMCs.  After the security team forward a report to a PMC they will reply to the reporter.  Therefore if you have reported an issue to us and not received any response after a week please send us a followup email.  Sometimes reporters send reports attaching large PDF files or even movies of exploitation that don’t make it to us, so please ensure any follow ups are a simple plain text email.


Investigation: Once a report is sent to the private list of the projects management committee, the process of triage and investigation varies in time depending on the project, availability of resources, and number of issues to be assessed.  As we send reports to this private list it does not reach every project committer, so there is a much smaller limited set of people in each project able to investigate and respond.  As a general guideline we try to ensure projects have triaged issues within 90 days of the report.  The ASF security team chase any untriaged issues over 90 days old.


Fix: Once a security issue is triaged and accepted, the timeline for the fixing of issues depends on the schedules of the projects themselves.  Issues of lower severity are most often held to future pre-planned releases.  


Announcement: Our process allows projects up to a few days between a fix release being pushed and the announcement of the vulnerability, to let mirrors catch up.  All vulnerabilities are announced via the announce@apache.org list.  We now aim to have them appear in the public Mitre list within a day of the announcement.

Conclusion

Apache Software Foundation projects are highly diverse and independent.  They have different languages, communities, management, and security models.  However one of the things every project has in common is a consistent process for how reported security issues are handled. The ASF Security Committee works closely with the project teams, communities, and reporters to ensure that issues get handled quickly and correctly.  This responsible oversight is a principle of The Apache Way and helps ensure Apache software is stable and can be trusted.


This report gave metrics for calendar year 2020 showing from the 18,000 emails received we triaged over 370 vulnerability reports relating to ASF projects, leading to fixing 151 (CVE) issues. The number of non-spam threads dealt with was up 53% from 2019 with the number of actual vulnerability reports up 13% and assigned CVE up 24%.


If you have vulnerability information you would like to share with or comments on this report please contact us.


# # #

Sunday January 24, 2021

Inside Infra: Chris Lambertus --Part II

Part II of the of the "Inside Infra" interview with Chris Lambertus, the last of the series of interviews with members of the ASF Infrastructure team, who share their experiences with Sally Khudairi, ASF VP Marketing & Publicity.


"...you want to limit your exposure... You have to keep that in mind as you move through the day to make sure that you are minimizing your risk and minimizing your security threat vectors."


So, in the scope of the team, I understand that you're a more "senior" developer. Not that you know better; it's not an issue of better or worse, but you're more seasoned. How does ASF compare to other groups that you've worked with? Are there special technical requirements or special security issues you have to be concerned with? Especially as we mentioned before, it seems like there's an unlimited number of project development environments. Are there certain things that you have to consider or accommodate or do that's so different with ASF that you've never experienced before? Can you give a little bit of a frame of reference for folks unfamiliar with how it is within the ASF?


First of all, I'm not a developer. I am terrible at programming. Absolutely, I'm awful at it. I don't consider myself a developer in any way, shape, or form. I am a system administrator, 100%.


...Administrator. Okay, so, you're a more "senior" sysadmin then.


I hesitate to use the word senior, because it has some implications in the industry that I don't necessarily feel are appropriate for the ASF. I believe that I have been doing it longer than most other people on the team just as a career. I'm guessing that's probably what you mean by that.


Right. That's why I used the word "seasoned" also. It's hard because some people go, "Are you saying I'm old, or are you saying hierarchical, that I'm above others?" It's a hard way of describing it, because some folks have been programming or dealing with computers since there were kids, others later in life, but you guys are all moving in the same direction. So, how does one describe it?


Yeah, I think seasoned is a good word. Just like I said, I've been working in the industry as a system administrator since 1992, pretty much continuously with some brief changes in the 2000s. It's not here nor there. So, it's not hierarchical. Everybody is equivalent in terms of the Infra team. Nobody's above anybody else or below anybody else, right?


...I was wondering how is the ASF different from other groups you've worked with.


All right. It's actually not all that different. There are a couple of things that make it unique. Well, a number of things that make it unique. One is that it's completely remote and completely geographically dispersed. Two is that the participants on the team are all from very different backgrounds and cultures and countries, which is fairly unusual for a system admin team, a small system admin team, I would say. But beyond that, it actually shares quite a lot of things that I typically see in system administration teams. There's a central job board, if you will, like the Jira stuff. There's a communications channel. We have Slack.


There's a nominal leader in Greg, that directs the general movement of the barge. Yeah, by and large, it's pretty similar with most environments that I've worked in. I mean, some are much different. Some are very corporate, some are very open. Yeah, now I remember one of your previous questions --one of the biggest challenges that I found is the openness.


The ASF for quite some time has been incredibly public with its configurations, with its systems, with its documentation. These types of things are very unusual in the corporate world or in commercial IT. Typically, you would never make that stuff public. The fact that it is and has been at the ASF, that's been a challenge for me. It's an unusual way to maintain systems. It's got some downsides. Having that stuff available can be concerning at times.


...How so? Help me understand this, because I've been with the ASF forever. What you're mentioning right now reminds me of about 10 years ago, something failed in Infrastructure. I can't remember what it was, but it was a big thing. People were talking about it. It was even in the press at the time. It wasn't catastrophic, but it was big. We actually wrote a blog post about it and we presented about it at ApacheCon. From a marketing perspective and a media perspective, I was uncomfortable, because from a corporate perspective, you don't do that. The fact that we not only encouraged it but published it and educated everyone about it, admitted it, ate it all, we took responsibility, 100%: "Here's what failed. Here's what happened. Here's what we did." People found this to be extremely refreshing, extremely helpful, and it was totally eye opening for me. I had no concept of anything like that before, and I'd been with the ASF for like 10 years already. I've never seen us opening the kimono at that capacity. So, I'm coming at it from a slightly different perspective as you. I understand you don't want to have your config files public. Obviously, that can put you at a different level of exposure and risk.


Exactly.


...Is that required, or is that just part of our culture saying, "This is what we do"?


It's definitely part of the culture. My background is heavily in computer security. Coming on board to the ASF and seeing all this stuff out in the open to me was... I couldn't believe my eyes. "You're doing what?" So, I've actually worked quite a lot to reel that into some extent, because even 10 years ago was nothing like what's happening today in the world of computer security, in the terms of the threats, in terms of what people are looking for, what people are doing, and what people are capable of doing, right? Even to benevolent organizations like ASF, it's distressing.


So, one of the things that I've really tried to encourage is it's okay to be open to some extent, but you have to have some common sense about your security exposure. That's what I've been trying to do just for the entire time that I've been here is just to try and reel some of that in without losing the culture, because I think the culture is valuable. Like you said, the incident that happened whenever that was, I think it was a right decision for the time. Would you do that today? Probably not.


It's not because you wanted to cover something up, but it's because you want to limit your exposure. Yeah, so it's a different culture now, not the ASF, but the world in general. You have to keep that in mind as you move through the day to make sure that you are minimizing your risk and minimizing your security threat vectors.


All right. Have you had instances where a project has basically treated you as their dedicated resource? Has anyone made unusual demands of the team? I’m not asking you to name names, but I can imagine it can get out of hand with all these different projects, especially the corporate ones.


Absolutely. Yeah, the corporate ones are typically the biggest problems, because they come in with a much different mindset than somebody who's come in from developing an Open Source package and has brought it to the ASF. The corporate projects that we've seen really are the ones who are the purveyors of that mentality. They feel Infra is their personal resource, because they don't really have an understanding of the scope of the Foundation. They don't have an understanding of the amount of projects that Infra supports. So, I don't really fault them for that, because it's just a matter of education. They just need to understand where they are placed in terms of the Foundation, in terms of Infra's availability and scalability.


Once we've explained that to people, they get it. We typically don't have any problems after that. But there are a few projects that have come in and just persisted in wanting weird stuff. Some of the things you can provide. Some of the things, you just got to kick back and say, "Hey, this is not something..." Like I mentioned earlier, if it doesn't have a broad benefit to the Foundation, if it's something really specific to your project. Infra is probably not going to support that for you, because we can't support all these one-offs.


So, we'll say, "We'll give you a VM. You can do it yourself." That's worked out pretty well, but there’ve been a few cases even where people like Greg and David have had to go and talk to these projects and say, "Look, how you're approaching this is not appropriate. You need to pull it back. You need to rein it in." But that's really pretty uncommon. I would say just a basic education as people come through the Incubator is sufficient to dispel most of that.


Those kinds of projects... Do they stand down or they wind up hiring their own committers to do their Infra work? Do you have any idea as to how that works? I'm seeing more projects coming in with more diversity in their committership to take care of marketing stuff, for example. That's expected especially as they scale, but from the site administration side of things, Website stuff, it's a very interesting thing to observe. Some project sites’ information is stagnant ... they're focused on specifically developing code. Others are super productive in terms of getting stuff done. I'm always wondering how are they able to handle all this? Curious to see if you had ideas as to what's going on there ...


I will say this, documentation is hard, right? Writing code is comparatively easy, and it's a lot more fun. So, when you're developing a product, your natural instinct is to develop the product, not develop the documentation. So, you get a project that's only got a couple of active members. They're probably not going to spend most of their time writing documentation. They're going to spend most of their time trying to advance the code base. Even within Infra, that's been a huge challenge for us.


Now that we've hired Andrew (ASF Infra team member and technical writer Andrew Wetmore) to help us work on some of this documentation, it's becoming extremely clear as we work through it how much of that documentation has been untouched. It's been stale, for all the same reasons as these projects. Yeah. Some projects will say, "Hey, we need a documentation guy. That's what Infra said, we need a documentation guy." They'll find one. Maybe somebody will volunteer or maybe it's a corporate thing, whatever. So, yeah, I think it really depends on the project. Some people have the resources. Some projects have the resources, and some don't.


Yeah, it's interesting. Again, since day one, since the '90s, documentation has always been an issue for all projects, even when we started with just HTTPd. It's a constant issue. 


If I was going to have money to do anything in a project, I would use it on documentation.


Documentation is often the thing we need the most. I mean, how is it going to work otherwise?


Yeah, I agree. Even from just a cognitive aspect, writing code and writing documentation are about polar opposites. The type of mind that goes and writes code isn't usually the type of mind that can write documentation or can write meaningful documentation. I'm guilty of it myself. I can't write documentation, I find it quite difficult. Where building packages and tying things together, and Puppet configuration management, is not difficult for me. So, it's a huge mind split between those two types of things. I absolutely agree that hiring somebody to do documentation is a great use of resources.


We've grown a lot during the time you've been with us, now six plus years. Other than scale, how has Infra changed over the years? What's unusual is that the team is getting smaller. I would presume as the Foundation is scaling upwards, you would have more team members. It's some crazy number: five people, six people, it's so small. It’s hard to understand how you guys handle everything.


Yes, six people, including Andrew and then Greg, right?


Including Andrew, that’s six, but Andrew doesn't handle the day-to-day Jira stuff anyway. He doesn't handle tickets. So, you really are a tiny group. From your perspective and your experience, would you say that that's a small group, considering the workload and the demand?


Yeah, I would say so. Probably based on my experience in other organizations, about half the size that it would be in a commercial environment. Well, to go to your original question there, in terms of what's changed, I think prior to David Nalley, I would say that Infra was extremely reactive. I think that's changed quite a lot. I think David has really brought an element of customer service and customer focus to the team that really had been somewhat lacking in the past.


So that was a proactive decision to go in and say, "We have to better serve our projects," right?


Yeah. I really do credit David with that. I think he brought a huge amount of that to the team and that mindset. It's really improved our relationships, Infra's relationships, with the projects. It's helped us develop tooling like Self-Service.The more that we can move off into those projects, do-it-yourself tooling, the better off we are, because it's less tickets that we have to handle. It's a constant juggle for us between dealing with legacy code, dealing with technical debt from years and years and years and years ago to doing modern things to bring out new tools, and all the while supporting projects.


In what areas are you guys experiencing bursts of growth or demand? Everyone has a slightly different perspective. I know CI comes up a lot in this arena. Greg's always saying (since I deal with ASF’s Sponsors), "We need more." Where do you feel Infra's growing at the highest rate or the most interesting rate? Where do you feel like that's happening?


Yeah, continuous integration was the first thing that came to mind when you said that. The more projects we have, the more need there is for CI. That's fairly linear. Other growth places are things like Infra VMs, machines that we run to support Infra services internally. Prior to the resources that we have now, we used to have a lot of monolithic systems, systems that would run a lot of things. Think of a machine like Minotaur, which used to run two dozen services on one machine. That's not a best practice at all.


Moving to aggressive use of configuration management Puppet, and making sure that systems are easily replicable with the configuration management, has allowed us to really build -- not quite micro services, but single purpose systems, which are a lot easier to maintain, a lot easier to scale than some of those monolithic systems. So, that's been a big growth area for us. Just the number of VMs, number of systems that we're maintaining, it's got to be in the hundreds at this point. I haven't counted. Yeah.


...These microservices that you're mentioning also reduce the single point of failure, which is critical. That keeps you guys scalable and keeps you up and running. That's important.


Yeah, that's right.


I'm curious when was the last time you guys had a fire drill type of thing, where everyone's hands on. You had something recently, right? A couple months ago, all hands on deck, there was something broken. You guys were able to resolve it pretty quickly, but that's uncommon, where something breaks in its entirety.


I don't want to say anything about this, because it's going to cause a problem.


...We can go off the record.


What I mean is I'm going to say it's fine, right? And then something's going to break.


...[laughing] You don't want to jinx it. Okay.


We have failures from time to time. We've had some situations where there's been a problem at a colo. One of our VM providers had an issue and we lost machines. We had to rebuild them with Puppet, our configuration management, and restore stuff from backup. It sucked, but it wasn't a disaster, right? Because we have the backups. We have the capacity. We have the configuration management. So, nobody had to wrack their brains: “How did this work? How did this go together?” We’ve made very, very big strides in avoiding that old mindset of ‘one guy set this up 10 years ago and nobody else knows how it works.’ We're very much trying to avoid that these days.


...Right, bus factor.


Yeah, yeah, yeah. The configuration management systems have been absolutely critical with that. So, that continues to grow. We continue to add to configuration management wherever possible and just make sure that those systems are able to be reconstituted wherever, whenever it's needed.


Cool, cool. Okay. What do you think people would be surprised to know about ASF Infra?


The other guys probably said the same thing, but probably the amount of stuff that we support from the number of people we have. I think that would probably surprise most people in the industry.


That's one answer. I think it was (Infra team member) Chris Thistlethwaite who said "that we exist", that you guys exist. People don't know how it happens. It's like magic. I've always talked about how Infra is this crazy-magic-impossible story. It's like The Little Engine That Could, because you guys are such a tiny group. You have such a good working relationship, and everyone is connected. From the outside, it seems like a completely seamless operation. There's this magic thing behind the scenes, and then you find it's only five, six people running it. That's mind blowing. It's incredible.


I hope that people have that perception. We do try to provide a unified front. In reality, there's not really any infighting in the team. We all generally know what needs to be done. We all generally agree on how to do it. So, the disagreements are fairly minor and not all that common.


Well, that in itself is unusual, right? Think about it. I mean, there's a lot of factions and politics and weirdness, but that tends to happen with larger groups. So, you guys make it work in a way that's awesome.


I think one of the things that makes that the way it is, is because we're all supporting the ASF, right? We're all here, because we support the Foundation, and we want the Foundation to succeed. So, that drives, I think, a lot of the direction and the way that we approach how we support the Foundation.


You guys have a very different common goal, right? You're there for the benefit of the Foundation with a capital F; Projects are there to work on their own thing. Of course, if they can help everybody else, that's good, too. But the focus is different. 

...What is your favorite part of the job?


I have to say, the flexibility and the remote aspect of it, along with the constantly changing technology. There are a lot of opportunities to learn new things, and work on new technologies. 


...You are all on call for certain periods throughout the week, right? So, because of your 7:00 to 11:00, are you ever on call overnight, or does that just not work out with schedules, or it doesn't matter?


Well, we rotate on call. So, you're on call for a week at a time, starting at, I think, 10:30 or 11:00 Pacific AM and then going through the following week. So, typically, what happens is you'll get the pages when you're on call, regardless of the time of day or night. But the way that it works out, typically, because we have folks in Europe, we have folks in the US, we have folks on the West Coast and the East Coast, that almost always there'll be somebody awake and available to answer.


Sometimes in the middle of the night, if my pager goes off at 2:00 in the morning, I'll look at my phone and I'll see that Humbedooh or Gavin is already working on it. Thanks, guys. Obviously, the same is reciprocated, right? If the phone goes off in the middle of their night and I see that they're on call but it's 3:00 in the morning, I'll grab a ticket if I can, I'll grab the call if I can. We just try to help each other out that way.


You guys are a true team: you have each other's backs in a way that again, is unusual to see. It's almost like family but even better, because even family has infighting and issues. You are there for each other, which is really, really cool to see.


Yeah, let's say we've had our disagreements, but it is a very familial atmosphere.


When you first came into the role, what was your biggest challenge? Was it what you thought it was? How was your experience?


It was an incredibly steep learning curve. When I first started here, we were in the middle of the transition from the "one guy who set up everything, a volunteer five years ago, nobody knows how it works" environment to a configuration management. We were  just starting to get into that, and shore up some of our documentation at the time. For me, just coming in and learning all the different systems and all the different processes and all the different edge cases and one-offs and locations for things and who's who and all these, that was incredibly difficult. It took me probably at least a couple of years before I felt comfortable with most of the systems.


Even today, there's stuff out there where I'll be like: "I'm not sure what this means. Do you have any idea what's going on?" Because there's so many little pockets and holes and places and things and historical legacy stuff. It's very complicated. It's been organically grown over a long, long time.


...With a lot of different personalities and a lot of different processes, that is what's unusual. The "quilt" that makes Apache is so diverse.


It is.


What are you most proud of with your career with Infra so far?


I'm not really sure, to be honest. I don't tend to think of things like that. I can't really single out one thing and say, "Hey, I'm really particularly proud of that," or whatever. I try and take pride with all my work. Building better backup systems, I think, is definitely a big one. Just getting through some of this mail project has been good as well. When I finally got everything working, that was a pretty proud moment there. I felt pretty good about that. That was a complicated system. It's still a complicated system. I'm still not sure it all works right. That's why we have to test it. By and large, I'm feeling pretty good about it.


That's great. How would your coworkers describe you?


[laughs] Grumpy.


...[laughs] The response is the same with everyone. Everyone laughs, but grumpy is the first one I've ever heard.


I don't really talk too much. I'm not a super verbal person. So, I always seem to come across as grumpy on the chat systems there. It's a schtick, I guess, but it is fun. I'm not really grumpy. Well, most of the time.


What are the biggest threats or concerns that sysadmins need to watch out for? I don’t mean doom-and-gloom unless there’s actually doom-and-gloom ...A lot of non-Apache folks are curious what the Apache guys think. So, is there anything that you could share in terms of advice or trends that are coming up or something that people should be aware of moving forward?


Security, backups, disaster recovery, those are the keystones of any organization that you absolutely must have in place to sleep at night. If you don't have any one of those three, you're in grave danger of doom-and-gloom.


That makes sense. What is your greatest piece of advice for someone looking to have a job like yours?


Oh, boy. Run for the hills [laughing]. Work with as many different things as you can, learn as many different things as you can, and try not to get stuck doing one specific thing. I think in my career I've been such a jack of all trades that it's really helped me to be able to see and build systems that work with a lot of different technologies. You get some people coming in, they're IBM guys, like a specific subset of IBM AIX expertise or something, right? That's all they do. And then when the situation comes around, well, nobody's really using that anymore, you run into a problem, because you're not really marketable anymore. So, the advice that I would give anybody who's trying to get into the system administration field, be broad and learn as much as you can about as many different things as you can.


If you had a magic wand, what would you see happen with ASF Infra?


I think I'd probably just give us more resources. I mean, I don't really have any complaints, to be honest. I think if we had more, then we would do more.


...More ...machines or more cash or more team or more what?


All of those ...I think more cash. Being able to buy more physical compute resources would go a long way for us. We do rely so much on donations and donated resources that it can be a little bit daunting when that donation goes away and you have to scramble to fill the void. Staffing is a complicated one, because it is familial.


Having somebody new come on board, it's challenging. It's nice to have an additional person be able to work on stuff, but going through the process of integrating them into the team and teaching everything else, it's daunting, it's challenging. So, I think having more resources would be more important at least to me than having more staff, because I think we're doing all right with the staff that we have now. So, that's just my perspective.


= = =


Chris is based in California on UTC -8. His favorite thing to drink during the workday is ice water and the occasional Diet Pepsi.

Friday January 22, 2021

The Apache News Round-up: week ending 22 January 2021

Happy Friday! Let's take a look at what the Apache community has been up to over the past week:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - The Apache Software Foundation Operations Summary: Q2 FY2021 (August - October 2020) https://s.apache.org/Q2FY2021
 - Next Board Meeting: 17 February 2021. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - all videos from ApacheCon@Home are available at https://www.youtube.com/c/TheApacheFoundation/  

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 100.00%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 385 Apache Committers changed 3,309,050 lines of code over 5,192 commits. Top 5 contributors, in order, are: Rohit Yadav, Wei Zhou, Kaxil Naik, Gary Gregory, and Andrea Cosentino.

Apache Project Announcements – the latest updates by category.

Big Data --
 - Apache Flink 1.12.1 released https://flink.apache.org/
 - Apache Qpid Broker J 7.1.11 and J 8.0.3 released https://qpid.apache.org/
 - The Apache Software Foundation Announces Apache® Superset™ as a Top-Level Project https://s.apache.org/scefo

Cloud Computing --
 - The Apache CloudStack Project Releases Apache® CloudStack® v4.15 https://s.apache.org/vi0v8

Content --
 - Apache Jackrabbit Oak 1.22.6 released http://jackrabbit.apache.org/
 - Apache Tika 2.0.0-ALPHA released https://tika.apache.org/

Integration --
 - Apache Camel 3.7.1 released https://camel.apache.org/

Network Client --
 - Apache Guacamole CVE-2020-11997: Inconsistent restriction of connection history visibility https://s.apache.org/i80o1

Servers --
 - Apache Tomcat CVE-2020-17527: Apache Tomcat HTTP/2 Request header mix-up https://s.apache.org/wqss6


Did You Know?

- Did you know that the Apache Maven projects has action cards for their community to promote their activities on social media? https://maven.apache.org/resource/branding/actioncards.html

- Did you know that US Top 10 retailer Target's enterprise-scale analytics (delivered to all levels of the organization) is powered by Apache Druid? http://druid.apache.org/

- Did you know that K&H Bank, one of the largest commercial banks in Hungary, uses Apache Wicket for their consumer banking and insurance site? http://wicket.apache.org/ 


Apache Community Notices

- Apache in 2020 - By The Digits https://s.apache.org/Apache2020Digits + Video highlights https://s.apache.org/Apache2020Digits-vid

- The Apache Software Foundation Operations Summary: 1 August - 31 October 2020 https://s.apache.org/Q2FY2021

- Apache Month In Review: December 2020 https://s.apache.org/Dec2020 

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet 
    Chris Thistlethwaite https://s.apache.org/InsideInfra-Chris
    Drew Foulks https://s.apache.org/InsideInfra-Drew
    Greg Stein Part I https://s.apache.org/InsideInfra-Greg
      ...Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3
    Daniel Gruno Part I https://s.apache.org/InsideInfra-Daniel1 and Part II https://s.apache.org/InsideInfra-Daniel2
    Gavin McDonald Part I https://s.apache.org/InsideInfra-Gavin and Part II https://s.apache.org/InsideInfra-Gavin2
    Andrew Wetmore Part I https://s.apache.org/InsideInfra-Andrew and Part II https://s.apache.org/InsideInfra-Andrew2
    Chris Lambertus Part I https://s.apache.org/InsideInfra-ChrisL

 - ASF Targeted Sponsor Manning Publications is offering special deals on the latest books on Apache Airflow, Pulsar, Spark, and Thrift, among other titles and eBooks https://deals.manning.com/the-latest-apache-innovations/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Thursday January 21, 2021

The Apache Software Foundation Announces Apache® Superset™ as a Top-Level Project

Open Source enterprise-grade Big Data visualization and business intelligence Web application in use at Airbnb, American Express, Dropbox, Lyft, Netflix, Nielsen, Rakuten Viki, Twitter, and Udemy, among others.

Wilmington, DE —21 January 2021— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® Superset™ as a Top-Level Project (TLP).

Apache Superset is a modern, Open Source data exploration and visualization platform that  enables users to easily and quickly build and explore dashboards using its simple no-code visualization builder and state-of-the-art SQL editor. The project originated at Airbnb in 2015 and entered into the Apache Incubator program in May 2017.

"It's been amazing to be an active part of growing a welcoming, diverse and engaged community over the past five years while following the ASF principles around inclusion, openness and collaboration," said Maxime Beauchemin, Vice President of Apache Superset. "At the scale and level of diversity that the Superset project has achieved, it's critical to have a solid governance model in place like the one prescribed by the ASF."

Apache Superset v1.0
Superset helps streamline the analytics process by providing an intuitive interface to rapidly explore and visualize datasets, create interactive dashboards, and model real-time business intelligence insights at scale. The platform integrates with most SQL speaking data sources, including modern cloud-native databases, data warehouses, and engines at petabyte scale. 

The Project also celebrates a major milestone with the release of Apache Superset 1.0. Features include: 

  • Rich library of visualizations with support for integrating custom visualizations
  • Thin caching layer to optimize performance of charts and dashboards 
  • Code-free visualization builder
  • State-of-the-art SQL editor and metadata workflow
  • Extensible enterprise authentication and security model 
  • Easy-to-use, lightweight semantic layer
  • Notification alerts and scheduled reports


"Apache Superset 1.0 is a solid, mature, self-standing solution that fully solves business intelligence and data visualization needs for modern data teams," added Beauchemin. "Superset not only covers the table stakes, but also offers guarantees, features and a fresh approach that existing BI solutions can't match."

Apache Superset is in use at Airbnb, American Express, Dropbox, Lyft, Netflix, Nielsen, Rakuten Viki, Twitter, and Udemy, among others. A list of known users is available at https://github.com/apache/superset/blob/master/INTHEWILD.md .

"Apache Superset helps Airbnb democratize data insights and make data-informed decisions," said Jeff Feng, Product Lead at Airbnb and member of the Apache Superset Project Management Committee. "Superset uniquely connects SQL analysis with data exploration for thousands of our employees each week. It also serves as a flexible and reliable platform for visualizing metrics, helping executives and knowledge workers see and understand data."

"We had an amazing journey with Superset at Dropbox," said Chloe Wang, Senior Product Manager, Data Insights Platform at Dropbox. "Superset got introduced in 2019 and soon became the most widely adopted query engine within the analytical organization. As a result, our analysts are able to make timely and high confidence product decisions."

"Before Superset, we were paying for a patchwork of proprietary tools and we kept running into limitations when it came to customizing charts and dashboards," said Amit Miran, Software Team Lead for Media Application Framework group at Nielsen. "Once the Superset project supported adding of custom visualizations, that was the turning point for us at Nielsen to start adopting Superset in large projects. We’re very excited about native dashboard filters and future support for cross filtering, which will make our viz plugins even more powerful. The excitement for the project drove me to become involved in my first open source project."

"Apache Superset is an amazing project that enables engineers to easily execute data analysis," said Grace Guo, member of the Apache Superset Project Management Committee. "I have been a Superset user and a Superset builder for a few years. I run queries in SQL Lab, visualize data using one of the many supported chart types, and build dashboards, specifically focusing on performance and product adoption metrics. As an engineer, I appreciate the ability to contribute to the product. If I see some area to improve, or need a feature which doesn’t exist, I am happy to create a PR to fix it for myself and benefit other users."

"Apache Superset’s strength lies in its community," added Beauchemin. "We invite those interested in data visualization to join our mailing lists and help shape future versions of Superset."

Learn more about the latest in v1.0 at the Apache Superset community global MeetUp on 28 January. Registration is open to all and free of charge https://s.apache.org/3cm4f 


Availability and Oversight
Apache Superset software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Superset, visit https://superset.apache.org/


About the Apache Incubator
The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects enter the ASF through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation is the world’s largest Open Source foundation, stewarding 227M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 813 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with nearly 8,000 Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Budget Direct, Capital One, Cloudera, Comcast, Didi Chuxing, Facebook, Google, Handshake, Huawei, IBM, Microsoft, Pineapple Fund, Red Hat, Reprise Software, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF


© The Apache Software Foundation. "Apache", "Superset", "Apache Superset", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Tuesday January 19, 2021

The Apache CloudStack Project Releases Apache® CloudStack® v4.15

Mature Open Source Enterprise Cloud platform powers billions of dollars in transactions for the world's largest Cloud providers, Fortune 5 multinationals, educational institutions, and more.

Wilmington, DE —19 January 2021— The Apache CloudStack Project announced today v4.15 of Apache® CloudStack®, the mature, turnkey Open Source enterprise Cloud orchestration platform.

Apache CloudStack is the proven, highly scalable IaaS platform of choice to rapidly and easily create private, public, and hybrid Cloud environments: it "just works".

Apache CloudStack powers mission-critical clouds for the world’s largest users and service providers, including Alcatel-Lucent, Apple, Autodesk, Bell Canada, BT, China Telecom, Dell, Disney, Fujitsu, Huawei, INRIA, Juniper Networks, Korea Telecom, Leaseweb, Melbourne University, Nokia, NTT, Orange, SAP, Schuberg Philis, Taiwan Mobile, Tata, TrendMicro, Verizon, WebMD, and countless others.

"We are pleased to announce our latest release, making CloudStack even easier to deploy full-featured public and private clouds," said Sven Vogel, Vice President of Apache CloudStack. "Apache CloudStack continues to grow from strength to strength, with upgraded software and powerful deployments, backed by a robust community."

Apache CloudStack v4.15
Apache CloudStack includes the entire "stack" of features in an IaaS cloud: compute orchestration, Network-as-a-Service, user and account management, full and open native API, resource accounting, and a first-class user interface. The new 4.15 release ships with more than 200 new features, improvements, and bug fixes that include:

  • A new, modern user interface at general availability
  • vSphere advanced storage capabilities to support VMware storage policies, vSAN, VMFS6, vVols and datastore clusters
  • VMware "deploy-as-is" templates with OVF properties support for deploying virtual appliances in CloudStack clouds
  • Secondary storage management tools
  • Roles based users in projects
  • Dynamic roles enhancements for more granular RBAC
  • Support for CentOS 8, Ubuntu 20.04, XCP-ng 8.1, and MySQL 8
  • noVNC console for performance improvements to VM console access
  • Redfish support for out of band management
  • Unmanaging guest VMs
  • PVLAN support for L2 networks
  • Boot into hardware setup (VMware)
  • Configure root disk via service offering

The full list of new features is available in the project release notes at https://docs.cloudstack.apache.org/en/4.15.0.0/releasenotes/about.html

"At NTT/Itelligence we were eagerly anticipating this latest version of Apache CloudStack as many of the features in the release are of importance to our Itelligence cloud solution," said Andre Walter, VP, Head of GMS Cloud Infrastructure Services at Itelligence Global Managed Services GmbH. "We are particularly excited about the vSphere advanced capabilities and full OVF properties support. It is important for us to see the Open Source community bringing more and more features that allow us to enhance our global cloud operations capabilities."

"Apache CloudStack continues to bring innovative features for public cloud providers like us,"  said Wido den Hollander, CTO of PCExtreme. "With the 4.15 release, we are very interested in the Redfish implementation for Out of Band Management which helps bring the next generation of server management to our data centres. The fact that the CloudStack community is driven by users of the software as opposed to vendors with competing interests means that time and time again we see these exciting features delivered In Apache CloudStack."

"Apache CloudStack continues to cement itself as the logical choice for reliable, open source IaaS orchestration," said Giles Sirett, CEO of ShapeBlue. "It is proven, hugely scalable and, most importantly, easy to deploy and operate. The 4.15 release brings many features that will allow both public and private cloud operators to further innovate on their service offerings. I’d like to thank everybody in the Apache CloudStack community for this latest release."

The Apache CloudStack community invites those interested to join its mailing lists and global events, including CloudStack Collaboration Conference and numerous regional user groups. To get started and for ways to contribute, visit http://cloudstack.apache.org/contribute.html

Availability and Oversight
Apache CloudStack software is released under the Apache License v2.0 and is overseen by a volunteer, self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases.

Apache CloudStack Resources

About Apache CloudStack
An Apache Software Foundation Top-Level Project since 2013, Apache CloudStack powers countless mission-critical elastic Cloud computing services and solutions for Fortune 5 multinational corporations, Gartner Magic Quadrant leaders, and, as reported by Forrester, "sits beneath hundreds of service provider clouds". Visit https://cloudstack.apache.org/ and https://twitter.com/CloudStack for more information.

© The Apache Software Foundation. "Apache", "CloudStack", "Apache CloudStack", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Friday January 15, 2021

The Apache News Round-up: week ending 15 January 2021

It's Friday already --the week has zipped by. Let's take a look at what the Apache community has been up to:

Inside Infra – the interview series featuring members of the ASF Infrastructure team. - Chris Lambertus --Part I https://s.apache.org/InsideInfra-ChrisL

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - The Apache Software Foundation Operations Summary: Q2 FY2021 (August - October 2020) https://s.apache.org/Q2FY2021
 - Next Board Meeting: 20 January 2021. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - all videos from ApacheCon@Home are available at https://www.youtube.com/c/TheApacheFoundation/  

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.94%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 395 Apache Committers changed 3,156,343 lines of code over 3,300 commits. Top 5 contributors, in order, are: Krzysztof Kopyściński, Gary Gregory, Andrea Cosentino, Duo Zhang, and Jean-Baptiste Onofré.  

Apache Project Announcements – the latest updates by category.

Application Performance Monitoring --
 - Apache SkyWalking Eyes v0.1.0 released https://skywalking.apache.org/

Big Data --
 - Apache Beam 2.27.0 released https://beam.apache.org/

Content --
 - Apache POI, XMLBeans CVE-2021-23926: XML Entity Expansion https://s.apache.org/vbzsd
 - Apache Jackrabbit 2.21.5 released http://jackrabbit.apache.org/

Enterprise Processes Automation / ERP --
 - Apache OFBiz 17.12.05 released https://ofbiz.apache.org/

Servers --
 - Apache Tomcat CVE-2021-24122: Information Disclosure https://s.apache.org/huz9p


Did You Know?

- Did you know that the Apache geospatial community is partnering with the Open Geospatial Consortium (OGC) and Open Source Geospatial Foundation (OSGeo) to hold a joint Virtual Code Sprint the last week of February 2021? Call for participation is open https://s.apache.org/kp6d8

- Did you know that DoorDash's Big Data platform is powered by Apache Beam, Cassandra, Druid, Flink, Pinot, Spark and other projects? https://projects.apache.org/projects.html?category

- Did you know that you can help Apache Pulsar better meet the needs of its user community? Complete the Pulsar user survey today https://s.apache.org/jvaji 


Apache Community Notices

- Apache in 2020 - By The Digits https://s.apache.org/Apache2020Digits + Video highlights https://s.apache.org/Apache2020Digits-vid

- The Apache Software Foundation Operations Summary: 1 August - 31 October 2020 https://s.apache.org/Q2FY2021

- Apache Month In Review: December 2020 https://s.apache.org/Dec2020 

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet 
    Chris Thistlethwaite https://s.apache.org/InsideInfra-Chris
    Drew Foulks https://s.apache.org/InsideInfra-Drew
    Greg Stein Part I https://s.apache.org/InsideInfra-Greg
      ...Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3
    Daniel Gruno Part I https://s.apache.org/InsideInfra-Daniel1 and Part II https://s.apache.org/InsideInfra-Daniel2
    Gavin McDonald Part I https://s.apache.org/InsideInfra-Gavin and Part II https://s.apache.org/InsideInfra-Gavin2
    Andrew Wetmore Part I https://s.apache.org/InsideInfra-Andrew and Part II https://s.apache.org/InsideInfra-Andrew2
    Chris Lambertus Part I https://s.apache.org/InsideInfra-ChrisL

 - ASF Targeted Sponsor Manning Publications is offering special deals on the latest books on Apache Airflow, Pulsar, Spark, and Thrift, among other titles and eBooks https://deals.manning.com/the-latest-apache-innovations/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Monday January 11, 2021

Inside Infra: Chris Lambertus --Part I

Part I of the last of the "Inside Infra" interview series with members of the ASF Infrastructure team features Chris Lambertus, who shares his experience with Sally Khudairi, ASF VP Marketing & Publicity.


"...The thing that we're fighting against is the safety and longevity of the old technology. For quite some time, our primary concern was that the hardware that was running this, which was 15 years old, was going to fail."


What's your name and how is it pronounced?


My name is Chris Lambertus (“Kris Lamb bert uhss”): it's pronounced exactly how it's spelled.



When and how did you get involved with the ASF?


I've been aware of the ASF probably at least since the inception of the ASF. I've been working in IT for quite a long while, and I've been very familiar with the ASF projects, because I use them daily in my career. I didn't actually get involved with the ASF until a buddy of mine who was working on the CloudStack project mentioned to me that (ASF VP Infrastructure) David Nalley was looking for somebody to do some contract Infra work. Long story short, I talked to David and I tossed in an application. I was eventually hired as a part time contractor.


...So CloudStack, we're talking about 2012, when CloudStack first came into the Apache Incubator, or was it after that?


I've been aware of the ASF probably since HTTPd, since the original Web server came out. I joined the team late 2014.


Explain your role within the Infra team —how did you get here? Were they looking for someone who specializes in something particular?


My understanding was David was really looking for somebody that had a background in production systems engineering and had been doing it for a long time in a production environment. That's something that I had been doing since 1992: I've been essentially a professional production systems administrator. I knew that skill set was definitely in line with what David was looking for. I think I brought that to the table pretty well. That's basically what I've been doing ever since as a contractor.


What are you responsible for specifically?


That's a complex question, because the ASF Infra sysadmins are essentially responsible for everything. All of us are all responsible for all of the things. We do tend to specialize a little bit. My current project is probably reengineering the mail system: it's the largest one I'm working on right now. I do tend to focus a lot of my efforts on backups. Beyond that, I do a lot of JIRA and Confluence work with Gavin (ASF Infra team member Gavin McDonald). But Puppet, configuration management, again, all these things are things that all the Infra guys support.


In past interviews everyone has basically said, “we do everything”. How does it work? There's no hierarchy. Everyone does everything. Do queries come in and everyone jumps on them? Do you have a round-robin way of getting stuff done? How do you manage with so much going on with Infra? How do you cope with that?


“Cope with it” is an accurate term. Each of us has... I don't really want to call it a specialty, but definitely a focus. If some question comes in about the new mail routing or something that I've been specifically working on, that would go into my bucket as a priority. Certain people have history with certain types of projects. Gavin (Infra team member Gavin McDonald), for example, has been heavily involved with the Continuous Integration infrastructure for many, many, many, many years. So, he tends to be the font of knowledge for all things CI-related.


We tend to break things up that way. Some of the team members definitely have skillsets above and beyond general system administration work. Humbedooh (Infra team member Daniel Gruno’s username) is a very skilled programmer. He then ends up owning a lot of the software that Infra has developed and he has developed. So, questions regarding that, and specialty configurations related to software that he's written tend to go into his bucket. Because of the nature of the team, and because of the nature of the time zones that we're all in, the responsibility of dealing with issues follows on whoever is on call, first of all, and then whoever is awake and available, to handle any situation that comes up regardless of who "owns" the technology.


Describe a typical workday for you.


Apache work for me is basically: I wake up in the morning, hopefully not at 3:00 in the morning, but get out of bed and plop down in front of the computer. Essentially, my lifestyle is I've always been a computer guy. I've always been really focused on computer system administration, not only as my work but also as a hobby. So, I spend the vast majority of my day behind the computer, whether I'm working on Apache stuff or working on other projects, things like that. That'll go on until 11:00 at night. So, my "workday" is essentially me living my life and doing tasks as they arrive and doing projects as necessary and getting things done that need to get done.


...All the things.


Regardless of the time of day, yeah.


How do you keep your workload organized? Folks have all sorts of different systems. Are you an Evernote type of person, or do you keep your own journal? Do you have a certain system to help manage your workload?


Jira is the primary basis for managing my workload with the ASF. We've done a lot of work in terms of building technologies around Jira. Our service level agreement reporting tools I find extremely useful for seeing what's in the queue, what needs to be done, what hasn't been touched in a while, things like that. That really drives a lot of my day-to-day efforts in terms of replying to tickets and servicing customers.


In addition to that, I also use Jira to track my projects. So, if I have a project going on, that's usually a Jira ticket. And then I can go back and refer to those and see where things are, what needs to be done. I've never been a big one for lists of notes. I do have notes that I keep, but by and large, the things that are on the top of my stack maintain on the top of my brain at the same time. So, I don't feel like I forget a lot of things, but I don't take a lot of notes, which is what it is.


…And then there's people who have everything except the monitors covered in Post-it's.


I don't do a Post-it thing, but I have little text files everywhere with notes and things in them.


So, you all have day-to-day tasks that you manage, as well as things that require your immediate attention, as well as long term projects. In my earlier interviews with other Infra team members, everyone's been saying that I have to talk to you, because you're handling “The Email Project”. For those who aren't aware, standard operating procedure at the ASF is “if it didn't happen on-list, it didn't happen”. So, you have, if I'm understanding this correctly, 21 years’ worth of email archives that you're working on. What's going on with this project? What are you handling? Why is it so important?


Well, as you know, email is the lifeblood of the Foundation. Everything that happens here happens on a list. Because of that, the Foundation has amassed a very large quantity of email archives. Those archives are fundamental to the provenance of the Foundation. So, maintaining those and keeping those safe and available is really a top goal of the Infra team.


The mail project, such as it is, is essentially to upgrade and migrate our existing legacy email system to a modern, more supported system. The current email system as it stands was engineered by folks, volunteers, some staffers, I would guess, over 10 years ago, maybe 15 years ago, running on FreeBSD, which we don't really use too much anymore. Actually, we don't really use it at all. They used technologies that were interesting at the time, but are perhaps not so well supported today. So, a lot of it is modernization.


A lot of it is taking a lot of that old tribal knowledge that really doesn't exist anymore and bringing it into the modern era, documenting all the weird little settings that we have and all the edge cases that we manage in email, management of the list systems, mailing lists and their configuration, and making sure that gets upgraded, migrated, modernized. Doing that all in such a way that we don't a) lose anything, or b) suffer any downtime. So, it's a large project. That's really what I've been working on probably for the better part of the last two years, bringing that up to the present era.


You’re like the Titan Atlas: carrying the heavens on your shoulders. That's a massive, massive undertaking. Is there like a deadline for this —where's the end for this project? Is it never ending?


I feel more like Sisyphus than Atlas, but the deadline is as soon as possible. The thing that we're fighting against is the safety and longevity of the old technology. For quite some time, our primary concern was that the hardware that was running the old email system, which was 15 years old, was going to fail. In fact, it did. But fortunately, I basically copied the whole thing off to a separate colocation facility. So, we had an archive of it when it went down, and I was able to bring it all back up.


So, that wasn't a problem. I mean, it was a problem, but it wasn't a disaster as it could have been. So, the deadline is as soon as possible. But in reality, it's going to work until it stops working. I'm not sure how to better state that, because the technology is so old and we really need to get off of it and onto new technology. But there's no hard and fast timeline. Nobody's really cattle prodding me to get it done, but it's the absolute top priority that I have.


...That was actually my follow-up question. Is the “as soon as possible” official, or is this something you're setting for yourself because you just want to get it done?


Oh, that's definitely an official timeline. Yeah.


...I remember our first email servers were a machine under Brian Behlendorf’s desk at the Wired offices. So, we've come a long way since then.


We have, yes.


...You're handling this behemoth. Are you also dealing with the day-to-day putting out the fires, as well as everybody else?


Absolutely, yes.


The volume and scale of this project seems so huge. Again, the word 'cope' keeps coming to mind, because knowing what I know —and I don't even know— it's just scratching the tip of the iceberg: it seems astronomical in terms of scale and scope. Are you building everything from scratch for this project? Are you using any kind of commercial packages? This is a huge overhaul. Tell us more about it.


Multitasking has been in my blood for my entire life. I don't typically have a problem of splitting my time and my attention and my energies between multiple projects. You are absolutely right: this is a titanic project. It's one of the reasons why it's taken so long. Like I said, we've been working on this for several years at this point. The reason it's taken so long is twofold: One is I can't spend 100% of my attention on it or else I would go absolutely crazy. So I partition that. I partition my mind and my time, if you will. Just a little bit of time here working on this, working on this particular aspect of it, then I'll go work on some tickets. So, I'll go work on something else. If I was only working on the mail, then other things wouldn't get done, right?


I have to partition it that way. I think the main way I've tackled this type of project... Again, my experience in system administration going back so far, I've worked on a lot of very large scale projects. So, this is in the middle in terms of the scale. But the biggest thing is to break it down into multiple components as small a component as you really can. The first thing to do is to analyze the existing system. "What is it? How is it running? How is it tied together? How are these things all related? Where are the pieces? Where are the tendrils? How far do they go?" “Write that down.”


I started developing documentation that explained a lot of stuff. There was some documentation that existed. I take that and I carry it forward then into the new system. Okay, "what things do I want to keep? What things do I HAVE to keep? What things are legacy? What things don't we use anymore?" That process of discovery, of understanding how it was built, why it was built and what we're still using, and what we don't need to use anymore, is probably the vast majority of the work--just to understand it. Once that's done, we say, "Can we use the old technology, or do we need to use a different technology?"


In the case of the Foundation, we're extremely tied to the way that ezmlm, our mailing list system, works. ezmlm is extremely tied to Qmail. So, converting those into other tools, basically, I'll say, it's too complicated. With the amount of data that we have and the amount of dependence that we have on those configurations, migrating it to a different system would be incredibly difficult. So, what we've done is there are modern versions (and updates for) these pieces of software, ezmlm and Qmail.


What we've done is I've taken those packages and I built them for modern operating systems. I've patched them with current technology, TLS and various modern email stuff, and put that into configuration management and built a system that deploys all those packages in a reproducible fashion. So, at any time, I can just turn on a new machine. I could type in, "This machine is the new mail router," and run Puppet, our configuration management software, on it. It'll deploy all that software automatically.


That's probably the second part of this huge phase of developing this. The phase that we're in right now is testing it to make sure that it works the same way as the old one works. Once that's verified, then we can actually look at migrating the old data onto the new system and deploying it into production. I think that answered your question.


I think so, but it made me think of another question: How did this wind up being "your" project? Was this assigned to you? Did you jump on it going, "Yeah, I'm taking it"? How did you wind up with this?


That is a very good question. I don't really know. I think probably just because I had been working with... Back in, 2015 maybe, we were actually having this exact same discussion: "what do we need to do to migrate this EZMLM, all these mail archives, all this stuff to a new modern system?"


One of the things that we looked at was, "Can we transfer this? Can we translate this to something like Mailman or some newer type of mailing list management system?" We looked at a couple of options. The biggest problem we had was that the archivers were terrible. So, Humbedooh basically ended up writing this thing that became Pony Mail as the answer to that system. Ultimately, that turned out to be a great effort. I think it's going to take us a long way. But in the end, I was the one to continue to work on the email system. For whatever reason, I guess it just became my thing. Maybe because I was the only one willing to do it. I don't know.


...Is the legacy system going to be powered by Apache Pony Mail (incubating) at some point, or is it already in the process?


So yeah, lists.apache.org is our primary advertised archive system. That is what we're telling people to use. In terms of what happens to the old system, that remains a little bit under discussion. I don't know the ultimate disposition of that, but the current plan is lists.apache.org will be the primary access to the mail archives.


I noticed that Pony Mail goes back quite a bit, but it didn't originally go back as far as it does now in terms of the archives. I’m curious to see if everything eventually is going to be migrated to it.


Yes, yes, we actually have a plan to load the previous archives in there. We loaded a subset when we first started it up. I believe they go back to 2012 right now. So yes, we do have a plan to load the previous archives.


Great. I understand some Apache projects and their communities are always asking for new services. How does Infra decide which products you support? Who gets assigned to take the lead on introducing new services or new products? I understand that you develop your own custom solutions as well. How do these get divvied up? Is everything in queue? How does it get done?


When you're talking about a project requesting a service, I think the first thing we look at is, "Is this service extremely specific to this one project, or is it something that has broad appeal to the Foundation?" If it has broad appeal to the Foundation, we've got multiple requests for it, it's a service that we feel we can provide, given the amount of time that we have available, then it's something that we would consider doing.


Obviously, there's a lot of other thought that goes into that in terms of what it is, what it does, what it needs to do, who needs access to it, that we have to evaluate. But generally, if it has broad appeal to the Foundation, it would be something we would look into. If it doesn't, if it's something that's very specific to a certain project, what we typically recommend is that a project request their own VM. They can run the service themselves. That's typically how we’ve approached that in the past.


Has the team been in a situation where you're like, "Hey, this is a really cool thing, let's bring it in," and then throw it on projects or see if anybody wants to do it? Does the converse happen also, where you guys have insight as to something that's hot and new and you think that would be a great fit for Infra, but you have to find a "problem" to connect it to; or is that not something that you deal with? Is your work all reactive, or do you ever come into a situation where you say, "Long-term planning: we want to introduce something brand new"?


I think probably up until maybe five years ago, the work was almost entirely reactive. But the team and the processes that David (Nalley) now put together have really pushed us more in a direction of future planning, of taking the time and taking the mindset of, "What can we do long term to better support projects?" I think selfserve.apache.org is a great example of that. That's something that grew out of a small subset of tools. We got very positive feedback from Committers and Projects about using selfserve.apache.org.


That tool has grown extensively since it was developed. I think one of the best things that we provided recently is the .asf.yaml system, which allows projects to essentially set up 90% of their project metadata in GitHub. It lets them set labels. It lets them set notifications. It lets them set all kinds of things, all self-service. So, it's taken a huge load off of Infra in terms of responding to tickets, and also put a lot of that control in the hands of the projects. That's been incredibly well received. It's definitely, I think, one of the best things that we've done for projects in a while. I think it's a fantastic tool.


That's great. Now, it's also a new way of institutionalizing, so to speak, of "scratch your own itch", but in a way that that's a common deployment. You can do your own thing, but there's a common mechanism or method of doing it, because before —it was like the Wild West, back in the '90s— everyone's just doing their own thing. It didn't really matter, but it wouldn't scale properly: you guys can’t really support them because everyone's doing something and it was a one-off. It's interesting to see that selfserve.apache.org has standardized or unified that process.


Yeah, and one of the things that I really like about it too is because we have so many different projects --they're so varied-- the people that work on them are so varied in their skill sets and their desires and their interest level and their skill level and all this. What we want to be able to do is empower projects to use the tooling and take advantage of the skill sets that they have available. So, we don't want to arbitrarily enforce, "Oh, you must use this particular technology," but we also don't want random technologies to proliferate, like you alluded to, the Wild West. So, it's a very refined balance between, "How do you allow projects to do their own thing in a way that's scalable and supportable?" That's a complex task. It's difficult to manage. I think Self-Serve (selfserve.apache.org) goes a long way to support that.


Speaking of Self-Serve and other solutions the team is providing, the strategic process of figuring out where to go —direction— I know you have David (Nalley), I know you have Greg (ASF Infrastructure Administrator Greg Stein). Does the entire team participate in this? How does this work: is it top-down, or is it bottom-up? Are you guys saying, "Hey, there's a new thing that we should do"? I presume you don't have an annual strategy, but rather an ongoing rolling process; how do strategic decisions get made?


It's a collaborative effort, for sure. I think we do have an annual —when we get together at ApacheCon, we do tend to have a lot of discussions about strategy and about future direction. That's one of the things that we try to do as Infra with our team meetups, and with ApacheCon as well, to get together in person in a room and talk about where we're going to go, what we want to do. I say the process is collaborative, because sometimes it comes from the direction of Greg, or the Board, or David, or whoever. Sometimes it comes from a staffer saying, "Hey, it'd be cool if we could do this."


Sometimes it comes from Projects, or Committers, and they say, "Hey, can we go in this direction? I think it would be useful for X reasons." It just depends. By and large, the decisions for a future strategy are brought up by whoever thinks of it and are discussed within the team at a peer-to-peer level, right? We have very few situations where Greg or David or somebody will come down and say, "Thou shalt do it this way." Yeah, very uncommon to have that happen. It's a very collaborative environment, which I appreciate and works well for me.


So, in light of the pandemic, you guys didn't have your face-to-face. Did you do a virtual annual meeting? Or did it just not happen?


Well, we have a weekly team meeting. Yeah, we didn't do any virtual thing beyond that.


With so many projects at the ASF now, with 350 projects and initiatives and growing and so few of you in Infra, you must be constantly learning new things. How do you keep abreast of what's new? How do you close your skills gap? How do you stay ahead of everything?


I follow a few mailing lists, discussion boards, Reddit, and other similar sources. I typically learn new things when I need to implement new technology to solve a problem. "How do we provide 'X'?" I’ll go research it and learn that way. I also find out about new things from my hobby projects or other work.  



...It's not like "I want to take Blah University to become certified in X" or anything like that, right? I mean, you’d do that from your own interest, but it's not something that's required of the job unless it comes up, right?


No, that's never been required of the job. Personally, I'm very much a self-directed learner. If I'm interested in something, I will absolutely seek out the resources to do so. I will say that there's not a lot of time for that stuff, at least not for me. I got a lot going on, right? So, having the time to sit down and take a class or go through that process, I find very difficult. I don't really learn that way very well either. So, class-based learning has never been for me.


...Not linear. Yeah.


Yeah. So, typically, if I want to learn something new —I've been trying to learn Python, because it's definitely a gap for me— I find it incredibly difficult, because it's very hard for me to sit down and watch a video on programming, right? I got to have a reason. I got to have a thing to do. I need to have a project that requires it. And then I go and I figure it out.


...Got it. So, it's purpose-driven education. You need an end result.


Yeah, exactly. That's how I've always operated.

[END OF PART I]

Friday January 08, 2021

The Apache News Round-up: week ending 8 January 2021

Happy Friday! Let's take a look at what the Apache community has been up to over the past week:

Apache in 2020 - By The Digits – a look at the achievements from the Apache Community over the past 12 months.
 - Summary and stats at https://s.apache.org/Apache2020Digits
 - Video highlights https://s.apache.org/Apache2020Digits-vid

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - The Apache Software Foundation Operations Summary: Q2 FY2021 (August - October 2020) https://s.apache.org/Q2FY2021
 - Next Board Meeting: 20 January 2021. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - all videos from ApacheCon@Home are available at https://www.youtube.com/c/TheApacheFoundation/  

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 100%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 348 Apache Committers changed 1,594,281 lines of code over 2,987 commits. Top 5 contributors, in order, are: Jean-Baptiste Onofré, Gary Gregory, Mark Thomas, Kartik Khare, and Andrea Cosentino.

Apache Project Announcements – the latest updates by category.

Application Performance Monitoring --
 - Apache SkyWalking NodeJS v0.1.0 released https://skywalking.apache.org/

Big Data --
 - Apache ShardingSphere ElasticJob UI 3.0.0-RC1 released http://shardingsphere.apache.org/elasticjob/
 - Apache Rya 4.0.1 released http://rya.apache.org/
 - Apache Flink CVE-2020-17518: Directory traversal attack: remote file writing through the REST API https://s.apache.org/qxl48 , and
   CVE-2020-17519: Directory traversal attack: reading remote files through the REST API https://s.apache.org/gith7

Network Client --
 - Apache Guacamole 1.3.0 released https://guacamole.apache.org/

Servers --
 - Apache Tomcat Native 1.2.26 released https://tomcat.apache.org/


Did You Know?

- Did you know that some of the latest podlings undergoing development in the Apache Incubator include BlueMarlin (advertising), HOP (orchestration), Pegasus (Big Data), Sedona (geospatial data processing), and Wayang (analytics)? http://incubator.apache.org/projects/

- Did you know that Apache Kafka is amongst the most popular streaming platform for disseminating COVID-19 related clinical data, test results, and caseload updates in real-time? http://kafka.apache.org/

- Did you know that the New Zealand Treasury Department, Prime Minister and Cabinet, National Emergency Management Agency, and Climate Change Commission's eRecruitment platform is powered by Apache Wicket? http://wicket.apache.org/


Apache Community Notices

- Apache Month In Review: December 2020 https://s.apache.org/Dec2020 

- Apache in 2020 - By The Digits https://s.apache.org/Apache2020Digits

- Video highlights: Apache in 2020 - By The Digits https://s.apache.org/Apache2020Digits-vid

- The Apache Software Foundation Operations Summary: 1 August - 31 October 2020 https://s.apache.org/Q2FY2021

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet 
    Chris Thistlethwaite https://s.apache.org/InsideInfra-Chris
    Drew Foulks https://s.apache.org/InsideInfra-Drew
    Greg Stein Part I https://s.apache.org/InsideInfra-Greg
      ...Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3
    Daniel Gruno Part I https://s.apache.org/InsideInfra-Daniel1 and Part II https://s.apache.org/InsideInfra-Daniel2
    Gavin McDonald Part I https://s.apache.org/InsideInfra-Gavin and Part II https://s.apache.org/InsideInfra-Gavin2
    Andrew Wetmore Part I https://s.apache.org/InsideInfra-Andrew and Part II https://s.apache.org/InsideInfra-Andrew2

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Friday January 01, 2021

The Apache News Round-up: week ending 1 January 2021

Welcome, 2021! We hope that you have had a festive holiday season and are excited to kick off the new year. Here's what happened over the past week:

Apache in 2020 - By The Digits – a look at the achievements from the Apache Community over the past 12 months.
 - Summary and stats at https://s.apache.org/Apache2020Digits
 - Video highlights https://s.apache.org/Apache2020Digits-vid

The Apache Month in Review – highlights of what we've accomplished over the past month. 
- December 2020 https://s.apache.org/Dec2020

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - The Apache Software Foundation Operations Summary: Q2 FY2021 (August - October 2020) https://s.apache.org/Q2FY2021
 - Next Board Meeting: 20 January 2021. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - all videos from ApacheCon@Home are available at https://www.youtube.com/c/TheApacheFoundation/  

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.95%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 214 Apache Committers changed 1,634,010 lines of code over 2,290 commits. Top 5 contributors, in order, are: Gary Gregory, Andreas Veithen, Chesnay Schepler, Rene Cordier, and Sylwester Lachiewicz.

Apache Project Announcements – the latest updates by category.

Application Performance Monitoring --
 - Apache SkyWalking Python v0.5.0 released https://skywalking.apache.org/

Big Data --
 - Apache ShardingSphere ElasticJob 3.0.0-RC1 released http://shardingsphere.apache.org/elasticjob/
 - Apache Accumulo 1.10.1 and 2.0.1 released http://accumulo.apache.org/
 - Apache Accumulo CVE-2020-17533: Improper Handling of Insufficient Permission https://s.apache.org/ixwwc

Data Management Platform --
 - Apache Ignite 2.9.1 released http://ignite.apache.org/


Did You Know?

- Did you know that the following Apache projects are celebrating anniversaries this month? Many happy returns to Apache Cocoon, James, and Web Services (17 years); Lucene (15 years); ActiveMQ (13 years); Hadoop (12 years); River (9 years); Empire-db and Gora (8 years); OpenMeetings (7 years); Samza (5 years); Arrow (4 years); and Ranger (3 years)! https://projects.apache.org/committees.html?date 

- Did you know that the Top Ten of Fortune's "Future 50" companies --ServiceNow, Veeva Systems, Atlassian, Workday, Splunk, Adyen, MercadoLibre, DexCom, Square, and Spotify-- are all Powered by Apache? Everyone is welcome to use ASF and Apache Project badges to show that your projects are Powered by Apache http://apache.org/foundation/press/kit/#poweredby

- Did you know that ASF Targeted Sponsor Manning Publications is offering special deals on the latest books on Apache Airflow, Pulsar, Spark, and Thrift, among other titles and MEAP (Manning Early Access Program) eBooks? https://deals.manning.com/the-latest-apache-innovations/


Apache Community Notices

- Apache Month In Review: November 2020 https://s.apache.org/Nov2020

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet 
    Chris Thistlethwaite https://s.apache.org/InsideInfra-Chris
    Drew Foulks https://s.apache.org/InsideInfra-Drew
    Greg Stein Part I https://s.apache.org/InsideInfra-Greg
      ...Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3
    Daniel Gruno Part I https://s.apache.org/InsideInfra-Daniel1 and Part II https://s.apache.org/InsideInfra-Daniel2
    Gavin McDonald Part I https://s.apache.org/InsideInfra-Gavin and Part II https://s.apache.org/InsideInfra-Gavin2
    Andrew Wetmore Part I https://s.apache.org/InsideInfra-Andrew and Part II https://s.apache.org/InsideInfra-Andrew2

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Apache in 2020 - By The Digits


Whilst 2020 has been quite a challenging year world-wide, the all-volunteer Apache community has demonstrated commendable strength, resilience, and commitment to our tenet of "Community Over Code" — 


  • 238 Apache Projects, sub-projects, incubating podlings, and their communities produced nearly 3,500 releases across dozens of categories. Release Categories: API Gateways, Application Performance Management, Big Data, Blockchain, Build Management Cloud Computing, Content, Cryptography, Customer Profile Platform, Databases, eMail, Enterprise Resource Planning, FinTech, Identity Management, Integrated Development Environments, Integration, IoT, Libraries, Logging, Machine Learning, Messaging, Natural Language Processing, Operating Systems, Programming Languages, Remote Desktop Gateway, Search, Security Frameworks, Servers, Services Framework, Templating, Testing, Version Control, Web Conferencing, Web Crawlers, Web Frameworks, and more.



  • Apache events moved online, and attracted our most diverse and greatest number of participants. ApacheCon@Home drew nearly 5,750 participants from more than 150 countries, who enjoyed 300+ sessions across 27 tracks. A staggering 1.5M+ viewers tuned in to the Apache Roadshow/China over its 2-day online event.


Additional highlights:


Apache Projects —https://projects.apache.org/


  • Total number of projects + sub-projects - 342
  • Top-Level Projects - 199
  • Podlings undergoing development in the Apache Incubator - 41
  • New Top-Level Projects that graduated from the Incubator - 10 


Community/People —http://home.apache.org/


The ASF’s merit-driven "Contributor-Committer-Member" progression is the central governing process across the Apache ecosystem. The core Apache Group of 21 individual Members grew with developers who contributed code, patches, or documentation. Some of these contributors were subsequently granted Committer status by the Membership, and provided access to: 1) commit code directly to Apache repositories; 2) vote on community-related decisions; and 3) propose an active user for Committership. Today, ASF Committers contribute not just code and documentation, but also an array of initiatives that provide value across the greater Apache ecosystem, including Project promotion and community development through mentoring, events, and diversity and inclusion programs. Those Committers who demonstrate merit in the Foundation's growth, evolution, and progress are nominated for ASF Membership by existing members.


The Apache community continues to grow: 


  • We welcomed 3,612 contributors in 2020, 51.87% of whom were newcomers to Apache
  • 905 individuals earned Committer status, totalling 8,022. 
  • 34 individuals were elected as new ASF Members, totalling 813.


Apache Projects/Code —https://projects.apache.org/statistics.html


3,258 Apache Committers changed 117,350,563 lines of code over 247,451 commits.


Top 5 Committers

  • Andrea Cosentino (6,357 commits; 2,003,123 lines changed)
  • Jean-Baptiste Onofré (3,120 commits; 735,656 lines changed)
  • Claus Ibsen (2,838 commits; 1,919,860 lines changed)
  • Mark Thomas (2,360 commits; 185,548 lines changed)
  • Gary Gregory (2,188 commits; 234,845 lines changed)


Top 5 Apache Project Repositories by Size (Lines of Code)


  • Tuweni (incubating; 7,822,771 --Tuweni is Apache's first project in the Blockchain space)
  • Flex (7,007,693)
  • NetBeans (6,582,707)
  • OpenOffice (6,376,683)
  • Hadoop (3,521,559)

Top 5 Apache Project Repositories by Commits


  • Camel
  • Flink
  • Airflow
  • Lucene/Solr
  • Spark


GitHub: Top 5 Most Active Apache Project Sources (clones)


  • Thrift
  • Beam
  • Arrow
  • Geode
  • Cordova


GitHub: Top 5 Most Active Apache Project Sources (visits)


  • Spark
  • Flink
  • Kafka
  • Beam
  • Camel



Mailing Lists —https://lists.apache.org/


"If it didn’t happen on-list, it didn’t happen"


The ASF’s day-to-day operations, including Apache project and community development, takes place on ~1,450 public and ~700 private mailing lists. 


In 2020, 18,388 authors sent 2,139,458 emails on 774,364 topics.


Top 5 most active Apache Project user@ mailing lists


  • Flink
  • Lucene-Solr
  • OpenMeetings
  • Ignite
  • Tomcat


Top 5 most active Apache Project dev@ mailing lists


  • Tomcat
  • Flink
  • Royale
  • James
  • Beam


Contributor License Agreements and Software Grants —https://www.apache.org/licenses/


Individuals who are granted write access to the Apache repositories must submit an Individual Contributor License Agreement (ICLA). Corporations that have assigned employees to work on Apache projects as part of an employment agreement may sign a Corporate CLA (CCLA) for contributing intellectual property via the corporation. Individuals or corporations donating a body of existing software or documentation to one of the Apache projects need to execute a formal Software Grant Agreement (SGA) with the ASF. Over the past year, the ASF had received: 


  • ICLAs - 708
  • CCLAs - 35
  • Grants - 35


Sponsorship and Individual Support —http://apache.org/foundation/contributing.html


The ASF benefits from the generosity of hundreds of individual donors and corporate Sponsors, whose support helps offset the ASF's day-to-day expenses for Accounting, Fundraising, Infrastructure, Legal, Marketing & Publicity, and other services.


ASF Sponsors provide financial backing for the ASF's operations. They are:


PLATINUM: Amazon Web Services, Facebook, Comcast, Google, Huawei, Pineapple Fund, Tencent, and Verizon Media.


GOLD: Anonymous, Baidu, Bloomberg, Cloudera, Handshake, IBM, Reprise Software, Union Investment, and Workday.


SILVER: Aetna, Alibaba Cloud Computing, Budget Direct, Capital One, Cerner, Inspur, Red Hat, and Target.


BRONZE: Airport Rentals, The Blog Starter, Bookmakers. Cash Store, Bestecasinobonussen.nl, Casino2k, Curity, The Economic Secretariat, Gundry MD, Host Advice, HostChecka.com, Indian Online Casino, Journal Review, LeoVegas, Miro-Kredit AG, Mutuo Kredit AG, Online Holland Casino, ProPrivacy, PureVPN, RX-M, SCAMS.info, SevenJackpots.com, Software Guru, Start a Blog by Ryan Robinson, Talend, The Best VPN, Top10VPN, Twitter, and Xplenty.


ASF Targeted Sponsors provide the Foundation with non-financial contributions for specific operational activities or programs. They include:


TARGETED PLATINUM: Amazon Web Services, CloudBees, DLA Piper, JetBrains, LeaseWeb, Microsoft, OSU Open Source Labs, Sonatype, and Verizon Media.


TARGETED GOLD: Atlassian, The CrytpoFund, Datadog, PhoenixNAP, and Quenda.


TARGETED SILVER: HotWax Systems, Manning Publications, and Rackspace.


TARGETED BRONZE: Bintray, Education Networks of America, Friend of Apache Cordova, Google, Hopsie, No-IP, PagerDuty, Peregrine Computer Consultants Corporation, Sonic.net, SURFnet, and Virtru.



Apache Members, Committers, contributors, users, supporters, and Sponsors further the ASF’s mission of providing Open Source software for the public good. Help keep Apache software accessible to everyone by making a contribution* to the ASF https://donate.apache.org/ , becoming a Sponsor, or adding us to your Corporate Giving program. Please visit http://apache.org/foundation/contributing.html for more information.


Best wishes for a stellar 2021!



* The ASF is a US 501(c)(3) not-for-profit charitable organization, whose tax identification number is 47-0825376. The ASF is recognized by Charity Navigator and cited with the Gold Seal of Transparency by GuideStar.


# # #

Apache Month in Review: December 2020

Welcome to the latest monthly overview of events from the Apache community. Here's a summary of what happened in December:

Support Apache --

When we founded the ASF 21 years ago, we made a commitment to ensure Apache software is freely available to everyone worldwide at 100% no cost. Today the ASF provides more than $21B worth of software developed by an all-volunteer community. 

 - from Individual and Corporate donations to online shopping, Corporate Charitable Giving, Matching Gifts, and Sponsorship, There are many ways to help the ASF with a tax-deductible contribution https://s.apache.org/2020SupportApache


New this month --

 - ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
   -- Videos of all ApacheCon@Home sessions, including Plenaries and Keynotes, are available https://www.youtube.com/c/TheApacheFoundation/

 - Apache Software Foundation Operations Summary: Q2 FY2021 (August - October 2020) https://s.apache.org/Q2FY2021

 - "Inside Infra" – the interview series featuring members of the ASF Infrastructure team
   -- Meet Andrew Wetmore --Part I https://s.apache.org/InsideInfra-Andrew and Part II https://s.apache.org/InsideInfra-Andrew2

 - Apache Month in Review: November 2020 https://s.apache.org/Nov2020


Important Dates --

  - Next Board Meeting: 20 January 2021. Board calendar and minutes http://apache.org/foundation/board/calendar.html


Infrastructure --

Our seven-member Infrastructure team on three continents oversees our highly-reliable, distributed network under the leadership of VP Infrastructure David Nalley and Infrastructure Administrator Greg Stein. ASF Infrastructure supports 300+ Apache projects and their communities across ~200 individual machines, 1,400+ repositories, 5-6PB in traffic annually, ~75M downloads per month, and 2-3M daily emails on 2,000+ lists. ASF Infra performs 7M+ weekly checks to ensure services are available around the clock. The average uptime in December was 99.95%. http://www.apache.org/uptime/

Committer Activity --

In December, 837 Apache Committers changed 11,192,118 lines of code over 18,775 commits. The Committers with the top 5 highest contributions, in order, were: Andrea Cosentino, Xiang Xiao, Hugh Miles, Andi Huber, and Gary Gregory.

Project Releases and Updates --

New releases from Apache Accumulo (Big Data); Airflow (Big Data); APISIX (API); Avro (Big Data); Beam (Big Data); Bigtop (Big Data); Camel (Integration); Flink (Big Data); Groovy (Programming Languages); HBase (Big Data); HttpComponents Core (Servers); IoTDB (IoT); Jackrabbit (Content); JMeter (Testing); JSPWiki (Content); Kafka (Big Data); Knox (Big Data); OpenMeetings (Web Conferencing); PDFBox (Content); Pulsar (Messaging); Rya (Big Data); ShardingSphere (Big Data); SINGA (Machine Learning); Skywalking (Application Performance Management); Struts (Web Frameworks); Syncope (Identity Management); Tika (Big Data); Tomcat (Servers); Traffic Control (Servers); Traffic Server (Servers); Yetus (Library).

The Apache Incubator is the primary entry path for projects wishing to become an official part of the ASF. New to the Apache Incubator in December: Wayang (Big Data). We invite you to review the many projects currently in development in the Apache Incubator http://incubator.apache.org/ .

# # #

To see our Weekly News Round-ups (published every Friday), visit https://blogs.apache.org/foundation/ and click on the calendar or hop directly to https://blogs.apache.org/foundation/category/Newsletter . For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. We appreciate your support!

Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation