Success at Apache: I Became an Apache Solr Committer in 4,662 Days. Here’s how you can do it faster!

by Eric Pugh

On April 6th, 2020 I was invited to become a committer on the Apache Solr project.  My journey to becoming a committer started in earnest 4662 days before that!  On July 2nd, 2007, I opened SOLR-284, a ticket for adding content extraction to Solr. 

A committer on an open source project under the Apache Foundation umbrella is someone who is trusted to contribute code to the project and to help manage and drive its ongoing development. It’s an honour to have been asked and I was very proud to accept the invitation!

So, you did the math, and you realized that it took me 153 months, or 13 years (rounding up), to become a committer, and you’re wondering “What if I don’t want to wait that long?” So here’s my quick cheat sheet on ways to become a committer on an open source project, illustrated by my own journey:

1. Start by learning the culture of the project. How are decisions made? What tools do people use? What do the various acronyms mean? Join the mailing lists and read every commit.
2. Start small and work your way in.  Some great ways to do this are to:
   • Take existing patches and test them.  Update them to the latest code base.  Document what you’ve learned. 
   • Take advantage of being new to a project to bring fresh eyes to the documentation.  Every time you find yourself scratching your head on how something works, contribute a fix to the docs.   It’s a powerful way to immediately contribute.  This is the fastest way to get involved and involves the least cognitive load!  See SOLR-2232 or this email thread.
   • Answer questions on the mailing list! Being able to articulate reasonable responses to questions demonstrates how much you have learned.
   • Bug fix, bug fix, bug fix! Pick bugs that have an obvious answer so that the “correct” solution is easy to figure out. If the right approach to solving it is very ambiguous, you probably won’t get much traction. Remember to remind committers to apply your fixes when they have the time! See SOLR-13965 and SOLR-11480 and SOLR-2611 and SOLR-2263.
3. Ready to start slinging some code?   Don’t go and refactor the core foundations of the project (at least not yet).   Instead, be like a pilot fish and latch onto one of the core committers who is being very active in the project.

Embrace their vision, and start picking up tasks related to whatever major chunk of work they are doing. Write some unit tests. See about opportunities for refactoring. Do some manual testing over multiple platforms. Once they see that you’re contributing (and accelerating what they are pushing), then work to get some of your own tickets assigned to you under that vision. I’ve seen this lead directly to committership many times, and if I had followed this route, I might have joined sooner!

Here’s to the next 4,662 days of being active in the Apache Solr project!

Eric Pugh is a member of the ASF and a committer Apache Solr. He co-authored the book Apache Solr Enterprise Search Server. Eric is co-founder and CEO of OpenSource Connections, where he helps OSC clients, especially those in the ecommerce space, build their own search teams by leading projects and by acting as a trusted advisor. He also stewards Quepid, a platform for assessing and improving your search relevance.

[this post first appeared at https://opensourceconnections.com/blog/2020/07/10/i-became-a-solr-committer-in-4662-days-heres-how-you-can-do-it-faster/ ]

= = =

"Success at Apache" is a monthly blog series that focuses on the processes behind why the ASF "just works" https://blogs.apache.org/foundation/category/SuccessAtApache 

Posted at 12:21PM Aug 03, 2020 by Sally in SuccessAtApache  |   | 

Apache Month in Review: July 2020

Welcome to the latest monthly overview of events from the Apache community. Here's a summary of what happened in July:

New this month --

 - ASF Annual Report – a look back at our many achievements during the 2020 Fiscal Year
 -- Press release https://s.apache.org/FY2020AnnualReport-PR
 -- Full report https://s.apache.org/FY2020AnnualReport

 - "Trillions and Trillions Served" – the feature documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019, watch “Apache Innovation”, the fourth and final segment of the series https://s.apache.org/ApacheInnovation

 - ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
  -- Announcing ApacheCon @Home 2020: ApacheCon North America and Europe have been combined and will be held online 29 September - 1 October 2020. Join us! https://apachecon.com/acah2020

 - "Inside Infra" – a new interview series with members of the ASF Infrastructure team
  -- Meet Greg Stein --Part III https://s.apache.org/InsideInfra-Greg3

 - Apache Month in Review: June 2020 https://s.apache.org/June2020

Important Dates --

 - Next Board Meeting: 19 August 2020. Board calendar and minutes http://apache.org/foundation/board/calendar.html

 - ApacheCon @Home 29 September - 1 October 2020 https://apachecon.com/acah2020

Infrastructure --

In July, 846 Apache Committers changed 14,227,571 lines of code over 16,062 commits. The Committers with the top 5 highest contributions, in order, were: Mark Miller, Andrea Cosentino, Gary Gregory, Sebastian Bazley, and Zhang Yonglun.  

Project Releases and Updates --

New releases from Apache Annotator (Libraries); APISIX (API); Arrow (Big Data); Atlas (Big Data); Avro (Big Data); Beam (Big Data); BVal (Libraries); Calcite (Big Data); Commons Lang and Numbers (Libraries); Commons Text (Libraries); Curator (Messaging); Daffodil (Libraries); Druid (Big Data); Flink (Big Data); Geometry (Libraries); Groovy (Programming Languages); Guacamole (Network Client); HBase (Big Data); IoTDB (IoT); Jackrabbit (Content); Kylin (Big Data); Lucene (Search); MyFaces (Web Frameworks); NiFi (Big Data); Nutch (Web Crawler); OFBiz (Enterprise Processes Automation / ERP); Qpid (Messaging); Skywalking (Application Performance Management); Storm (Big Data); Tomcat (Servers);Tuweni (Blockchain); TVM (Machine Learning); Wicket (Web Frameworks).

The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. Congratulations to Apache APISIX, which graduated as a Top-Level Project this month https://s.apache.org/29wd9. We invite you to review the many projects currently in development in the Apache Incubator http://incubator.apache.org/

# # #

To see our Weekly News Round-ups, visit https://blogs.apache.org/foundation/ and click on the calendar in the upper-right side (published every Friday) or hop directly to https://blogs.apache.org/foundation/category/Newsletter . For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. We appreciate your support!

Posted at 03:09PM Aug 01, 2020 by Swapnil M Mane in Newsletter  |   | 

The Apache News Round-up: week ending 31 July 2020

Farewell, July --we're wrapping up the month with another great week. Here are the latest updates on the Apache community's activities:

ASF Annual Report – a look back at our many achievements during the 2020 Fiscal Year
 - Press release https://s.apache.org/FY2020AnnualReport-PR
 - Full report https://s.apache.org/FY2020AnnualReport

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 19 August 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is OPEN (and free) for ApacheCon@Home taking place online 29 September - 1 October. Join us! https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 100%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 414 Apache Committers changed 4,407,598 lines of code over 3,792 commits. Top 5 contributors, in order, are: Mark Miller, Andrea Cosentino, Gary Gregory, Guillaume Nodet, and Shen Yi.  

Apache Project Announcements – the latest updates by category.

Application Performance Monitor --
 - Apache SkyWalking CLI 0.3.0 and SkyWalking Python 0.2.0 released https://skywalking.apache.org/

Big Data --
 - Apache Arrow 1.0.0 released https://arrow.apache.org/
 - Apache Calcite 1.24.0 released https://calcite.apache.org/
 - Apache Beam 2.23.0 released https://beam.apache.org/

Content --
 - Apache Jackrabbit 2.21.3 released https://jackrabbit.apache.org/

Libraries --
 - Apache Commons Text 1.9 released https://commons.apache.org/proper/commons-text/

Messaging --
 - Apache Qpid Proton-J 0.33.6 and JMS 0.53.0 released https://qpid.apache.org/

Did You Know?

 - Did you know that there were 150TB+ in source code downloads from Apache mirrors (excludes convenience binaries) in FY2020? https://s.apache.org/FY2020AnnualReport 

 - Did you know that workflows for the Creative Commons Catalog project are powered by Apache Airflow? https://opensource.creativecommons.org/blog/entries/data-flow-API-to-DB/ 

 - Did you know that, after two years in development, Apache Cayenne v4.1 GA is now available? https://cayenne.apache.org/ 

Apache Community Notices

- Apache Month In Review: June 2020 – overview of events that have taken place within the Apache community https://s.apache.org/June2020 

- "Trillions and Trillions Served" – the documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 have been released: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" short https://s.apache.org/ApacheEverywhere 3) "Why Apache" teaser https://s.apache.org/ASF-Trillions 4) “Apache Innovation” shorts https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

Posted at 07:12AM Jul 31, 2020 by Swapnil M Mane in Newsletter  |   | 

The Apache® Software Foundation Announces Annual Report for 2020 Fiscal Year

World's largest Open Source foundation provides 227M+ lines of code, valued at more than $20B, to the public-at-large at 100% no cost.

Wakefield, MA —29 July 2020— The Apache^® Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the availability of the annual report for its 2020 fiscal year (1 May 2019 - 30 April 2020).

Now in its 21st year, the world's largest Open Source foundation’s "Apache Way" of community-driven development is the proven process behind thousands of developers successfully collaborating on hundreds of Apache projects. The Apache Way has directly influenced the InnerSource methodology of applying Open Source and open development principles to an organization. The Apache Way has been adopted by countless organizations, including Capital One, Comcast, Ericsson, HP, IBM, Google, Microsoft, PayPal, SAP, T-Mobile, Walmart, and countless others.

Valued at more than $20B —and provided to the public-at-large at 100% no cost— Apache software is used in every Internet-connected country on the planet.

Apache software comprises 227M+ lines of code, is integral to nearly every end user computing device, manages exabytes of data, executes teraflops of operations, and stores billions of objects in virtually every industry. Countless mission-critical projects in Artificial Intelligence, Big Data, build management, Cloud Computing, content management, DevOps, Deep Learning, IoT and Edge computing, mobile, servers, Web frameworks, and many other categories are powered by Apache. [Learn more about the ASF’s reach and influence at https://s.apache.org/ApacheEverywhere ]

Every Apache Top-Level Project (and its sub-projects, if applicable) is overseen by a Project Management Committee (PMC) that guides its day-to-day operations, including community development and product releases.

At the close of FY2020, 201 PMCs managed 339 Top-Level Projects and dozens of sub-projects; 9 projects were newly-graduated Top-Level Projects from the Apache Incubator, and 8 projects retired to the Apache Attic. Apache Incubator PMCs mentored 45 projects under development in the Apache Incubator; 6 projects were new entrants to the Incubator, and 3 were retired.

FY2020 highlights include:

• ~8M lines of Apache code added, valued at approximately $600M worth of work; total code value exceeding $20B (CoCoMo model);
• Stewardship of 227M+ lines of code in the Apache repositories;
• Foundation operations supported by contributions from 10 Platinum Sponsors, 9 Gold Sponsors, 11 Silver Sponsors, 25 Bronze Sponsors, 6 Platinum Targeted Sponsors, 5 Gold Targeted Sponsors, 3 Silver Targeted Sponsors, 10 Bronze Targeted Sponsors, and more than 500 individual donors;
• 34 new individual ASF Members elected, totalling 813;
• Exceeded 7,700 code Committers;
• 206 Top-Level communities overseeing 339+ Apache projects, plus dozens of sub-projects and initiatives;
• 9 newly-graduated Top-Level Projects from the Apache Incubator;
• 45 projects currently undergoing development in the Apache Incubator;
• Web requests received from every Internet-connected country on the planet;
• 35M+ page views per week across apache.org;
• ~2 Petabytes source code downloads from Apache mirrors;
• Top 5 most active/visited Apache projects: Kafka, Hadoop, Lucene, POI, ZooKeeper; 
• Top 5 Apache repositories by number of commits: Camel, Flink, Beam, HBase, Lucene Solr;
• Top 5 Apache repositories by lines of code: NetBeans, OpenOffice, Flex (combined), Mynewt (combined), Trafodion;
• 2,892 Committers changed 60,132,710 lines of code over 174,889 commits;
• 12,413 people created 63,172 new issues; 2,868 people closed 54,633 issues
• 19,396 authors sent 2,137,560 emails on 907,870 topics across 1,417 mailing lists;
• Top 5 most active mailing lists (user@ + dev@): Flink, Tomcat, Royale, Beam, Lucene Solr;
• Top Senders: (Apache Projects + Committers): GitBox, AsterixDB, Whimsy, Andrea Cosentino, Mark Thomas
• 2,045 git repositories, containing ~250GB of code and repository history;
• GitHub traffic: Top 5 most active Apache sources --clones: Thrift, Beam, Cordova, Arrow, Geode;
• GitHub traffic: Top 5 most active Apache sources --visits: Spark, Flink, Camel, Kafka, Beam;
• 25th anniversary of the Apache HTTP Server (21 years under the ASF umbrella);
• 748 Individual Contributor License Agreements (ICLAs) signed;
• 33 Corporate Contributor License Agreements signed;
• 40 Software Grant Agreements signed; and
• ASF was a mentoring organization in Google Summer of Code for the 15th consecutive year.

The full report is available online at https://s.apache.org/FY2020AnnualReport 

About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation (ASF) is the world’s largest Open Source foundation, stewarding 227M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 813 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 7,800+ Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Indeed, Inspur, Leaseweb, Pineapple Fund, Private Internet Access, Red Hat, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF 

© The Apache Software Foundation. "Apache", "Arrow", "Apache Arrow", "AsterixDB", "Apache AsterixDB", "Beam", "Apache Beam", "Camel", "Apache Camel", "Cordova", "Apache Cordova", "Flex", "Apache Flex", "Flink", "Apache Flink", "Geode", "Apache Geode", "Apache GitBox", "Hadoop", "Apache Hadoop", "HBase", "Apache HBase", "Apache HTTP Server", "Ignite", "Apache Ignite", "Kafka", "Apache Kafka", "Lucene Solr", "Apache Lucene Solr", "Mynewt", "Apache Mynewt", "NetBeans", "Apache NetBeans", "OpenOffice", "Apache OpenOffice", "POI", "Apache POI", "Royale", "Apache Royale", "Spark", "Apache Spark", "Thrift", "Apache Thrift", "Tomcat", "Apache Tomcat", "Trafodion", "Apache Trafodion", "Whimsy", "Apache Whimsy", "ZooKeeper", "Apache ZooKeeper", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Posted at 12:00PM Jul 29, 2020 by Sally in General  |   | 

The Apache News Round-up: week ending 24 July 2020

Happy Friday! We've had a great week within the Apache community. Here's what happened:

Inside Infra – the interview series with members of the ASF Infrastructure team.
 - Meet Greg Stein --Part III https://s.apache.org/InsideInfra-Greg3

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 19 August 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is OPEN (and free) for ApacheCon@Home taking place online 29 September - 1 October. Join us! https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.96%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 407 Apache Committers changed 4,357,953 lines of code over 3,609 commits. Top 5 contributors, in order, are: Gary Gregory, Andrea Cosentino, Mark Miller, Zhang Yonglun, and Sebastian Bazley.      

Apache Project Announcements – the latest updates by category.

Big Data --
 - Apache Atlas 2.1.0 released https://atlas.apache.org/
 - Apache Calcite Avatica Go 5.0.0 released https://calcite.apache.org/avatica
 - Apache Druid 0.19.0 released https://druid.apache.org/
 - Apache NiFi Registry 0.7.0 released http://nifi.apache.org/registry
 - Apache Flink 1.11.1 released https://flink.apache.org/

Content --
 - Apache Jackrabbit Oak 1.22.4 released https://jackrabbit.apache.org/oak

Libraries --
 - Apache Commons Lang 3.11 released https://commons.apache.org/proper/commons-lang/
 - Apache Geometry 1.0-beta1 released https://commons.apache.org/proper/commons-geometry/

Programming Languages --
 - Apache Groovy 2.4.20, 2.5.13, and 3.0.5 released http://groovy.apache.org/

Search --
 - Apache Lucene 8.6.0 and Solr 8.6.0 released http://lucene.apache.org/

Web Frameworks --
 - Apache Wicket 7.17.0 released https://wicket.apache.org/

Did You Know?

 - Did you know that Apache projects' ongoing sustainability is ensured through the generosity of our Sponsors and individual donors, whose support helps ensure that the ASF continues to provide more than $20B worth of software to the public-at-large at 100% no cost? http://donate.apache.org

 - Did you know that projects undergoing development in the Apache Incubator are in a variety of categories that include AI, Annotation, Big Data, Blockchain, Cryptography, Data Visualization, Distributed Computing, Email, Embedded Systems, Geospatial Data, Graphing, Hardware Acceleration, IoT, Messaging, Monitoring, Natural Language Processing, Scheduling, Streaming, Training, Usability Testing, and more? http://incubator.apache.org/projects/

 - Did you know that you can learn about Apache Beam, Calcite, Camel, CarbonData, Groovy, Hadoop, Karaf, Labs, NetBeans, OFBiz, OpenOffice, PLC4X, Rya, Spark, Tomcat, Unomi, and more in the "Apache Innovation" short? https://s.apache.org/ApacheInnovation

Apache Community Notices

- Apache Month In Review: June 2020 – overview of events that have taken place within the Apache community https://s.apache.org/June2020 

- "Trillions and Trillions Served" – the documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 have been released: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" short https://s.apache.org/ApacheEverywhere 3) "Why Apache" teaser https://s.apache.org/ASF-Trillions 4) “Apache Innovation” shorts https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

Posted at 08:17AM Jul 24, 2020 by Swapnil M Mane in Newsletter  |   | 

Inside Infra: Greg Stein --Part III

We were talking about ensuring that the team is up to speed with everything required of them...

So there certainly are skill gaps; this is one of the things I want to help motivate the team with, where if somebody says, "Hey, I want to go and investigate Ansible as a potential Puppet replacement," I say, "Go forward." 

This would be similar to Google having their 20% projects. I'm sure you've heard of that.

Oh, yeah.

It's almost the same where it's not 20%, maybe 5%, but it's the same as Google, no matter what they want to tell you, because everybody's got their job and you have to be really rigorous to carve out 20% of your time. And strictly speaking, it does actually make your Google manager a little upset if you carve out the entire 20%. But anyways, the concept is similar.

So for us it’s like, "Well, go in and investigate Ansible, see if it'll work for us and put your notes into the Wiki." That's how we make forward progress, up our game, and learn new skills. If someone says, "I want to go and figure this out," the response is almost always, "Okay. You go do it." There's certainly an allowance for people to learn new skills. But most of the time we simply rely on, say, Gavin (ASF Infrastructure team member Gavin McDonald), knowing more about JIRA configuration than the other guys.

That added component of sharing what you know, and adding it to the JIRA or to the Wiki actually is great because then everyone's learning. This is like the rising tide: everybody's learning about this, whether they're doing it perfectly or not. I think this is a very interesting process.

Yes, and that's also where Andrew (technical writer Andrew Wetmore) is helping us out. He’s organizing that information that we have learned, that we have documented, that we memorialized into the Wiki.

Because our (ASF’s) legacy is quite Medusa-like over all these years, it's interesting to see how everyone can get caught up and also contribute...you have to go back and deal with the legacy, but you also have to be able to move forward. To be able to bring others with you is brilliant. That's really cool.

The infrastructure has grown organically over 25 years from when Brian Behlendorf first said, "Hey, I have this server called hyperreal.org: you can run a CVS repository on it for the Web server."

That computer was under his desk at the Wired offices way back when, wasn’t it...

Yes it was. And it's just grown organically over those 25 years. Then we had Minotaur and it did six different things ... now it only does half of one and we've moved the stuff out onto newer machines and newer processes and this and that. But the organic growth means that we've got some really hairy stuff. Our move to Puppet --first Puppet 3, and now to Puppet 6-- at each step we're improving it and making it less hairy and more manageable and something that somebody can come along, look at, pick up and run with it from there. That makes it a lot easier, so that we don't have to spend 100% of our time cross training.

What are your thoughts on products, the hype cycle, where everyone's demanding Kubernetes, to use that as an example. Do you decide which products to provide support for, or is that up to Apache projects in the communities? You mentioned Ansible, just not too long ago, that was your internal decision to move. But I remember not long ago, GitHub entered into the landscape. How did that happen? How did you decide to make a move like that? That's a significant thing. Can you tell me a little bit about that?

It's a lot based on community input. So if we see a lot of people asking for a particular tool, we'll like, "Oh, hey, David, can you go and take a look at that and see if that's something…” Not David (ASF VP Infrastructure David Nalley), but Chris (Infrastructure team member Chris Lambertus) or somebody else. "Can you go take a look. Is that something that we can support? Because we're getting some queries about it."

And there's a little chicken and egg problem there that if the communities don't know to ask for the egg, we don't know whether to prep the chicken. It's like, “okay, wait, they don't even know to ask for a tool because we haven't said we will make this tool available, because we're not going to make the tool available until somebody asks”. But sometimes people file tickets like, "Can I get this set up?" and we'll go, "No."

Then six months later, somebody else will file a ticket: "Can I get this set up?" and we'll go," No." But after enough of those, we're like, "Maybe that's something that we really want to do." For GitHub, specifically that’s what happened there. Well, even before that Git, where we ran our own Git server, that was a volunteer that made that happen. That was, six years ago or so.

Well...the volunteer came along and said, "Well, I'll do this. I'm not going to take any time from Infra." There's been a couple things for the past few years where I've told people, "No, Infra will not work on that. But if you want to volunteer or find a volunteer, then we'll stand it up for testing." You know what I mean? Why not? So there's a couple things where people have stood up for test examples and there hasn't really been a lot of usage.

So, we're not going to support that. But something like Ansible is our own internal workflow and the tool we’ll experiment with, then to see if it'll improve our stuff. But from the community, they pretty much have to ask and it has to be a sustained ask. That's how we ended up with Travis CI: we actually pay for capacity in Travis CI, and that's based on community input.

So many people wanted to do their continuous integration through Travis that eventually we decided to pay for it. But it's tricky because some of these systems like Travis CI and others require certain permissions that we don't want to provide to the community. So we will want to hold those only within Infra. And so it gets hard to integrate certain tools. We've had to say no, but then again, we've found other ways to improve that so that we can lock down the permissions or use a proxy or other ways that we can route around some of these issues and then integrate the requested tool.

So further to that, have you been in a situation where a project or a community has made unreasonable demands of Infra or have expectations, where it's like, so over the top or so out of scope, it totally surprised you? Have you had something like this?

Nothing surprises me.

Nothing surprises you? Okay. Have you been in this situation? Like “was never going to happen”...

Yes, yes. There's been several times where one of the guys on the team is like, "Oh man, I got this ticket. I don't know what we want to do with this. Greg, go take a look." And I go and look at it and that's where I make that call: "Okay, is the Infra team going to take this on, or do I just say ‘no’ right now?"

So, yeah, there's been a number of times where I've said no and probably two or three times where I've gotten a little bit of pushback on that no. I say, "My answer is no, but here's how you escalate." I've had escalation a few times and I'm actually, mid-process --I'm dealing with one right now. So, I've said, "no, if you don't like my no, you can go to VP Infra and VP Infra is, probably going to tell you the same thing. And then you can go to the President. Right now those are actually the same person."

The same person is a double "no".

That really is the true escalation path. I have to describe that to people and say, "I don't think you're going to get what you want." If I'm the one that says, no, you probably are not going to get it because VP Infra and President, and after that is the Board. They're probably not going to say, "Greg is wrong. Yes, we'll give that to you." But it's there. There's been a couple of times where I said "No, you have to ask the Board for the budget for those additional virtual machines." They went to the board and said, "Can we have budget for three machines?" and the Board said, "Yes."

So Infra went ahead and gave them the three VMs that they had initially requested. Strictly speaking, we would track those machines against their budget, but that detail is more than what the actual budget was. So we don't spend that time doing that, but I have had to say, no. I have had to... There was Apache Maven: they were keeping a copy of Maven Central, and Maven Central is run by Sonatype...

Which is a commercial product...

Yes. They're using the trademark “Maven”, essentially a licensing agreement from us, a MOU. So with Maven Central, you could imagine if someone decides to just turn it off one day ...we wanted a copy. Apache Maven was making a copy of it, and it just started consuming so much disk space. We were like, "We can't support that growth rate. We can't support that even for the next six months. If you want to keep doing it, go ask the Board for money to keep doing it." They never did. We turned it off.

I wouldn't call that a ridiculous request --it was something where we didn't have to just say, "No, not going to do it. Bye." A lot of the requests are mostly just, "We aren't going to run that extra software. If you want: ask for a VM and you can run it, but we're not going to take responsibility for it."

Over the years, obviously ASF Infra has changed. Was this all reactive or was it also proactive? Do you plan for those changes as you go or has it all been in response to Project X or in response to X emergency?

The growth of Infrastructure and its movement from volunteer-only to paid staff was part of just the growth of Apache. The volunteers could no longer keep up and things, like account creation, used to take sometimes four weeks to get an account. You’d put in a request for an account, four weeks later, it would finally get created.

My gosh, that queue was crazy, huh?

Well, it wasn't even a long queue, it was simply that we didn't have volunteers making sure the queue stayed empty. Today it's down to one, two, maybe three days, and the account is created, because every day a staff member goes and creates the accounts first thing in the morning.

It was how I said that my day starts with looking at messages on Slack and then reading emails to see if there's stuff to handle. Well, one of the guys on staff, first thing he does in the morning is go and look at account creation. So he's been off and on pondering on a tool to make that easier for himself; he hasn't finished the tool, so he still has to do it manually. That's his incentive.

“Work quickly”...

This is Chris Thistlethwaite. I say, "Chris, we can do something about that." And he says, "No, no, this is still my project. And every day when I run the script, it just makes me remember, I need to finish this."

So when the volunteers could not keep up with the amount of work, that's when we hired Joe Schaefer, then we hired another person, and hired another person. And so it was just trying to keep up with the rate of requests. 

That's how we ended up with hiring six people. And then I'm half a person, like I said, I'm part-time. So, it's just the growth of Apache. I think we're in much better shape than when I started. We're ahead of the curve. We can stay ahead of the curve because one of the things that I can do because I don't fight the fires every day ... that's for all the guys who know their stuff. They fight the fires and I can look at if I need to go and ask for another head count. And that's how we ended up with Andrew (technical writer Andrew Wetmore): “Well, you know, what we really need is somebody to manage all this documentation.” This was part of Sam's (former ASF President Sam Ruby), “If you had some money, what would you do with it?” That's how the technical writer/editor came around, because we've got 20 years of organic growth. We had...let's just call it “organic documentation”. That revamping project is going really well, I think.

So, in what areas are you guys experiencing your biggest growth? As I was asking Chris and Drew, is there like a geographic influence on the demand? We’ve had a huge influx of users in China. Does any of that change the way or what you guys are doing? Or is it just more of everything?

Our biggest pain point, I would say, is continuous integration/continuous development: CI/CD. Jenkins, Travis, CircleCI, and things like this, where people make a change and they want that change built and tested. The more projects we get and the larger the communities get, the more changes and the more testing and the more building and the more this, more, more, more. It's kind of one of those things where it's “expand-to-fit”. So if we gave people 100 machines, they'd use 100 machines. If we doubled it to 200, they'd use all 200. It's just this rapacious need for CI machines. It's very hard to figure out how to plan around that other than just telling the communities, “No: we just don't have that much capacity: if you want to build it, do it on your own machine. You just can't use Apache hardware to do it.”

That's an unsatisfactory answer. That's been one of our hard problems and it's also kind of a newer problem: the development workflow that uses CI probably is just maybe five years old. Before that, certainly, automated building and testing was a thing, but it's really kind of grown into community workflow much, much more over the past five years, and more and more people are wanting to do it. The communities are growing. Apache is growing: we're just seeing the demand explode and it's a hard problem for us to solve.

China is the one case where we see regional issues, and that's because of the great firewall of China. Not because we're getting more Chinese developers, but because they have problems accessing our servers because they're located outside of China, and so we're looking at CDNs, a content distribution network to essentially make our content available closer to China. We've found that even with one of those CDN drop points in Hong Kong, they still have problems just reaching it there in Hong Kong, and so ... and we don't want to buy or lease or rent a server in China because doing business in China is too high of a hurdle for the Foundation. 

Oh? 

You know, Microsoft and Google have to do business in China and they've got a pack of lawyers and a giant vault of money to deal with all the barriers. The Foundation does not, so it's also a hard problem to solve. We think we might be able to do it through Microsoft Azure, that they have a CDN that resides in China that Microsoft has done all that paperwork, so we're looking at that, but as far as regional things, it's not so much that we run into issues. We see Open Source communities in Europe and Brazil and Australia and Sri Lanka: none of them really have any problems because they don't have that firewall. It's not really about the Chinese people, but about the China firewall. 

That's bigger than us. And that’s not something we can fire hose.

 

We do see little engagement from Japan and Brazil, and that is partly for language reasons and partly because the Brazil community is more about Free Software than Open Source software. 

Yeah. They're very pro-FOSS.

Not OSS. But pro-free. And so, they're going to deal with the Free Software Foundation rather than the Apache Software Foundation.

I see. That’s an important distinction. 

And then you also have the Portuguese language barrier. People contributing from Europe and India, Sri Lanka, etc., they pretty much know English and that's fine. A lot of the Brazilian developers do not know English...this is the same with the Japanese Open Source developers. Japanese and Brazilian, they tend to not know English, and so that kind of isolates them from the larger Open Source world, or Free Software world, in the case of Brazil.

Would we consider localizing anything that we do, or are we going to continue as-is, as the ASF is all English?

The Infrastructure team will not translate our documents to serve those other languages. That's just too high of a bar.

There are a couple groups that have user mailing lists that are not English and that's totally fine, and Infrastructure will... well, you don't have to file a ticket anymore. It's, again, back to selfserve.apache.org: “self-serve” on Apache will create a mailing list for users communicating in Brazilian Portuguese, for example, or communicating in Japanese. But Infra doesn't do anything about that, that's just the self-serve tools. We certainly can't support non-English, and I don't think that the Foundation itself is going to make any moves towards that.

Fair enough. So a lot of companies are really struggling to accommodate their teams working from home in response to the Coronavirus and all that. These stay-at-home orders are kind of shaking companies, but from day one, the ASF has always been a virtual organization. Has anything changed with your operation on that front? Has anything impacted the ASF's day-to-day, from this pandemic?

(chuckling) Not at all. I shouldn't laugh, but no. It really hasn't changed. We've been on our team channel for all three years, three and a half years that I've been here, and the world is burning down around us, but we still sit on the team channel.

Now, that said, (Infra team member) Daniel Gruno got stranded in Canada.

Right! He’s still there?

He's still doing work from Canada. This is why when he travels to Canada for two months at a time, I don't care, you know? Because if his butt is in a chair in Denmark or in a chair in Canada, it's the same butt, so, you know...

As long as you have connectivity and a computer, you can do it. 

Right. But if he has to be offline for two months, I'd say no. Or if you want unpaid time off, well, I'm not going to pay you, of course. Certainly the discussions have changed, you know? I mean, going shopping. You know, some members are immuno-compromised and that had an effect on our team meeting that we were planning in Nashville: they were the first to say, “No way. I'm not going,” so, there’s that, but our day to day hasn't changed.

That's more of a social thing versus an operational thing. Safety first.

So the notion of, “Oh, I got to run out to the grocery store. I need to strap on a mask,” changes, but not the operation.

Right. Right. So...what do you think people would be surprised to know about ASF Infra?

I don't know if it'd be surprising, but we are global. We've got four people in the United States, one in Canada, one in Denmark, one used to be in Australia, but is now in the UK, which actually kind of hurt a little bit, because in Australia, that meant that we always had somebody in that time zone, but now we have kind of this gap of Australia/Asia time zones when...

A “Gavin” gap.

Yeah, well, I might be awake at that time, but I can't go and fix a MySQL server, so it does mean that we don't have that straight-up 24-hour coverage.

The notion that we are worldwide is kind of a neat thing about our team, and is what makes us pretty unique relative to other IT departments. I don't like being called an IT department, but that is essentially what we are. 

Surprise.

What's the name of that TV show? The one that's about IT...

“The IT Crowd”, is that what you’re referring to? The British show?

Yeah. So, you know, that's a funny show, but mostly when you think “IT department”, you think of some corporate people with button-up shirts, but ...most of us, we're in our pajamas.

Good one. What's your favorite part of the job?

I definitely like the team and that's why, nominally I'm part-time, but I'm pretty much constantly on the team channel and interacting, and so I think I just put that down as volunteer hours, where before I might work on Apache Subversion, but now I hang out with the team or I write some little tool or something like that. That's definitely been one of the more rewarding changes. Up until I started with this, I'd been a director for 15-and-a-half years, and that was kind of how I contributed to Apache. Now my work for Infrastructure is a new way to contribute to the Foundation. I'm also part of a new community, where before I would hang out with the httpd community, APR community, the Subversion people ...now it's the Infra people and my hobby time is kind of blended in with my work time, and vice versa. I mean, when your work time can also be seen as a hobby time, that's pretty cool.

I do think it's the team that makes it interesting. That's what I like the most, and that I'm working with a new, interesting community to contribute to the Foundation. 

Not only did you switch roles, you switched communities. What was your biggest challenge going into this new role?

I would say probably trying to delineate what I was going to handle for the guys and that I wasn't going to tell them what to do or how to do it. It's like, “OK, I'm here to assist, to unblock things, to enable you guys, rather than to block you or micromanage you.”

To earn that trust, that I wasn't going to be some pointy-haired boss telling them how to do their work. Now, I don't know if that was ever a problem for them, but that was certainly one of my initial concerns: how to properly create my role. This was the first time Apache's even had somebody fill in this role, so I also had to find the role, which is, again, why I came up with “Infrastructure Administrator”, is because I wanted to define it as an enabler role, as an administrator, so they could get their work done but I would not be their manager. I would not be their boss: I was simply there to enable them.

So, what are you most proud of in your infra career to date?

Ooh. I don't know. I would say by being hands-on, being the “hands” of Infra, it means that VP Infra didn't run away screaming.

David said in January 2016, maybe earlier, he was like, “No way. I'm out.” And after I was on the job for about two months, he said, “Huh. All right.”

“I'm in!”

And so I get that feedback from him, “You know, you make the VP Infra hat quite easy for me.” I think that's probably what I really like about taking on the role, is that one of our volunteers got to stay rather than drop it because it was just causing so much anxiety and pain and time and frustration. Otherwise, most of the stuff I do is really boring. Not to me, but I don't have “accomplishments”. I push paperwork, basically, so the other guys can do accomplishments.

Speaking of the other guys, how would your co-workers describe you?

I have no idea. I don't know. I really don't know. (laughing)

Where I just got done talking about what I saw as an issue, trying to frame what my role would be, it might have been fine with them and I was overly worried about it, but it’s hard for me to know. We don't do 360 reviews in Infra, so I don't get any feedback, really, from the team on what they think about myself or how I'm doing my job, so you'd have to ask them. 

I have. Just kidding. So...what are the biggest “threats” that infrastructure managers or infrastructure administrators need to watch out for? What do you think is a “big thing” that people should be aware of, or is ASF so unique that you don’t feel like anyone really experiences what you experience?

There's our capacity issue with things like Travis, but I think you're asking a different question.

I am, but that's fine. What's your greatest piece of advice? What would you tell aspiring infra administrators?

Actually, one of my greatest fears is really, as a small charitable foundation, it's hard for us to compete with well-funded corporations and some well-funded start-ups.

Related to that, I touched on it earlier, is career development ...you go into Google or Microsoft and there's a career ladder; we simply don't have a career ladder. There's salary growth. There's bonuses. If you want to have a resume or a LinkedIn profile that shows changes in growth and titles and career ladder, we can't offer that, and that's going to cut out some people. It's a very hard problem for me to solve. You know, there's things I can maybe do, but I also want to keep the team egalitarian and sort of level, rather than, “Oh, well, this guy is now the team lead.”

Given what I talked about, our social aspects, because we are all equal peers, keeping everybody with the same title, same position on the ladder means that we are peers and it's a little easier to interact that way. It's a real, real difficult problem. You ask what's scary: that's scary.

But there's a counterpoint to that. You may not have a traditional career ladder path, but to say that you've worked in Infra for Apache carries weight. That's significant. 

I believe it does, especially when you can demonstrate the hundred different types of tasks...

Well, that's exactly it. The breadth of work and the scale of what you guys do and the skill sets that you have to have and the fact that you have to play nice in the sandbox, all of it. The demand is immense, so to be able to be there and thrive and develop something from yourself in terms of a career is tremendous. Our team is exceptional. I mean, they're not expecting a linear ladder or something that others have.

You know, in other jobs, somebody might say, “I was a MySQL administrator.” Here, you're a MySQL administrator, PostgreSQL administrator… They had one role; here you've got dozens. 

If you had a magic wand, what would you see happen with ASF infra?

I'd like to solve that CI problem. The other magic wand would be upgrading our mail server from 10-year-old technology to modern technology.

Is that happening or is that literally a wish list issue?

It's happening, but it's been happening for three years. The thing is that email is so central to the Foundation that we can't really experiment with that. There are certain things we can do, but most of it, not so much, and so it means that we're being super-careful. There's about 10-12 different moving parts to it, and we're upgrading each of those a little bit by a little bit, until we can finally pull that big, scary, Young Frankenstein lever to hit the lightning bolt, you know?

Yeah: I see the visual of that.

The magic wand would be to just make that all happen and make it work. Without the wand, it's going to take another 6-12 months.

Right. What else do we need to know that I haven't asked? What should I be aware of or what should I be sharing?

Oh, I don't know. This is where my creativity ends. Ask me a coding question.

Oh no coding questions. All right. Our time has also ended. Before we go, who should I be interviewing next? 

I would say Daniel (Gruno), because his role ... he's 20-30% system administration. The rest is tool development, so that makes his role rather unique in the team.

Perfect. Thanks so much, Greg. I really appreciate it. 

= = =

Greg is based in Austin on UTC -5. His favorite thing to drink during the workday is a big 32oz cup of Diet Mountain Dew.

Posted at 02:42PM Jul 17, 2020 by Sally in SuccessAtApache  |   | 

The Apache News Round-up: week ending 17 July 2020

Happy Friday! Let's take a look at what the Apache community has been up to over the past week:

"Trillions and Trillions Served" – the feature documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019
 - Watch “Apache Innovation”, the fourth and final segment of the series https://s.apache.org/ApacheInnovation

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 19 August 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is OPEN (and free) for ApacheCon@Home taking place online 29 September - 1 October. Join us! https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.88%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 405 Apache Committers changed 3,321,417 lines of code over 3,483 commits. Top 5 contributors, in order, are: Mark Miller, Shad Storhaug, Andi Huber, Andrea Cosentino, and Gary Gregory.  

Apache Project Announcements – the latest updates by category.

API --
 - The Apache Software Foundation Announces Apache® APISIX™ as a Top-Level Project https://s.apache.org/29wd9

Big Data --
 - Apache HBase 2.3.0 released https://hbase.apache.org/

Blockchain --
 - Apache Tuweni (Incubating) 1.1.0 released https://tuweni.apache.org/

Content --
 - Apache Jackrabbit Oak 1.32.0 released https://jackrabbit.apache.org/oak

Enterprise Processes Automation / ERP --
 - Apache OFBiz 17.12.04 released https://ofbiz.apache.org/

Libraries --
 - Apache Annotator (Incubating) 0.1.0 released https://annotator.apache.org/
 - Apache BVal 2.0.4 released https://bval.apache.org/
 - Apache Daffodil (Incubating) 2.7.0 https://daffodil.apache.org/

Machine Learning --
 - Apache TVM (Incubating) 0.6.1 released https://tvm.apache.org/

Messaging --
 - Apache Curator 5.1.0 released https://curator.apache.org/

Web Frameworks --
 - The Apache Software Foundation Announces Apache® Wicket™ v9 https://s.apache.org/lpsmm
 - Apache Wicket 8.9.0 released https://wicket.apache.org/
 - Apache MyFaces Core 2.2.13 released http://myfaces.apache.org/

Did You Know?

 - Did you know that, as the world's largest Open Source foundation, the ASF is 7,800 Committers strong? https://projects.apache.org/timelines.html 

 - Did you know that the "Trillions and Trillions Served" feature documentary is now available with Chinese subtitles? https://www.bilibili.com/video/BV1Uz411i7MH 

 - Did you know that your employer can join companies such as American Express, Bloomberg, IBM, and Microsoft in matching contributions and volunteer hours made by their employees? http://apache.org/foundation/contributing.html ?

Apache Community Notices

 - "Trillions and Trillions Served" – the documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 have been released: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" short https://s.apache.org/ApacheEverywhere 3) "Why Apache" teaser https://s.apache.org/ASF-Trillions 4) “Apache Innovation” shorts https://s.apache.org/ApacheInnovation 

 - Apache Month In Review: June 2020 – overview of events that have taken place within the Apache community https://s.apache.org/June2020

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19 

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

Posted at 05:09AM Jul 17, 2020 by Swapnil M Mane in Newsletter  |   | 

The Apache Software Foundation Announces Apache® APISIX™ as a Top-Level Project

Open Source, Cloud-native microservices API gateway handles interface traffic for Websites, mobile and IoT applications in Cloud Computing, FinTech, Insurance, Marketplaces, Real Estate, Security, Speech Recognition, and Travel, among other industries.

Wakefield, MA —15 July 2020— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® APISIX™ as a Top-Level Project (TLP).

Apache APISIX is a Cloud-native API gateway used to handle interface traffic for Websites, mobile and IoT applications. The project was first developed at ZhiLiu Technology, was open-sourced in June 2019, and entered the Apache Incubator in October 2019.

"Thanks to the help of our mentors, contributors and the Apache Incubator, Apache APISIX has now graduated as a Top-Level Project," said Ming Wen, Vice President of Apache APISIX. "After entering the Apache incubator, APISIX evolved from being an Open Source project led by a commercial company to a community-led project guided by the Apache Way."

Apache APISIX consists of the following three parts:

• Data Plane, to dynamically control the request traffic, and implement traffic processing and distribution;

• Control Plane, to store and synchronize gateway data configuration; and

• AI Plane(TODO), to orchestrate plugins, as well as real-time analysis and processing of request traffic.

With more than 30 functions, Apache APISIX includes traffic control, analytics, observability, monitoring, and logging plugins. Features include:

• Dynamic routing and plug-in hot loading --particularly suitable for API management under micro-service systems;
  
  

• Built-in high availability, multiple security plugins --puts stability and security at the forefront with identity authentication and interface verification;

• Simple, powerful development interface --easy-to-use, built-in dashboard and a powerful and flexible interface for faster development;

• Designed and implemented to meet the highest performance requirements --including routing, IP matcher, JSON schema, built-in plugins, and more; and

• Multi-protocol and multi-platform support --HTTP(s), TCP, UDP,  HTTP to gRPC transcoding, Websocket, gRPC, Apache Dubbo, and MQTT proxy, as well as ARM64 and others.

Apache APISIX is in use at dozens of organizations that include Airwallex, AISpeech, api7.ai, ke.com, Qihoo 360, taikang Cloud, Tencent Cloud, TravelSky, and more.

"Congratulations to Apache APISIX!" said Ryan Cao, Principal Architect at Airwallex. "As a global fintech that is transforming the way businesses move and manage money for collections, FX and digital payments, and our financial infrastructure provides a modern tech stack for businesses of all sizes to operate internationally. We have implemented our API gateway based on APISIX, and smoothly evolved our system to a multi-cloud distributed, microservices architecture, with thanks to APISIX's highly optimised, scalable and extensible platform and support from its developer community!"

"Our cloud AI technology is open to the world through its API gateway," said Shun Zhang, Senior R&D Director at AISpeech. "We developed Kubernetes Ingress controllers based on Apache APISIX to replace the Kubernetes native Ingress to handle all north-south container clusters and part of east-west traffic. APISIX's high-performance routing, flexible plugin mechanism, API management and design concepts are just the needs of Cloud-Native architecture. I wish APISIX continued success as the best and most easy-to-use API gateway with the support of the Apache Software Foundation."

"I am very happy to see Apache APISIX flourish," said Hui Wang, Senior Engineer at ke.com. "The fast and stable adoption of Apache APISIX within ke.com confirms that APISIX is an excellent project. Congratulations to Apache APISIX and the community for successfully graduating from the Apache Incubator."

"Congratulations to Apache APISIX for graduating as an Apache Top-Level Project," said Hui Li, Engineer at Tencent Cloud. "Recent growth in demand for interconnection between mobile applications, enterprise interoperability, and the Internet of Things have expanded backend service support objects from single Web applications to a variety of usage scenarios. This increases both the access pressure and the complexity of backend services. A suitable solution for this issue is an API Gateway: in addition to basic request forwarding, protocol conversion, routing and other functions such as high performance and high stability, it also has good scalability and can continuously enhance the capabilities of the gateway. We evaluated many API gateways, and finally chose Apache APISIX as the core component of our new generation API gateway because of its high performance, high scalability, and active community. I hope to see APISIX's future development have a far-reaching impact on the microservices field."

"Congratulations to Apache APISIX for successfully graduating from the Apache Incubator," said Junteng Gao, Senior Engineer at Tencent IEG. "With the large-scale popularization of microservices, the scale of applications, the number of nodes and dependencies are growing rapidly, the demand for efficient and flexible, cloud-native API gateways is also increasing. We started to pay attention to Apache APISIX since the first version, and actively contributed to this project, so our team members were elected as committers to the project. With Apache APISIX becoming a Top-Level Project, look forward to seeing companies and developers participating and making the community more diverse."

"I am very pleased to see that Apache APISIX has graduated as a Top-Level Project in a very short period of time," said Wei Liu, Senior Technical Expert at Kuaishou and member of the Apache APISIX Project Management Committee. "Promoting Community Over Code, we encourage more developers to join the community and help us build future versions of Apache APISIX."

"Apache APISIX is a very active and diverse community, with more than 90 contributors from all over the world participating," added Wen. "We welcome those interested in getting involved with APISIX to connect through GitHub and our mailing lists, and become part of the community the Apache Way!"

Catch the Apache APISIX interview on Feathercast at https://feathercast.apache.org/2020/06/15/apache-apisix-nirojan-selvanathan/ 

Availability and Oversight Apache APISIX software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache APISIX, visit http://apisix.apache.org/ and https://twitter.com/ApacheAPISIX

About the Apache Incubator The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects enter the ASF through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/  

About The Apache Software Foundation (ASF) Established in 1999, The Apache Software Foundation (ASF) is the world’s largest Open Source foundation, stewarding 200M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 813 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 7,800+ Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, CarGurus, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Indeed, Inspur, Leaseweb, Pineapple Fund, Private Internet Access, Red Hat, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF

© The Apache Software Foundation. "Apache", "APISIX", "Apache APISIX", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Posted at 02:00PM Jul 15, 2020 by Sally in General  |   | 

The Apache Software Foundation Announces Apache® Wicket™ v9

Popular Open Source component-oriented server-side Java Web framework used to create robust Websites with faster and more maintainable code.

Wakefield, MA —15 July 2020— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, today announced Apache® WicketTM v9.0.0, the latest version of the Open Source Java framework for creating rich Websites and applications more quickly using less code.

Since 2006, Apache Wicket has been the go-to framework for elegant, responsive, and simple HTML pages that are well suited for Web Designers seeking to test the applications they are building. Apache Wicket was listed amongst the "10 Best Java Web Frameworks to Use in 2019 (100% Future-Proof)" by JavaPipe.

"Java has deeply changed in the last few years," said Andrea Del Bene, Vice President of Apache Wicket and Apache Wicket v9.0 Release Manager. "In addition to the new release policy, starting with version 9 Java platform went through a massive refactoring aimed to modularize its code base and remove legacy classes and packages. With Wicket 9 we fully embraced this new Java course migrating our codebase to Java 11 LTS, offering a fundamental tool to keep your code up to date with Java evolution."

Reflecting the tagline, "Bring the Web into the modern Java world", Apache Wicket 9 features include:

• Java 11 is required as a minimum version; CI machines are available to test the Wicket codebase with later versions. For more details see  https://wiki.openjdk.java.net/display/quality/Quality+Outreach
  
  

• Added support for CSP (Content Security Policy) and activated by default. CSP allows Web developers to protect their apps against malicious scripts and unauthorized code execution. Wicket 9 exposes CSP with a flexible API to allow custom levels of CSP.

• Wicket has been part of the OpenJDK Quality Outreach (that promotes the testing of open source projects with JDK Early Access builds and with the latest GA version, since January 2019). As of today, Apache Wicket is are one of few projects tested with OpenJDK 11, 14, and 15 (the latter in Early Access)
  
  

• The internal page storing mechanism has been reworked to implement a better and simpler design. For more details see https://issues.apache.org/jira/browse/WICKET-6563
  
  

• A new modern implementation of modal window component, called ModalDialog. The old ModalWindow component is still here but it is deprecated and will be removed in a future version of Wicket.
  
  

• JUnit has been upgraded to version 5. WicketTester uses it internally. It still supports JUnit 4 via JUnit 5 Vintage Engine

• Update CDI (Context and Dependency Injection) to version 2.0

Apache Wicket is widely deployed across numerous organizations worldwide, such as Access Canberra, Apress, Brazilian 4th Regional Labor Court, Burger King, DHL, Facturación Electrónica, Lindenbaum, OneDev, SAP, TVH Group, UK Sciences, and countless others. For a comprehensive list of Apache Wicket implementations, see https://builtwithwicket.tumblr.com/

"Apache Wicket is the most suitable framework for OneDev," said Robin Shen, owner of the OneDev project, "With Wicket I can work with the same set of code from front-end to back-end, with Java's mature libraries and toolings. I must say I gained great productivity with Wicket."

"At ParnasSys we work with very private data of millions of students in our student information system," said Robert Kromkamp, manager of software development at ParnasSys. "Since we are very keen about the security and privacy of our customers, we immediately adopted the new content security policy (CSP) feature of Wicket 9, so we can deploy an improved, more secure ParnasSys when the final release hits. Wicket has proven to be resilient and secure through the years, and we are very happy that Wicket continues to adopt new security standards in a developer friendly way."

"At ValueCare we use Apache Wicket to build our main interactive web-application, which offers our users insight into their business-data easily," said Rob Audenaerde, Technical Lead at ValueCare. "Apache Wicket is a well structured, object-oriented Java framework that allows for quick extension and customization. This reduces our time to market for new features, because we can rapidly prototype and develop new components as needed."

"With Wicket 9 we want to bring Web development into the post-Java 8 world," added Del Bene. "We rewrote our code base to comply with the new Java architecture, taking advantage of all the improvements and new features introduced from Java 8 to 11. Developers can now leave with no worry the safe harbor that Java 8 has been for all these years, and plunge into the modern Java world."

Availability and Oversight

Apache Wicket software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Wicket, visit http://wicket.apache.org/ and https://twitter.com/apache_wicket

About The Apache Software Foundation (ASF)

Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server —the world's most popular Web server software. Through the ASF's merit-based process known as "The Apache Way," more than 813 individual Members and 7,800 Committers across six continents successfully collaborate to develop freely available enterprise-grade software, benefiting billions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Indeed, Inspur, Leaseweb, Microsoft, ODPi, Pineapple Fund, Pivotal, Private Internet Access, Red Hat, Target, Tencent, Union Investment, Workday, and Verizon Media. For more information, visit http://apache.org/ and https://twitter.com/TheASF

© The Apache Software Foundation. "Apache", "Wicket", "Apache Wicket", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Posted at 12:01PM Jul 15, 2020 by Sally in General  |   | 

The Apache News Round-up: week ending 10 July 2020

We're more than halfway through the year, and the Apache community is wrapping up another great week. Let's take a look:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 15 July 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is OPEN (and free!) for ApacheCon@Home: combining ApacheCon North America and ApacheCon Europe in a new, online format! Taking place 29 September - 1 October. Join us! https://www.apachecon.com/acna2020/
 - FINAL CALL: CFP for ApacheCon@Home closes on 13 July https://www.apachecon.com/acna2020/cfp.html

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.95%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 303 Apache Committers changed 752,461 lines of code over 2,537 commits. Top 5 contributors, in order, are: Jean-Baptiste Onofré, Andrea Cosentino, Ioan Eugen Stan, Sebastian Bazley, and Paul King.

Apache Project Announcements – the latest updates by category.

Content --
 - Apache Jackrabbit 2.21.2 released https://jackrabbit.apache.org/

Servers --
 - Apache Tomcat 7.0.105, 8.5.57, 9.0.37, and 10.0.0-M7 released https://tomcat.apache.org/

Did You Know?

 - Did you know that more than 100 developers at University Hospitals Leuven, one of Belgium's largest hospitals, created software (powered by Apache Subversion, Ant, Ivy, and many Apache Java libraries) that is used across the nexuzhealth joint venture of 25 hospitals to meet their rapidly-evolving needs as impacted by the COVID-19 pandemic? http://subversion.apache.org/ | http://ant.apache.org/ | https://ant.apache.org/ivy/  

 - Did you know that those wishing to support Apache may do so with a donation towards their ApacheCon@Home registration or by sponsoring the event? https://www.apachecon.com/acna2020/ 

 - Did you know that Airflow Summit continues over the next week? https://airflowsummit.org/

Apache Community Notices

 - "Trillions and Trillions Served" – three parts of the documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 have been released: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" short https://s.apache.org/ApacheEverywhere 3) "Why Apache" teaser https://s.apache.org/ASF-Trillions 

 - Apache Month In Review: June 2020 – overview of events that have taken place within the Apache community https://s.apache.org/June2020

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19 

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

Posted at 11:42AM Jul 10, 2020 by Swapnil M Mane in Newsletter  |   | 

The Apache News Round-up: week ending 3 July 2020

Welcome, July! We've had a great week within the Apache community. Here's what happened:

Inside Infra – the third interview in the series with members of the ASF Infrastructure team.
 - Meet Greg Stein --Part II https://s.apache.org/InsideInfra-Greg2

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 15 July 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - NEW! ApacheCon@Home: combining ApacheCon North America and ApacheCon Europe in a new, online format! Taking place 29 September - 1 October, registration is OPEN and FREE, with donation options for those wishing to support the ASF. Join us! https://www.apachecon.com/acna2020/ 

- CFP for ApacheCon@Home has been re-opened. Hurry! Presentation proposals due on 13 July.
https://www.apachecon.com/acna2020/cfp.html

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 100%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – this week, 907 Apache contributors changed 1,367,670 lines of code over 3,904 commits. Top 5 contributors, in order, are: Gary Gregory, Kaxil Naik, Andrea Cosentino, Eugen Stan, and Sebastian Bazley.     

Apache Project Announcements – the latest updates by category.

Application Performance Monitor --
 - Apache Chart 3.0.0 and Python 0.1.0 released https://skywalking.apache.org/

Big Data --
 - Apache Avro 1.10.0 released https://avro.apache.org/
 - Apache Storm 2.2.0 released https://storm.apache.org/
 - Apache Kylin 3.1.0 released https://kylin.apache.org/

IoT --
 - Apache IoTDB (Incubating) 0.10.0 released https://iotdb.apache.org/

Network Client --
 - Apache Guacamole 1.2.0 released https://guacamole.apache.org/

Web Crawler --
 - Apache Nutch 1.17 released https://nutch.apache.org/

Did You Know?

 - Did you know that the following Apache projects are celebrating anniversaries this month? Three cheers to Tcl (20 years); DB (18 years); STeVe (8 years); JSPWiki (7 years); Celix and Tez (6 years); NiFi (5 years); Kudu (4 years); Fluo, MADlib, and Streams (3 years); OpenWhisk (1 year)! https://projects.apache.org/committees.html?date

 - Did you know that, as with all Apache software, registration to ApacheCon@Home is free of charge? We do have donation options for those who wish to support the ASF; thank you in advance for your participation. Join us! https://apachecon.com/acna2020/

 - Did you know that John Deere's data platform is powered by Apache Flink and Apache Spark to scalably receive and processes millions of sensor measurements per second, and adapt to continually increasing volumes of data? https://flink.apache.org/ https://spark.apache.org/

Apache Community Notices

 - "Trillions and Trillions Served" – the feature documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 https://s.apache.org/Trillions-Feature  

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache Month In Review: June 2020 – overview of events that have taken place within the Apache community https://s.apache.org/June2020

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u  

 - "Trillions and Trillions Served", the documentary on the ASF, is in post-production. Catch the teaser at https://s.apache.org/ASF-Trillions and "Apache Everywhere", the first "Trillions" "short" filmed onsite at ApacheCon Las Vegas and Berlin this past year https://youtu.be/nXtIti9jMFI

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - ASF Operations Summary: Q2 FY2020 (August - October 2019) https://s.apache.org/2kv2n

 - ASF Founders look back on 20 Years of the ASF https://blogs.apache.org/foundation/entry/our-founders-look-back-on

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - ApacheCon: Tomorrow's Technology Today since 1998 http://s.apache.org/ApacheCon

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

Posted at 11:25AM Jul 03, 2020 by Swapnil M Mane in Newsletter  |   | 

Apache Month in Review: June 2020

Welcome to the latest monthly overview of events from the Apache community. Here's a summary of what happened in June:

New this month --

 -"Trillions and Trillions Served" – the feature documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 https://s.apache.org/Trillions-Feature

 - ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
  -- Announcing ApacheCon @Home 2020: ApacheCon North America and Europe have been combined and will be held online 29 September - 1 October 2020. Join us! https://apachecon.com/acah2020

 - "Inside Infra" --a new interview series with members of the ASF Infrastructure team
  -- Meet Greg Stein --Part I https://s.apache.org/InsideInfra-Greg and Part II https://s.apache.org/InsideInfra-Greg2

 - ASF Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19

 - Apache Month in Review: May 2020 https://s.apache.org/May2020

Important Dates --

 - Next Board Meeting: 15 July 2020. Board calendar and minutes http://apache.org/foundation/board/calendar.html

 - ApacheCon @Home 29 September - 1 October 2020

Infrastructure --

In May, 860 Apache Committers changed 19,454,137 lines of code over 16,319 commits. The Committers with the top 5 highest contributions, in order, were: Gary Gregory, Jean-Baptiste Onofré, Sebastian Bazley, Andrea Cosentino, and Claus Ibsen.

Project Releases and Updates --

New releases from Apache Archiva (Build Management); Beam (Big Data); Calcite (Big Data); Commons IO (Libraries); Commons BCEL (Libraries); Curator (Messaging); CXF Fediz (Libraries); Flink (Big Data); Fortress (Identity Management); HttpComponents Client (Servers); HttpComponents Core (Servers); Hudi (Big Data); Jackrabbit (Content); JSPWiki (Content); Libcloud (Cloud Computing); NetBeans (Integrated Development Environment); PDFBox (Content); Pulsar (Messaging); Qpid (Messaging); ShardingSphere (Big Data); Skywalking and Nginx (Application Performance Management); Tomcat (Servers); Traffic Control (Servers).

The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. Congratulations to Apache Hudi, which graduated as a Top-Level Project this month https://s.apache.org/odtwv. Welcome to Apache Pegasus (incubating) as the latest podling to enter the Incubator! We invite you to review the many projects currently in development in the Apache Incubator http://incubator.apache.org/   

# # #

To see our Weekly News Round-ups, visit https://blogs.apache.org/foundation/ and click on the calendar in the upper-right side (published every Friday) or hop directly to https://blogs.apache.org/foundation/category/Newsletter . For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. We appreciate your support!

Posted at 01:33PM Jul 01, 2020 by Swapnil M Mane in Newsletter  |   | 

Inside Infra: Greg Stein --Part II

The "Inside Infra" interview continues with ASF Infrastructure Administrator Greg Stein, who shares his experience with Sally Khudairi, ASF VP Marketing & Publicity.

How or what would you describe the Infra "brand" to be?

I don't really know. I've never really thought about branding or marketing ourselves, so ...

Well, you guys have a certain persona, you have those funky t-shirts you wear at ApacheCon ...there's definitely some kind of street cred that's different from everybody else. I was curious to see if that's part of your natural sense of hip, or is that something that you guys deliberately planned for.

The t-shirts and other things go back to the team bonding kind of thing. We'll give ourselves an identity, but haven't tried to create or market ourselves. I think it is something that we do need to take some control over. We hired a part-time writer in December and he's been organizing our content to provide a better and more useful front to Infrastructure.

There were a lot of pages on www.apache.org that have now moved over to infra.apache.org. That creates a more coherent Web space, if you will. We can really talk about those different channels. "How do you reach Infrastructure? Do I go to the Slack channel or do I file a JIRA ticket: how do I decide?" So he's helping to, while I wouldn't say "market a new face", he's certainly helping people figure out who we are, what we do, what we can help with and getting that information organized.

Which is good. That's new. Even to have you guys featured in a project like this, it's unusual and it's refreshing. I'm personally curious, and I'm sure other people are also curious about what's behind Infra.

Right, right. Who are these crazy guys spread around the world that are keeping 200 machines up and running for all these different projects and committers and contributors?

So Andrew (technical writer Andrew Wetmore) is primarily going to work on the infrastructure docs until those are whipped into shape because a lot of the material that we have, a lot of the Webpages, is really infrastructure related. He has been working with the team on those pages. What's going to be harder though is when he's kind of at a stopping point for that, what to turn his focus to, and that would be www.apache. But then it gets a lot more difficult because when he wants to update the How It Works page, who does he talk to? Who's authoritative? He can do some edits for flow and word consistency, punctuation, clarity, right, but he can't really update the process.

Right. Right. That's the Foundation thing.

Yeah. But the problem is we don't really even have a concept of who's in charge of that How It Works page, who is, you know, it's just there's nobody that the foundation is willing to say, "That person controls that process." You know what I mean?

I totally do --I come across the same pages and people go, "Are they yours?" It's hard to determine not only evolving processes, but who signs off on this or who gets it. I hear you.

I've recommended for the past year, or three, that Marketing is the owner of DubDubDub (www.), but you know, that's the "face" of Apache. You know? But the raw content, as you point out, who approves the raw content.

One thing that I asked Drew and Chris, and I'm always curious with people who are super busy and juggling 50 things, is to describe a typical workday for you.

I wake up, I look for email first, generally, sometimes I'll hop onto Slack because sometimes people ask me directly for something. Then I go look at email and sort through a number of different categories between direct team stuff, operations, the Apache Board, and then Apache in general. And then of course, if there's any vendor email to deal with. So there's a bunch of different categories in priority order. After I get through that initial work, then it's go and read all the back scroll in the team channel, which is anywhere from 200 to 400 lines of back scroll ...

Can you get any work done? Beyond just catching up on the communications?

Yes. But it does take like 30 minutes to read that back scroll. For me there's a lot in there about what the guys are doing and what they're working on, how to solve a particular problem when they're asking somebody else, "Hey, can you look at this? Can you help me with this?" But I don't, for the most part, "serve", you know ...they are the technical staff... I can do it: I have technical chops, but I let them do their jobs as they know best. I do like reading the back scroll because I'm also looking at it from the angle of "how is the team working together? Is that going well? Is there something that I need to poke and prod to improve how they're working? Are they getting jammed up on something that I can unblock for them so that they can get their work done?"

Stuff like that. That's what I look for when I go through that back scrolling, so it's important to me to read that back scroll. Most of the guys do tend to, when they first sign in in the morning, go back and scan for stuff where they might be needed. I've never really asked them how detailed they get, but I think pretty much everybody reads all of it to catch up, but they're going to be looking at it with a different lens than how I look at it. Mostly I'm looking at unblocking --are they running into problems that I can ease for them?

How do you keep your workload organized?

I don't.

Fair enough. Again, there's a lot, so it's curious to me, like everything at Apache, with the exception of a handful of things, everything could be a priority, if you're always on fire and always running around, putting out fires, you know? It's funny when I've talked to the Infra guys and you also, you all have the same reaction to that question, which is the laugh. I think that's the nature of the beast with the ASF.

Yes. That really is the nature of system administration work. My career has been product development, and you can reasonably plot that out. You can say, "We're going to develop these five new features, which is going to take us between two and four months." We'll see...we might cut a feature to try and limit our time development. The feature is going to change, unless we'll plan in time for change. But system administration is very reactive, so it's a very different beast. This is where, like I said, we were kind of treading water with four people, but we could see as Apache was growing we were not going to be able to keep up. And we certainly weren't going to be able to move ahead of the curve and do things like selfserve.apache.org where, you know, before we would get a dozen tickets to create repositories and that took time. Now we don't have to do anything.

It's all selfserve.apache.org, but we had to write the tool first and have enough air time to get that tool written. So I think we're ahead of the curve. We're getting some of our longer-term initiatives done, but it is still a very reactive thing. For myself, my back office work is pretty straightforward and it's a lot of email and Website work, you know, going in, paying an invoice, putting in the infrastructure credit card, sending out a purchase order, stuff like verifying and improving payroll, that doesn't require me sitting down and writing Python scripts.

The other half of my job is being present on that channel because I also help to set priorities. When something comes up, I ask, "Is this a thing that we want to do? Do we want to take on this new task? Do we want to provide this new tool to the projects?" You know, like a project is going to say, "Well, we want to integrate this thing into our GitHub repository," and we go and review it. It may require permissions that we simply don't want to allow. So there's some of those kinds of policy kind of things that I also help with. And there's always being present to help set policies and priorities.

OK... so how do you work with (VP Infrastructure) David Nalley? Are you making the decisions? Infra is an unusual type of group as opposed to other areas of activity operationally at the ASF. How do you work together?

Correct: I'm the day-to-day, so I look at it like he's the brains and I'm the hands. That said, he's like the strategic brain and I do all the tactical decisions.

I make all the tactical decisions. I am an officer of the corporation. I can make any decision that I need to, related to Infrastructure. If I feel it's a little bit weird, then I'll bounce that off David, but for most of the stuff, he doesn't feel a need to inject himself in. He feels comfortable letting me go ahead and run with the things, and rely upon me asking when it seems a little sketchy.

That's good: that process suits both of your personalities, both your sensibilities. It sounds like a good fit.

I report to the VP of Infrastructure, and that is still David, even though he became Executive VP and is now (ASF) President. He still holds that title. He's asked me, "Well, Greg, maybe you should just be VP Infra," and I said, "No way." Because we're paid people, but the Foundation is all volunteers. I told him I do not want to be a VP, because I want to report to a volunteer. I think that I (and the team) should report to a volunteer that always has a volunteer eye on the Foundation's long-term goals.

Because I manage all the day-to-day, it's a very lightweight hat for him. That VP hat is a tiny aspect compared to his President hat. One day, he'll find somebody to take over that VP Infra hat, but I've essentially mandated to him that it has to be a volunteer position.

It's not that I see we're going to go all out of control and we need a check from a volunteer; I just want a volunteer to always be able to say, "Okay, you guys are a little bit crazy, let's redirect our long-term thinking more in line with what the Foundation wants," and have a volunteer interpret what the Foundation wants.

That perfectly dovetails into what folks referred to in our ("Trillions and Trillions Served") documentary, where they were talking about Greg Stein's famous "plan for the ASF for 50 years..." This super long-term vision, which again, everyone goes back and says, "Greg Stein said..." What does that mean exactly, and how does that translate to Infra, considering that you can't really plan that far out? How does that work?

Well, actually we can plan that far out. I wrote that "50 years" in one of my Director's statements, I think it was 2014 or 2012 ...maybe earlier. Where I was going in that Director statement was the Board doesn't deal with the communities. The Board is there to support the communities. So we want the Foundation to exist for 50 years so that these communities can continue to run and see through evolution.

Some communities are going to move to the Attic, new ones are going to come along, but we want the Foundation to be viable. To say "forever" is okay. Nobody can really put that in their brain. So I just said, "OK, we can think what 50 years means." That is long enough out, but still within people's brain capacity to think, "Okay, what _does_ 50 years mean?"

And so that's where I came up with that. What does the Board need to think about to ensure that we are here 50 years from now and our projects are successful and can run through their lifetime, lifecycles. Apache HTTP and Tomcat, I don't think they are ever going to go away, but you could see maybe in 30 years they might. There might be some other mechanism in computing that would obsolete them, but the model of Apache does need to exist for at least that long.

Now, within Infra, I think we actually can plan that far out because we have growth curves. We see what kinds of computing resources people need. So we can plan for project growth, for machine growth. We can do long-term planning on how we allocate machines among our various cloud resources that we have, and start to schedule those further out. None of that really affects our day to day, but it is something that we can project out a ways and think about what kinds of resources we are going to need two, three, five years from now.

There isn't anything really that we can do for 50 years, but we can keep it in mind. Okay, that is going to be a larger team. That is going to need a larger staff, a full time manager, a full time HR person, a full time... There's different things that will change over that time, but we can actually do some of that projection, although we haven't bothered.

I do the five year plans for the Board, but mostly that is a simple cost growth as opposed to actually changing the structure of the team or the role assignments, because like I said, I think probably within 10 years, we'll probably need to add one or two more staff on top of the head count of six that we have right now. And I think supporting that would still be fine for a part-time person like myself. But once it grows to 10 or 12, then I think it's going to need a real change. Where we need to have a full-time person managing and so, we'll need to adjust the budget considerably to make that happen.

But if we ever get there, the Foundation is going to be likely in a very different position. We're talking 10 years from now. And so, who knows.

So with more than 350 projects and initiatives as we've discussed before, how do you guys stay ahead of the demand? And again, if you're trying to plan for five, ten years out, you mentioned earlier cloud computing. Not so long ago, cloud computing was a novelty. How do you plan for this?

And that is where we try and move more things to selfserve.apache.org, where we look at the kinds of requests that we're getting. The kinds of tasks that we’re performing and find a way to automate that workflow and create more self-serve options for the kinds of tasks that we regularly get tickets on.

Where we used to get tickets on creating Git repositories, we get zero now and, and we can see over the past six months, we've had 20 tickets to do X, is there a way that we can automate that, so we don't have to get our hands on that ourselves and save our hands for doing things like machine upgrades, for rebalancing some of our computer resources, where things are running on an old operating system and we need to get that onto a newer version. Right now, all of our machines are managed by a system called Puppet, which does the basic configuration work for us. But today, we're on two different versions of Puppet, a really old one and a reasonably new one.

And we're trying to get everything migrated off the old stuff onto the new but once we finished that migration, we're going to have to start all over again, or maybe switch to a different tool. We're looking at a tool called Ansible to use instead of Puppet.

And so there's this never-ending ongoing set of tasks, but each time we do it, it reduces our workload by that much more. So when we upgrade from Puppet 3 to Puppet 6, we get an improvement in the maintainability of that server. And that means that we spend less time with that server going forward and have more time to do other things or to deal with project growth.

Regarding a scale of efficiency, how do you close your skills gaps? When I spoke to Chris and Drew, they both said, "We do everything." How do you do that? How do you know all of this? Do you look at this big picture and say, "Okay, we need a person to specialize in X and Y and Z," and then you send them out to learn about it? How do you cope with that?

The team definitely specializes. And the guys have specializations around different areas, but we do a little bit of cross training, but not a lot because as I mentioned, we've got like 200 machines, each individually doing their own thing. If we cross trained everybody in everything, we'd get nothing done. So, there's a little bit of cross training, but mostly some specialties. It does create a little bit of bus factor...

Which is very scary. I was just going to say, your bus factor is very scary. Talk about that.

The thing is that Puppet allows us to create configurations and that's in version control. If all of a sudden somebody leaves, another person can backfill them because if somebody leaves, it's not like they take their work with them: all the work is in version control. And so that work doesn't go with them, but we may need to backfill some education on that particular specialized area. For example, Chris (ASF Infra team member Chris Thistlethwaite) does a lot of our monitoring work. If he left, now we need somebody to get a little more familiar with NodePing and a little more familiar with Datadog, but that'll be like a week for somebody to pick that up.

It wouldn't be, "Oh my God, this is three years of expertise that we need to go backfill" ...we don't have anything that is that highly specialized.

Is that because the team is more well rounded or because you guys are more efficient or what about it? Because of technology evolution, or...

We don't deal with systems of that level of complexity. We've got 200 machines, like I said, each doing their thing, but it's not like we've got a cluster of 200 machines all trying to coordinate to create one particular outcome. It's, here's my SQL server, here's a JIRA server, here's a Puppet server. Things like that, where the amount of technology is pretty small in each little pocket ... but we just have a hundred pockets on our pants.

[END OF PART II]

Posted at 02:53PM Jun 29, 2020 by Sally in SuccessAtApache  |   | 

The Apache News Round-up: week ending 26 June 2020

Farewell, June --we're wrapping up the month with another great week. Here are the latest updates on the Apache community's activities:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 15 July 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Notice on Apache 2020 Conferences https://s.apache.org/zgm8m 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 100%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – this week, 917 Apache contributors changed 3,498,501 lines of code over 3,692 commits. Top 5 contributors, in order, are: Manfred Moser, Sebastian Bazley, Andrea Cosentino, Gary Gregory, and Claus Ibsen.      

Apache Project Announcements – the latest updates by category.

Application Performance Monitor --
 - Apache SkyWalking 8.0.1 and Nginx LUA 0.2.0 released https://skywalking.apache.org/

Big Data --
 - Apache Calcite Avatica 1.17.0 released https://calcite.apache.org/avatica

Build Management --
 - Apache Archiva 2.2.5 released https://archiva.apache.org/

Libraries --
 - Apache CXF Fediz 1.5.0 released http://cxf.apache.org/fediz.html

Messaging --
 - Apache Pulsar 2.6.0 released https://pulsar.apache.org/

Servers --
 - Apache Traffic Server 8.0.8 and 7.1.11 released https://trafficserver.apache.org/

Did You Know?

 - Did you know that Apache Cordova released OSX 6.0.0? https://cordova.apache.org/ 

 - Did you know that Apache Royale released the new, nifty Tour De Jewel component to demonstrate progress on pages? https://royale.apache.org/ 

 - Did you know that the Python agent for Apache SkyWalking is in development? http://skywalking.apache.org/ 

Apache Community Notices

 - "Trillions and Trillions Served" – the feature documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 https://s.apache.org/Trillions-Feature  

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache Month In Review: May 2020 – overview of events that have taken place within the Apache community https://s.apache.org/May2020

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u  

 - "Trillions and Trillions Served", the documentary on the ASF, is in post-production. Catch the teaser at https://s.apache.org/ASF-Trillions and "Apache Everywhere", the first "Trillions" "short" filmed onsite at ApacheCon Las Vegas and Berlin this past year https://youtu.be/nXtIti9jMFI

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - ASF Operations Summary: Q2 FY2020 (August - October 2019) https://s.apache.org/2kv2n

 - ASF Founders look back on 20 Years of the ASF https://blogs.apache.org/foundation/entry/our-founders-look-back-on

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - ApacheCon: Tomorrow's Technology Today since 1998 http://s.apache.org/ApacheCon

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

Posted at 09:35AM Jun 26, 2020 by Sally in Newsletter  |   | 

The Apache News Round-up: week ending 19 June 2020

Happy Friday! Let's take a look at what the Apache community has been up to over the past week:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 15 July 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Notice on Apache 2020 Conferences https://s.apache.org/zgm8m

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.72%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – this week, 902 Apache contributors changed 5,499,342 lines of code over 3,942 commits. Top 5 contributors, in order, are: Chunen Ni, Sebastian Bazley, Rupeng Wang, Gary Gregory, and Andrea Cosentino.      

Apache Project Announcements – the latest updates by category.

Cloud Computing --
 - Apache Libcloud 3.1.0 released http://libcloud.apache.org/

Servers --
 - Apache HttpComponents Client 5.0.1 GA released https://hc.apache.org/
 - Apache Traffic Control 4.1.0 released https://trafficcontrol.apache.org/

Did You Know?

 - Did you know that you can meet Apache APISIX (Incubating), catch up with Apache CloudStack, see what’s next with Apache HBaseas the project celebrates its 10th Anniversary, and more? Only on Feathercast --the voice of the ASF https://feathercast.apache.org

 - Did you know that Tencent uses Apache Pulsar to process tens of billions of dollars in financial transactions each day? http://pulsar.apache.org/ 

 - Did you know that Apache Cordova has a major release for iOS? https://cordova.apache.org/ 
 

Apache Community Notices

 - "Trillions and Trillions Served" – the feature documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 https://s.apache.org/Trillions-Feature  

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache Month In Review: May 2020 – overview of events that have taken place within the Apache community https://s.apache.org/May2020

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u  

 - "Trillions and Trillions Served", the documentary on the ASF, is in post-production. Catch the teaser at https://s.apache.org/ASF-Trillions and "Apache Everywhere", the first "Trillions" "short" filmed onsite at ApacheCon Las Vegas and Berlin this past year https://youtu.be/nXtIti9jMFI

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - ASF Operations Summary: Q2 FY2020 (August - October 2019) https://s.apache.org/2kv2n

 - ASF Founders look back on 20 Years of the ASF https://blogs.apache.org/foundation/entry/our-founders-look-back-on

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - ApacheCon: Tomorrow's Technology Today since 1998 http://s.apache.org/ApacheCon

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

Posted at 05:46AM Jun 19, 2020 by Swapnil M Mane in Newsletter  |   |