Entries tagged [foundation]

Friday September 18, 2020

The Apache News Round-up: week ending 18 September 2020

Greetings everyone --it's time to review the Apache community's activities from the past week:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 21 October 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is open (and free) for ApacheCon@Home 29 September - 1 October https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.89%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 382 Apache Committers changed 3,359,513 lines of code over 3,557 commits. Top 5 contributors, in order, are: Mark Thomas, Jarek Potiuk, Andrea Cosentino, Claus Ibsen, and Shad Storhaug.  

Apache Project Announcements – the latest updates by category.

Big Data --
 - Apache Kylin 4.0.0-alpha released https://kylin.apache.org/

IDE --
 - Apache NetBeans 12.1 released http://netbeans.apache.org/

Identity Management --
 - Apache Syncope 2.0.16 and 2.1.7 released https://syncope.apache.org/

Libraries --
 - Apache MXNet (Incubating) 1.7.0 released http://mxnet.incubator.apache.org
 - Apache Commons Daemon 1.2.3 released https://commons.apache.org/proper/commons-daemon/

Messaging --
 - Apache Qpid Dispatch 1.14.0 released https://qpid.apache.org/

Servers --
 - Apache HttpComponents Core 5.0.2 GA released https://hc.apache.org/
 - Apache Tomcat 8.5.58, 9.0.38, 10.0.0-M8 released https://tomcat.apache.org/


Did You Know?

- Did you know that ApacheCon@Home features keynotes by Jonathan Ellis of DataStax, Sam Lightstone of IBM, Gian Merlino of Imply, Anil Inamdar of Instaclustr, Thomas Huang of NASA Jet Propulsion Laboratory, Edmon Begoli of Oak Ridge National Labs, Kim Huang of Red Hat, Sheng Wu of Tetrate, Camille Fournier of Two Sigma, and Catherine McGarvey of VMWare? Sign up today https://hopin.to/events/apachecon-home   

- Did you know that Solr is de-coupling from Apache Lucene to be an independent Apache project? Learn more from Atri Sharma, as interviewed by Rich Bowen, exclusively on Feathercast https://feathercast.apache.org/2020/09/01/apache-solr-and-lucene-atri-sharma/

- Did you know that recent entries into the Apache Incubator include BlueMarlin (Data Streaming), Liminal (Machine Learning), Pegasus (Big Data), and Sedona (Geospatial)? http://incubator.apache.org/  


Apache Community Notices

- Apache Month In Review: August 2020 https://s.apache.org/Aug2020

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg , Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3 | Daniel Gruno --Part I https://s.apache.org/InsideInfra-Daniel1

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Friday September 11, 2020

The Apache News Round-up: week ending 11 September 2020

Happy Friday! Let's take a look at what the Apache community has been up to over the past week:

Inside Infra – the interview series featuring members of the ASF Infrastructure team.
 - Meet Daniel Gruno --Part I https://s.apache.org/InsideInfra-Daniel1

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 16 September 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is open (and free) for ApacheCon@Home 29 September - 1 October https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.93%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 396 Apache Committers changed 2,112,475 lines of code over 3,612 commits. Top 5 contributors, in order, are: Daniel Gruno, Mark Miller, Nick Vatamaniuc, Robert Newson, and Andrea Cosentino. 

Apache Project Announcements – the latest updates by category.

Big Data --
 - Apache Ranger 2.1.0 released http://ranger.apache.org/
 - Apache Accumulo 1.10.0 released http://accumulo.apache.org/

Content --
 -Apache Jackrabbit 2.12.11 and Oak 1.34.0 released https://jackrabbit.apache.org/

Database --
 - Apache Geode 1.13.0 released http://geode.apache.org/
 - Apache ZooKeeper 3.6.2 released https://zookeeper.apache.org/

Libraries --
 - Apache OpenJPA 3.1.2 released http://openjpa.apache.org/
 - Apache Commons IO 2.8.0 released https://commons.apache.org/proper/commons-io/
 - Apache OpenWebBeans-2.0.18 released http://openwebbeans.apache.org/

Servers --
 - Apache Tomcat Native 1.2.25 released https://tomcat.apache.org/

Web Conferencing --
 - Apache OpenMeetings 4.0.11 released https://openmeetings.apache.org/

Did You Know?

- Did you know that ApacheCon@Home keynoters include
  Edmon Begoli, Director of Scalable Protected Data Facilities at Oak Ridge National Laboratory;
  Ben Bromhead, CTO and Co-Founder at Instaclustr;
  Jonathan Ellis, Co-founder and CTO at DataStax;
  Camille Fournier, Head of Platform Engineering at Two Sigma;
  Kim Huang, Content Strategist at Red Hat;
  Thomas Huang, Technical Group Supervisor and Strategic Lead for Interactive Analytics at NASA Jet Propulsion Laboratory;
  Sam Lightstone, Chief Technology Officer for AI Strategy at IBM; and
  Catharine McGarvey, Development Engineering Lead for Tanzu at VMWare? https://www.apachecon.com/acah2020/tracks/keynotes.html

- Did you know that Apache Druid are holding a series of MeetUps in September? http://druid.apache.org/ 

- Did you know that Apache MXNet (incubating) was listed among the best machine learning and deep learning libraries for building and training machine learning and deep learning models? https://mxnet.apache.org/


Apache Community Notices

- Apache Month In Review: August 2020 https://s.apache.org/Aug2020

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg , Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Monday September 07, 2020

Inside Infra: Daniel Gruno --Part I

The fourth interview in the "Inside Infra" series with members of the ASF Infrastructure team. Meet Daniel Gruno, who shares his experience with Sally Khudairi, ASF VP Marketing & Publicity.




"...companies are not the same as ASF. They don't have 300 different departments that all have their own little tools that they want working in their specific way. And they want this to connect to that, and that's connected to some other thing. We are not afraid to create custom solutions, we're not afraid to get our hands dirty and we're not afraid to make mistakes."



What is your name and how is it pronounced? I have my official name and I have my user name and people usually ask about both of them. My name is "Dan-yell Gkhroo-no" or I will accept "Dan-yell Groo-no" which is as you read it in English. It's actually a Dutch name. So you would pronounce it "Hrooy-no" in Dutch, which I'm not even going to try to phoneticize that because, that's, well, Dutch. And my username is "Humbedooh" which is an onomatopoeia that I randomly made up in 2004 for a game called World of Warcraft, where you need a username for this character that you create. And I think I had just listened to "New York, New York", where Frank Sinatra sings "scooby doo bee doo", and I was like, "hum-be-doo-de-doo" and the name just came to me and it stuck ever since. And so for the past 15 years or 16 years, I've been primarily "Hum-beh-doo" online. By the way, Frank Sinatra sings "zoo-bee-doo-bee-doo", not "scooby-doo-bee-doo" in "Strangers in the Night", but I like your version better. Okay. Well today I learned that. When and how did you get involved with the ASF? That goes back to 2010, 2011? Again, this beautifully tied us into World of Warcraft because in that game you can make modules, add ons for the game that will do nifty things, like add ons for a Web browser. And this is written in a programming language called Lua, L-U-A, which is Portuguese for "moon". And so I started writing some programs for this game and I had great fun with it, and programing is not my official trade. I was educated in, or studied, human resource management at university actually. But it was my hobby and I had great fun doing it. And this Lua thing just got stuck in me. And then five years later or so I started writing a program for the Apache Web server called mod_pLua, the best way to describe it as if PHP and Lua had a baby. So it would be the same for people that know PHP. It would be the same structure with the less than equal sign and a question mark, and then the same thing to end it on the other end, but with the Lua language instead of the PHP language. So I wrote this program or interpreter for the Apache Web server. And I didn't really think much of it. Obviously it was mostly for my own edification if you will, and for my own use. But I had put this on a site called SourceForge, which at that time had a community manager named Rich Bowen (also Apache HTTP Server PMC Member) who took a liking to this program or this module for the Web server because the Apache Web server community, which he was a part of at that point, have been doing something similar called mod_lua or at that time mod_wombat. And that had stalled. People have interests and then the interests wane and people would move on to new jobs and the person in charge of this mod _lua had found other interests in life. And so this module was just sitting there and not really being worked on. And Rich said, "Why don't you come take a look at this program and maybe this is a place where we can collaborate." And he also got (ASF co-founder and Apache HTTP Server PMC Member) Jim Jagielski very interested in the work I was doing. And so I slowly started on my path to becoming an ASF Committer initially by fixing what's called 404s, which is basically a reference in a Webpage to a link or another page that doesn't exist. Either it never existed or it doesn't exist anymore. So I started fixing a bunch of those just to get on their good side and hopefully they would take me seriously. And I didn't have high hopes, but I think I was probably the fastest person to get committership at the Apache Web Server Project...perhaps the fastest in the 10 years preceding when I got it probably within a week. They had a vote going and I was voted in and… Within a week? Within a week. Unheard of. I was pretty much on the path to becoming a Committer. I couldn't believe it. Part of me wanted to believe it, because it was a very big validation for me. Because I had been using the Apache Web Server since 1998 and it always been a project that I looked up to and it had been this mythical "Father of the Web” program. And so to actually be a part of it and get your name on the page that says these are the Committers that actually have a say in the project and can commit code to it, that was I was quite a feat for me, especially at that time where I had stopped my studies at university and I didn't know, what am I going to do now? Because as happens with a lot of people that study something, they eventually found out that while, okay, this was interesting, but it's probably not what I want to be doing, if I'm honest. Because what I had fun with was programming. So while it was nice knowing a lot of stuff about statistics and economic models and psychology and so forth, it had started to get a little boring for me. I knew these things, what now? And so to get this validation to get an avenue of sorts where I could use my creativity in a new way that I hadn't studied for, but it naturally just came to me this programming inspiration, that was really nice, to use a very vague word. It was a tremendous opportunity for me. And then that's how I got started with Apache. Fantastic. You’re not only a Member of the ASF and an Infra team member. What other "hats" do you wear at the ASF? I have a couple of hats. I’m also the Vice President of the Apache Web Server Project, which is a great honor. And it's still to this day, three years in, fun to do. People think of it as this is a dictator role or you get to decide, but it's more of a glorified secretary really, where you keep tabs with everything or most of the things that are going on in the project. And you relay that information in a concise way to the Board of directors, whose job it is to look at these reports and say, "Are the projects doing okay? Do they need any help from us? Are they in trouble?" So basically VP is the watchdog --in these COVID days, I guess you can say it's a pulse oximeter of the project. And if you want to know if a project is still healthy and stable and progressing, the VP is the one to ask, because that's basically their job to know. As VP I don't get to decide who gets in or who gets kicked out or what direction we take in the project: I am just the person that ensures that the Board knows that the project is in good health. Do you wear any other hats or is it just the VP of Apache HTTP Server? I'm also VP of Apache STeVe. As I said, I have two VP-ships and STeVe is a whole other beast. Let's say it's very stable in that we have a code base that works and we don't really do much about it, we maintain it. In the Apache Web server, we have around 20 to 30 people actively contributing code every single quarter. And in Apache STeVe, we are basically twiddling our thumbs, waiting for something bad to happen. And it never happens. We have a program that works the way we like it. And we don't see the need for any large changes. And as long as there is sufficient oversight in a project, then the Board doesn't come in and say, "Hey, can you make this cool feature?" Because that's not the Board's job. The Board’s job is just to help us, as projects. And so if the project doesn't have anything that it feels it wants to add, but it's still there and the people are alive and well, then the Board will say, "You got it. We'll see you next quarter." And so two projects are very different and it also makes for very different reports. OK. Let’s drill into Infra, as that’s the focus of our interview series. How long have you been a member of the Infrastructure team? How did you get there? I am not sure. I think I've been a member of the Infrastructure Team since 2012. You can probably figure out when exactly I got my membership in the email archives . It started because the Apache Web server project needed a commenting system. Because we had been eyeballing the PHP project and they had a system where you could, on the documentation pages, you can enter, I have a comment about this documentation bit, or you could add some code snippets or ask a question and get an answer. And the only thing we had was send an email and get a reply and then the next person comes along and doesn't know that that email existed and sends the exact same question and gets the exact same reply and that can get tiresome in the long run. So we wanted someplace inside the documentation itself where you could go in and see, okay, I have an issue with this documentation, have other people encountered the same problem or are there some smart solutions that I can find here. And this type of software doesn't write itself, unfortunately. So I set about in writing that using the mod_lua that I had now invested a great deal of time in because A, we needed a comments system and B this was a good excuse to show off mod_lua in a production system. This could really do something, it's not only fast, but it's got a lot of features and it's got a lot of flexibility to it. And so I asked the Infra Team, which at that point was very daunting for me because they were, let's say our image has improved over time at Apache Infra, it was much more a, well basically an operator from hell vibe you got back in the early 2000s or early 2010s from the Infra Team, especially when you're someone of a more timid nature like I am. So anyway, I asked if I could get a place to set up a machine or borrow a Web server basically and put this commenting system on that I had been writing as a hobby. And they pretty much said "Sure." Which was surprising to me because normally when you go and ask for something at a company and it's very difficult, you can ask for meetings and meetings and meetings, but if you ask for actual resources, you will usually have to file a form J/99-B in triplicates and whatnot. And here they were just: “well it looks like he wants to help the project, just give him what he wants”. And so I got started on this commenting system. And other projects became aware of it and they wanted to use it as well. And then I became the comments guy, basically. And I started maintaining this system for, I think, it was seven different Apache projects at the time using it. And since you can't really maintain anything at the ASF without somehow being an infrastructure person, I was made an infrastructure person...and generally if you're a given something, you get a taste for it and you want more. And so I started volunteering for more and more infrastructure tasks. And then I became what is called infrastructure root. This was about two years later down the road. Which is a point where the Infrastructure Team says we have complete trust in what you do. Here are basically the keys to the kingdom. Do whatever you like, except don't do that. But you could do whatever you wanted to. And that was almost as awesome as becoming a member, which I had become just about a month prior. It needs to be said that at that point you could not become infrastructure root unless you were also an ASF member, because needless to say, when you have root access to an organization as wide and important as to the ASF, you get to be privy to a lot of information that you should keep to yourself. And so the logic at the time was, if you are an ASF member, you will already have access to most of this information because of your membership and so we can allow you to become an infrastructure root person. This has changed since then, we have cast a wider net when looking for new infrastructure people, this also includes a more thorough vetting process that we have now. So we feel more secure and not just requiring you got to be an Apache member before you come and help us. So we are able to look for a broader set of requirements that might not have been found within the, at that time, 400 and something members that were in the Foundation. What are you responsible for in ASF Infrastructure? Oh God. As with most infrastructure members, it's almost easier to see what are you not in charge of, which I usually say “Jenkins” with a big smile because that's things that are, I know this is going to sound silly to a lot of people reading the article, but things that are Java, I tend to shun like a vampire and sunlight. Any particular reason? Yes. I'm not accustomed to the way the output and stack traces and core dumps. And the thing about Java is it's very verbose: you can write 50 lines of code and you'll have a print Hello. And it doesn't appeal to me. So yeah, when things don't appeal to me immediately, this is one of my weaknesses, I try to not really understand it because it's easier not to. Fortunately we have some very talented people at the Infrastructure Team that knows pretty much everything there is about Jenkins and JIRA and Confluence and all the other big Java powered mod lists we have at the Foundation so I can spend my time elsewhere. What I mostly do at the Foundation day to day work aside -- because we all have basic maintenance tasks and disasters that can crop up from time to time -- is product development of the glue that binds The Apache Software Foundation together and its software infrastructure. And I'll tell you about a new thing that we've been doing, which is something called PyPubSub. I can spell it, it's P-Y-P-U-B-S-U-B, so it's a Python publisher subscriber service for the ASF. You can basically think of it as a newspaper where you have a publisher, you have an audience, you have the readers, and then you have topics of interest. Some might want the sports section or the funnies, or someone might want the financial news. And then you have, of course, the writers or journalists that make up the contents in these sections. And at ASF, these sections, they would be Subversion commit or Git commit or a new email being written, or someone got added as a Committer or someone filed a pull request, someone filed a new bug or issue, or some are discussing an issue. And the writers and journalists would be all these systems where you send an email to, or you open up a new ticket or you commit some code to it. The readers will then be either users, or there will be a lot of different software components that rely on these messages in order to operate themselves and do what they're supposed to do. So in essence, PyPubSub is, again, some glue that binds the majority of our services together. And it does so by dispatching events to basically whomever wants to read about them. We actually have something called a Pub/Sub Explorer, in real time shows every single event that happens at the ASF technology wise. So if someone sends an email to us, if someone commits something, if someone opens the poll request, if someone comments on a discussion, it all shows up in this Explorer that will update in real time. And it's very cool. (ASF Infrastructure Administrator) Greg (Stein) was saying that you do things that are uniquely different from other team members. In addition to the PyPubSub, what other things are you working on? Currently, one of the main things we manage is called technical debt, which is basically, the longer you don't maintain and upgrade a system, the more expensive it's going to become once you finally have to do it. And so I'm dealing with some technical debt that is moving the service that we have called GitBox from an old, pretty ancient set up to a brand spanking new 2020 machine and software, which also means moving from Python 2.7 to Python 3.8 for every single component that is in the service called GitBox. And that is a lot of components. GitBox is the ASF side of where a committer would commit code to if a project uses the Git version control system. The other side would then be GitHub, if a project chooses to use GitHub. And GitBox and GitHub, they kind of talk together and figure out, okay, someone pushed to me, I'm going to synchronize this with you. And I'm also going to make sure that everyone gets an email on the mailing list saying "something just happened." It's rather unique in that you can choose to either use a GitHub account, or you can choose to say, "I'm not going to use GitHub. I'll just use my Apache credentials on the Apache server instead.'' Not a lot, very, very few, in fact, organizations have this kind of interconnectivity between GitHub and a locally hosted git server. And what we have done very neatly is, we have managed to link our LDAP directory of all our committers to GitHub. Meaning that, if you go in and say, "This is me on GitHub.'' We automatically figured out, okay, that means you get wide access to this and this and this repository. And that is updated in real time. How did these out-of-the box projects come about? I remember when you first approached me about five years ago with these fantastic stats just before I was going to publish the Annual Report. I’d never seen anything like that at the ASF. It's difficult to explain. It's like asking a painter, ''Where do you get your inspiration from?'' It just happens. A lot of time --I will tell a little secret-- a lot of the time that I spend in my day-to-day work is not spent actually typing code or reading up on new fun things. A lot of it is spent what you would call idling. And by that I mean not particularly engaged in any specific task, but kind of just all over the place casually ... Like how, and I hope not to cause any offense here, but how a standard office worker would spend a lot of time on Facebook catching up on friends and family. I'll just spend mine to see whatever I'm interested in the moment that has to do with programming or mathematics or psychology. And in the back of my mind, there's always, how can I take this information that I'm reading about and apply it in a software world? My mind has a tendency to see structures that may or may not be there. And I think almost exclusively in structures. Whenever I see something I want to understand not just how does it work, but how is it basically designed? And can I replicate that? And so, a lot of my day-to-day work is, I see something cool, it might not be anything that has to do with software, or the internet, or anything. It might just be a cool gadget, or a painting, or a chart in a newspaper. And I'll be like, ''What can I use that for that would benefit the foundation? Or whatever hobby project that I'm working on?'' And then you get these aha moments where you're like, ''This I can actually use this way to fix a problem that we are having, or that problem that we could have.'' Sometimes you just make up problems that will potentially happen in the future, just so you can have an excuse to get started on something. And for some strange reason, these fictitious problems very often tend to be not so fictitious at all. And once you show three or four people, hey, I thought of this thing that's not actually a problem. And I thought of a solution. They'll be like, ''That is actually a problem for us.'' And suddenly you have a solution to a problem that you didn't think existed in real life, but it actually does. So, a lot of the things I do are “for the fun of that”. But there's always a work-related starting point in that, is this something that can be used within the software world? Or within the managerial world of software? Which is where I primarily tend to focus my energy. In terms of your day-to-day work with the Infra team, you said that you’re hands-on, not necessarily coding specific tactical solutions, but solving other problems --do you participate with the firefighting as do the other team members? You often respond to my queries about mailing lists --is that your specialty? Chris and Drew shared that everyone specializes in at least one thing. What do you specialize in? My focus is primarily, and there's this kind of a self-made problem. My focus is all the programs and services that I, unfortunately, created. You create it, you own it. Yes. There are a lot of services at Apache Infrastructure that either I made from scratch, or they have a very big thumbprint of mine on them. And so, when I started at Infrastructure, the Infrastructure team, it was expected that we do our fair bit of firefighting. We do a fair bit of the tasks that every single member of the infrastructure team knows how to complete. And I will go through tickets and I find tickets that I find manageable and complete those. I will participate in firefighting. I will do whatever I feel needs to be done right away. If there's something important, or if there's something where I feel like this should have been dealt with by now, I will do that. But it was also the expectation that I come in and help develop and maintain a lot of new features we were looking at creating for the committers and for the end users of Apache software. Simply to make for a better user experience and an easier workflow for our committers and contributors. So, a lot of what I do is maintaining and assisting with services that I have either initially authored or helped expand upon. Tell us about the structure of the Infra team --how did your work come about in a formal way? You were saying that you're creating these tools and then they just kind of got integrated. But were they looking for your sort of skill set? Or was it more of, “hey, we need another Java guy”? What clicked there? Your background is really different. Your expertise is different. Your insight is different. It's an unusual scenario to have a traditional department embrace someone like you and say, ''Hey, we're going to have a whole new type of services offered based on this one guy's vision.'' That's very unusual. Can you elaborate on that a bit? I don't think they were looking for someone like me. But I think they got someone like me and it was completely happenstance. The Infrastructure team at that point, that was early 2013. They were looking to expand with one more staffing spot. This was a part time job. And this was probably about a year and a half after I started doing things for the Infrastructure team. And they had a very narrow list of candidates at the time, because it was a very closed circle. And kind of still is because when you're a staffer, you get the keys to a very mighty kingdom. And so, they had a few people that they could consider, but I was probably by far the one putting in the most hours. And I will, gladly admit that, at that time, I did not have a job. So, I was able to put in a lot of hours. This was when Sam Ruby was VP infrastructure. When Sam initially took me aside and said, ''Hey, we are looking for this part time opening, are you interested?'' I was like, ''No, this, surely you're not, you can't be serious. There's got to be someone that's actually qualified for this job.'' I didn't consider myself qualified at all. And... But you were doing the work? I was doing the work, I just didn't have any confidence in the work I was doing. You can be creative, you can do a lot of interesting things and still have this incredible imposter syndrome going on at the back of your head saying, ''Someone else is doing this work. It's not you.'' So, I politely turned him down and said, ''Thank you, but I'm not insisted because you'll just find out I'm a fraud.'' It actually took two other Infrastructure at that point, current staffers, two other sector members to yank me aside and say, ''What are you doing? We want you for this job.'' And they had actually pretty much all internally, independently been rooting for me and trying to position me to become this new member of the team, to my great surprise. After, I think it was after a very long talk with (former Infra team member) Joe Schaeffer, I was finally convinced, maybe I should give it a shot. And I'm very glad that he convinced me. I'm very glad that the other people at that time also convinced me because it's now been, to this month, seven years since I started. And it's not been fun every day because there can be such a thing as too much firefighting going on. But it's been interesting every single day. You're never bored and you never think, ''I need to find a new job.'' Because you are respected for what you do. You are rewarded in more ways than money, honestly, and you can probably agree with this, at The Apache Software Foundation you get a very unique sense of loyalty. Not to the Board of Directors, or to the specific projects, or anything else, but to the community as a whole. To the mission that we're doing. So, I am honestly very content being where I am. I'm very happy that these people ganged up on me and, basically, forced me to get a job that was... It was kind of silly in hindsight because it's a well paying job, it's part time. So, you don't have to spend nine hours a day on it. You can work whenever you want to and... There were no setbacks except for this nagging doubt that people are going to find out the real me. Which, as I discovered myself, it turned out the real me was actually kind of awesome at this job. It's interesting because the Apache community tends to not want someone if they're not good. So, it's testament to your skill set, and who you are as a person, you're liked. You're very well liked. Thank you very much. And, you're right. The Apache community seems to be very good at finding talent, and also very good at rewarding it in ways that make that talent stick, and make them interested and continue working within the ASF community. I think that's a thing that you don't see in all software communities. We learned from (Infra team members) Chris (Thistlethwaite), Drew (Foulks), and Greg (Stein) about the scope of the work that Infra does. How is the ASF different from other Open Source foundations from an Infra perspective --are there other people doing what you do, or how our group performs, or the services that our group provides. Is this common in other Open Source foundations? It is not common in other foundations. We are different in that the breadth of the amount of services that we provide for each project. And especially at the budget that we provide it at. I think we did a count back in 2015 and it was something around 52 different distinct unique services that we had, that we were running for all projects to use. And in between these, there are possibly more than 300 machines each running, some of them running the same thing on 10 machines. And then you have another 10 machines that are running 10 different things. This is all handled by what? A team of what, seven people now? Six people actually, five of us and Greg (Stein). Greg is a bit of an übermensch, so yeah. That's amazing, in terms of the workload. It can get hectic, and I will not deny that, but we have a very, very strong cohesion. I don't want to say we finish each other's sentences, but when someone has a problem, the others know when to step in and help, when to back off, and what to do while someone else is doing their thing. We compliment each other really well. And we have a nice set of tools to help us with managing things, making sure that everything is up and running, diagnosing when something goes wrong. We have a lot of, again, by the hand of me, a lot of custom tiny services that you never even hear of or see if you're not within the infrastructure team. But that goes on automatically. Let's say you're abusing someone in a ticket multiple times, or you're spamming, whatever. We have a lot of microprocesses that go in and detect abusive behavior, both in terms of spam, but also what you would call technical hardware abuse, where someone is repeatedly using all of our bandwidth, for example, or causing the CPU to spike. We can go and detect that automatically and pull a systemwide ban on you, which it's very custom, but it saves us a lot of money. I will say that we've saved a lot of money at the Foundation by being smart about what we do and not being afraid of making a few mistakes while we make new things... Because a lot of what we do is custom-based, custom-made. Because there is not, unless you're talking about something big like Kubernetes or something at that scale, it's often very difficult to find the tools that do what we want them to do with the problem that we have. Because other companies, especially companies, are not the same as ASF. They don't have 300 different departments that all have their own little tools that they want working in their specific way. And they want this to connect to that, and that's connected to some other thing. We are not afraid to create custom solutions, we're not afraid to get our hands dirty and we're not afraid to make mistakes. That doesn't mean we make mistakes all the time, or that we're okay with all sorts of risks. How do you interact with the team? How do you stay motivated? I stay motivated by interacting with my team, I would say. Interaction is mostly on Slack, which is, for those that either don't know it or pretend they don't know it, is an instant messaging platform. We have an account for the Foundation; we have our staff channel where everything gets discussed, whether that be, the mail servers are a big backlog, or this prime rib I just sous vide-ed at 105 Fahrenheit four or five hours is awesome. I think one of the tricks or keys to success for teams like us is to really mix up the subjects and not be all business and not be all fun because you don't want it to be too boring, you don't want it to be too relaxed. I think we've somehow managed to hit a pretty good ratio of fun and serious items that we discuss on a day to day basis. So, it's fun talking to your colleagues about real-life stuff that isn't work, but it's also rewarding talking about work and learning from them and their experiences, and you being able to give them some work experiences and wisdom from your many years of being a sysadmin or infrastructure architect. I think we've hit a really good ratio there. It's an interesting perspective with that because everyone I’ve interviewed thus far has given the same answer. Can you describe your typical workday: now, I know some people don't have an exact schedule, some people do. What's a day like in Daniel Gruno's life? My typical workday is very atypical for a worker. I don't have a set schedule. I don't have a set time. I don't have a minimum amount of hours I work. I don't have, unfortunately, a maximum amount of hours I work. It all depends on the day and what happens during that day. As said earlier, a lot of what I do is developing new services for the Foundation. As such, I spend a lot of time getting inspiration, and that's done through various means of... From idling, I can be working at noon and then I'll be like, ''I should watch a movie.'' And then I'll go watch a movie. My significant other will tell you that's a lie, I don't watch movies. But that was just an example. I can't sit through two hours, I get too fidgety. And that's actually the real truth about me. I can't sit still and do something for a specific amount of hours, unless I'm in a really inspired mood. So, my typical workday is finding things to do that don't take more than half an hour to do, in between suddenly getting the greatest inspiration from up high. I'll be looking at tickets that are easy for me, not absolutely speaking, easy to fix, but tickets that I know how to fix and I'll go in and fix those. I'll catch up on every single email that I receive, which is thousands of emails every single day. I have a mania about inbox zero. If there's an email, I have to read it and sort it. Otherwise, I can't get past the inbox. I can't even close down the mail client unless I know that there is nothing in my inbox. Yeah, it's the same with Slack and IRC and all that. If there's a message pending for me, I have to check it. But that's beside the point. It gives me something to multitask between. Because there will always be a new email, there will always be someone saying something on Slack. So, a lot of my time is spent just multitasking between that, between reading up on news. And then, at some point, the inspiration that I need for that day will hit me and then comes the manic in a few hours where I just code like crazy because I have the inspiration. I tend to form fully thought out ideas which is terrible because if you have a fully formed idea in your head, you know it's going to take eight hours to complete it. But you also know that if you stop, you might forget that fully formed thought. Sometimes a work hour day can be four or five hours and sometimes it can be 10, 12, 13 hours because my muse has sung to me and the inspiration just has to be translated through the keyboard and into some sort of code or what page or documentation or just a specification for a new idea. Having said that though, don't pity me because I work 12 hours a day and don't be jealous because I work five hours a day. Because it adds up to a lot of hours on average per month. But I'm also happy to do it because it brings me joy. With this constant flow of concepts and code and inspiration, how do you keep your workload organized? You might be hammering away on a solution and imaging and envisioning something to develop --there's a lot of things happening simultaneously. A lot of people have a hard time multitasking, or focusing on one thing and managing the thousands of emails coming into the mailbox, et cetera. How do you manage that? I would say I don't manage it, but luckily I have family that helps manage it. I have a boss that helps manage it. I'm a very ... I'm on the autism spectrum and some would say that I probably have ADD as well. So I get very easily distracted and can lose focus, but I am surrounded by people that are very good at a) knowing that I lose focus very easily and b) guiding me back to the right path for that day. I think in terms of my boss, Greg's point of view, I think it's a win win because I get guided back on my path and I get to actually do something useful and not just 20 unfinished projects. And he gets some services that are working and are improving the use of experience of the people that we are there to support: the committers. So serving 350 Apache projects, initiatives and their communities, like how busy are you? How many requests do you receive a day? How do you prioritize these requests? How do you do this? Greg, Drew, and Chris talked to me about JIRA systems, et cetera. Your work, as I understand it, is not necessarily responding to user requests. How do you fit the creativity in with this process? How do you mitigate that? How do you fit everything in? I do respond to users to keep me busy because if I am, I don't want to say stalling, but if I am really idling then I lose interest so I have to always keep busy with something. So I will grab a lot of tickets just to keep busy with that. That's the thing that I had to teach myself how to do. And I don't have the recipe for it, and yet I have somehow taught myself. The thing where you have to not click on every single new ticket that opens up. And not read every single ... Well, you can read email, you just don't have to write a reply to every single email. It took a few years, I think, for me to stop doing every single ticket that came in within five minutes of it coming in. Because at that point, if you do that, plus you have 10 different projects on the side, you get burnout very quickly. And I've had a few burnouts, where I've been unproductive and doing nothing for the next week because I'd lost all hope in humanity because of the amount of tickets and angry users. So a lot of it just letting go and knowing that there are team members who know just as well as you do what this is about and how to solve it. And if they don't then they will ask you and you can help them then. So a lot of managing the workload is learning to let go of the workload. And if someone creates a ticket saying, "My forwarding address doesn't work." It's probably okay to wait more than five minutes before you fix that if you're in the middle of something. I used to be of the, not opinion, but yeah, I used to be of the opinion that this must be fixed right away. The minute I saw someone had a problem, I wanted to help them. But there comes a point where the more you try to help someone, the less you're actually helping them in the end because of the overhead of dealing with too many tasks and being burned out. I think some of it is ... Stefano… Mazzocchi? Yeah. Right. The Mazzocchi equilibrium. There is a certain point where in the effort you put in and the effort comes out of that starts to not align anymore. And so if you're not good at holding back and letting things slide just a bit, then you cross that threshold and you end up putting in maybe, I don't know, 12 hours of work. And really what you are doing is five hours of work or four hours or one hour of work because you're so not interested in what you are doing. I know that some of my colleagues use Trello or If This Then That, or other tools to organize their day, but I want to say that's not for me. I don't think it factors in the creativity that is needed in the role I have. I think without any scientific evidence whatsoever, that if your job is to think up new ideas and think of new ways to do something. These tools, they don't necessarily account for where creativity comes in because you can't put in your calendar: step one, “be creative”; or at 9:00, “be creative”. Creativity is something that just happens. I've found that it happens for me when I am idling, when I am doing a lot of non-work related things, switching between and then switching back to work. And then switching back to non work items and switching back to work. And then suddenly, a link appears between these two things and they're like, yeah, this idea could actually be used for work. But the things I am doing are not something that you can put into plan because you don't know ... I mean if I knew how to be creative, if I knew to just go to this Website, then I would be a millionaire by now. So I don't know *how* to be creative: I know that I can be creative and I know it happens when I let it happen. You have to make space for that to happen, right? You have to allow for that to happen. It's great that you have flexibility to be able to do that in your job --that part of your work is to be able to conceptualize and visualize and come up with things. It takes a while because sometimes you're not going to know the problem unless you're in the middle of it: "so, oh this is an issue … here's an opportunity for us to come up with something that'll help." It's great. And it's especially great because I think honestly if I was stuck and let's say I was doing human resource management or whatever that I studied for, if I was stuck doing Excel spreadsheets, for example, all day long ... Not that that's a bad thing, but when creativity suddenly hits me, I have to get it down on paper or it's going to haunt me to an extent where I just can't stand myself. So I'm very fortunate to have a job where I can, fire fighting aside, I can say, "Boom, I have this inspiration suddenly. I need to focus on that." And then I can go and focus on that. And I have a boss and I have a boss's boss and I have my colleagues that are understanding so that suddenly, "Oh Daniel got inspired. He's probably going to manic for the next eight hours just working on this idea he’s got." It's really wonderful being given that space to be creative because I think no matter what job I have, there would be an urge to be creative and to think up ideas. And again, when I think of an idea, it forms itself completely in my head. Some people will start with half an idea or a fingertip of an idea. For me, it's mostly been the entire idea presents itself to me right away, and I have to get as much of that as possible down on paper before it's lost. To have that opportunity is really wonderful. [END OF PART ONE]

Friday September 04, 2020

The Apache News Round-up: week ending 4 September 2020

Welcome, September! We've had a great week within the Apache community. Here's what happened:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 16 September 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is open (and free) for ApacheCon@Home 29 September - 1 October https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.76%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 387 Apache Committers changed 2,230,138 lines of code over 3,461 commits. Top 5 contributors, in order, are: Mark Miller, Hervé Boutemy, Tellier Benoit, Andrea Cosentino, and Claus Ibsen.

Apache Project Announcements – the latest updates by category.

Content --
 - Apache Jackrabbit 2.18.6 and Oak 1.8.23 released https://jackrabbit.apache.org/

Libraries --
 - Apache Commons Codec 1.15 released http://commons.apache.org/codec/

Messaging --
 - Apache Proton 0.32.0 released https://qpid.apache.org/

Search --
 - Apache Lucene 8.6.2 and Solr 8.6.2 released http://lucene.apache.org/

Web Frameworks --
 - Apache MyFaces Core 2.3-next-M4 released http://myfaces.apache.org/


Did You Know?

- Did you know that details about the online/virtual format, presentation sessions, Apache projects and communities featured, schedule selection, languages available, sponsors, Slack channel & more are available at https://blogs.apache.org/conferences/entry/apachecon-home-2020-is-just ?

- Did you know that Target uses Apache Druid to ingest more than 3 trillion rows of data across 3,500+ data sources? http://druid.apache.org/ 

- Did you know that Apache Kylin enables SQL developers and BI users to achieve sub-second response times for queries against petabytes of data? http://kylin.apache.org/ 


Apache Community Notices

- Apache Month In Review: August 2020 https://s.apache.org/Aug2020

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg , Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Tuesday September 01, 2020

Apache Month in Review: August 2020

Welcome to the latest monthly overview of events from the Apache community. Here's a summary of what happened in August:

New this month --

 - ASF Annual Report – a look back at our many achievements during the 2020 Fiscal Year
    -- Press release https://s.apache.org/FY2020AnnualReport-PR
    -- Full report https://s.apache.org/FY2020AnnualReport

 - ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
   -- ApacheCon @Home 2020: ApacheCon North America and Europe have been combined and will be held online and free of charge 29 September - 1 October 2020. Sponsorship opportunities available. Join us! https://apachecon.com/acah2020

 - "Trillions and Trillions Served" – the feature documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019. 
   -- “Apache Innovation”, the fourth and final segment of the series now available https://s.apache.org/ApacheInnovation

 - Success at Apache: the monthly blog series that focuses on the people and processes behind why the ASF "just works".
   -- I Became an Apache Solr Committer in 4,662 Days. Here’s how you can do it faster! by Eric Pugh https://s.apache.org/hney3

 - Apache Month in Review: July 2020 https://s.apache.org/July2020


Important Dates --

 - Next Board Meeting: 16 September 2020. Board calendar and minutes http://apache.org/foundation/board/calendar.html

 - ApacheCon @Home 29 September - 1 October 2020 https://apachecon.com/acah2020


Infrastructure --

Our seven-member Infrastructure team on three continents oversees our highly-reliable, distributed network under the leadership of VP Infrastructure David Nalley and Infrastructure Administrator Greg Stein. ASF Infrastructure supports 300+ Apache projects and their communities across ~200 individual machines, 1,400+ repositories, 5-6PB in traffic annually, ~75M downloads per month, and 2-3M daily emails on 2,000+ lists. ASF Infra performs 7M+ weekly checks to ensure services are available around the clock. The average uptime in August was 99.79%. http://www.apache.org/uptime/

Committer Activity --

In August, 822 Apache Committers changed 16,077,266 lines of code over 14,003 commits. The Committers with the top 5 highest contributions, in order, were: Mark Miller, Daniel Klco, Jean-Baptiste Onofré, Gary Gregory, and Andrea Cosentino.

Project Releases and Updates --

New releases from Apache Calcite (Big Data); Cocoon (Web Frameworks); Commons Imaging (Libraries); Commons NET (Libraries); Commons Pool (Libraries); Commons Validator (Libraries); Commons JCS (Libraries); CXF (Libraries); Flink (Big Data); HBase (Big Data); HTTP Server (Servers); Kafka (Big Data); Log4cxx (Libraries); Lucene (Search); NiFi (Big Data); OpenMeetings (Web Conferencing); OpenNLP (Machine Learning); Parquet (Big Data); PDFBox (Content); Qpid (Messaging); ShardingSphere (Big Data); Skywalking (Application Performance Management); and Solr (Search).

The Apache Incubator is the primary entry path for projects we invite you to review the many projects currently in development in the Apache Incubator http://incubator.apache.org/ . New releases from incubating podlings include: Apache APISIX (API) and IoTDB (IoT). 

# # #

To see our Weekly News Round-ups, visit https://blogs.apache.org/foundation/ and click on the calendar in the upper-right side (published every Friday) or hop directly to https://blogs.apache.org/foundation/category/Newsletter . For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. We appreciate your support!

Friday August 28, 2020

The Apache News Round-up: week ending 28 August 2020

Farewell, August --we're wrapping up the month with another great week. Here are the latest updates on the Apache community's activities:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 16 September 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is open (and free) for ApacheCon@Home 29 September - 1 October https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.76%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 396 Apache Committers changed 3,762,638 lines of code over 3,973 commits. Top 5 contributors, in order, are: Daniel Klco, Radu Cotescu, Jean-Baptiste Onofré, Mark Thomas, and Andrea Cosentino.  

Apache Project Announcements – the latest updates by category.

Big Data --
 - Apache HBase 2.3.1 released https://hbase.apache.org/
 - Apache Calcite 1.25.0 released https://calcite.apache.org/
 - Apache Flink 1.10.2 released https://flink.apache.org/

Content --
 - Apache PDFBox 2.0.21 released https://pdfbox.apache.org/

IoT --
 - Apache IoTDB (Incubating) 0.10.1 released https://iotdb.apache.org/

Libraries --
 - Apache Commons JCS 3.0 released https://commons.apache.org/jcs/
 - Apache Log4cxx 0.11.0 released https://logging.apache.org/log4cxx/
 - Apache CXF 3.4.0 released https://cxf.apache.org

Web Conferencing --
 - Apache OpenMeetings 5.0.0-M4 released https://openmeetings.apache.org/

Did You Know?

- Did you know that ApacheCon@Home keynoters include Thomas Huang of NASA Jet Propulsion Lab; Camille Fournier of Two Sigma; and Edmon Begoli of Oak Ridge National Labs? Learn more at https://www.apachecon.com/acah2020/tracks/ 

- Did you know that Shopify is building a new streaming analytics pipeline powered by Apache Beam, Druid, and Kafka? https://projects.apache.org/projects.html?category#big-data 

- Did you know that Apache Royale Jewel UI set already provides 60+ components to quickly build Web applications? https://royale.apache.org/ 


Apache Community Notices

- Apache Month In Review: July 2020 https://s.apache.org/July2020 

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg , Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3

 - Did you know that Beam Summit 2020 will be held 24-28 August online and free of charge? https://beamsummit.org/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.


Friday August 21, 2020

The Apache News Round-up: week ending 21 August 2020

Happy Friday! Let's take a look at what the Apache community has been up to over the past week:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 16 September 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is open (and free) for ApacheCon@Home 29 September - 1 October https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.76%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 233 Apache Committers changed 1,491,530 lines of code over 1,662 commits. Top 5 contributors, in order, are: Luca Burgazzoli, Kaxil Naik, Liang Zhang, Nick Vatamaniuc, and Boris Zhu.

Apache Project Announcements – the latest updates by category.

API --
 - Apache APISIX Dashboard 1.5 (Incubating) released https://apisix.apache.org/

Application Performance Monitor --
 - Apache SkyWalking Chart 3.1.0 released https://skywalking.apache.org/

Big Data --
 - Apache ShardingSphere ElasticJob UI 3.0.0-alpha released https://shardingsphere.apache.org/elasticjob/
 - Apache NiFi 1.12.0 released http://nifi.apache.org/

Messaging --
 - Apache Qpid JMS 0.54.0 and Dispatch 1.13.0 released https://qpid.apache.org/

Search --
 - Apache Lucene 8.6.1 and Solr 8.6.1 released http://lucene.apache.org/


Did You Know?

- Did you know that Apache projects' ongoing sustainability is ensured through the generosity of our Sponsors and individual donors, whose support helps ensure that the ASF continues to steward 227M+ lines of code and provide more than $20B worth of software to the public-at-large at 100% no cost? http://donate.apache.org 

- Did you know that you can learn about Apache Beam, Calcite, Camel, CarbonData, Groovy, Hadoop, Karaf, Labs, NetBeans, OFBiz, OpenOffice, PLC4X, Rya, Spark, Tomcat, Unomi, and more in the "Apache Innovation" short segment of the "Trillions and Trillions Served" documentary? https://s.apache.org/ApacheInnovation  

- Did you know that projects undergoing development in the Apache Incubator are in a variety of categories that include AI, Annotation, Big Data, Blockchain, Cryptography, Data Visualization, Distributed Computing, Email, Embedded Systems, Geospatial Data, Graphing, Hardware Acceleration, IoT, Messaging, Monitoring, Natural Language Processing, Scheduling, Streaming, Training, Usability Testing, and more? http://incubator.apache.org/projects/ 


Apache Community Notices

- Apache Month In Review: July 2020 https://s.apache.org/July2020 

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg , Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3

 - Did you know that Beam Summit 2020 will be held 24-28 August online and free of charge? https://beamsummit.org/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.


Friday August 14, 2020

The Apache News Round-up: week ending 14 August 2020

Hooray for Friday! The Apache community has been busy over the past week; let's review what happened:

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 19 August 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is open (and free) for ApacheCon@Home 29 September - 1 October https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.76%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 151 Apache Committers changed 4,892,020 lines of code over 678 commits. Top 5 contributors, in order, are: Sebastian Bazley, Gary Gregory, Nick Vatamaniuc, Liang Zhang, and Kaxil Naik.    

Apache Project Announcements – the latest updates by category.

Big Data --
 - Apache Kafka 2.5.1 and 2.6.0 released https://kafka.apache.org/
 - Apache ShardingSphere ElasticJob-3.0.0-alpha released https://shardingsphere.apache.org/elasticjob/
 - Apache Parquet 1.11.1 released https://parquet.apache.org/

Libraries --
 - Apache Commons Imaging 1.0-alpha2 released https://commons.apache.org/imaging/
 - Apache Commons NET 3.7 released https://commons.apache.org/net/
 - Apache Commons Validator 1.7 released https://commons.apache.org/validator/

Servers --
 - Apache HTTP Server 2.4.46 released https://httpd.apache.org/

Web Frameworks --
 - Apache Struts CVE-2019-0230 (Possible RCE) and CVE-2019-0233 (DoS) security issues https://lists.apache.org/thread.html/r42ff5f78b46f751c48eee1fe4201a49ab48524f33ac8f8975f2029b8%40%3Cannounce.apache.org%3E 


Did You Know?

 - Did you know that tracks for ApacheCon@Home include Big Data, Community, Content Delivery, Fintech, Integration, IoT, Machine Learning, Messaging, Mobile, Natural Language Programming, Observability, Programming Languages, Search, Servers, Streaming, and Usability, as well as emerging innovations from the Apache Incubator? Learn more, including featured projects, at https://www.apachecon.com/acah2020/tracks/  

 - Did you know that mobile transportation platform Didi Chuxing uses Apache Kylin to return exact distinct count on billions of rows of data with sub-second latency to generate the most accurate picture of its business? http://kylin.apache.org/ 

 - Did you know that the "Trillions and Trillions Served" documentary on The Apache Software Foundation comprised 65 hours of filming over 8 terrabytes of footage? https://s.apache.org/Trillions-Feature

Apache Community Notices

- Apache Month In Review: July 2020 https://s.apache.org/July2020 

- ASF FY2020 Annual Report https://s.apache.org/FY2020AnnualReport 

- "Trillions and Trillions Served" documentary on the ASF: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" https://s.apache.org/ApacheEverywhere 3) "Why Apache" https://s.apache.org/ASF-Trillions 4) “Apache Innovation” https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg , Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3

 - Did you know that Beam Summit 2020 will be held 24-28 August online and free of charge? https://beamsummit.org/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Friday August 07, 2020

The Apache News Round-up: week ending 7 August 2020

Welcome, August! We've had a great week within the Apache community. Here's what happened:

Success at Apache – the monthly blog series that focuses on the people and processes behind why the ASF "just works".
 - "I Became an Apache Solr Committer in 4,662 Days. Here’s how you can do it faster!" by Eric Pugh https://s.apache.org/hney3

ASF Annual Report – a look back at our many achievements during the 2020 Fiscal Year
 - Press release https://s.apache.org/FY2020AnnualReport-PR
 - Full report https://s.apache.org/FY2020AnnualReport

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 19 August 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is OPEN (and free) for ApacheCon@Home taking place online 29 September - 1 October. Join us! https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.86%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 399 Apache Committers changed 2,461,830 lines of code over 3,074 commits. Top 5 contributors, in order, are: Andrea Cosentino, Gary Gregory, Sebastian Bazley, Jean-Baptiste Onofré, and Claus Ibsen.    

Apache Project Announcements – the latest updates by category.

Application Performance Monitor --
 - Apache SkyWalking 8.1.0 released https://skywalking.apache.org/

Libraries --
 - Apache Commons Pool 2.8.1 released https://commons.apache.org/proper/commons-pool/

Machine Learning --
 - Apache OpenNLP 1.9.3 released https://opennlp.apache.org/

Web Frameworks --
 - Apache Cocoon 2.1.13 released https://cocoon.apache.org/


Did You Know?

 - Did you know that the ASF stewards more than 227M+ lines of code, valued at more than $20B? https://s.apache.org/FY2020AnnualReport-PR

 - Did you know that the top 5 most active Apache projects over FY2020 were, in order: Apache Kafka, Hadoop, Lucene, POI, and ZooKeeper? https://s.apache.org/FY2020AnnualReport

 - Did you know that Apache projects' ongoing sustainability is ensured through the generosity of our Sponsors and individual donors, whose support helps ensure that the ASF continues to provide more than $20B worth of software to the public-at-large at 100% no cost? http://donate.apache.org

Apache Community Notices

- Apache Month In Review: July 2020 – overview of events that have taken place within the Apache community https://s.apache.org/July2020 

- "Trillions and Trillions Served" – the documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 have been released: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" short https://s.apache.org/ApacheEverywhere 3) "Why Apache" teaser https://s.apache.org/ASF-Trillions 4) “Apache Innovation” shorts https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg , Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3

 - Did you know that Beam Summit 2020 will be held 24-28 August online and free of charge? https://beamsummit.org/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Monday August 03, 2020

Success at Apache: I Became an Apache Solr Committer in 4,662 Days. Here’s how you can do it faster!

by Eric Pugh

On April 6th, 2020 I was invited to become a committer on the Apache Solr project.  My journey to becoming a committer started in earnest 4662 days before that!  On July 2nd, 2007, I opened SOLR-284, a ticket for adding content extraction to Solr. 

A committer on an open source project under the Apache Foundation umbrella is someone who is trusted to contribute code to the project and to help manage and drive its ongoing development. It’s an honour to have been asked and I was very proud to accept the invitation!

So, you did the math, and you realized that it took me 153 months, or 13 years (rounding up), to become a committer, and you’re wondering “What if I don’t want to wait that long?” So here’s my quick cheat sheet on ways to become a committer on an open source project, illustrated by my own journey:

  1. Start by learning the culture of the project. How are decisions made? What tools do people use? What do the various acronyms mean? Join the mailing lists and read every commit.
  2. Start small and work your way in.  Some great ways to do this are to:
    • Take existing patches and test them.  Update them to the latest code base.  Document what you’ve learned
    • Take advantage of being new to a project to bring fresh eyes to the documentation.  Every time you find yourself scratching your head on how something works, contribute a fix to the docs.   It’s a powerful way to immediately contribute.  This is the fastest way to get involved and involves the least cognitive load!  See SOLR-2232 or this email thread.
    • Answer questions on the mailing list! Being able to articulate reasonable responses to questions demonstrates how much you have learned.
    • Bug fix, bug fix, bug fix! Pick bugs that have an obvious answer so that the “correct” solution is easy to figure out. If the right approach to solving it is very ambiguous, you probably won’t get much traction. Remember to remind committers to apply your fixes when they have the time! See SOLR-13965 and SOLR-11480 and SOLR-2611 and SOLR-2263.
  3. Ready to start slinging some code?   Don’t go and refactor the core foundations of the project (at least not yet).   Instead, be like a pilot fish and latch onto one of the core committers who is being very active in the project.

Embrace their vision, and start picking up tasks related to whatever major chunk of work they are doing. Write some unit tests. See about opportunities for refactoring. Do some manual testing over multiple platforms. Once they see that you’re contributing (and accelerating what they are pushing), then work to get some of your own tickets assigned to you under that vision. I’ve seen this lead directly to committership many times, and if I had followed this route, I might have joined sooner!

Here’s to the next 4,662 days of being active in the Apache Solr project!

Eric Pugh is a member of the ASF and a committer Apache Solr. He co-authored the book Apache Solr Enterprise Search Server. Eric is co-founder and CEO of OpenSource Connections, where he helps OSC clients, especially those in the ecommerce space, build their own search teams by leading projects and by acting as a trusted advisor. He also stewards Quepid, a platform for assessing and improving your search relevance.

[this post first appeared at https://opensourceconnections.com/blog/2020/07/10/i-became-a-solr-committer-in-4662-days-heres-how-you-can-do-it-faster/ ]

= = =

"Success at Apache" is a monthly blog series that focuses on the processes behind why the ASF "just works" https://blogs.apache.org/foundation/category/SuccessAtApache 

Saturday August 01, 2020

Apache Month in Review: July 2020

Welcome to the latest monthly overview of events from the Apache community. Here's a summary of what happened in July:

New this month --

 - ASF Annual Report – a look back at our many achievements during the 2020 Fiscal Year
 -- Press release https://s.apache.org/FY2020AnnualReport-PR
 -- Full report https://s.apache.org/FY2020AnnualReport

 - "Trillions and Trillions Served" – the feature documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019, watch “Apache Innovation”, the fourth and final segment of the series https://s.apache.org/ApacheInnovation

 - ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
  -- Announcing ApacheCon @Home 2020: ApacheCon North America and Europe have been combined and will be held online 29 September - 1 October 2020. Join us! https://apachecon.com/acah2020

 - "Inside Infra" a new interview series with members of the ASF Infrastructure team
  -- Meet Greg Stein --Part III https://s.apache.org/InsideInfra-Greg3

 - Apache Month in Review: June 2020 https://s.apache.org/June2020


Important Dates --

 - Next Board Meeting: 19 August 2020. Board calendar and minutes http://apache.org/foundation/board/calendar.html

 - ApacheCon @Home 29 September - 1 October 2020 https://apachecon.com/acah2020


Infrastructure --

Our seven-member Infrastructure team on three continents oversees our highly-reliable, distributed network under the leadership of VP Infrastructure David Nalley and Infrastructure Administrator Greg Stein. ASF Infrastructure supports 300+ Apache projects and their communities across ~200 individual machines, 1,400+ repositories, 5-6PB in traffic annually, ~75M downloads per month, and 2-3M daily emails on 2,000+ lists. ASF Infra performs 7M+ weekly checks to ensure services are available around the clock. The average uptime in July was 99.95%. http://www.apache.org/uptime/

Committer Activity --

In July, 846 Apache Committers changed 14,227,571 lines of code over 16,062 commits. The Committers with the top 5 highest contributions, in order, were: Mark Miller, Andrea Cosentino, Gary Gregory, Sebastian Bazley, and Zhang Yonglun.  

Project Releases and Updates --

New releases from Apache Annotator (Libraries); APISIX (API); Arrow (Big Data); Atlas (Big Data); Avro (Big Data); Beam (Big Data); BVal (Libraries); Calcite (Big Data); Commons Lang and Numbers (Libraries); Commons Text (Libraries); Curator (Messaging); Daffodil (Libraries); Druid (Big Data); Flink (Big Data); Geometry (Libraries); Groovy (Programming Languages); Guacamole (Network Client); HBase (Big Data); IoTDB (IoT); Jackrabbit (Content); Kylin (Big Data); Lucene (Search); MyFaces (Web Frameworks); NiFi (Big Data); Nutch (Web Crawler); OFBiz (Enterprise Processes Automation / ERP); Qpid (Messaging); Skywalking (Application Performance Management); Storm (Big Data); Tomcat (Servers);Tuweni (Blockchain); TVM (Machine Learning); Wicket (Web Frameworks).

The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. Congratulations to Apache APISIX, which graduated as a Top-Level Project this month https://s.apache.org/29wd9. We invite you to review the many projects currently in development in the Apache Incubator http://incubator.apache.org/

# # #

To see our Weekly News Round-ups, visit https://blogs.apache.org/foundation/ and click on the calendar in the upper-right side (published every Friday) or hop directly to https://blogs.apache.org/foundation/category/Newsletter . For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. We appreciate your support!

Friday July 31, 2020

The Apache News Round-up: week ending 31 July 2020

Farewell, July --we're wrapping up the month with another great week. Here are the latest updates on the Apache community's activities:

ASF Annual Report – a look back at our many achievements during the 2020 Fiscal Year
 - Press release https://s.apache.org/FY2020AnnualReport-PR
 - Full report https://s.apache.org/FY2020AnnualReport

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 19 August 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is OPEN (and free) for ApacheCon@Home taking place online 29 September - 1 October. Join us! https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 100%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 414 Apache Committers changed 4,407,598 lines of code over 3,792 commits. Top 5 contributors, in order, are: Mark Miller, Andrea Cosentino, Gary Gregory, Guillaume Nodet, and Shen Yi.  

Apache Project Announcements – the latest updates by category.

Application Performance Monitor --
 - Apache SkyWalking CLI 0.3.0 and SkyWalking Python 0.2.0 released https://skywalking.apache.org/

Big Data --
 - Apache Arrow 1.0.0 released https://arrow.apache.org/
 - Apache Calcite 1.24.0 released https://calcite.apache.org/
 - Apache Beam 2.23.0 released https://beam.apache.org/

Content --
 - Apache Jackrabbit 2.21.3 released https://jackrabbit.apache.org/

Libraries --
 - Apache Commons Text 1.9 released https://commons.apache.org/proper/commons-text/

Messaging --
 - Apache Qpid Proton-J 0.33.6 and JMS 0.53.0 released https://qpid.apache.org/


Did You Know?

 - Did you know that there were 150TB+ in source code downloads from Apache mirrors (excludes convenience binaries) in FY2020? https://s.apache.org/FY2020AnnualReport 

 - Did you know that workflows for the Creative Commons Catalog project are powered by Apache Airflow? https://opensource.creativecommons.org/blog/entries/data-flow-API-to-DB/ 

 - Did you know that, after two years in development, Apache Cayenne v4.1 GA is now available? https://cayenne.apache.org/ 

Apache Community Notices

- Apache Month In Review: June 2020 – overview of events that have taken place within the Apache community https://s.apache.org/June2020 

- "Trillions and Trillions Served" – the documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 have been released: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" short https://s.apache.org/ApacheEverywhere 3) "Why Apache" teaser https://s.apache.org/ASF-Trillions 4) “Apache Innovation” shorts https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg , Part II https://s.apache.org/InsideInfra-Greg2 and Part III https://s.apache.org/InsideInfra-Greg3

 - Did you know that Beam Summit 2020 will be held 24-28 August online and free of charge? https://beamsummit.org/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

Wednesday July 29, 2020

The Apache® Software Foundation Announces Annual Report for 2020 Fiscal Year

World's largest Open Source foundation provides 227M+ lines of code, valued at more than $20B, to the public-at-large at 100% no cost.

Wakefield, MA —29 July 2020— The Apache® Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the availability of the annual report for its 2020 fiscal year (1 May 2019 - 30 April 2020).

Now in its 21st year, the world's largest Open Source foundation’s "Apache Way" of community-driven development is the proven process behind thousands of developers successfully collaborating on hundreds of Apache projects. The Apache Way has directly influenced the InnerSource methodology of applying Open Source and open development principles to an organization. The Apache Way has been adopted by countless organizations, including Capital One, Comcast, Ericsson, HP, IBM, Google, Microsoft, PayPal, SAP, T-Mobile, Walmart, and countless others.

Valued at more than $20B —and provided to the public-at-large at 100% no cost— Apache software is used in every Internet-connected country on the planet.

Apache software comprises 227M+ lines of code, is integral to nearly every end user computing device, manages exabytes of data, executes teraflops of operations, and stores billions of objects in virtually every industry. Countless mission-critical projects in Artificial Intelligence, Big Data, build management, Cloud Computing, content management, DevOps, Deep Learning, IoT and Edge computing, mobile, servers, Web frameworks, and many other categories are powered by Apache. [Learn more about the ASF’s reach and influence at https://s.apache.org/ApacheEverywhere ]

Every Apache Top-Level Project (and its sub-projects, if applicable) is overseen by a Project Management Committee (PMC) that guides its day-to-day operations, including community development and product releases.

At the close of FY2020, 201 PMCs managed 339 Top-Level Projects and dozens of sub-projects; 9 projects were newly-graduated Top-Level Projects from the Apache Incubator, and 8 projects retired to the Apache Attic. Apache Incubator PMCs mentored 45 projects under development in the Apache Incubator; 6 projects were new entrants to the Incubator, and 3 were retired.

FY2020 highlights include:

  • ~8M lines of Apache code added, valued at approximately $600M worth of work; total code value exceeding $20B (CoCoMo model);
  • Stewardship of 227M+ lines of code in the Apache repositories;
  • Foundation operations supported by contributions from 10 Platinum Sponsors, 9 Gold Sponsors, 11 Silver Sponsors, 25 Bronze Sponsors, 6 Platinum Targeted Sponsors, 5 Gold Targeted Sponsors, 3 Silver Targeted Sponsors, 10 Bronze Targeted Sponsors, and more than 500 individual donors;
  • 34 new individual ASF Members elected, totalling 813;
  • Exceeded 7,700 code Committers;
  • 206 Top-Level communities overseeing 339+ Apache projects, plus dozens of sub-projects and initiatives;
  • 9 newly-graduated Top-Level Projects from the Apache Incubator;
  • 45 projects currently undergoing development in the Apache Incubator;
  • Web requests received from every Internet-connected country on the planet;
  • 35M+ page views per week across apache.org;
  • ~2 Petabytes source code downloads from Apache mirrors;
  • Top 5 most active/visited Apache projects: Kafka, Hadoop, Lucene, POI, ZooKeeper; 
  • Top 5 Apache repositories by number of commits: Camel, Flink, Beam, HBase, Lucene Solr;
  • Top 5 Apache repositories by lines of code: NetBeans, OpenOffice, Flex (combined), Mynewt (combined), Trafodion;
  • 2,892 Committers changed 60,132,710 lines of code over 174,889 commits;
  • 12,413 people created 63,172 new issues; 2,868 people closed 54,633 issues
  • 19,396 authors sent 2,137,560 emails on 907,870 topics across 1,417 mailing lists;
  • Top 5 most active mailing lists (user@ + dev@): Flink, Tomcat, Royale, Beam, Lucene Solr;
  • Top Senders: (Apache Projects + Committers): GitBox, AsterixDB, Whimsy, Andrea Cosentino, Mark Thomas
  • 2,045 git repositories, containing ~250GB of code and repository history;
  • GitHub traffic: Top 5 most active Apache sources --clones: Thrift, Beam, Cordova, Arrow, Geode;
  • GitHub traffic: Top 5 most active Apache sources --visits: Spark, Flink, Camel, Kafka, Beam;
  • 25th anniversary of the Apache HTTP Server (21 years under the ASF umbrella);
  • 748 Individual Contributor License Agreements (ICLAs) signed;
  • 33 Corporate Contributor License Agreements signed;
  • 40 Software Grant Agreements signed; and
  • ASF was a mentoring organization in Google Summer of Code for the 15th consecutive year.


The full report is available online at https://s.apache.org/FY2020AnnualReport 

About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation (ASF) is the world’s largest Open Source foundation, stewarding 227M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 813 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 7,800+ Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Indeed, Inspur, Leaseweb, Pineapple Fund, Private Internet Access, Red Hat, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF 

© The Apache Software Foundation. "Apache", "Arrow", "Apache Arrow", "AsterixDB", "Apache AsterixDB", "Beam", "Apache Beam", "Camel", "Apache Camel", "Cordova", "Apache Cordova", "Flex", "Apache Flex", "Flink", "Apache Flink", "Geode", "Apache Geode", "Apache GitBox", "Hadoop", "Apache Hadoop", "HBase", "Apache HBase", "Apache HTTP Server", "Ignite", "Apache Ignite", "Kafka", "Apache Kafka", "Lucene Solr", "Apache Lucene Solr", "Mynewt", "Apache Mynewt", "NetBeans", "Apache NetBeans", "OpenOffice", "Apache OpenOffice", "POI", "Apache POI", "Royale", "Apache Royale", "Spark", "Apache Spark", "Thrift", "Apache Thrift", "Tomcat", "Apache Tomcat", "Trafodion", "Apache Trafodion", "Whimsy", "Apache Whimsy", "ZooKeeper", "Apache ZooKeeper", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Friday July 24, 2020

The Apache News Round-up: week ending 24 July 2020

Happy Friday! We've had a great week within the Apache community. Here's what happened:

Inside Infra – the interview series with members of the ASF Infrastructure team.
 - Meet Greg Stein --Part III https://s.apache.org/InsideInfra-Greg3

ASF Board – management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 19 August 2020. Board calendar and minutes https://apache.org/foundation/board/calendar.html

ApacheCon™ – the ASF's official global conference series, bringing Tomorrow's Technology Today since 1998.
 - Registration is OPEN (and free) for ApacheCon@Home taking place online 29 September - 1 October. Join us! https://www.apachecon.com/acna2020/ 
 - Sponsorships available for ApacheCon@Home https://www.apachecon.com/acna2020/sponsors.html 

ASF Infrastructure – our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield uptime at 99.96%. Performance checks across 50 different service components spread over more than 250 machines in data centers around the world. http://www.apache.org/uptime/

Apache Code Snapshot – Over the past week, 407 Apache Committers changed 4,357,953 lines of code over 3,609 commits. Top 5 contributors, in order, are: Gary Gregory, Andrea Cosentino, Mark Miller, Zhang Yonglun, and Sebastian Bazley.      

Apache Project Announcements – the latest updates by category.

Big Data --
 - Apache Atlas 2.1.0 released https://atlas.apache.org/
 - Apache Calcite Avatica Go 5.0.0 released https://calcite.apache.org/avatica
 - Apache Druid 0.19.0 released https://druid.apache.org/
 - Apache NiFi Registry 0.7.0 released http://nifi.apache.org/registry
 - Apache Flink 1.11.1 released https://flink.apache.org/

Content --
 - Apache Jackrabbit Oak 1.22.4 released https://jackrabbit.apache.org/oak

Libraries --
 - Apache Commons Lang 3.11 released https://commons.apache.org/proper/commons-lang/
 - Apache Geometry 1.0-beta1 released https://commons.apache.org/proper/commons-geometry/

Programming Languages --
 - Apache Groovy 2.4.20, 2.5.13, and 3.0.5 released http://groovy.apache.org/

Search --
 - Apache Lucene 8.6.0 and Solr 8.6.0 released http://lucene.apache.org/

Web Frameworks --
 - Apache Wicket 7.17.0 released https://wicket.apache.org/


Did You Know?

 - Did you know that Apache projects' ongoing sustainability is ensured through the generosity of our Sponsors and individual donors, whose support helps ensure that the ASF continues to provide more than $20B worth of software to the public-at-large at 100% no cost? http://donate.apache.org

 - Did you know that projects undergoing development in the Apache Incubator are in a variety of categories that include AI, Annotation, Big Data, Blockchain, Cryptography, Data Visualization, Distributed Computing, Email, Embedded Systems, Geospatial Data, Graphing, Hardware Acceleration, IoT, Messaging, Monitoring, Natural Language Processing, Scheduling, Streaming, Training, Usability Testing, and more? http://incubator.apache.org/projects/

 - Did you know that you can learn about Apache Beam, Calcite, Camel, CarbonData, Groovy, Hadoop, Karaf, Labs, NetBeans, OFBiz, OpenOffice, PLC4X, Rya, Spark, Tomcat, Unomi, and more in the "Apache Innovation" short? https://s.apache.org/ApacheInnovation

Apache Community Notices

- Apache Month In Review: June 2020 – overview of events that have taken place within the Apache community https://s.apache.org/June2020 

- "Trillions and Trillions Served" – the documentary on the ASF filmed onsite at ApacheCon Las Vegas and Berlin in 2019 have been released: 1) full feature https://s.apache.org/Trillions-Feature 2) "Apache Everywhere" short https://s.apache.org/ApacheEverywhere 3) "Why Apache" teaser https://s.apache.org/ASF-Trillions 4) “Apache Innovation” shorts https://s.apache.org/ApacheInnovation 

 - The Apache Software Foundation Statement on the COVID-19 Coronavirus Outbreak https://s.apache.org/COVID-19  

 - The Apache Software Foundation Celebrates 21 Years of Open Source Leadership https://s.apache.org/21stAnniversary

 - The Apache Software Foundation Operations Summary: Q3 FY2020 (November 2019 - January 2020) https://s.apache.org/r6s5u

 - Apache in 2019 - By The Digits https://s.apache.org/Apache2019Digits

 - The Apache Way to Sustainable Open Source Success https://s.apache.org/GhnI

 - Foundation Reports and Statements http://www.apache.org/foundation/reports.html

 - "Success at Apache" focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Inside Infra: the new interview series with members of the ASF infrastructure team --meet Christ Thistlethwaite https://s.apache.org/InsideInfra-Chris | Drew Foulks https://s.apache.org/InsideInfra-Drew | Greg Stein Part I https://s.apache.org/InsideInfra-Greg and Part II https://s.apache.org/InsideInfra-Greg2 

 - Did you know that Beam Summit 2020 will be held 24-28 August online and free of charge? https://beamsummit.org/

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter (https://twitter.com/TheASF) and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/ and Twitter account https://twitter.com/ApacheCommunity

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.


Friday July 17, 2020

Inside Infra: Greg Stein --Part III

The close of the "Inside Infra" interview with ASF Infrastructure Administrator Greg Stein, who shares his experience with Sally Khudairi, ASF VP Marketing & Publicity. 




"Apache is growing: we're just seeing the demand explode and it's a hard problem for us to solve."



PART THREE.


We were talking about ensuring that the team is up to speed with everything required of them...


So there certainly are skill gaps; this is one of the things I want to help motivate the team with, where if somebody says, "Hey, I want to go and investigate Ansible as a potential Puppet replacement," I say, "Go forward." 


This would be similar to Google having their 20% projects. I'm sure you've heard of that.


Oh, yeah.


It's almost the same where it's not 20%, maybe 5%, but it's the same as Google, no matter what they want to tell you, because everybody's got their job and you have to be really rigorous to carve out 20% of your time. And strictly speaking, it does actually make your Google manager a little upset if you carve out the entire 20%. But anyways, the concept is similar.


So for us it’s like, "Well, go in and investigate Ansible, see if it'll work for us and put your notes into the Wiki." That's how we make forward progress, up our game, and learn new skills. If someone says, "I want to go and figure this out," the response is almost always, "Okay. You go do it." There's certainly an allowance for people to learn new skills. But most of the time we simply rely on, say, Gavin (ASF Infrastructure team member Gavin McDonald), knowing more about JIRA configuration than the other guys.


That added component of sharing what you know, and adding it to the JIRA or to the Wiki actually is great because then everyone's learning. This is like the rising tide: everybody's learning about this, whether they're doing it perfectly or not. I think this is a very interesting process.


Yes, and that's also where Andrew (technical writer Andrew Wetmore) is helping us out. He’s organizing that information that we have learned, that we have documented, that we memorialized into the Wiki.


Because our (ASF’s) legacy is quite Medusa-like over all these years, it's interesting to see how everyone can get caught up and also contribute...you have to go back and deal with the legacy, but you also have to be able to move forward. To be able to bring others with you is brilliant. That's really cool.


The infrastructure has grown organically over 25 years from when Brian Behlendorf first said, "Hey, I have this server called hyperreal.org: you can run a CVS repository on it for the Web server."


That computer was under his desk at the Wired offices way back when, wasn’t it...


Yes it was. And it's just grown organically over those 25 years. Then we had Minotaur and it did six different things ... now it only does half of one and we've moved the stuff out onto newer machines and newer processes and this and that. But the organic growth means that we've got some really hairy stuff. Our move to Puppet --first Puppet 3, and now to Puppet 6-- at each step we're improving it and making it less hairy and more manageable and something that somebody can come along, look at, pick up and run with it from there. That makes it a lot easier, so that we don't have to spend 100% of our time cross training.


What are your thoughts on products, the hype cycle, where everyone's demanding Kubernetes, to use that as an example. Do you decide which products to provide support for, or is that up to Apache projects in the communities? You mentioned Ansible, just not too long ago, that was your internal decision to move. But I remember not long ago, GitHub entered into the landscape. How did that happen? How did you decide to make a move like that? That's a significant thing. Can you tell me a little bit about that?


It's a lot based on community input. So if we see a lot of people asking for a particular tool, we'll like, "Oh, hey, David, can you go and take a look at that and see if that's something…” Not David (ASF VP Infrastructure David Nalley), but Chris (Infrastructure team member Chris Lambertus) or somebody else. "Can you go take a look. Is that something that we can support? Because we're getting some queries about it."


And there's a little chicken and egg problem there that if the communities don't know to ask for the egg, we don't know whether to prep the chicken. It's like, “okay, wait, they don't even know to ask for a tool because we haven't said we will make this tool available, because we're not going to make the tool available until somebody asks”. But sometimes people file tickets like, "Can I get this set up?" and we'll go, "No."


Then six months later, somebody else will file a ticket: "Can I get this set up?" and we'll go," No." But after enough of those, we're like, "Maybe that's something that we really want to do." For GitHub, specifically that’s what happened there. Well, even before that Git, where we ran our own Git server, that was a volunteer that made that happen. That was, six years ago or so.


Well...the volunteer came along and said, "Well, I'll do this. I'm not going to take any time from Infra." There's been a couple things for the past few years where I've told people, "No, Infra will not work on that. But if you want to volunteer or find a volunteer, then we'll stand it up for testing." You know what I mean? Why not? So there's a couple things where people have stood up for test examples and there hasn't really been a lot of usage.


So, we're not going to support that. But something like Ansible is our own internal workflow and the tool we’ll experiment with, then to see if it'll improve our stuff. But from the community, they pretty much have to ask and it has to be a sustained ask. That's how we ended up with Travis CI: we actually pay for capacity in Travis CI, and that's based on community input.


So many people wanted to do their continuous integration through Travis that eventually we decided to pay for it. But it's tricky because some of these systems like Travis CI and others require certain permissions that we don't want to provide to the community. So we will want to hold those only within Infra. And so it gets hard to integrate certain tools. We've had to say no, but then again, we've found other ways to improve that so that we can lock down the permissions or use a proxy or other ways that we can route around some of these issues and then integrate the requested tool.


So further to that, have you been in a situation where a project or a community has made unreasonable demands of Infra or have expectations, where it's like, so over the top or so out of scope, it totally surprised you? Have you had something like this?


Nothing surprises me.


Nothing surprises you? Okay. Have you been in this situation? Like “was never going to happen”...


Yes, yes. There's been several times where one of the guys on the team is like, "Oh man, I got this ticket. I don't know what we want to do with this. Greg, go take a look." And I go and look at it and that's where I make that call: "Okay, is the Infra team going to take this on, or do I just say ‘no’ right now?"


So, yeah, there's been a number of times where I've said no and probably two or three times where I've gotten a little bit of pushback on that no. I say, "My answer is no, but here's how you escalate." I've had escalation a few times and I'm actually, mid-process --I'm dealing with one right now. So, I've said, "no, if you don't like my no, you can go to VP Infra and VP Infra is, probably going to tell you the same thing. And then you can go to the President. Right now those are actually the same person."


The same person is a double "no".


That really is the true escalation path. I have to describe that to people and say, "I don't think you're going to get what you want." If I'm the one that says, no, you probably are not going to get it because VP Infra and President, and after that is the Board. They're probably not going to say, "Greg is wrong. Yes, we'll give that to you." But it's there. There's been a couple of times where I said "No, you have to ask the Board for the budget for those additional virtual machines." They went to the board and said, "Can we have budget for three machines?" and the Board said, "Yes."


So Infra went ahead and gave them the three VMs that they had initially requested. Strictly speaking, we would track those machines against their budget, but that detail is more than what the actual budget was. So we don't spend that time doing that, but I have had to say, no. I have had to... There was Apache Maven: they were keeping a copy of Maven Central, and Maven Central is run by Sonatype...


Which is a commercial product...


Yes. They're using the trademark “Maven”, essentially a licensing agreement from us, a MOU. So with Maven Central, you could imagine if someone decides to just turn it off one day ...we wanted a copy. Apache Maven was making a copy of it, and it just started consuming so much disk space. We were like, "We can't support that growth rate. We can't support that even for the next six months. If you want to keep doing it, go ask the Board for money to keep doing it." They never did. We turned it off.


I wouldn't call that a ridiculous request --it was something where we didn't have to just say, "No, not going to do it. Bye." A lot of the requests are mostly just, "We aren't going to run that extra software. If you want: ask for a VM and you can run it, but we're not going to take responsibility for it."


Over the years, obviously ASF Infra has changed. Was this all reactive or was it also proactive? Do you plan for those changes as you go or has it all been in response to Project X or in response to X emergency?


The growth of Infrastructure and its movement from volunteer-only to paid staff was part of just the growth of Apache. The volunteers could no longer keep up and things, like account creation, used to take sometimes four weeks to get an account. You’d put in a request for an account, four weeks later, it would finally get created.


My gosh, that queue was crazy, huh?


Well, it wasn't even a long queue, it was simply that we didn't have volunteers making sure the queue stayed empty. Today it's down to one, two, maybe three days, and the account is created, because every day a staff member goes and creates the accounts first thing in the morning.


It was how I said that my day starts with looking at messages on Slack and then reading emails to see if there's stuff to handle. Well, one of the guys on staff, first thing he does in the morning is go and look at account creation. So he's been off and on pondering on a tool to make that easier for himself; he hasn't finished the tool, so he still has to do it manually. That's his incentive.


“Work quickly”...


This is Chris Thistlethwaite. I say, "Chris, we can do something about that." And he says, "No, no, this is still my project. And every day when I run the script, it just makes me remember, I need to finish this."


So when the volunteers could not keep up with the amount of work, that's when we hired Joe Schaefer, then we hired another person, and hired another person. And so it was just trying to keep up with the rate of requests. 


That's how we ended up with hiring six people. And then I'm half a person, like I said, I'm part-time. So, it's just the growth of Apache. I think we're in much better shape than when I started. We're ahead of the curve. We can stay ahead of the curve because one of the things that I can do because I don't fight the fires every day ... that's for all the guys who know their stuff. They fight the fires and I can look at if I need to go and ask for another head count. And that's how we ended up with Andrew (technical writer Andrew Wetmore): “Well, you know, what we really need is somebody to manage all this documentation.” This was part of Sam's (former ASF President Sam Ruby), “If you had some money, what would you do with it?” That's how the technical writer/editor came around, because we've got 20 years of organic growth. We had...let's just call it “organic documentation”. That revamping project is going really well, I think.


So, in what areas are you guys experiencing your biggest growth? As I was asking Chris and Drew, is there like a geographic influence on the demand? We’ve had a huge influx of users in China. Does any of that change the way or what you guys are doing? Or is it just more of everything?


Our biggest pain point, I would say, is continuous integration/continuous development: CI/CD. Jenkins, Travis, CircleCI, and things like this, where people make a change and they want that change built and tested. The more projects we get and the larger the communities get, the more changes and the more testing and the more building and the more this, more, more, more. It's kind of one of those things where it's “expand-to-fit”. So if we gave people 100 machines, they'd use 100 machines. If we doubled it to 200, they'd use all 200. It's just this rapacious need for CI machines. It's very hard to figure out how to plan around that other than just telling the communities, “No: we just don't have that much capacity: if you want to build it, do it on your own machine. You just can't use Apache hardware to do it.”


That's an unsatisfactory answer. That's been one of our hard problems and it's also kind of a newer problem: the development workflow that uses CI probably is just maybe five years old. Before that, certainly, automated building and testing was a thing, but it's really kind of grown into community workflow much, much more over the past five years, and more and more people are wanting to do it. The communities are growing. Apache is growing: we're just seeing the demand explode and it's a hard problem for us to solve.


China is the one case where we see regional issues, and that's because of the great firewall of China. Not because we're getting more Chinese developers, but because they have problems accessing our servers because they're located outside of China, and so we're looking at CDNs, a content distribution network to essentially make our content available closer to China. We've found that even with one of those CDN drop points in Hong Kong, they still have problems just reaching it there in Hong Kong, and so ... and we don't want to buy or lease or rent a server in China because doing business in China is too high of a hurdle for the Foundation. 


Oh? 


You know, Microsoft and Google have to do business in China and they've got a pack of lawyers and a giant vault of money to deal with all the barriers. The Foundation does not, so it's also a hard problem to solve. We think we might be able to do it through Microsoft Azure, that they have a CDN that resides in China that Microsoft has done all that paperwork, so we're looking at that, but as far as regional things, it's not so much that we run into issues. We see Open Source communities in Europe and Brazil and Australia and Sri Lanka: none of them really have any problems because they don't have that firewall. It's not really about the Chinese people, but about the China firewall. 


That's bigger than us. And that’s not something we can fire hose.

 

We do see little engagement from Japan and Brazil, and that is partly for language reasons and partly because the Brazil community is more about Free Software than Open Source software. 


Yeah. They're very pro-FOSS.


Not OSS. But pro-free. And so, they're going to deal with the Free Software Foundation rather than the Apache Software Foundation.


I see. That’s an important distinction. 


And then you also have the Portuguese language barrier. People contributing from Europe and India, Sri Lanka, etc., they pretty much know English and that's fine. A lot of the Brazilian developers do not know English...this is the same with the Japanese Open Source developers. Japanese and Brazilian, they tend to not know English, and so that kind of isolates them from the larger Open Source world, or Free Software world, in the case of Brazil.


Would we consider localizing anything that we do, or are we going to continue as-is, as the ASF is all English?


The Infrastructure team will not translate our documents to serve those other languages. That's just too high of a bar.


There are a couple groups that have user mailing lists that are not English and that's totally fine, and Infrastructure will... well, you don't have to file a ticket anymore. It's, again, back to selfserve.apache.org: “self-serve” on Apache will create a mailing list for users communicating in Brazilian Portuguese, for example, or communicating in Japanese. But Infra doesn't do anything about that, that's just the self-serve tools. We certainly can't support non-English, and I don't think that the Foundation itself is going to make any moves towards that.


Fair enough. So a lot of companies are really struggling to accommodate their teams working from home in response to the Coronavirus and all that. These stay-at-home orders are kind of shaking companies, but from day one, the ASF has always been a virtual organization. Has anything changed with your operation on that front? Has anything impacted the ASF's day-to-day, from this pandemic?


(chuckling) Not at all. I shouldn't laugh, but no. It really hasn't changed. We've been on our team channel for all three years, three and a half years that I've been here, and the world is burning down around us, but we still sit on the team channel.


Now, that said, (Infra team member) Daniel Gruno got stranded in Canada.


Right! He’s still there?


He's still doing work from Canada. This is why when he travels to Canada for two months at a time, I don't care, you know? Because if his butt is in a chair in Denmark or in a chair in Canada, it's the same butt, so, you know...


As long as you have connectivity and a computer, you can do it. 


Right. But if he has to be offline for two months, I'd say no. Or if you want unpaid time off, well, I'm not going to pay you, of course. Certainly the discussions have changed, you know? I mean, going shopping. You know, some members are immuno-compromised and that had an effect on our team meeting that we were planning in Nashville: they were the first to say, “No way. I'm not going,” so, there’s that, but our day to day hasn't changed.


That's more of a social thing versus an operational thing. Safety first.


So the notion of, “Oh, I got to run out to the grocery store. I need to strap on a mask,” changes, but not the operation.


Right. Right. So...what do you think people would be surprised to know about ASF Infra?


I don't know if it'd be surprising, but we are global. We've got four people in the United States, one in Canada, one in Denmark, one used to be in Australia, but is now in the UK, which actually kind of hurt a little bit, because in Australia, that meant that we always had somebody in that time zone, but now we have kind of this gap of Australia/Asia time zones when...


A “Gavin” gap.


Yeah, well, I might be awake at that time, but I can't go and fix a MySQL server, so it does mean that we don't have that straight-up 24-hour coverage.


The notion that we are worldwide is kind of a neat thing about our team, and is what makes us pretty unique relative to other IT departments. I don't like being called an IT department, but that is essentially what we are. 


Surprise.


What's the name of that TV show? The one that's about IT...


“The IT Crowd”, is that what you’re referring to? The British show?


Yeah. So, you know, that's a funny show, but mostly when you think “IT department”, you think of some corporate people with button-up shirts, but ...most of us, we're in our pajamas.


Good one. What's your favorite part of the job?


I definitely like the team and that's why, nominally I'm part-time, but I'm pretty much constantly on the team channel and interacting, and so I think I just put that down as volunteer hours, where before I might work on Apache Subversion, but now I hang out with the team or I write some little tool or something like that. That's definitely been one of the more rewarding changes. Up until I started with this, I'd been a director for 15-and-a-half years, and that was kind of how I contributed to Apache. Now my work for Infrastructure is a new way to contribute to the Foundation. I'm also part of a new community, where before I would hang out with the httpd community, APR community, the Subversion people ...now it's the Infra people and my hobby time is kind of blended in with my work time, and vice versa. I mean, when your work time can also be seen as a hobby time, that's pretty cool.


I do think it's the team that makes it interesting. That's what I like the most, and that I'm working with a new, interesting community to contribute to the Foundation. 


Not only did you switch roles, you switched communities. What was your biggest challenge going into this new role?


I would say probably trying to delineate what I was going to handle for the guys and that I wasn't going to tell them what to do or how to do it. It's like, “OK, I'm here to assist, to unblock things, to enable you guys, rather than to block you or micromanage you.”


To earn that trust, that I wasn't going to be some pointy-haired boss telling them how to do their work. Now, I don't know if that was ever a problem for them, but that was certainly one of my initial concerns: how to properly create my role. This was the first time Apache's even had somebody fill in this role, so I also had to find the role, which is, again, why I came up with “Infrastructure Administrator”, is because I wanted to define it as an enabler role, as an administrator, so they could get their work done but I would not be their manager. I would not be their boss: I was simply there to enable them.


So, what are you most proud of in your infra career to date?


Ooh. I don't know. I would say by being hands-on, being the “hands” of Infra, it means that VP Infra didn't run away screaming.


David said in January 2016, maybe earlier, he was like, “No way. I'm out.” And after I was on the job for about two months, he said, “Huh. All right.”


“I'm in!”


And so I get that feedback from him, “You know, you make the VP Infra hat quite easy for me.” I think that's probably what I really like about taking on the role, is that one of our volunteers got to stay rather than drop it because it was just causing so much anxiety and pain and time and frustration. Otherwise, most of the stuff I do is really boring. Not to me, but I don't have “accomplishments”. I push paperwork, basically, so the other guys can do accomplishments.


Speaking of the other guys, how would your co-workers describe you?


I have no idea. I don't know. I really don't know. (laughing)


Where I just got done talking about what I saw as an issue, trying to frame what my role would be, it might have been fine with them and I was overly worried about it, but it’s hard for me to know. We don't do 360 reviews in Infra, so I don't get any feedback, really, from the team on what they think about myself or how I'm doing my job, so you'd have to ask them. 


I have. Just kidding. So...what are the biggest “threats” that infrastructure managers or infrastructure administrators need to watch out for? What do you think is a “big thing” that people should be aware of, or is ASF so unique that you don’t feel like anyone really experiences what you experience?


There's our capacity issue with things like Travis, but I think you're asking a different question.


I am, but that's fine. What's your greatest piece of advice? What would you tell aspiring infra administrators?


Actually, one of my greatest fears is really, as a small charitable foundation, it's hard for us to compete with well-funded corporations and some well-funded start-ups.


Related to that, I touched on it earlier, is career development ...you go into Google or Microsoft and there's a career ladder; we simply don't have a career ladder. There's salary growth. There's bonuses. If you want to have a resume or a LinkedIn profile that shows changes in growth and titles and career ladder, we can't offer that, and that's going to cut out some people. It's a very hard problem for me to solve. You know, there's things I can maybe do, but I also want to keep the team egalitarian and sort of level, rather than, “Oh, well, this guy is now the team lead.”


Given what I talked about, our social aspects, because we are all equal peers, keeping everybody with the same title, same position on the ladder means that we are peers and it's a little easier to interact that way. It's a real, real difficult problem. You ask what's scary: that's scary.


But there's a counterpoint to that. You may not have a traditional career ladder path, but to say that you've worked in Infra for Apache carries weight. That's significant. 


I believe it does, especially when you can demonstrate the hundred different types of tasks...


Well, that's exactly it. The breadth of work and the scale of what you guys do and the skill sets that you have to have and the fact that you have to play nice in the sandbox, all of it. The demand is immense, so to be able to be there and thrive and develop something from yourself in terms of a career is tremendous. Our team is exceptional. I mean, they're not expecting a linear ladder or something that others have.


You know, in other jobs, somebody might say, “I was a MySQL administrator.” Here, you're a MySQL administrator, PostgreSQL administrator… They had one role; here you've got dozens. 


If you had a magic wand, what would you see happen with ASF infra?


I'd like to solve that CI problem. The other magic wand would be upgrading our mail server from 10-year-old technology to modern technology.


Is that happening or is that literally a wish list issue?


It's happening, but it's been happening for three years. The thing is that email is so central to the Foundation that we can't really experiment with that. There are certain things we can do, but most of it, not so much, and so it means that we're being super-careful. There's about 10-12 different moving parts to it, and we're upgrading each of those a little bit by a little bit, until we can finally pull that big, scary, Young Frankenstein lever to hit the lightning bolt, you know?


Yeah: I see the visual of that.


The magic wand would be to just make that all happen and make it work. Without the wand, it's going to take another 6-12 months.


Right. What else do we need to know that I haven't asked? What should I be aware of or what should I be sharing?


Oh, I don't know. This is where my creativity ends. Ask me a coding question.


Oh no coding questions. All right. Our time has also ended. Before we go, who should I be interviewing next? 


I would say Daniel (Gruno), because his role ... he's 20-30% system administration. The rest is tool development, so that makes his role rather unique in the team.


Perfect. Thanks so much, Greg. I really appreciate it. 


= = =

Greg is based in Austin on UTC -5. His favorite thing to drink during the workday is a big 32oz cup of Diet Mountain Dew.


Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation