The Apache Software Foundation Blog

Friday March 01, 2019

The Apache News Round-up: week ending 1 March 2019

Hurrah for March! Let's review what the Apache Community has achieved this week:

ASF Board –management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - The Apache Software Foundation Operations Summary: November 2018 - January 2019 https://s.apache.org/nUxz
 - Next Board Meeting: 20 March. Board calendar and minutes http://apache.org/foundation/board/calendar.html

ApacheCon™ –the ASF's official global conference series, now in its 21st year.
 - Join us: Apache Roadshow/DC and Open Source Career Fair 25 March 2019 http://www.apachecon.com/usroadshow18/
 - Join us: Apache Roadshow/Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html
 - Save the Date: ApacheCon North America 9-13 September in Las Vegas http://apachecon.com/
 - Save the Date: ApacheCon Europe 22-24 October in Berlin http://www.apachecon.com/aceu19/

ASF Infrastructure –our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield 99.93% uptime. http://www.apache.org/uptime/

Apache Code Snapshot –this week, 493 Apache contributors changed 1,098,430 lines of code over 2,927 commits. Top 5 contributors, in order, are: Vishal Suvagia, Andrea Cosentino, Tzu-Li Tai, Zhang Duo, and Claus Ibsen.

Apache Project Announcements –the latest updates by category.

Big Data --
 - Apache Trafodion 2.3.0 released https://trafodion.apache.org/

Content --
 - Apache UIMA Ruta 2.7.0 released http://uima.apache.org/
 - Apache PDFBox 2.0.14 released http://pdfbox.apache.org/

Databases --
 - Apache Jackrabbit Oak 1.10.1 released http://jackrabbit.apache.org/

 Library --
- Apache Daffodil (incubating) 2.3.0 https://daffodil.apache.org/

Messaging --
 - Apache Pulsar 2.3.0 released https://pulsar.apache.org/

Network-Server --
 - Apache Tomcat 7.0.93 released http://tomcat.apache.org/

Did You Know?

 - Did you know that the following projects are celebrating anniversaries this month? Apache mod_perl (19 years); Apache Maven (16 years); Apache Struts (15 years); Apache Jackrabbit (13 years); Apache Felix (12 years); Apache Archiva (11 years); Apache UIMA (9 years); Apache Accumulo and Apache Sqoop (7 years); Apache Bloodhound, Apache CloudStack, and Apache cTAKES (6 years); Apache Allura, Apache Olingo, and Apache Tajo (5 years); Apache Aurora (4 years); Apache Sentry (3 years); and Apache FreeMarker (1 year) -- many happy returns! https://projects.apache.org/committees.html?date

 - Did you know that Flink Forward San Francisco 2019 is promoting collaboration between Apache projects by are inviting 50 Apache Committers to attend for FREE? Use "ASFCommitters19" code PLUS your @apache.org email when registering at https://sf-2019.flink-forward.org/register

 - Did you know that Apache Myriad are welcoming contributors, with an initial request to review the new v0.3.0-rc1? https://www.mail-archive.com/dev@myriad.incubator.apache.org/msg02661.html

Apache Community Notices:

 - Apache in 2018 - By The Digits https://s.apache.org/Apache2018Digits

 - ASF Operations Summary: Q2 FY2019 https://s.apache.org/d2Fq

 - ASF Annual Report for FY2018 https://s.apache.org/FY2018AnnualReport

 - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3

 - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA

 - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity

 - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html

 - The Apache Big Data community will be at DataWorks Summit 18-21 March 2019 in Barcelona and 20-23 May 2019 in Washington DC https://dataworkssummit.com/

 - Future dates for Spark + AI Summit 2019 announced: 23-25 April/San Francisco and 15-17 October/Amsterdam https://databricks.com/sparkaisummit/

 - CFP open for Berlin Buzzwords: 16-18 June 2019 https://berlinbuzzwords.de/

 - Save the date: ApacheCon North America 9-13 September 2019 http://apachecon.com/

 - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache HTTP Server, Avro, ComDev (community development), Directory, Incubator, OODT, POI, Polygene, Syncope, Tika, Trafodion, and more! https://helpwanted.apache.org/

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

# # #

The Apache Software Foundation Operations Summary: November 2018 - January 2019

FOUNDATION OPERATIONS SUMMARY

Third Quarter, Fiscal Year 2019 (November 2018 - January 2019)

"Happy to be a sponsor of one of the most important foundations in the world!"
Randy Abernethy, Managing Partner, RX-M LLC (ASF Bronze Sponsor)

> Conferences and Events: During this period, the Events team was mostly focused on preparations for upcoming events, as reported in our previous quarterly report, so there's not much to add. Our current status, as of 31 January, is as below:

  • Apache Roadshow, DC
    After several setbacks, the DC roadshow has been rescheduled for March 25th at George Mason University. Event details, and the full schedule of talks, may be found at http://apachecon.com/usroadshowdc19

    The DC roadshow will also feature a job fair.

  • Apache Roadshow, Chicago
    The planning for the Chicago roadshow is moving along nicely. It will be held May 13-14 in Logan Square, and details may be found at http://apachecon.com/chiroadshow19/index.html

  • ApacheCon North America 2019
    During the period of this report, we signed agreements to hold ApacheCon North America 2019 in Las Vegas, Nevada, September 9-12, 2019. This event celebrates the 20th anniversary of the Apache Software Foundation. The call for presentations will open soon, and sponsor opportunities will also be available soon.

  • ApacheCon Europe 2019
    Planning for ApacheCon Europe 2019 is well underway, and you can expect announcements and further details very soon.


> Community Development
One of the main themes raised this quarter was around ways to encourage new contributors by lowering the barriers to entry and other techniques that could help onboard them.

Some new contributor onboarding material tailored specifically tailored around Apache and participation in Apache projects is currently being developed and the community feedback has been very positive.

Discussion has also started about how to recognize non technical contributions. This is an area where Community Development could really help in the development of materials and processes, as well as providing guidance to projects on how it could be implemented.

In December, we participated at the Paris Open Source Summit, one of the main Open Source events in France. We had an ASF booth and over the two day summit saw lots of attendees who were keen to know more about the ASF. 

We were also invited to participate in a workshop to talk about how the administrative sector could help increase collaboration with Open Source foundations and communities.

The ASF has once again has applied to be a mentoring organisation for GSoC. In preparation, we have asked all ASF projects interested in participating to start recording their ideas.

We are continually receiving requests to participate at events and would like to be more organised in selecting and planning our event participation. As a first step we have asked our community for their feedback on which events they think are the most important ones for us to be involved. Based on this we hope to create a high level event participation plan for 2019.

Our mailing list subscriptions have increased this quarter and even though we have had a lot of active discussions, the traffic has decreased, probably due to the December holiday period.

> Committers and Contributions: Over the past quarter, 1,615 contributors committed 45,054 changes that amount to 18,612,331 lines of code across Apache projects. The top 5 contributors during this timeframe were: Andrea Cosentino (1,142 commits), Jean-Baptiste Onofré (747 commits), Tilman Hausherr (599 commits), Mark Thomas (594 commits), and Tellier Benoit (512 commits).


All individuals who are granted write access to the Apache repositories must submit an Individual Contributor License Agreement (ICLA). Corporations that have assigned employees to work on Apache projects as part of an employment agreement may sign a Corporate CLA (CCLA) for contributing intellectual property via the corporation. Individuals or corporations donating a body of existing software or documentation to one of the Apache projects need to execute a formal Software Grant Agreement (SGA) with the ASF. 

During Q3 FY2019, the ASF Secretary processed 232 ICLAs, 3 CCLAs, and 10 Software Grants. History of Apache committer growth can be seen at https://projects.apache.org/timelines.html

> Brand Management: Operations — The work of the Brand Management team falls broadly into one of three categories:

  • trademark transfers and registrations
  • granting permission to use our marks
  • addressing potential infringements of our marks

The volume of work fell slightly this quarter which was probably due to the end of year holiday period. The budget request for next financial year was finalized and submitted.

This quarter has seen the usual collection of requests to use Apache marks for user groups, events, merchandise and publications with nearly all requests being granted, subject to our Trademark Usage Policy. 

Registrations — The US registration of OPENOFFICE completed this quarter and we started the process to register IGNITE in the EU and China. We also worked with counsel to respond to office actions for a number of in-progress registrations.

With the assistance of counsel, we addressed a number of external registrations that overlap with our own with those external registrations being amended to avoid the overlap.

Infringements — Potential infringements are brought to our attention from both internal and external sources. The majority of infringements we see are accidental and our project communities are able to resolve these quickly and informally with occasional input from the Brand Management team. A small number of issues take longer to resolve. After last quarter's successes in resolving two of these more complex issues, this quarter has seen a small uptick in the number of complex issues we are tracking.

We saw an increase in projects reaching out to speakers and conference organizers this quarter to ensure that Apache projects were referred to correctly in conference session titles and abstracts.

And finally…

The Brand Management team welcomes your comments and suggestions as well as any questions you might have. Please see https://www.apache.org/foundation/marks/contact for our contact details.


> Infrastructure:
 It has been a very busy quarter for the Infrastructure team. We have been migrating many services to new approaches and systems, including gitbox, backups, and configuration management.

We have been reporting on the move to "Gitbox" for a while now, and this quarter the testing, development, and evolution is nearly at a close. This quarter, many hundreds of repositories have migrated from our old git provisioning to our new gitbox system. A mere hundred repositories are left, and will be completed in February, allowing us to shut down old hardware, old systems, and reduce our maintenance efforts. Meanwhile, the many tools that GitHub brings to development will be available to our projects.

This quarter has also seen a revamp of our backup systems to a simpler, straightforward mechanism across our multitude of virtual machines. We had a mix of commercial backup and in-house systems built on standard F/OSS tools. By concentrating on the in-house tooling, we have dramatically reduced costs and simplified our ongoing maintenance.

Our final, large system migration involves our use of Puppet for configuration management. For the past several years, we've been migrating our very old, organically-constructed systems onto Puppet version 3. While that process of bringing modern configuration management into our operations, Puppet has released many versions, now up to version 6. This quarter, we have begun to deploy new systems using this v6 setup. Looking forward, we'll begin shifting our operations to this new platform, to improve our operations.

In addition to these larger system moves, the team has kept busy with larger scale planning. Hiring more staff, longer term migration of our core email handling, five year budget planning, monitoring improvements, and decommissioning of our old hardware.

> Financial Statement:



> Fundraising:
 Fundraising has continued to go well this past quarter. We are pleased to have had several renewals from our awesome sponsors and we've even grown the ASF family of Sponsors and Targeted Sponsors! Welcome Amazon Web Services at the Platinum level, and RX-M LLC and Online Holland Casino at the Bronze level. We've recently begun our quarterly cycles of outreach via our Sponsor Ambassadors and look forward to hearing from our sponsors.

Work on reviewing the concept of a potential endowment or endowment-like project for the ASF has been transferred to VP Finance. The Fundraising committee remains engaged and eager to support however possible.

Fundraising support for ASF events (ApacheCon and Apache Roadshow) has picked up with the development of event prospectuses and assignments of Event Sponsor Ambassadors. We have also established and documented a new model for handling event fundraising.

It was wonderful to spend some time with our ASF family and friends at FOSDEM and look forward to doing the same at Apache Roadshow DC in March.

Internally the Fundraising committee continues to focus on improvements. Processes, documentation, record keeping and tracking is being reviewed and bolstered across the board. The team also thanks Kevin A McGrail for his hard work and dedication (VP and Co-VP Fundraising) over the past two years as he steps away from Fundraising.

= = = 

Thank you to all our Sponsors!

  • PLATINUM: Amazon Web Services, Cloudera, Comcast, Facebook, Google, LeaseWeb, Microsoft, Pineapple Fund, Oath, Tencent
  • GOLD: Anonymous, ARM, Bloomberg, Handshake, Hortonworks, Huawei, IBM, Indeed, ODPi, Pivotal
  • SILVER: Aetna, Alibaba Cloud Computing, Baidu, Budget Direct, Capital One, Cerner, Inspur, Private Internet Access, Red Hat, Target, Union Investment
  • BRONZE: Airport Rentals, Best VPN, The Blog Starter, Bookmakers, Cash Store, Casino Bonus, Casino2k, Cloudsoft, Emerio, Holland Online Casino, HostChecka.com, HostingAdvice.com, HostPapa Web Hosting, The Linux Foundation, Mobile Slots, RX-M LLC, SCAMS.info, Site Builder Report, Talend, The Best VPN, Twitter, Web Hosting Secret Revealed
  • TARGETED PLATINUM: DLA Piper, Microsoft, Oath, OSU Open Source Labs, Sonatype
  • TARGETED GOLD: Atlassian, The CrytpoFund, Datadog, PhoenixNAP, Quenda
  • TARGETED SILVER: Amazon Web Services, HotWax Systems, Rackspace
  • TARGETED BRONZE: Bintray, Education Networks of America, Google, Hopsie, No-IP, PagerDuty, Peregrine Computer Consultants Corporation, Sonic.net, SURFnet, Virtru

To Sponsor The Apache Software Foundation, visit http://apache.org/foundation/sponsorship.html . To make a one-time or monthly recurring donation, please visit https://donate.apache.org/

# # #

Report prepared by Sally Khudairi, Vice President Marketing & Publicity, with contributions by Rich Bowen, Vice President Conferences; Sharan Foga, Vice President Community Development; Mark Thomas, Vice President Brand Management; Greg Stein, ASF Infrastructure Administrator; Tom Pappas, Vice President Finance; and Daniel Ruggeri, Vice President Fundraising.

For more information, subscribe to the announce@apache.org mailing list and visit http://www.apache.org/, the ASF Blog at http://blogs.apache.org/, the @TheASF on Twitter, and https://www.linkedin.com/company/the-apache-software-foundation.

(c) The Apache Software Foundation 2019.

Friday February 22, 2019

The Apache News Round-up: week ending 22 February 2019

We hope you are having a happy Friday. The Apache Community has had a productive week, as always. Let's review:

ASF Board –management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 20 March. Board calendar and minutes http://apache.org/foundation/board/calendar.html

ApacheCon™ –the ASF's official global conference series, now in its 21st year.
 - Join us: Apache Roadshow/DC and Open Source Career Fair 25 March 2019 http://www.apachecon.com/usroadshow18/
 - CFP CLOSING TODAY: Apache Roadshow/Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html
 - Save the Date: ApacheCon North America 9-13 September in Las Vegas http://apachecon.com/

ASF Infrastructure –our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield 99.94% uptime. http://www.apache.org/uptime/

Apache Code Snapshot –this week, 457 Apache contributors changed 794,949 lines of code over 2,520 commits. Top 5 contributors, in order, are: Colm Ó hÉigeartaigh, Michael Stack, Iñigo Goiri, Claus Ibsen, and Andrea Cosentino.

Apache Project Announcements –the latest updates by category.

Big Data --
 - The Apache® Software Foundation Announces Apache Arrow™ Momentum https://s.apache.org/zhaP
 - Apache Beam 2.10.0 released https://beam.apache.org/
 - Apache Flink 1.7.2 released https://flink.apache.org/
 - Apache Kafka 2.1.1 released https://kafka.apache.org/ - Apache NiFi 1.9.0 released https://nifi.apache.org/

Library --
 - Apache Commons Codec 1.12 released http://commons.apache.org/
 - Apache Olingo 2.0.11 released http://olingo.apache.org/

Messaging --
 - Apache Pulsar 2.3.0 released https://pulsar.apache.org/

Testing --
 - Apache JMeter 5.1 released https://jmeter.apache.org/

Did You Know?

 - Did you know that Amazon Music Recommendations is powered by Apache MXNet (incubating)? http://mxnet.apache.org/

 - Did you know that China Merchants Bank, CMBChina Mall, and China Eastern Airline use Apache SkyWalking (incubating)? http://skywalking.apache.org/

 - Did you know that you can review and share your feedback on the first milestone of Apache Camel 3? http://camel.apache.org/

Apache Community Notices:

 - Apache in 2018 - By The Digits https://s.apache.org/Apache2018Digits

 - ASF Operations Summary: Q2 FY2019 https://s.apache.org/d2Fq

 - ASF Annual Report for FY2018 https://s.apache.org/FY2018AnnualReport

 - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3

 - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA

 - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity

 - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html

 - The Apache Big Data community will be at DataWorks Summit 18-21 March 2019 in Barcelona and 20-23 May 2019 in Washington DC https://dataworkssummit.com/

 - Future dates for Spark + AI Summit 2019 announced: 23-25 April/San Francisco and 15-17 October/Amsterdam https://databricks.com/sparkaisummit/

 - CFP open for Berlin Buzzwords: 16-18 June 2019 https://berlinbuzzwords.de/

 - Save the date: ApacheCon North America 9-13 September 2019 http://apachecon.com/

 - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache HTTP Server, Avro, ComDev (community development), Directory, Incubator, OODT, POI, Polygene, Syncope, Tika, Trafodion, and more! https://helpwanted.apache.org/

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

# # #

Tuesday February 19, 2019

The Apache® Software Foundation Announces Apache Arrow™ Momentum

Open Source Big Data in-memory columnar layer adopted by dozens of Open Source and commercial technologies; exceeded 1,000,000 monthly downloads within first three years as an Apache Top-Level Project

Wakefield, MA —19 February 2019— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, today announced momentum with Apache® Arrow™, the Open Source Big Data in-memory columnar layer.

Since the founding of the project in January 2016, Apache Arrow has quickly become the defacto standard for representing and processing analytical data in memory, accelerating analytical processing and interchange by more than 100x.

"When we became a Top-Level Project, we projected that the majority of the world's data will be processed through Arrow within the next decade," said Jacques Nadeau, Vice President of Apache Arrow. "In just three years time, we are proud to see Arrow's substantial industry adoption and increased value across a wide range of analytical, machine learning, and artificial intelligence workloads."

Highlights of Apache Arrow's success include:

Industry Adoption —more than 20 major technologies adopted Arrow to accelerate in-memory analytics, including Apache Spark, NVIDIA RAPIDS, pandas, and Dremio, among others. A list of known Open Source and commercial implementations can be found at https://arrow.apache.org/powered_by/

Millions of Downloads —leveraging and integrating Apache Arrow into many other technologies has bolstered downloads to more than 1,000,000 each month.

New Language Support —as a cross-language development platform, supporting multiple programming languages is paramount. Apache Arrow has grown from supporting one language to eleven different languages today; they include C++, Java, Python, R, C#, Javascript, and Ruby, among others.

Seamless Data Format Support —Arrow supports different data types, both simple and nested, located in arbitrary memory such as regular system RAM, memory-mapped files or on-GPU memory. In addition, it can ingest data from popular storage formats such as Apache Parquet, CSV files, Apache ORC, JSON, and more.

Major Code Donations —Apache Arrow's new features and expanded functionality are due in part to code and component donations that include:
  • C# Library
  • Gandiva LLVM-based Expression Compiler
  • Go Library
  • Javascript Library
  • Plasma Shared Memory Object Store
  • Ruby Libraries (Apache Arrow and Apache Parquet)
  • Rust Libraries (Parquet and DataFusion Query Engine)
Community and Contributor Growth —over the past 12 months, nearly 300 individuals have submitted more than 3,000 contributions that have grown the Apache Arrow code base by 300,000 lines of code. The Arrow community is welcoming approximately 10 new contributors each month.


In January the project announced its most recent release, Apache Arrow 0.12.0, which reflects more than 600 enhancements developed during Q4 2018. The Apache Arrow community is actively working on a number of impactful new initiatives that include solving high performance analytical problems and allowing for more efficient data distribution across entire clusters.

"Apache Arrow's rapid industry adoption and developer community growth supports our original thesis of the importance of a language-independent open standard for columnar data," said Wes McKinney, member of the Apache Arrow Project Management Committee, and creator of Python's pandas project. "Additionally, we are seeing productive collaborations take place not only between programming languages but also between the database systems and data science worlds. We look forward to welcoming more data system developers into our community."

About Apache Arrow
Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and interprocess communication. Languages currently supported include C, C++, C#, Go, Java, JavaScript, MATLAB, Python, R, Ruby, and Rust.

Availability and Oversight
Apache Arrow software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Arrow, visit http://arrow.apache.org/

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 730 individual Members and 7,000 Committers across six continents successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official global conference series. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Hortonworks, Huawei, IBM, Indeed, Inspur, LeaseWeb, Microsoft, Oath, ODPi, Pineapple Fund, Pivotal, Private Internet Access, Red Hat, Target, Tencent, Union Investment, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF

© The Apache Software Foundation. "Apache", "Arrow", "Apache Arrow", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Friday February 15, 2019

The Apache News Round-up: week ending 15 February 2019

Mid-month already! Let's see what Apache Community worked on last week:

A little Valentine's appreciation to all our Sponsors, whose support demonstrates how they #LoveApache http://apache.org/foundation/thanks

ASF Board –management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 20 February. Board calendar and minutes http://apache.org/foundation/board/calendar.html

ApacheCon™ –the ASF's official global conference series, now in its 21st year.
 - Join us: Apache Roadshow/DC and Open Source Career Fair 25 March 2019 http://www.apachecon.com/usroadshow18/
 - CFP CLOSING TODAY: Apache Roadshow/Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html
 - Save the Date: ApacheCon North America 9-13 September in Las Vegas http://apachecon.com/

ASF Infrastructure –our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield 99.88% uptime. http://www.apache.org/uptime/

Apache Code Snapshot –this week, 477 Apache contributors changed 776,105 lines of code over 2,899 commits. Top 5 contributors, in order, are: Sean Busbey, Duo Zhang, Andrea Cosentino, Claus Ibsen, and Sebastian Bazley.

Apache Project Announcements –the latest updates by category.

Library --
 - Apache Apache Commons Collections 4.3 released http://commons.apache.org/

Logging --
 - Apache Log4j 2.11.2 released https://logging.apache.org/

Network/Server --
 - Apache Tomcat 8.5.38 released http://tomcat.apache.org/ - Apache Qpid Proton 0.27.0 released http://qpid.apache.org/

Search --
 - Apache Lucene 7.7.0 and Solr 7.7.0 released http://lucene.apache.org/
 - CVE-2017-3164 SSRF issue in Apache Solr http://mail-archives.apache.org/mod_mbox/www-announce/201902.mbox/%3CCAECwjAVjBN%3DwO5rYs6ktAX-5%3D-f5JDFwbbTSM2TTjEbGO5jKKA%40mail.gmail.com%3E

Did You Know?

 - Did you know that the following Apache projects are celebrating anniversaries this month? Apache HTTP Server (24 years --established prior to the ASF's incorporation!); Gump and Portals (15 years); Directory, MyFaces, and Xerces (14 years); Tapestry (13 years); Roller (12 years); Cassandra and Subversion (9 years); Chemistry (8 years); BVa and OpenNLP (7 years); Clerezza and Crunch (6 years); Knox, Open Climate Workbench, and Spark (5 years); and DataFu (1 year). Many happy returns! https://projects.apache.org/committees.html?date

 - Did you know that Apache Groovy was downloaded 30 million times during Q4 2018? http://groovy.apache.org/

 - Did you know that Apache Edgent (incubating) can be embedded in gateways and small footprint edge devices to enable real-time IoT data analytics? http://edgent.incubator.apache.org/

Apache Community Notices:

 - Apache in 2018 - By The Digits https://s.apache.org/Apache2018Digits

 - ASF Operations Summary: Q2 FY2019 https://s.apache.org/d2Fq

 - ASF Annual Report for FY2018 https://s.apache.org/FY2018AnnualReport

 - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3

 - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA

 - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity

 - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html

 - The Apache Big Data community will be at DataWorks Summit 18-21 March 2019 in Barcelona and 20-23 May 2019 in Washington DC https://dataworkssummit.com/

 - Future dates for Spark + AI Summit 2019 announced: 23-25 April/San Francisco and 15-17 October/Amsterdam https://databricks.com/sparkaisummit/

 - CFP Now Open: Apache Roadshow Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html

 - CFP now open for Berlin Buzzwords: 16-18 June 2019 https://berlinbuzzwords.de/

 - Save the date: ApacheCon North America 9-13 September 2019 http://apachecon.com/

 - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache HTTP Server, Avro, ComDev (community development), Directory, Incubator, OODT, POI, Polygene, Syncope, Tika, Trafodion, and more! https://helpwanted.apache.org/

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

# # #

Friday February 08, 2019

The Apache News Round-up: week ending 8 February 2019

Hooray for Friday! It's time to review the activities of the Apache Community over the past week:

FINAL CALL: please share your thoughts on The Apache Way in our quick survey https://s.apache.org/oxTr

Success at Apache –the monthly blog series that focuses on the people and processes behind why the ASF "just works".
 - For Love or Money: Volunteer vs. Professional Open Source by Rich Bowen https://s.apache.org/ePHl

ASF Board –management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 20 February. Board calendar and minutes http://apache.org/foundation/board/calendar.html

ApacheCon™ –the ASF's official global conference series, now in its 21st year.
 - Join us: Apache Roadshow/DC and Open Source Career Fair 25 March 2019 http://www.apachecon.com/usroadshow18/
 - CFP Now Open: Apache Roadshow/Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html
 - Save the Date: ApacheCon North America 9-13 September in Las Vegas http://apachecon.com/

ASF Infrastructure –our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield 99.98% uptime. http://www.apache.org/uptime/

Apache Code Snapshot –this week, 409 Apache contributors changed 780,780(!) lines of code over 2,663 commits. Top 5 contributors, in order, are: Jean-Baptiste Onofré, Sean Busbey, Andrea Cosentino, Duo Zhang, and Mark Thomas.

Apache Project Announcements –the latest updates by category.

BigData -- 
 - Apache BookKeeper 4.9.0 released http://bookkeeper.apache.org/

Cloud --
 - Apache jclouds 2.1.2 released https://jclouds.apache.org/

Content --
 - Apache Jackrabbit 2.19.1 released http://jackrabbit.apache.org/

IoT --
 - Apache PLC4X (incubating) 0.3.0 released http://plc4x.apache.org

Library --

 - Apache Commons VFS 2.3 released http://commons.apache.org/
 - Apache Groovy (incubating) 2.5.6 released https://groovy.apache.org/

Network/Server --
 - Apache Tomcat 9.0.16 and Tomcat Native 1.2.21 released http://tomcat.apache.org/

Web Framework --
 - Apache MyFaces Core v2.3.3 released http://myfaces.apache.org/
 - Apache Wicket 7.12.0 released https://wicket.apache.org

Did You Know?

 - Did you know that the European Commission's new bug bounty program target projects include Apache Kafka and Apache Tomcat? https://joinup.ec.europa.eu/news/eur-3000-eur-25000

 - Did you know that you can participate in the Call for Entries for the ASF's 20th Anniversary logo? https://s.apache.org/pK8x

 - Did you know that the Los Angeles County Department of Public Social Services uses Apache Wicket? https://wicket.apache.org

Apache Community Notices:

 - Apache in 2018 - By The Digits https://s.apache.org/Apache2018Digits

 - ASF Operations Summary: Q2 FY2019 https://s.apache.org/d2Fq

 - ASF Annual Report for FY2018 https://s.apache.org/FY2018AnnualReport

 - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3

 - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA

 - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity

 - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html

 - The Apache Big Data community will be at DataWorks Summit 18-21 March 2019 in Barcelona and 20-23 May 2019 in Washington DC https://dataworkssummit.com/

 - Future dates for Spark + AI Summit 2019 announced: 23-25 April/San Francisco and 15-17 October/Amsterdam https://databricks.com/sparkaisummit/

 - CFP Now Open: Apache Roadshow Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html

 - CFP now open for Berlin Buzzwords: 16-18 June 2019 https://berlinbuzzwords.de/

 - Save the date: ApacheCon North America 9-13 September 2019 http://apachecon.com/

 - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache HTTP Server, Avro, ComDev (community development), Directory, Incubator, OODT, POI, Polygene, Syncope, Tika, Trafodion, and more! https://helpwanted.apache.org/

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

# # #

Tuesday February 05, 2019

Success at Apache: For Love or Money: Volunteer vs. Professional Open Source

EDITOR'S NOTE: "Success at Apache" reflects diverse, personal experiences from our community members, with particular focus on the people and processes behind why the ASF "just works". The post below is the result of a discussion with the author that originated in early September 2018 and remained unpublished as its tone deviates from the general tenor of this series.Over the past few months, this topic has increased in visibility and relevance with the greater community, and we have made an exception in publishing due its timeliness and representation of the evolution of Open Source communities, both within and without Apache.

by Rich Bowen

A few weeks ago, a colleague asked me what I believed to be the biggest threat facing Open Source today. I answered that I think it's full-time Open Source developers, and the effect they have on part-time volunteer developers.

Long ago (it actually hasn't been very long, it just seems that way sometimes) Open Source was developed primarily by part-time hobbyist developers, working on their evenings and weekends on things that they were passionate about. Sure, there were full-time developers, but they were in the minority. Those of us working a few hours on the weekends envied them greatly. We wished that we, too, could get paid to do the thing that we love.

Now, 20 years on, the overwhelming majority of Open Source development is done by full-timers, working 9-5 on Open Source software. And those who are working nights and weekends are often made to feel that they are less important than those that are putting in the long hours.

Most of the time, this is unintentional. The full-timers are not intentionally marginalizing the part-timers. It just happens as a result of the time that they're able to put into it.

Imagine, if you will, that you're an "evenings-and-weekends" contributor to a project. You have an idea to add a new feature, and you propose it on the mailing list, as per your project culture. And you start working on it, a couple of hours on Friday evening, and then a few more hours on Saturday morning before you have to mow the lawn and take your kids to gymnastics practice. Then there's the cross country meet, and next thing you know, it's Monday morning, and you're back at work.

All week you think about what you're going to do next weekend.

But, Friday evening comes, and you `git pull`, and, lo and behold, one of the full-timers has taken your starting point, turned it in a new direction, completed the feature, and there's been a new release of the project. All while you were punching the clock on your unrelated job.

This is great for the product, of course. It moves faster. Users get features faster. Releases come out faster.

But, meanwhile, you have been told pretty clearly that your contribution wasn't good enough. Or, at the very least, that it wasn't fast enough.

The Cost of Professionalism

And of course there are lots of other benefits, too. Open Source code, as a whole, is probably better than it used to be, because people have more time to focus. The features are more driven by actual use cases, since there's all sorts of customer feedback that goes into the road map. But the volunteerism that made Open Source work in the first place is getting slowly squelched.

This is happening daily across the Open Source world, and MOST of it is unintentional. People are just doing their jobs, after all.

We are also starting to see places where projects are actively shunning the part timers, because they are not pulling their weight. Indeed, in recent weeks I've been told this explicitly by a prominent developer on a project that I follow. He feels that the part timers are stealing his glory, because they have their names on the list of contributors, but they aren't keeping up with the volume of his contributions.

But, whether or not it is intentional, I worry about what this will do to the culture of open source as a whole. I do not in any way begrudge the full-timers their jobs. It's what I dreamed of for *years* when I was an evenings-and-weekends open source developer, and it's what I have now. I am *thrilled* to be paid to work full time in the Open Source world. But I worry that most new Open Source projects are completely corporate driven, and have none of the passion, the scratch-your-own-itch, and the personal drive with which Open Source began.

While most of the professional Open Source developers I have worked with in my years at Red Hat have been passionate and personally invested in the projects that they work on, there's a certain percentage of them for whom this is just a job. If they were reassigned to some other project tomorrow, they'd switch over with no remorse. And I see this more and more as the years go by. Open Source projects as passion is giving way to developers that are working on whatever their manager has assigned, and have no personal investment whatsoever.

This doesn't in any way mean that they are bad people. Work is honorable.

I just worry about what effect this will have, and what Open Source will look like 20 years from now.


Rich Bowen has been doing open source-y stuff since about 1995, and has been a member of the Apache Software Foundation since 2002. He currently serves on the ASF Board of Directors. By day, he's the CentOS Community Manager, working for Red Hat. Read Rich's earlier post, "Success at Apache: Wearing Small Hats".

# # #

"Success at Apache" is a monthly blog series that focuses on the people and processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

Friday February 01, 2019

The Apache News Round-up: week ending 1 February 2019

Happy February! We've had quite a productive week, so let's review what the Apache Community has been up to:

Have you shared your thoughts on The Apache Way? We appreciate your input! https://s.apache.org/oxTr

ASF Board –management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 20 February. Board calendar and minutes http://apache.org/foundation/board/calendar.html

ASF Fundraising –oversees ASF Sponsorship as well as Individual and Corporate Giving programs.
 - The Apache Software Foundation Welcomes Amazon Web Services (AWS) as its Newest Sponsor https://s.apache.org/7UKJ

ApacheCon™ –the ASF's official global conference series, now in its 21st year.
 - Join us: Apache Roadshow/DC and Open Source Career Fair 25 March 2019 http://www.apachecon.com/usroadshow18/
 - CFP Now Open: Apache Roadshow/Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html
 - Save the Date: ApacheCon North America 9-13 September in Las Vegas http://apachecon.com/

ASF Infrastructure –our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield 99.98% uptime. http://www.apache.org/uptime/

Apache Code Snapshot –this week, 469 Apache contributors changed 954,152 lines of code over 3,244 commits. Top 5 contributors, in order, are: Radu Cotescu, Robert Munteanu, Andrea Cosentino, Jean-Baptiste Onofré, and Duo Zhang.

Apache Project Announcements –the latest updates by category.

Content --
 - Apache JSPWiki 2.11.0.M1 released http://jspwiki.apache.org/
 - [CVE-2018-20242] Apache JSPWiki Cross-site scripting vulnerability on Apache JSPWiki http://mail-archives.apache.org/mod_mbox/www-announce/201901.mbox/%3CCAMufup5Mj5hvg3%2BRFUYRTOXsrj%2BcY5mpU_1h9WS%2Be7Q3SW7zOg%40mail.gmail.com%3E

HTTP --
 - Apache Traffic Server v8.0.2 released https://trafficserver.apache.org/

Identity Management --
 - Apache Kerby 2.0.0 released http://directory.apache.org/kerby

Library --
 - Apache Commons BCEL 6.3 released https://commons.apache.org/

Web Framework --
 - Apache Wicket 8.3.0 released https://wicket.apache.org

Did You Know?

 - Did you know that Apache Beam and Apache Kafka won InfoWorld 2019 Technology of the Year Awards? https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html

 - Did you know that you can graph analytics using Apache MADlib? http://madlib.apache.org/

 - Did you know that you can participate in the Call for Entries for the ASF's 20th Anniversary logo? https://s.apache.org/pK8x

Apache Community Notices:

 - Apache in 2018 - By The Digits https://s.apache.org/Apache2018Digits

 - ASF Operations Summary: Q2 FY2019 https://s.apache.org/d2Fq

 - ASF Annual Report for FY2018 https://s.apache.org/FY2018AnnualReport

 - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3

 - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA

 - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity

 - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html

 - The Apache Big Data community will be at DataWorks Summit 18-21 March 2019 in Barcelona and 20-23 May 2019 in Washington DC https://dataworkssummit.com/

 - Future dates for Spark + AI Summit 2019 announced: 23-25 April/San Francisco and 15-17 October/Amsterdam https://databricks.com/sparkaisummit/

 - CFP Now Open: Apache Roadshow Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html

 - CFP now open for Berlin Buzzwords: 16-18 June 2019 https://berlinbuzzwords.de/

 - Save the date: ApacheCon North America 9-13 September 2019 http://apachecon.com/

 - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache HTTP Server, Avro, ComDev (community development), Directory, Incubator, OODT, POI, Polygene, Syncope, Tika, Trafodion, and more! https://helpwanted.apache.org/

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

# # #

Monday January 28, 2019

The Apache Software Foundation Welcomes Amazon Web Services (AWS) as its Newest Sponsor

The Apache Software Foundation (ASF) today welcomed Amazon Web Services (AWS), the latest company to sponsor the ASF at the Platinum level.

"We are pleased to have AWS as a Platinum Sponsor," said ASF Chairman Phil Steitz. "In addition to helping support the ASF’s day-to-day operations through a Platinum sponsorship, AWS has been a Silver-level Targeted Sponsor providing the ASF Infrastructure team with AWS Cloud credits over the past two years. More than 350 Apache projects and their communities directly benefit from the generosity of ASF Sponsors."

AWS joins the following Sponsors:

Platinum level --Cloudera, Comcast, Facebook, Google, LeaseWeb, Microsoft, Oath, Pineapple Fund, and Tencent;

Gold level --Anonymous, ARM, Bloomberg, Handshake, Hortonworks, Huawei, IBM, Indeed, Pivotal, and Union Investment;

Silver level --Aetna, Alibaba Cloud Computing, Baidu, Budget Direct, Capital One, Cerner, Inspur, ODPi, Private Internet Access, Red Hat, and Target;

Bronze level --Airport Rentals, The Blog Starter, Bookmakers, Cash Store, Casino Bonus, Casino2k, Cloudsoft, Emerio, Footprints Recruiting, HostChecka.com, HostingAdvice.com, HostPapa Web Hosting, The Linux Foundation, Mobile Slots, Mutuo Kredit, Online Holland Casino, RX-M, SCAMS.info, Site Builder Report, Talend, The Best VPN, Twitter, and Web Hosting Secret Revealed.

For more information on becoming a Sponsor of the ASF, please see http://apache.org/foundation/sponsorship.html

# # #

Friday January 25, 2019

The Apache News Round-up: week ending 25 January 2019

It's the final Friday of January already, and the Apache community is closing out the week with the following activities:

Thanks in advance for helping the ASF by taking a very short survey on The Apache Way https://s.apache.org/oxTr

ASF Board –management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 20 February. Board calendar and minutes http://apache.org/foundation/board/calendar.html

ApacheCon™ –the ASF's official global conference series, now in its 21st year.
 - Join us: Apache Roadshow/DC and Open Source Career Fair 25 March 2019 http://www.apachecon.com/usroadshow18/
 - CFP Now Open: Apache Roadshow/Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html
 - Save the Date: ApacheCon North America 9-13 September in Las Vegas http://apachecon.com/

ASF Infrastructure –our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield 99.98% uptime. http://www.apache.org/uptime/

Apache Code Snapshot –this week, 453 Apache contributors changed 713,027 lines of code over 2,592 commits. Top 5 contributors, in order, are: Andrea Cosentino, Ravindra Pesala, Claus Ibsen, Tellier Benoit, and Duo Zhang.

Apache Project Announcements –the latest updates by category.

Big Data --
 - Apache Arrow 0.12.0 released https://arrow.apache.org/
 - The Apache Software Foundation Announces Apache® Hadoop® v3.2.0 https://s.apache.org/HK21

Build Management --
 - [CVE-2018-11803] Apache Subversion Denial of Service Vulnerability http://mail-archives.apache.org/mod_mbox/www-announce/201901.mbox/%3C60d59530-6950-35c5-d118-69e5549b7bf1%40apache.org%3E

HTTP --
 - HttpComponents Core 4.4.11 and Client 4.5.7 GA released http://hc.apache.org/

Identity Management --
 - Apache Syncope 2.0.12 and 2.1.3 released http://syncope.apache.org/

Library --
 - Apache Yetus released https://yetus.apache.org/

Network Client/Server --
 - [SECURITY] CVE-2018-1340: Secure flag missing from Apache Guacamole session cookie http://mail-archives.apache.org/mod_mbox/www-announce/201901.mbox/%3CCALKeL-O%2B%3DRxbd0y%2BhSB9%3DY0N400A8sV2BiKgZfNsjGxZipA-uQ%40mail.gmail.com%3E


Did You Know?

 - Did you know that there are 52 podlings under development in the Apache Incubator? http://incubator.apache.org/projects/#current

 - Did you know that the first bank of French Polynesia, Banque SOCREDO, uses Apache Wicket? http://wicket.apache.org/

 - Did you know that you can learn more about ASF Gold Sponsor Bloomberg and their involvement in Open Source? https://s.apache.org/jKNc

Apache Community Notices:

 - Apache in 2018 - By The Digits https://s.apache.org/Apache2018Digits

 - ASF Operations Summary: Q2 FY2019 https://s.apache.org/d2Fq

 - ASF Annual Report for FY2018 https://s.apache.org/FY2018AnnualReport

 - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3

 - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA

 - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity

 - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html

 - The ASF will be at FOSDEM 2-3 February/Brussels https://fosdem.org/2019/

 - The Apache Big Data community will be at DataWorks Summit 18-21 March 2019 in Barcelona and 20-23 May 2019 in Washington DC https://dataworkssummit.com/

 - Future dates for Spark + AI Summit 2019 announced: 23-25 April/San Francisco and 15-17 October/Amsterdam https://databricks.com/sparkaisummit/

 - CFP Now Open: Apache Roadshow Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html

 - CFP now open for Berlin Buzzwords: 16-18 June 2019 https://berlinbuzzwords.de/

 - Save the date: ApacheCon North America 9-13 September 2019 http://apachecon.com/

 - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache HTTP Server, Avro, ComDev (community development), Directory, Incubator, OODT, POI, Polygene, Syncope, Tika, Trafodion, and more! https://helpwanted.apache.org/

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

# # #

Wednesday January 23, 2019

The Apache Software Foundation Announces Apache® Hadoop® v3.2.0

Pioneering Open Source distributed enterprise framework powers US$166B Big Data ecosystem

Wakefield, MA —23 January 2019— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, today announced Apache® Hadoop® v3.2.0, the latest version of the Open Source software framework for reliable, scalable, distributed computing.

Now in its 11th year, Apache Hadoop is the foundation of the US$166B Big Data ecosystem (source: IDC) by enabling data applications to run and be managed on large hardware clusters in a distributed computing environment. "Apache Hadoop has been at the center of this big data transformation, providing an ecosystem with tools for businesses to store and process data on a scale that was unheard of several years ago," according to Accenture Technology Labs.

"This latest release unlocks the powerful feature set the Apache Hadoop community has been working on for more than nine months," said Vinod Kumar Vavilapalli, Vice President of Apache Hadoop. "It further diversifies the platform by building on the cloud connector enhancements from Apache Hadoop 3.0.0 and opening it up for deep learning use-cases and long-running apps."

Apache Hadoop 3.2.0 highlights include:
  • ABFS Filesystem connector —supports the latest Azure Datalake Gen2 Storage;
  • Enhanced S3A connector —including better resilience to throttled AWS S3 and DynamoDB IO;
  • Node Attributes Support in YARN —helps to tag multiple labels on the nodes based on its attributes and supports placing the containers based on expression of these labels;
  • Storage Policy Satisfier  —supports HDFS (Hadoop Distributed File System) applications to move the blocks between storage types as they set the storage policies on files/directories; 
  • Hadoop Submarine —enables data engineers to easily develop, train and deploy deep learning models (in TensorFlow) on very same Hadoop YARN cluster;
  • C++ HDFS client —helps to do async IO to HDFS which helps downstream projects such as Apache ORC;
  • Upgrades for long running services —supports in-place seamless upgrades of long running containers via YARN Native Service API (application program interface) and CLI (command-line interface).

"This is one of the biggest releases in Apache Hadoop 3.x line which brings many new features and over 1,000 changes," said Sunil Govindan, Apache Hadoop 3.2.0 release manager. "We are pleased to announce that Apache Hadoop 3.2.0 is available to take your data management requirements to the next level. Thanks to all our contributors who helped to make this release happen."

Apache Hadoop is widely deployed at numerous enterprises and institutions worldwide, such as Adobe, Alibaba, Amazon Web Services, AOL, Apple, Capital One, Cloudera, Cornell University, eBay, ESA Calvalus satellite mission, Facebook, foursquare, Google, Hortonworks, HP, Huawei, Hulu, IBM, Intel, LinkedIn, Microsoft, Netflix, The New York Times, Rackspace, Rakuten, SAP, Tencent, Teradata, Tesla Motors, Twitter, Uber, and Yahoo. The project maintains a list of educational and production users, as well as companies that offer Hadoop-related services at https://wiki.apache.org/hadoop/PoweredBy

Global Knowledge hails, "...the open-source Apache Hadoop platform changes the economics and dynamics of large-scale data analytics due to its scalability, cost effectiveness, flexibility, and built-in fault tolerance. It makes possible the massive parallel computing that today's data analysis requires."

Hadoop is proven at scale: Netflix captures 500+B daily events using Apache Hadoop. Twitter uses Apache Hadoop to handle 5B+ sessions a day in real time. Twitter’s 10,000+ node cluster processes and analyzes more than a zettabyte of raw data through 200B+ tweets per year. Facebook’s cluster of 4,000+ machines that store 300+ petabytes is augmented by 4 new petabytes of data generated each day. Microsoft uses Apache Hadoop YARN to run the internal Cosmos data lake, which operates over hundreds of thousands of nodes and manages billions of containers per day.

Transparency Market Research recently reported that the global Hadoop market is anticipated to rise at a staggering 29% CAGR with a market valuation of US$37.7B by the end of 2023.

Apache Hadoop remains one of the most active projects at the ASF: it ranks #1 for Apache project repositories by code commits, and is the #5 repository by size (3,881,797 lines of code).

"The Apache Hadoop community continues to go from strength to strength in further driving innovation in Big Data," added Vavilapalli. "We hope that developers, operators and users leverage our latest release in fulfilling their data management needs."

Catch Apache Hadoop in action at the Strata conference, 25-28 March 2019 in San Francisco, and dozens of Hadoop MeetUps held around the world, including on 30 January 2019 at LinkedIn in Sunnyvale, California.

Availability and Oversight
Apache Hadoop software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Hadoop, visit http://hadoop.apache.org/ and https://twitter.com/hadoop

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 730 individual Members and 7,000 Committers across six continents successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official global conference series. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Hortonworks, Huawei, IBM, Indeed, Inspur, LeaseWeb, Microsoft, Oath, ODPi, Pineapple Fund, Pivotal, Private Internet Access, Red Hat, Target, Tencent, and Union Investment. For more information, visit http://apache.org/ and https://twitter.com/TheASF

© The Apache Software Foundation. "Apache", "Hadoop", "Apache Hadoop", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Friday January 18, 2019

The Apache News Round-up: week ending 18 January 2019

It's been a great week! Let's review the Apache community's activities:

Please take a few minutes to help the ASF by providing your feedback on The Apache Way https://s.apache.org/oxTr

Sponsor Case Study –insight from our Sponsors, whose support ensures our 300+ community-driven software products remain available to billions of users around the world at no cost.
 - Apache Software Foundation Gold Sponsor Profile: Bloomberg https://s.apache.org/jKNc

ASF Board –management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 20 February. Board calendar and minutes http://apache.org/foundation/board/calendar.html

ApacheCon™ –the ASF's official global conference series, now in its 21st year.
 - Join us: Apache Roadshow/DC and Open Source Career Fair 25 March 2019 http://www.apachecon.com/usroadshow18/
 - CFP Now Open: Apache Roadshow/Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html
 - Save the Date: ApacheCon North America 9-13 September in Las Vegas http://apachecon.com/

ASF Infrastructure –our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - 7M+ weekly checks yield 99.97% uptime. http://www.apache.org/uptime/

Apache Code Snapshot –this week, 479 Apache contributors changed 1,041,597 lines of code over 2,850 commits. Top 5 contributors, in order, are: Andrea Cosentino, Tellier Benoit, Kaxil Naik, Claus Ibsen, and Xiaoyu Yao.

Apache Project Announcements –the latest updates by category.

Big Data --
 - Apache Kylin 2.6.0 released https://kylin.apache.org/

Database --
 - Apache Jackrabbit Oak 1.6.16, 1.8.11, and 1.10.0 released http://jackrabbit.apache.org/

FinTech --
 - Apache Fineract 1.2.0 released http://fineract.apache.org/

Library --
 - Apache Commons FileUpload 1.4 released http://commons.apache.org/

Web Framework --
 - Apache Struts 2.5.20 GA and 2.3.37 GA released https://struts.apache.org/


Did You Know?

 - Did you know that Twitter uses Apache Hadoop to handle 5B+ sessions a day in real time? http://hadoop.apache.org/

 - Did you know that Apache Tinkerpop Gremlin now offers text based predicates? http://tinkerpop.apache.org/

 - Did you know that members of the Apache JMeter Project Management Committee have published a new book, "Master Apache JMeter: From load testing to DevOps"? https://leanpub.com/master-jmeter-from-load-test-to-devops

Apache Community Notices:

 - Apache in 2018 - By The Digits https://s.apache.org/Apache2018Digits

 - ASF Operations Summary: Q2 FY2019 https://s.apache.org/d2Fq

 - ASF Annual Report for FY2018 https://s.apache.org/FY2018AnnualReport

 - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3

 - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA

 - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity

 - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html

 - The ASF will be at FOSDEM 2-3 February/Brussels https://fosdem.org/2019/

 - The Apache Big Data community will be at DataWorks Summit 18-21 March 2019 in Barcelona and 20-23 May 2019 in Washington DC https://dataworkssummit.com/

 - Future dates for Spark + AI Summit 2019 announced: 23-25 April/San Francisco and 15-17 October/Amsterdam https://databricks.com/sparkaisummit/

 - CFP Now Open: Apache Roadshow Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html

 - CFP now open for Berlin Buzzwords: 16-18 June 2019 https://berlinbuzzwords.de/

 - Save the date: ApacheCon North America 9-13 September 2019 http://apachecon.com/

 - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache HTTP Server, Avro, ComDev (community development), Directory, Incubator, OODT, POI, Polygene, Syncope, Tika, Trafodion, and more! https://helpwanted.apache.org/

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

# # #

Tuesday January 15, 2019

Apache Software Foundation Gold Sponsor Profile: Bloomberg

with Kevin Fleming, Head of Open Source Community Engagement and Member of the CTO Office at Bloomberg


Kevin has spent more than 20 years in the technology industry. In 2004, he started a VOIP service provider company and chose Asterisk as his platform. 9 months later he was offered a position at Digium to work on Asterisk full-time. After seven years of developing and managing the Asterisk project, and helping to design and build the Asterisk SCF project, he moved on to Bloomberg LP, where he works with various teams to help produce and support Bloomberg's open software, used by its customers and partners to integrate with the Bloomberg Terminal. Follow @realkpfleming and @TechAtBloomberg on Twitter.


“ASF's very explicit statement that every participant

in a project is participating personally is really

a big differentiating factor to the other models.”


How did Bloomberg's work with Open Source begin?

Bloomberg has been a consumer of Open Source software for decades, but our involvement as a community collaborator and contributor began in earnest about seven years ago. That was a result of a company-wide decision to begin using Open Source tools in the development of our applications and infrastructure when possible, instead of commercial or proprietary tools.

Open Source tools are important to us because, when a tool’s source code is available, you have the flexibility to understand how it works, to modify it, and to support it by yourself, and you don’t have to rely on a vendor who might go away or whose priorities might change. That's not to say that there aren’t vendors who support their Open Source tools, because there are, but we're not locked into a channel model. Open Source tools give us control over our own destiny. If something becomes important enough to us, we can form an internal team to provide support and enhancements for that tool and the team can contribute those enhancements back to the Open Source community, if appropriate.

Why Apache? How is Bloomberg involved with the ASF, and for how long?

Bloomberg has been involved with the ASF for almost all of the seven years that we've been an active participant in the Open Source community. Apache is the home of dozens of Open Source projects that are fundamentally important to us. These tools support our data science workflows, data processing workflows, and web services, as well as other internal and external services that the company operates.

It's important to us that the organization providing a project's home is able to effectively support the project, and we want developers that work on projects in the Apache organization to focus on actually developing software, without having to spend time managing infrastructure (like bug trackers, source code repositories, and related tools). Contributors to Apache projects can focus on their own project because of the support from the foundation that these projects receive.

Why is support for foundations such as the ASF important? How does helping the ASF help Bloomberg?

Supporting organizations like the ASF is important to Bloomberg because of its governance model. Every participant in one of these Open Source projects has equal representation and footing, and developers are valued based on the merits of their contribution. Projects operating under a different governance model might not offer same type of participation for developers unless their company has made a significant financial contribution to the organization. Bloomberg participates in those types of organizations as well, but we strongly prefer those that allow everyone to participate. Not only can input come from a broader community, but also this allows contributors with a varied level of experience to participate in projects.

Contributors on these projects aren't only developers; they also include people with varied skills like documentation, project management, and marketing. Sometimes a project's decision-makers don’t write any code. While useful tools are developed within both governance models, the way an Apache project's roadmap is set, planned, and decided upon is what's important to us.

What sets the ASF apart from other software foundations or consortia?

The one really significant difference from other charitable foundations that we also support is the ASF's governance model. ASF's very explicit statement that every participant in a project is participating personally is really a big differentiating factor to the other models. 

What does 'The Apache Way' mean to Bloomberg? What makes 'The Apache Way' special?

As a result, meetings held to discuss the next major phase of an Apache project's development don't feel like any company has a representative group. ASF's policy states that you're there representing your own personal interests and not anyone else's. Even though I may know where everyone in the room works, it doesn’t matter and is not relevant. That doesn’t mean you can't say 'the company I work for uses this piece of software in a certain way,' and obviously the way you use the software will impact your opinion, but you don’t make suggestions because that’s what your company wants. Your decisions about a project should be influenced by what you think is best for the broader community. A contributor's entire currency is based on their contributions and how the community values their participation.

Do you have any guidelines for promoting innovation? There is no limit with Open Source: how do you stay focused?

We often have to consider different solutions because of changing business requirements, new types of systems to manage, or some other reason. We strongly encourage people to take a few steps back from their daily work of maintaining our systems to think about what the service may need to support in the future. We want to know if we're using the right tool, if we should use something else, or if we should improve an existing tool.

We strongly encourage people to consider that they're not just delivering a service today and that they don't want that service to be irrelevant to clients six months down the road. They need to build applications and provide databases and services that have the right functionality that meet our clients' present needs. But, you also have to choose solutions that can grow with you and that you can grow, and you have to be able to take technology in the direction you need it to go. You can do this with Open Source easier than you can with a commercial, off-the-shelf solution.

How does Apache fit into Bloomberg's long-term strategy/plans?

It's strategically important to us to ensure that the ASF continues to be a place where new, interesting, and exciting Open Source projects want to go to as their home. As these projects evolve beyond just a few people putting code up on GitHub, they can receive the benefits and support that ASF has to offer – this is a good thing for Bloomberg and the greater Open Source community.

Money is just one way to support the ASF. How else do you contribute? What recommendations do you have for others to participate?

We participate in helping the ASF decide how best to support organizations like us, and we look for ways to further help the ASF through regular conversations with them. We help spread the word about how Apache projects are run and why that's valuable and important – marketing and evangelizing is another way we contribute to the overall health of the community and the projects to which our developers contribute.

There are a few different models for how people can contribute to Apache projects. We have some employees who contribute outside of work, in their personal time. On the other end of the spectrum, we have staff that spend most of their day working on Open Source projects, including reviewing patches that aren't contributed by Bloomberg employees. These Open Source projects are important to Bloomberg, and this work ensures that the projects move forward. Of course we also have team members who contribute to projects as the need arises, which may mean that they contribute to a dozen (or more) projects in any given year.

We also host collaborative weekend sprints that are focused on a specific Open Source project or group of projects. Between 50 and 100 people attend these events with project leaders so they can learn how to become contributors to that specific project, and we provide the support system so they can participate. We ask people to break up into teams so they can, while working as part of a small group, tackle open items on the project's 'to do' list. The groups dive in and try to figure out how to solve each problem, and experts are available to help when people get stuck or to merge participants' PRs on the spot.

For some of those attending, this may be their initial contribution experience, and it's important that this experience is very collaborative, friendly, and productive. The goal of these events is to train people to be Open Source contributors. While the sprint might not be for an Apache project, some participants end up contributing to Apache projects in the future. Once someone becomes a contributor, they become more comfortable with the process and know what to expect, and they're able to translate that experience to subsequent projects.

How does it feel to be able to offer this level of support?

We're pleased to be able to participate and encourage more companies to sponsor the Apache Software Foundation. Everyone wins when everyone collaborates.

# # #

Sponsors of The Apache Software Foundation such as Bloomberg enable the all-volunteer ASF to ensure its 300+ community-driven software products remain available to billions of users around the world at no cost, and to incubate the next generation of Open Source innovations. For more information sponsorship and on ways to support the ASF, visit http://apache.org/foundation/contributing.html . 

Friday January 11, 2019

The Apache News Round-up: week ending 11 January 2019

Happy Friday --with the year off to a hot start, we're picking up the pace with a new Top-Level Project, more events, and 80% more commits over the past 7 days. Let's take a look at what's happening with the Apache community:

Do you have a few minutes to help the ASF? In advance of our 20th Anniversary, we'd love your insight on The Apache Way. Our community is integral to our success, and we'd appreciate your feedback on this quick survey https://s.apache.org/oxTr

Success at Apache –a monthly blog series that focuses on the processes behind why the ASF "just works".
 - Success at Apache: Accidentally Finding Awesome by Daniel Ruggeri https://s.apache.org/e3is

ASF Board –management and oversight of the business affairs of the corporation in accordance with the Foundation's bylaws.
 - Next Board Meeting: 16 January. Board calendar and minutes http://apache.org/foundation/board/calendar.html

ApacheCon™ –the ASF's official global conference series, now in its 21st year.
 - Join us: Apache Roadshow/DC and Open Source Career Fair at George Mason University 25 March 2019 http://apachecon.com/
 - CFP Now Open: Apache Roadshow/Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html
 - Save the Date: ApacheCon North America 9-13 September in Las Vegas http://apachecon.com/

ASF Infrastructure –our distributed team on three continents keeps the ASF's infrastructure running around the clock.
 - NEW statistics and visualizations for 7M+ weekly checks: 99.95% uptime. http://www.apache.org/uptime/

Apache Code Snapshot –this week, 440 Apache contributors changed 1,027,772 lines of code over 3,027 commits. Top 5 contributors, in order, are: Kaxil Naik, Ash Berlin-Taylor, Andrea Cosentino, Chesnay Schepler, and Tellier Benoit.

Apache Project Announcements –the latest updates by category.

Big Data --
 - The Apache Software Foundation Announces Apache® Airflow™ as a Top-Level Project https://s.apache.org/LeeE
 - Apache Flume 1.9.0 released http://flume.apache.org/
 - Apache HBase 2.0.4 and 2.1.2 released https://hbase.apache.org/

Network Client/Server --
 - Apache Guacamole 1.0.0 released http://guacamole.apache.org/
 - Apache Jackrabbit 2.10.9 released http://jackrabbit.apache.org/
 - Apache Qpid Broker-J 7.1.0 released http://qpid.apache.org

Did You Know?

 - Did you know that the following Apache projects are celebrating anniversaries in January? Apache Cocoon, James, and Web Services (16 years); Lucene (14 years); ActiveMQ (12 years); Hadoop (11 years); River (8 years); Empire-db and Gora (7 years); OpenMeetings (6 years); Samza (4 years); Arrow (3 years); and Ranger (2 years). Many happy returns! https://projects.apache.org/committees.html?date

 - Did you know that blogs.apache.org is powered by Apache Roller? v5.2.2 just released today! http://roller.apache.org/

 - Did you know that NTT, the world's fourth largest telecommunications company, built a Kappa architecture for network flows using Apache Kafka? http://kafka.apache.org/

Apache Community Notices:

 - Apache in 2018 - By The Digits https://s.apache.org/Apache2018Digits

 - ASF Operations Summary: Q2 FY2019 https://s.apache.org/d2Fq

 - ASF Annual Report for FY2018 https://s.apache.org/FY2018AnnualReport

 - The Apache Software Foundation 2018 Vision Statement https://s.apache.org/zqC3

 - Foundation Statement –Apache Is Open. https://s.apache.org/PIRA

 - "Success at Apache" focuses on the processes behind why the ASF "just works". https://blogs.apache.org/foundation/category/SuccessAtApache

 - Please follow/like/re-tweet the ASF on social media: @TheASF on Twitter and on LinkedIn at https://www.linkedin.com/company/the-apache-software-foundation

 - Do friend and follow us on the Apache Community Facebook page https://www.facebook.com/ApacheSoftwareFoundation/and Twitter account https://twitter.com/ApacheCommunity

 - The list of Apache project-related MeetUps can be found at http://events.apache.org/event/meetups.html

 - The ASF will be at FOSDEM 2-3 February/Brussels https://fosdem.org/2019/

 - The Apache Big Data community will be at DataWorks Summit 18-21 March 2019 in Barcelona and 20-23 May 2019 in Washington DC https://dataworkssummit.com/

 - Future dates for Spark + AI Summit 2019 announced: 23-25 April/San Francisco and 15-17 October/Amsterdam https://databricks.com/sparkaisummit/

 - CFP Now Open: Apache Roadshow Chicago 13-14 May 2019 http://apachecon.com/chiroadshow19/index.html

 - CFP now open for Berlin Buzzwords: 16-18 June 2019 https://berlinbuzzwords.de/

 - Save the date: ApacheCon North America 9-13 September 2019 http://apachecon.com/

 - Find out how you can participate with Apache community/projects/activities --opportunities open with Apache HTTP Server, Avro, ComDev (community development), Directory, Incubator, OODT, POI, Polygene, Syncope, Tika, Trafodion, and more! https://helpwanted.apache.org/

 - Are your software solutions Powered by Apache? Download & use our "Powered By" logos http://www.apache.org/foundation/press/kit/#poweredby

= = =

For real-time updates, sign up for Apache-related news by sending mail to announce-subscribe@apache.org and follow @TheASF on Twitter. For a broader spectrum from the Apache community, https://twitter.com/PlanetApache provides an aggregate of Project activities as well as the personal blogs and tweets of select ASF Committers.

# # #

Tuesday January 08, 2019

The Apache Software Foundation Announces Apache® Airflow™ as a Top-Level Project

Open Source Big Data workflow management system in use at Adobe, Airbnb, Etsy, Google, ING, Lyft, PayPal, Reddit, Square, Twitter, and United Airlines, among others.

Wakefield, MA —8 January 2019— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® Airflow™ as a Top-Level Project (TLP).

Apache Airflow is a flexible, scalable workflow automation and scheduling system for authoring and managing Big Data processing pipelines of hundreds of petabytes. Graduation from the Apache Incubator as a Top-Level Project signifies that the Apache Airflow community and products have been well-governed under the ASF's meritocratic process and principles.

"Since its inception, Apache Airflow has quickly become the de-facto standard for workflow orchestration," said Bolke de Bruin, Vice President of Apache Airflow. "Airflow has gained adoption among developers and data scientists alike thanks to its focus on configuration-as-code. That has gained us a community during incubation at the ASF that not only uses Apache Airflow but also contributes back. This reflects Airflow’s ease of use, scalability, and power of our diverse community; that it is embraced by enterprises and start-ups alike, allows us to now graduate to a Top-Level Project."

Apache Airflow is used to easily orchestrate complex computational workflows. Through smart scheduling, database and dependency management, error handling and logging, Airflow automates resource management, from single servers to large-scale clusters. Written in Python, the project is highly extensible and able to run tasks written in other languages, allowing integration with commonly used architectures and projects such as AWS S3, Docker, Apache Hadoop HDFS, Apache Hive, Kubernetes, MySQL, Postgres, Apache Zeppelin, and more. Airflow originated at Airbnb in 2014 and was submitted to the Apache Incubator March 2016.

Apache Airflow is in use at more than 200 organizations, including Adobe, Airbnb, Astronomer, Etsy, Google, ING, Lyft, NYC City Planning, Paypal, Polidea, Qubole, Quizlet, Reddit, Reply, Solita, Square, Twitter, and United Airlines, among others. A list of known users can be found at https://github.com/apache/incubator-airflow#who-uses-apache-airflow

"Adobe Experience Platform is built on cloud infrastructure leveraging open source technologies such as Apache Spark, Kafka, Hadoop, Storm, and more," said Hitesh Shah, Principal Architect of Adobe Experience Platform. "Apache Airflow is a great new addition to the ecosystem of orchestration engines for Big Data processing pipelines. We have been leveraging Airflow for various use cases in Adobe Experience Cloud and will soon be looking to share the results of our experiments of running Airflow on Kubernetes." 

"Our clients just love Apache Airflow. Airflow has been a part of all our Data pipelines created in past 2 years acting as the ring-master and taming our Machine Learning and ETL Pipelines," said Kaxil Naik, Data Engineer at Data Reply. "It has helped us create a Single View for our client's entire data ecosystem. Airflow's Data-aware scheduling and error-handling helped automate entire report generation process reliably without any human-intervention. It easily integrates with Google Cloud (and other major cloud providers) as well and allows non-technical personnel to use it without a steep learning curve because of Airflow’s configuration-as-a-code paradigm."

"With over 250 PB of data under management, PayPal relies on workflow schedulers such as Apache Airflow to manage its data movement needs reliably," said Sid Anand, Chief Data Engineer at PayPal. "Additionally, Airflow is used for a range of system orchestration needs across many of our distributed systems: needs include self-healing, autoscaling, and reliable [re-]provisioning."

"Since our offering of Apache Airflow as a service in Sept 2016, a lot of big and small enterprises have successfully shifted all of their workflow needs to Airflow," said Sumit Maheshwari, Engineering Manager at Qubole. "At Qubole, not only are we a provider, but also a big consumer of Airflow as well. For example, our whole Insight and Recommendations platform is built around Airflow only, where we process billions of events every month from hundreds of enterprises and generate insights for them on big data solutions like Apache Hadoop, Apache Spark, and Presto. We are very impressed by the simplicity of Airflow and ease at which it can be integrated with other solutions like clouds, monitoring systems or various data sources."

"At ING, we use Apache Airflow to orchestrate our core processes, transforming billions of records from across the globe each day," said Rob Keevil, Data Analytics Platform Lead at ING WB Advanced Analytics. "Its feature set, Open Source heritage and extensibility make it well suited to coordinate the wide variety of batch processes we operate, including ETL workflows, model training, integration scripting, data integrity testing, and alerting. We have played an active role in Airflow development from the onset, having submitted hundreds of pull requests to ensure that the community benefits from the Airflow improvements created at ING.  We are delighted to see Airflow graduate from the Apache Incubator, and look forward to see where this exciting project will be taken in future!"

"We saw immediately the value of Apache Airflow as an orchestrator when we started contributing and using it," said Jarek Potiuk, Principal Software Engineer at Polidea. "Being able to develop and maintain the whole workflow by engineers is usually a challenge when you have a huge configuration to maintain. Airflow allows your DevOps to have a lot of fun and still use the standard coding tools to evolve your infrastructure. This is 'infrastructure as a code' at its best."

"Workflow orchestration is essential to the (big) data era that we live in," added de Bruin. "The field is evolving quite fast and the new data thinking is just starting to make an impact. Apache Airflow is a child of the data era and therefore very well positioned, and is also young so a lot of development can still happen. Airflow can use bright minds from scientific computing, enterprises, and start-ups to further improve it. Join the community, it is easy to hop on!"

Availability and Oversight
Apache Airflow software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Airflow, visit http://airflow.apache.org/ and https://twitter.com/ApacheAirflow

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 730 individual Members and 7,000 Committers across six continents successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Hortonworks, Huawei, IBM, Indeed, Inspur, LeaseWeb, Microsoft, Oath, ODPi, Pineapple Fund, Pivotal, Private Internet Access, Red Hat, Target, Tencent, and Union Investment. For more information, visit http://apache.org/ and https://twitter.com/TheASF

© The Apache Software Foundation. "Apache", "Airflow", "Apache Airflow", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #

Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation