Entries tagged [source]
The Apache Software Foundation Announces Apache® Hudi™ as a Top-Level Project
Open Source data lake technology for stream processing on top of Apache Hadoop in use at Alibaba, Tencent, Uber, and more.
Wakefield, MA —4 June 2020— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® Hudi™ as a Top-Level Project (TLP).
Apache Hudi (Hadoop Upserts Deletes and Incrementals) data lake technology enables stream processing on top of Apache Hadoop compatible cloud stores & distributed file systems. The project was originally developed at Uber in 2016 (code-named and pronounced "Hoodie"), open-sourced in 2017, and submitted to the Apache Incubator in January 2019.
"Learning and growing the Apache way in the incubator was a rewarding experience," said Vinoth Chandar, Vice President of Apache Hudi. "As a community, we are humbled by how far we have advanced the project together, while at the same time, excited about the challenges ahead."
Apache Hudi is used to manage petabyte-scale data lakes using stream processing primitives like upserts and incremental change streams on Apache Hadoop Distributed File System (HDFS) or cloud stores. Hudi data lakes provide fresh data while being an order of magnitude efficient over traditional batch processing. Features include:
- Upsert/Delete support with fast, pluggable indexing
- Transactionally commit/rollback data
- Change capture from Hudi tables for stream processing
- Support for Apache Hive, Apache Spark, Apache Impala and Presto query engines
- Built-in data ingestion tool supporting Apache Kafka, Apache Sqoop and other common data sources
- Optimize query performance by managing file sizes, storage layout
- Fast row based ingestion format with async compaction into columnar format
- Timeline metadata for audit tracking
Apache Hudi is in use at organizations such as Alibaba Group, EMIS Health, Linknovate, Tathastu.AI, Tencent, and Uber, and is supported as part of Amazon EMR by Amazon Web Services. A partial list of those deploying Hudi is available at https://hudi.apache.org/docs/powered_by.html
"We are very pleased to see Apache Hudi graduate to an Apache Top-Level Project. Apache Hudi is supported in Amazon EMR release 5.28 and higher, and enables customers with data in Amazon S3 data lakes to perform record-level inserts, updates, and deletes for privacy regulations, change data capture (CDC), and simplified data pipeline development," said Rahul Pathak, General Manager, Analytics, AWS. “We look forward to working with our customers and the Apache Hudi community to help advance the project."
"At Uber, Hudi powers one of the largest transactional data lakes on the planet in near real time to provide meaningful experiences to users worldwide," said Nishith Agarwal, member of the Apache Hudi Project Management Committee. "With over 150 petabytes of data and more than 500 billion records ingested per day, Uber’s use cases range from business critical workflows to analytics and machine learning."
"Using Apache Hudi, end-users can handle either read-heavy or write-heavy use cases, and Hudi will manage the underlying data stored on HDFS/COS/CHDFS using Apache Parquet and Apache Avro," said Felix Zheng, Lead of Cloud Real-Time Computing Service Technology at Tencent.
"As cloud infrastructure becomes more sophisticated, data analysis and computing solutions gradually begin to build data lake platforms based on cloud object storage and computing resources," said Li Wei, Technical Lead on Data Lake Analytics, at Alibaba Cloud. "Apache Hudi is a very good incremental storage engine that helps users manage the data in the data lake in an open way and accelerate users' computing and analysis."
"Apache Hudi is a key building block for the Hopsworks Feature Store, providing versioned features, incremental and atomic updates to features, and indexed time-travel queries for features," said Jim Dowling, CEO/Co-Founder at Logical Clocks. "The graduation of Hudi to a top-level Apache project is also the graduation of the open-source data lake from its earlier data swamp incarnation to a modern ACID-enabled, enterprise-ready data platform."
"Hudi's graduation to a top-level Apache project is a result of the efforts of many dedicated contributors in the Hudi community," said Jennifer Anderson, Senior Director of Platform Engineering at Uber. "Hudi is critical to the performance and scalability of Uber's big data infrastructure. We're excited to see it gain traction and achieve this major milestone."
"Thus far, Hudi has started a meaningful discussion in the industry about the wide gaps between data warehouses and data lakes. We have also taken strides to bridge some of them, with the help of the Apache community," added Chandar. "But, we are only getting started with our deeply technical roadmap. We certainly look forward to a lot more contributions and collaborations from the community to get there. Everyone’s invited!"
Catch Apache Hudi in action at Virtual Berlin Buzzwords 7-12 June 2020, as well as at MeetUps, and other events.
Availability and Oversight
Apache Hudi software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Hudi, visit http://hudi.apache.org/ and https://twitter.com/apachehudi
About the Apache Incubator
The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects enter the ASF through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/
About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 813 individual Members and 7,800 Committers across six continents successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Budget Direct, Capital One, CarGurus. Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Inspur, Leaseweb, Microsoft, Pineapple Fund, Red Hat, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF
© The Apache Software Foundation. "Apache", "Hudi", "Apache Hudi", "Hadoop", "Apache Hadoop", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 01:00PM Jun 04, 2020
by Sally in General |
|
The Apache Software Foundation Announces Apache® Subversion® 1.14.0-LTS
Community-led Version Control Software and Source Code Management Tool Available on Most Integration Servers, Integrated Development Environments, Issue Tracking Systems, and more.
Wakefield, MA —28 May 2020— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® Subversion® 1.14.0-LTS, the latest release of the popular centralized software version control system.
Apache Subversion ("SVN") provides a version controlled backing store for any kind of data. It records an accurate log of changes made to that data over time, and keeps track of who made them. Subversion allows users to commit files and directories, recover previous revisions, and even maintain multiple variations of their work in parallel. Able to service projects of any size, from individuals up to large scale collaborative efforts, Subversion is ideal for work in vast swaths of industries, from software development to semiconductor design, scientific research to medical technology. An Apache Top-Level Project for over a decade, Subversion celebrated its 20th Anniversary earlier this year.
"First and foremost, I'd like to thank all of our developers and community members who helped make this release possible," said Nathan Hartman, Vice President of Apache Subversion. "We are excited to publish our latest LTS release, and the first in the 1.14 line."
As an LTS release, the focus is on stability and availability. These are achieved through the project's policies. For any change in core code to be included in updates to 1.14.x, the change must first undergo a process of nomination and voting for backport. At least three Subversion developers must support the change, with none having concerns about it.
LTS (Long Term Support) is an industry designation that a particular release line is planned to be maintained for a longer period of time than regular, non-LTS releases. For the Subversion project, this means that later updates to the 1.14.x series may contain bug fixes and security updates only. Any bleeding edge new features, even if developed during the lifetime of 1.14.x, will have to be introduced in a separate release line. Server operators and system administrators usually prefer LTS releases for stability, while end users often choose the latest release (LTS or not) to get the newest features.
Numerous third parties provide Subversion install packages for Windows, macOS, Linux, OpenBSD, FreeBSD, and other operating systems. To maximize platform independence, Subversion is implemented with strict conformance to ISO C90, one of the most widely supported software coding standards worldwide. In addition, the Subversion developers provide bindings that enable integrations with software coded in popular web languages: Java, Ruby, Perl, and Python.
Particularly noteworthy for this release, Subversion's language bindings for Python received significant attention. Python 3 is supported, up from Python 2 in prior Subversion releases, an oft requested improvement that keeps Subversion 1.14.0-LTS current with the changing Python landscape. While this was a major undertaking, the project also tackled the challenge of maintaining compatibility with the older Python 2. This legacy support is expected to phase out gradually, as Python 3 continues to gain mindshare across the computing industry, but the Subversion project has a long tradition of maintaining compatibility wherever practical, giving operators of legacy systems some much-needed breathing room as they make the transition.
Among Subversion's strengths are its extensive support for working with giant repositories. The bedrock of this support is its centralized model, which allows users to check out only the portions of a repository that they need. The ASF uses Subversion this way in its own infrastructure, housing more than 80 of its Apache Top-Level Projects and sub-projects comprising millions of lines of code, including Subversion itself, in a single Subversion repository that makes all 1.8 million revisions of that information available to collaborators worldwide.
When dealing with such vast amounts of data, including all of its revisions, one might wonder about storage costs. Subversion uses a variety of techniques to minimize storage, including temporal compression, spacial compression, and data deduplication.
Another improvement in Subversion 1.14.0-LTS is a new tool in support of deduplication that could help some administrators reduce future storage costs. The deduplication feature uses an internal database named rep-cache.db. If deduplication was previously disabled, the database may not contain all necessary entries. The new feature, known as the 'svnadmin build-repcache' command, allows re-adding such missing entries and provides a way for those who had previously turned off deduplication to regain some of its benefits.
The release also includes several experimental features. One of these, Shelving and Checkpointing, allows users to save, restore, and roll back snapshots of their work, without making commits to the central repository. This is useful for setting aside a work in progress to work on something else, or for taking temporary snapshots when a network connection to the server is unavailable. Another experimental feature, Viewspec, allows users who create different cross-sections or "views" into their version controlled data, to save the layouts of those views and easily recreate them later. These experimental features are designated as such because they are not yet considered feature-complete. In Subversion 1.14.0-LTS, they are turned off and hidden by default, but are made available on an opt-in basis to entice open source community members to help further their development.
Subversion users, developers, and other stakeholders routinely communicate with each other through email lists. One ongoing discussion taking place there centers around a proposal to make Subversion even stronger at handling big files. The discussion thread, titled "Who else is using SVN for large-binary-asset storage?" has already generated some enthusiasm.
"Apache Subversion is more than code, it's a community," added Hartman. "As an open source and purely volunteer-driven effort, we thrive on participation from enthusiastic users and developers worldwide. We welcome their involvement in the future of Subversion and on our email lists."
Subversion 1.14.0-LTS is available now. The complete software source code can be downloaded from https://subversion.apache.org/download.html , with a list of install packages which are maintained by numerous third parties at https://subversion.apache.org/packages.html .
Over its 20-year history, Subversion has grown to become the most popular version control system on the market, and remains the leading centralized versioning and revision control software today. Millions of users worldwide depend on the collaboration-friendly system to easily access all files and historical data simultaneously without code conflicts or corruption.
Apache Subversion is used for mission-critical code distribution and collaboration workflow by Adobe Dreamweaver, Eclipse, Google, Halliburton, Microsoft Visual Studio, Python, Ruby, Skype, SourceForge, and WordPress, among many others. The ASF's infrastructure uses Apache Subversion across millions of lines of code and nearly two million commits by more than 300 Apache projects.
Availability and Oversight
Apache Subversion software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Subversion, visit http://subversion.apache.org/ .
About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 813 individual Members and 7,800 Committers across six continents successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Budget Direct, Capital One, CarGurus. Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Inspur, Leaseweb, Microsoft, Pineapple Fund, Red Hat, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF
© The Apache Software Foundation. "Apache", "Subversion", "Apache Subversion", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 12:00PM May 28, 2020
by Sally in General |
|
The Apache Software Foundation Announces the 10th Anniversary of Apache® HBase™
Open Source distributed, scalable Big Data store celebrates a decade of processing zettabytes of data across highly scalable large tables for the Apache Hadoop ecosystem
Wakefield, MA —13 May 2020— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the tenth Anniversary of Apache® HBase™, the distributed, scalable data store for the Apache Hadoop Big Data ecosystem.
"The success of Apache HBase is the success of Open Source," said Duo Zhang, Vice President of Apache HBase. "Ten years after graduating as a TLP, HBase is still among the most active projects at the ASF. We have hundreds of contributors all around the world. We speak different languages, we have different skills, but we all work together to make HBase better and better. Ten year anniversary is not the end, but a new beginning, I believe our strong community will lead the project to a bright future."
HBase originated at Powerset in 2006 as an Open Source system to run on Apache Hadoop’s Distributed File System (HDFS), similar to how BigTable ran on top of the Google File System. In 2007, a significant code contribution was added to the Apache Hadoop codebase and was integrated into the Apache Hadoop 0.15.0 release later that year. Development on HBase continued as a sub-project of Apache Hadoop, and graduated as an Apache Top-Level Project (TLP) in April 2010.
An Open Source, versioned, non-relational database, Apache HBase provides low latency random access to very large tables —billions of rows and millions of columns— atop clusters of non-specialized, commodity hardware. HBase reads, writes, and processes structured, semi-structured, and unstructured data in real-time environments.
Apache HBase is in use at thousands of organizations, including Adobe, Airbnb, Alibaba, Bloomberg, Flipkart, Huawei, HP, Hubspot, IBM, Microsoft, NetEase, Pinterest, Salesforce, Shopee, Tencent, Twitter, Xiaomi, and Yahoo! (now Verizon Media), among others.
Testimonials
"Congratulations on the 10th birthday of Apache HBase! Alibaba started to use HBase since January 2011 and has witnessed its growth and come along with the community through the years. The Apache HBase community has always been an open and powerful team that produced many stable, production-ready and widely used versions. Today at Alibaba, we have HBase clusters with more than 10k nodes serving hundreds of petabytes of data, as well as more than 1,000 enterprise HBase users on Alibaba Cloud. We will continue collaborating with and contributing to the HBase community and wish us all ongoing success in future!"
—Chunhui Shen and Yu Li, members of the HBase team at Alibaba
"I have worked with Apache HBase for many years and I think it is a great product. it does what it says on the tin so to speak. Ironically if you look around the NoSQL competitors, most of them are supported by start-ups, whereas HBase is only supported as part of Apache suite of products by vendors like Cloudera, Hortonworks, MapR, etc. For those who would prefer to use SQL on top, there is Apache Phoenix around which makes life easier for the most SQL-savvy world to work on HBase: problem solved. For TCO, HBase is still value for money compared to others. You don't need expensive RAM or SSD with HBase. That makes it easy to onboard it in no time. Also HBase can be used in a variety of different business applications, whereas other commercial ones are focused on narrower niche markets. Least but last happy 10th anniversary and hope HBase will go from strength to strength and we will keep using it for years to come!"
—Dr. Mich Talebzadeh, Chief Data Architect, Big Data
"Congratulations on the 10th anniversary of Apache HBase! Xiaomi started to use HBase in 2012, when our business started booming. Many key Xiaomi products and services, as well as Xiaomi's data analytics platform, require a new system to provide quick and random access to billions of rows of structured and semi-structured data. Traditional solutions are not able to handle the large volume of data brought by the quickly increasing Xiaomi user base. Among several available options, we choose HBase not only because it provides a rich set of features and excellent performance specs, but also because it has a very active, open and friendly community. Embracing open source has been part of Xiaomi's engineering culture, and our deep involvement in the development of Apache HBase demonstrates the best practices of Xiaomi's open source strategy. In the past several years, we have contributed tons of bug fixes and important features to HBase, and, in the meantime, we have contributed 9 committers and 3 PMC members to the HBase community. Looking forward, we will continue to work closely with the Apache HBase community to help the project grow, and we wish Apache HBase a wonderful future!"
—Dr. Baoqiu Cui, Vice President of Xiaomi Corporation and Technical Committee Chairman
“Congratulations on the 10th anniversary of Apache HBase, it’s great to see how the project has developed over the years and continues to have good community support around it! Salesforce has a large global footprint of Apache HBase in production storing multiple petabytes of customer data and serving several billions of queries per day for a wide variety of use cases including security, monitoring, collaboration portals, and performance caches to scale over RDBMS limitations. HBase has played a major role in Salesforce’s customer success in the BigData storage space and we continue to invest in it as one of the pillars of our multi-substrate database strategy along with Apache Phoenix for SQL access to data stored in HBase. We have contributed many features and bug fixes to HBase over the last several years, and we look forward to continue working with the Apache HBase community to develop the project further. Here’s to many more successful years for Apache HBase!”
—Sanjeev Lakshmanan, Senior Director, Software Development, Salesforce
“Happy 10th Apache HBase! It was around 8 years ago that we started looking at HBase to include as part of our Hosted Big Data Services stack. Fast-forward to today and it continues to be a critical offering in our stack, powering a diverse set of use cases and workloads such as ad targeting, content personalization, analytics, security, monitoring, etc. HBase enables these diverse workloads thanks to it’s high-scalability, feature set and performance, all of which have been continuously refined through the years. In turn our footprint continues to grow storing petabytes of data across thousands of machines. Our success is in part thanks to the project’s success as we benefit from our collaborations, the contributions and other efforts by the community (eg mailing list, meetups, HBaseCon, etc). This is a testament to the open, friendly and dedicated community around Apache HBase which is necessary for the success of any open source project. We wish the project continued success for years to come as we continue to collaborate with and be part of the community cultivating the project.”
—Francis Liu and Thiruvel Thirumoolan, HBase Big Data Team Members, Yahoo! (now Verizon Media)
“Congratulations on the 10th anniversary of Apache HBase! It’s great to see how this project has evolved from a big data project to one that runs business critical systems and continues to accelerate with a growing community and increasing pace of development! Cloudera has over 500 customers in production using it for a range of use cases ranging from mission critical transactional applications to supporting data warehousing. Our largest customers have footprints in excess of 7,000 nodes storing over 70PB of data. Our customers choose HBase because of its resilience with some customers able to realize 100% application uptime using HBase (over the past 3 years). We plan to continue to invest in HBase (and Apache Phoenix) to ensure that we can continue to both broaden support for a variety of hybrid transactional and analytical use cases and deepen support for existing use cases. Here's to many more successful years!"
—Arun C. Murthy, Chief Product Officer, Cloudera
“Many Congratulations to the Apache HBase community on the 10th anniversary. Apache HBase provides rich functions and excellent performance, and has an open and friendly community. Huawei started using HBase since 2010: HBase is widely used by multiple solutions of Huawei running on more than 10,000 nodes, storing hundreds of PBs data to meet our requirements. Huawei FusionInsight provides the Best Practices of Huawei for HBase, which serves a lot of customers across many industries such as finance, operators, government, energy, medical, manufacturing, and transportation. Meanwhile, Huawei team members contributed a lot of bug fixes and features to HBase, successfully hosted the first HBase Asia Technology Conference HBaseCon Asia 2017 at Shenzhen. Going forward, Huawei will continue to work closely with the Apache HBase community to promote community development.”
—Wei Zhi, Kai Mo and Pankaj Kumar, members of the HBase team at Huawei
“Happy 10th anniversary, HBase! At Ultra Tendency, you have been the backbone of our Dual Lambda Streaming Architecture for many years! You have served billions of queries to our customers without interruption and at low latency. Your architecture guaranteed that you were always there when we needed you, never letting us or our customers down. You are the reason why our European clients today are running flourishing new business models backed by low-latency streaming products. Our committers and contributors will continue to fix bugs and provide feature enhancements. Ultra Tendency wishes you a bright and successful future!”
—Jan Hentschel, Chief Information Officer, Ultra Tendency
“Congratulations on the 10th anniversary of Apache HBase, I can't believe it's been 10 years since the first day when I tried to use Apache HBase and its ecosystem to help the business and company. Also, it is so great to see many colleagues and friends work, discuss, cooperate together to make this system become better. Some of them also make great career development and some are still progress. Shopee, one of the biggest e-commerce platforms in Southeast Asia, has several large Apache HBase clusters in production to support businesses that depend on several billions of queries per day. Apache HBase has played a significant role in Shopee and it is still in expansion along with the business growth of Shopee. Apache HBase, as well as the community, helps us a lot and we also will continue to make contributions to Apache HBase. Looking forward to keeping working with the Apache HBase community to develop the project and its ecosystem further.”
—Li Luo, Manager of Data Infra department, Shopee
”At Microsoft, our mission is to empower every person and every organization on the planet to achieve more, and it’s this mission that drives our commitment to open source. Congratulations to the Apache HBase community on its 10th anniversary. Microsoft has been part of the vibrant HBase community since 2014, today we are proud to serve the numerous enterprise customers across industries who are leveraging HBase in Azure HDInsight for their most critical business applications.”
—Tomas Talius, Director of Engineering, Azure Data Services, Microsoft
Availability and Oversight
Apache HBase software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache HBase, visit http://hbase.apache.org/ and https://twitter.com/HBase
About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation (ASF) is the world’s largest Open Source foundation, stewarding 200M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 813 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 7,600+ Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, CarGurus, Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Indeed, Inspur, Leaseweb, Microsoft, ODPi, Pineapple Fund, Private Internet Access, Red Hat, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF
© The Apache Software Foundation. "Apache", "HBase", "Apache HBase", "Hadoop", "Apache Hadoop", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 01:00PM May 13, 2020
by Sally in General |
|
The Apache® Software Foundation Celebrates 21 Years of Open Source Leadership
World’s largest Open Source foundation advances community-led innovation "The Apache Way"
Wakefield, MA —26 March 2020— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today its 21st Anniversary.
Advancing its mission of providing software for the public good, the ASF's all-volunteer community grew from 21 original Members overseeing the development of the Apache HTTP Server to 765 individual Members, 206 Apache Project Management Committees, and 7,600+ Committers shepherding 300 projects and 200M+ lines of Apache code valued at more than $20B.
Apache’s breakthrough technology touches every aspect of modern computing, powering most of the Internet, managing exabytes of data, executing teraflops of operations, and storing trillions of objects in virtually every industry. Apache projects are all freely-available, at 100% no cost, and with no licensing fees.
“Over the past two decades, The Apache Software Foundation has served as a trusted home for vendor-neutral, community-led collaboration,“ said David Nalley, Executive Vice President at The Apache Software Foundation. “Today, the ASF is a vanguard for Open Source, fostering project communities large and small, with a portfolio of best-in-class innovations upon which the world continues to rely.“
The Apache Way
As a community-led organization, the ASF is strictly vendor-neutral. Its independence ensures that no organization, including ASF Sponsors and those who employ contributors to Apache projects, is able to control a project's direction or has special privileges of any kind.
The ASF’s community-focused development process known as "The Apache Way" guides existing projects and their communities, and continues to inspire a new generation of innovations from around the world. The Apache Way edict involves:
Earned Authority: all individuals are given the opportunity to participate based on publicly earned merit, i.e., what they contribute to the community.
Community of Peers: individuals participate at the ASF, with merit gained by the individual everlasting and free from association of employment status or employer.
Open Communications: all communications related to code and decision-making are publicly accessible to ensure asynchronous collaboration within the ASF’s globally-distributed communities.
Consensus Decision Making: Apache Projects are overseen by a self-selected team of active volunteers who are contributing to their respective projects.
Responsible Oversight: The ASF governance model is based on trust and delegated oversight.
The Apache Way has been a forerunner in collaborative computing, and has directly influenced the InnerSource methodology of applying Open Source and open development principles to an organization. The Apache Way has been adopted by countless organizations, including Capital One, Comcast, Ericsson, HP, IBM, Google, Microsoft, PayPal, SAP, T-Mobile, and many others.
The ASF’s focus on community is so integral to the Apache ethos that the maxim, "Community Over Code" is an unwavering tenet. Vibrant, diverse communities keep code alive, however, code, no matter how well written, cannot thrive without a community behind it. Members of the Apache community share their thoughts on “Why Apache” in the teaser for “Trillions and Trillions Served”, the upcoming documentary on the ASF https://s.apache.org/Trillions-teaser
Powerhouse Projects
Dozens of enterprise-grade Apache projects have defined industries and serve as the backbone for some of the most visible and widely used applications in Artificial Intelligence and Deep Learning, Big Data, Build Management, Cloud Computing, Content Management, DevOps, IoT and Edge Computing, Mobile, Servers, and Web Frameworks, among many other categories.
No other software foundation serves the industry with such a wide range of projects. Examples of the breadth of applications that are "Powered by Apache" include:
China’s second largest courier, SF Express, uses Apache SkyWalking to ship critical COVID-19 coronavirus supplies worldwide;
Apache Guacamole’s clientless remote desktop gateway is helping thousands of individuals, businesses, and universities worldwide safely work from home without needing to be tied to a specific device, VPN, or client;
Alibaba uses Apache Flink to process more than 2.5 billion records per second for its merchandise dashboard and real-time customer recommendations;
the European Space Agency’s Jupiter spacecraft mission control is powered by Apache Karaf, Apache Maven, and Apache Groovy;
British Government Communications Headquarters (GCHQ)’s application Gaffer stores and manages petabytes of data using Apache Accumulo, Apache HBase, and Apache Parquet;
Netflix uses Apache Druid to manage its 1.5 trillion-row data warehouse to manage what users see when tapping the Netflix icon or logging in from a browser across platforms;
Uber's 100-petabyte data lake is powered in near real-time using Apache Hudi (incubating), supporting everything from warehousing to advanced machine learning;
Boston Children's Hospital uses Apache cTAKES to link phenotypic and genomic data in electronic health records for the Precision Link Biobank for Health Discovery;
Amazon, DataStax, IBM, Microsoft, Neo4j, NBC Universal and many others use Apache Tinkerpop in their graph databases and to write complicated traversals;
the Global Biodiversity Information Facility uses Apache Beam, Hadoop, HBase, Lucene, Spark, and others to integrate biodiversity data from nearly 1,600 institutions and more than a million species and nearly 1.4 billion location records freely available for research;
the European Commission developed its new API Gateway infrastructure using Apache Camel;
China Telecom Bestpay uses Apache ShardingSphere (incubating) to scale 10 billion datasets for mobile payments distributed across more than 30 applications;
Apple’s Siri uses Apache HBase to complete full ring replication around the world in 10 seconds;
the US Navy uses Apache Rya to power smart drones, autonomous small robot swarms, manned-unmanned team advanced tactical communications, and more; and
hundreds of millions of Websites worldwide are powered by the Apache HTTP Server.
Additional Milestones
In addition to the ASF’s 21st Anniversary, the greater Apache community are celebrating milestone anniversaries of the following projects:
25 Years - Apache HTTP Server
21 Years - Apache OpenOffice (at the ASF since 2011), Xalan, Xerces
20 Years - Apache Jakarta (Apache Open Source Java projects), James, mod_perl, Tcl, APR/Portable Runtime, Struts, Subversion (at the ASF since 2009), Tomcat
19 Years - Apache Avalon, Commons, log4j, Lucene, Torque, Turbine, Velocity
18 Years - Apache Ant, DB, FOP, Incubator, POI, Tapestry
17 Years - Apache Cocoon, James, Logging Services, Mavin, Web Services
16 Years - Apache Gump, Portals, Struts, Geronimo, SpamAssassin, Xalan, XML Graphics
15 Years - Apache Lucene, Directory, MyFaces, Xerces, Tomcat
The chronology of all Apache projects can be found at https://projects.apache.org/committees.html?date
The Apache Incubator is home to 45 projects undergoing development, spanning AI, Big Data, blockchain, Cloud computing, cryptography, deep learning, hardware, IoT, machine learning, microservices, mobile, operating systems, testing, visualization, and many other categories. The complete list of projects in the Incubator is available at http://incubator.apache.org/
Support Apache
The ASF advances the future of open development by providing Apache projects and their communities bandwidth, connectivity, servers, hardware, development environments, legal counsel, accounting services, trademark protection, marketing and publicity, educational events, and related administrative support.
As a United States private 501(c)(3) not-for-profit charitable organization, the ASF is sustained through tax-deductible corporate and individual contributions that offset day-to-day operating expenses. To support Apache, visit http://apache.org/foundation/contributing.html
About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation (ASF) is the world’s largest Open Source foundation, stewarding 200M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 765 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 7,600 Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an Open Source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, CarGurus, Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Indeed, Inspur, Leaseweb, Microsoft, ODPi, Pineapple Fund, Private Internet Access, Red Hat, Target, Tencent, Union Investment, Verizon Media, and Workday. For more information, visit http://apache.org/ and https://twitter.com/TheASF
© The Apache Software Foundation. "Apache", "Accumulo", "Apache Accumulo", "Camel", "Apache Camel", "cTAKES", "Apache cTAKES", "Druid", "Apache Druid", "Flink", "Apache Flink", "Groovy", "Apache Groovy", "Guacamole", "Apache Guacamole", "HBase", "Apache HBase", "Apache HTTP Server", "Karaf", "Apache Karaf", "Maven", "Apache Maven", "Parquet", "Apache Parquet", "Rya", "Apache Rya", "SkyWalking, "Apache SkyWalking", "Tinkerpop", "Apache Tinkerpop", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 09:59AM Mar 26, 2020
by Sally in Milestones |
|
The Apache Software Foundation Announces Apache® Brooklyn(TM) v1.0
Advanced Open Source framework for modelling, monitoring, and managing applications used by global systems integrators, Cloud software and service providers, and major enterprises across financial services, supply chain management, and more.
Wakefield, MA —3 March 2020— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, today announced Apache® BrooklynTM v1.0, the latest version of the Open Source framework for modelling, monitoring, and managing applications.
"I am excited to see the 1.0 release of Apache Brooklyn," said Geoff Macartney, Vice President of Apache Brooklyn. "This reflects the maturity and stability that Brooklyn has reached after nearly five years as a Top-Level Apache project."
Apache Brooklyn provides a single tool that includes a REST API and GUI for:
- managing provisioning and application deployment;
- monitoring an application’s health and metrics;
- understanding the dependencies between components; and
- applying complex policies to manage the application.
Apache Brooklyn uses declarative YAML blueprints to describe an application and all its components. Blueprints can be treated as an integral part of the application, and as modular components that can be composed and reused in many ways. Brooklyn blueprints incorporate policies that actively manage a deployed application by reacting to sensor data such as application health or load, and take actions such as replacing nodes or growing a cluster. Brooklyn’s design is influenced by Autonomic computing and promise theory and implements the OASIS CAMP and TOSCA standards.
Apache Brooklyn 1.0 highlights include:
- Support for public and private clouds, available out-of-the-box thanks to integrated Apache jclouds, as well as private infrastructure
- A modern, user-friendly, web-based UI including the drag-and-drop Blueprint Composer
- REST API and CLI tools, suitable for power users, automation and scripting
- A stable blueprint language and API
- “Batteries included” entities and policies covering clusters, auto-scaling, replacing unhealthy components, and more
"Apache Brooklyn has been in use for some time in production environments," said Richard Downer, Apache Brooklyn 1.0 release manager. "I’m delighted we can now announce our 1.0 release. Everyone should feel confident building on and deploying Apache Brooklyn 1.0 and know that the Brooklyn Project Management Committee has prioritised the long-term stability of Brooklyn."
Apache Brooklyn is in use by global systems integrators, providers of Cloud software and services, as well as mission-critical applications for major enterprises in financial services, supply chain management, and more.
"We are delighted to see Apache Brooklyn reach this milestone," said David Cairns, CTO for innovation at Fujitsu Digital Technology Services. "Apache Brooklyn powers Fujitsu AIOps solutions with policy-based autonomics to detect service deterioration or outage and can automatically re-locate Cloud applications and services from one cloud provider to another to elevate resilience and uptime."
"Reaching v1.0 reflects the maturity of Apache Brooklyn and we appreciate the community’s effort," said Ross Gray, CEO at Cloudsoft. "Cloudsoft AMP is built on Apache Brooklyn and helps customers eliminate manual processes, cut effort by 75%, and reduce infrastructure spend by as much as 66%."
Apache Brooklyn blueprints for many well-known applications and tools, including ElasticSearch, clustered MySQL, and DNS management, as well as Apache projects such as Cassandra, CouchDB, Kafka, Solr, Storm, ZooKeeper and more, are all freely available under the Apache License v2. The Apache Brooklyn community warmly welcomes new code, testing, blueprints, documentation, presentations, and other contributions.
"Brooklyn is a powerful tool for unified modelling, deployment and lifetime management of applications," added Macartney. "This latest release is a great opportunity for a wider audience to try Brooklyn for themselves and find out how it can help them create and manage their applications, be it in the Cloud, on-premise, or in a hybrid environment. We look forward to growing our community as people discover all that Brooklyn can do."
Availability and Oversight
Apache Brooklyn software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Brooklyn, visit https://brooklyn.apache.org/ and https://twitter.com/ApacheBrooklyn
About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation (ASF) is the world’s largest Open Source foundation, stewarding 200M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 765 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 7,600 Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 has become an industry standard within the Open Source world, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, CarGurus, Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Indeed, Inspur, Leaseweb, Microsoft, ODPi, Pineapple Fund, Pivotal, Private Internet Access, Red Hat, Target, Tencent, Union Investment, Workday, and Verizon Media. For more information, visit http://apache.org/ and https://twitter.com/TheASF
© The Apache Software Foundation. "Apache", "Brooklyn", "Apache Brooklyn", "Cassandra", "Apache Cassandra", "CouchDB", "Apache CouchDB", "jclouds", "Apache jclouds", "Kafka", "Apache Kafka", "Solr", “Apache Solr", "Storm", “Apache Storm", "ZooKeeper", and "Apache ZooKeeper" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 11:00AM Mar 03, 2020
by Sally in General |
|
The Apache Software Foundation Announces 20th Anniversary of Apache® Subversion®
Community-led Version Control Software and Source Code Management Tool Available on Most Integration Servers, Integrated Development Environments, Issue Tracking Systems, and more.
Wakefield, MA —27 February 2020— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the 20th Anniversary of Apache® Subversion®, the popular centralized software version control system.
Apache Subversion ("SVN") allows users to commit code, manage changes, and recover previous versions of all sorts of data across files and directories. Subversion is ideal for distributed teams who need to easily audit and act on modification logs and versioning history across projects. Subversion originated at CollabNet in 2000 as an effort to create an Open Source version-control system similar to the then-standard CVS (Concurrent Versions System) but with additional features and functionality. Subversion was submitted to the Apache Incubator In November 2009, and became an Apache Top-Level Project in February 2010.
"We are very proud of Subversion's long history, and remain committed to our mission statement," said Stefan Sperling, Vice President of Apache Subversion. "Subversion has moved well beyond its initial goal of creating a compelling replacement for CVS. In 2010 our mission statement was updated to ‘Enterprise-class centralized version control for the masses’.”
Over its 20-year history, Subversion has grown to become the most popular version control system on the market, and remains the leading centralized versioning and revision control software today. Millions of users worldwide depend on the collaboration-friendly system to easily access all files and historical data simultaneously without code conflicts or corruption. Subversion accommodates a wide variety of integrated development environments (IDEs), and is well-suited for large projects.
Apache Subversion has been broadly adopted for mission-critical code distribution and collaboration workflow by Adobe Dreamweaver, Eclipse, Google, Halliburton, Microsoft Visual Studio, Python, Ruby, Skype, SourceForge, and WordPress, among many organizations and development communities. The ASF uses Apache Subversion in its own infrastructure, housing millions of lines of code in more than 1.8 Million commits across 300 Apache Top-Level Projects and sub-projects.
"One of the best decisions of my life was emailing up Karl (Fogel) to see if he was interested in moving the Open Source community beyond CVS," said Brian Behlendorf, co-founder of CollabNet and co-founder of The Apache Software Foundation. "Essential to Subversion's success was the core team of Karl, Ben (Collins-Sussman), and Mike (Pilato) working publicly, spending the difficult time on design docs and helping newbies up the learning curve, with the goal of building as a community what three people (even the best) alone could not do. 20 years later I'm not surprised to see it continuing to innovate, to add features, to fix bugs, and to push the envelope forward. Git still needs competition :) But it's also the best example, and essential example, for why community matters more than code. It's the Subversion community that made it successful, that made the code continuously better, that left no CVS user behind, and that did so with the technical precision and super-human decency all other projects should aspire to."
"Twenty years later, Subversion is no longer the upstart -- it is mature software, and still going strong," said Karl Fogel, original founding developer of Subversion, and Partner at Open Tech Strategies. "Subversion continues to be widely used, especially in enterprise settings, because of its reliability, the simplicity of its conceptual model, its ability to handle large files, and features like path-based access control and optional file-locking. In situations where Subversion's centralized model is the right tool for the job, it really shines: we use it for our entire internal corporate tree, for example, because the path-based authorization is crucial. To get some other viewpoints on where Subversion has come over 20 years, I took a walk through the main project's support forums and the forums of TortoiseSVN, the popular open source SVN client application for Windows. I was delighted by what I saw: a diversity of uses and users, fast and helpful responses, and a focus on practical needs. Starting two decades ago, Subversion helped bring version control beyond developers to a wider audience, and it continues to do that today."
"Today we've got a plethora of fast, reliable, and efficient version control systems, but twenty years ago we had exactly zero: CVS was the only widely used version control system and it still failed in unpredictable ways (including bitrot that was undetectable until you tried to check out old code)," said Brian Fitzpatrick, one of Subversion’s earlier developers. "Even though most people use Git today in the Open Source world, Subversion was the catalyst that allowed folks to move from CVS to Git and so many other modern day version control systems. While the core team wrote a great deal of Subversion's code, we also spent a great deal of time communicating outside of our office in Chicago in an effort to build a larger Subversion community--an effort that eventually paid off more than tenfold."
"When we gathered in my basement in early 2000, thinking about what paths Subversion should follow, none of us imagined what would be accomplished over the next twenty years," said Greg Stein, an early developer of Subversion, and former Vice President of Apache Subversion. "We focused on improving the experience of CVS users and administrators. We overshot our own expectations within just a few years, creating a system that millions have found worthy. From our humble beginnings, I couldn't be more proud of what the community has accomplished."
"Technology is at its best when it brings people together," said Matt Mullenweg, Founder and Lead Developer at the WordPress Foundation. "SVN has brought countless people together over the years and I wish it much continued success."
"Reliable and powerful version management is essential for our product development. Today, more than 100 of our employees regularly use Apache Subversion with several million lines of source code in our Subversion repository," said Roland Wagner, Head of Product Marketing at CODESYS Group. "Our success with Subversion convinced us to become the first company to develop a connected product for the area of industrial automation with the launch of CODESYS SVN. Many of the over 100,000 CODESYS users worldwide work with CODESYS SVN whichsignificantly simplifies the development of their industrial IEC 61131-3 application software, when realizing automation projects for factories and plants, mobile machines, buildings and energy systems. We thank and congratulate the Subversion community on its 20th anniversary!"
"After 20 years, Apache Subversion continues to deliver on our goal with a stable and portable version control system that powers software projects of all sizes being developed on any of the popular operating system platforms," added Sperling. "Apache Subversion repositories store valuable mission-critical assets of companies and organizations across the globe. Subversion remains an essential source code management tool for developers at every level --we welcome their participation on our lists and community."
Availability and Oversight
Apache Subversion software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Subversion, visit http://subversion.apache.org/
About The Apache Software Foundation (ASF)
Established in 1999, The Apache Software Foundation is the world’s largest Open Source foundation, stewarding 200M+ lines of code and providing more than $20B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 765 individual Members and 206 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 7,200 Committers through the ASF’s meritocratic process known as "The Apache Way". Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 has become an industry standard within the Open Source world, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors including Aetna, Alibaba Cloud Computing, Anonymous, ARM, Baidu, Bloomberg, Budget Direct, Capital One, CarGurus, Cerner, Cloudera, Comcast, Facebook, Google, Handshake, Huawei, IBM, Indeed, Inspur, Leaseweb, Microsoft, ODPi, Pineapple Fund, Pivotal, Private Internet Access, Red Hat, Target, Tencent, Union Investment, Workday, and Verizon Media. For more information, visit http://apache.org/ and https://twitter.com/TheASF
© The Apache Software Foundation. "Apache", "Subversion", "Apache Subversion", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 11:00AM Feb 27, 2020
by Sally in General |
|
Launch of the 2020 ASF Community Survey
Find more information about the 2020 ASF Community Survey on its page on Confluence [3], including the privacy policy governing this initiative. If you are part of our community, either as a user, contributor, or both, your participation is paramount to the success of this project! Please consider filling out the survey, and share this blog on social media, send it to your fellow Apache contributors. As individuals form the Apache community, your opinion matters: we need to hear your voice.
Links
[1] https://cwiki.apache.org/confluence/display/COMDEV/ASF+Committer+Diversity+Survey+-+2016
[2] https://blog.bitergia.com/tag/openstack/
[3] https://cwiki.apache.org/confluence/display/EDI/Launch+Plan+-+The+2020+ASF+Community+Survey
Posted at 07:40PM Dec 05, 2019
by Sally in General |
|
The Apache Software Foundation Announces Apache® SINGA™ as a Top-Level Project
"We are excited that SINGA has graduated from the Apache Incubator," said Wei Wang, Vice President of Apache SINGA and Assistant Professor at the National University of Singapore. "The SINGA project started at the National University of Singapore, in collaboration with Zhejiang University, focusing on scalable distributed deep learning. In addition to scalability, during the incubation process, built multiple versions to improve the project’s usability and efficiency. Incubating SINGA at the ASF brought opportunities to collaborate, grew our community, standardize the development process, and more."
Apache SINGA is a distributed machine learning library that facilitates the training of large-scale machine learning (especially deep learning) models over a cluster of machines. Various optimizations on efficiency, memory, communication and synchronization are implemented to speed it up and scale it out. Currently, the Apache SINGA project is working on SINGA-lite for deep learning on edge devices with 5G, and SINGA-easy for making AI usable by domain experts (without deep AI background).
Apache SINGA is in use at organizations such as Carnegie Technologies, CBRE, Citigroup, JurongHealth Hospital, National University of Singapore, National University Hospital, NetEase, Noblis, Shentilium Technologies, Singapore General Hospital, Tan Tock Seng Hospital, YZBigData, and others. Apache SINGA is used across applications in banking, education, finance, healthcare, real estate, software development, and other categories.
"So glad to see the first Apache project focusing on distributed deep learning become a Top-Level Project," said Beng Chin Ooi, Distinguished Professor of National University of Singapore who initialized the SINGA project, and a member of the Apache SINGA Project Management Committee. "It is essential to scale deep learning via distributed computing as the deep learning models are typically large and trained over big datasets, which may take hundreds of days using a single GPU."
Posted at 01:00PM Nov 04, 2019
by Sally in General |
|
The Apache Software Foundation Announces Apache® Dubbo™ as a Top-Level Project
"This day is not only the success of Apache Dubbo project itself, but also another success of The Apache Way," said Ian Luo, Vice President of Apache Dubbo. "Back to the time when Dubbo started to incubate at The Apache Software Foundation, there was a small number of initial committers to the project, but today the number of the Dubbo committers has increased by five times, and we are proud of having far more contributors in this project by now. It is indeed a great journey."
The Dubbo framework specifies the methods that can be called remotely across distributed and microservice systems. Its primary functionalities are: interface based remote call; fault tolerance and load balancing; and automatic service registration and discovery. Features include:
- Transparent interface based RPC – provides high performance interface based RPC, which is transparent to users.
- Intelligent load balancing – supports multiple load balancing strategies out of the box, which perceives downstream service status to reduce overall latency and improve system throughput.
- Automatic service registration and discovery – supports multiple service registries that can detect service online/offline instantly.
- High extensibility – micro-kernel and plugin design ensures that it can easily be extended by third party implementation across core features like protocol, transport, and serialization.
- Runtime traffic routing – can be configured at runtime so that traffic can be routed according to different rules, which makes it easy to support features such as blue-green deployment, data center aware routing, etc.
- Visualized service governance – provides rich tools for service governance and maintenance such as querying service metadata, health status, and statistics.
"Apache Dubbo is one of the most highly visible projects that was open-sourced by Alibaba," said Jiangwei Jiang, Principal Engineer at Alibaba Cloud Intelligence. "Dubbo is widely used in Alibaba and many other companies. It is one of the best designed Open Source frameworks to develop microservices with high-throughput, complicated business logic, and sophisticated governance."
“Congratulations to Dubbo for becoming an Apache Top-Level Project," said Ruimin Jin, Technical Director at Youzan Cloud. "Youzan's large-scale microservice cluster is built on Apache Dubbo. Dubbo's outstanding features, flexible design, and extensive experience in the community has helped us to build distributed systems quickly. In the past three years, we have done a lot of service governance based on Dubbo and achieved very good business results. I hope that Apache Dubbo will have more achievements in multi-language in the future, and the Open Source community can also contribute more plugins to Dubbo."
"Qunar.com first chose Apache Dubbo as service infrastructure in 2012 --doing so helped us take fewer detours in the framework selection when we deployed it at scale," said Zhaohui Yu, Former Senior Director, Infrastructure Department, Qunar.com. "Our team was happy to participate in the community by fixing bugs and contributing new features over the years. Congratulations to Dubbo for graduating as an Apache Top-Level Project: it is a great honor for all Dubbo users."
© The Apache Software Foundation. "Apache", "Dubbo", "Apache Dubbo", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 02:00AM May 21, 2019
by Sally in General |
|
The Apache Software Foundation Announces Apache® SkyWalking™ as a Top-Level Project
"This is a special day for the SkyWalking project and its community. We thank our mentors, contributors, and the Apache Incubator for helping us achieve this goal," said Sheng Wu, Vice President of Apache SkyWalking. "The original agenda behind SkyWalking was to help newcomers understand what is distributed tracing, and the community has grown bigger and stronger since we entered the Apache Incubator. Through The Apache Way, SkyWalking has a very active and diverse community, is used by over 70 companies, and has over 100 source contributors from dozens of different organizations."
Apache SkyWalking provides tracing, service mesh telemetry analysis, metric aggregation and visualization for the distributed system. The project landscape has expanded from a pure tracing system, to an observability analysis platform, and application performance management/monitoring system. Features include:
- Distributed tracing-based APM: 100% traces collected with low payload for original system;
- Cloud-native friendly: observe distributed system powered by service mesh, Istio and Envoy;
- Automated source code change: multiple language agents provided, especially with auto instrumentation supported, in Java, .NET and Nodejs;
- Easy to operate: doesn’t require Big Data in monitoring large scale distributed system; and
- Advanced visualization: used in traces, metrics and topology map.
Apache SkyWalking is in use at dozens of organizations that include 5i5j Group, Alibaba, autohome.com, China Eastern Airlines, China Merchants Bank, Daocloud, dangdang.com, guazi.com, Huawei, ke.com, iFLYTEK, primeton.com, Sinolink Securities, tetrate.io, tuhu.cn, tuya.com, WeBank, Yonghui Superstores, youzan.com, and more.
"Instrumentation is unquestionably the most time-consuming part of establishing a distributed tracing solution into an existing platform. I had the chance to code with some of the SkyWalking community earlier on and could see the quality being invested back then," said Mick Semb Wever, ASF Member and Apache SkyWalking Incubating Mentor. "When they were looking for mentors and a champion to help them create a proposal to become an Apache project, I was excited at the opportunity to help bring the project to the Apache Incubator, and was pleasantly surprised to see how prepared, and ASF-like, the SkyWalking community and project had already become. As was the case with Apache Kylin, SkyWalking has not only been a model project during the incubation process, they have also become ambassadors on open development The Apache Way to the greater Open Source community in China. Congratulations on graduating as an Apache Top-Level Project."
Posted at 10:00AM Apr 24, 2019
by Sally in General |
|
The Apache Software Foundation Announces Apache® PLC4X™ as a Top-Level Project
"There is a lot of hype about 'Industry 4.0,' 'industrial IoT,' and 'digitization', however, there's unfortunately been no viable way to access industrial PLCs from the Open Source world," said Christofer Dutz, Vice President of Apache PLC4X. "Apache PLC4X is the missing link between Open Source and the reality on today's industrial shop floor. We are proud that PLC4X is helping redefine how industrial software is created, and graduating as an Apache Top-Level Project is an important milestone in solidifying our project's maturity."
Connectivity and integration across industrial hardware and IoT edge gateways is often impossible with closed-source, proprietary legacy systems, incompatible protocols, and locked-in vendor solutions. Not only is Apache PLC4X an API for communicating with industrial hardware, it also provides a set of drivers for communicating using a variety of industrial protocols. In addition, PLC4X ships with a wide variety of integration modules to make it easy to use in other Open Source solutions and frameworks.
"We build industrial IoT solutions and do edge computing in the industrial ecosystem, thus PLC connection is an important concern for us but no part of our core business," said Julian Feinauer, CEO at pragmatic industries GmbH. "As there were no open alternatives, we used our own home-made stack for PLC communication and regularly had issues with edge cases or bugs. We moved to Apache PLC4X about half a year ago and already see the payoff of the initial efforts. Since moving to PLC4X we see a massive performance boost, have connectors to more types of PLCs, and reached very good stability."
© The Apache Software Foundation. "Apache", "PLC4X", "Apache PLC4X", "Brooklyn", "Apache Brooklyn", "Camel", "Apache Camel", "Edgent", "Apache Edgent", "Kafka", "Apache Kafka", "Mynewt", "Apache Mynewt", "NiFi", "Apache NiFi", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 10:00AM Apr 23, 2019
by Sally in General |
|
The Apache Software Foundation Announces Apache® Unomi™ as a Top-Level Project
"I am truly thankful to our community, especially our mentors, who have helped us achieve this milestone," said Serge Huber, Vice President of Apache Unomi. "The original vision behind Unomi was to ensure true privacy by making the technologies handling customer data completely Open Source and independent. Since it was submitted to the Apache Incubator, developing Unomi using the Apache Way will ensure the project grows its community to be more diverse and welcome new users and developers."
Apache Unomi is versatile, and features privacy management, user/event/goal tracking, reporting, visitor profile management, segmentation, personas, A/B testing, and more. It can be used as:
- a personalization service for a Web CMS;
- an analytics service for native mobile applications;
- a centralized profile management system with segmentation capabilities; and
- a consent management hub
Apache Unomi is the industry's first reference implementation of the upcoming OASIS CDP specification (established by the OASIS CXS Technical Committee, which sets standards as a core technology for enabling the delivery of personalized user experiences). As a reference implementation, Apache Unomi serves as a real world example of how the standard will be stable, and is quickly gaining traction by those interested in truly open and transparent customer data privacy. Apache Unomi is in use at organizations such as Al-Monitor, Altola, Jahia, Yupiik, and many others to create and deliver consistent personalized experiences across channels, markets, and systems.
"When Serge and I announced the launch of the Apache Unomi project at the 2015 ApacheCon Budapest, Apache Unomi, at that time, was the first proposal among the rising Customer Data Platform industry's segment, positioned as an 'ethical data-driven marketing' product that would respect the privacy of customers while leveraging the power of unified customers data," said Elie Auvray, Head of Business Development at Jahia. "Jahia's digital experience management solutions are based on Apache Unomi, and we can't wait to see how the project will now evolve with its growing community. Seeing today Apache Unomi becoming a Top-Level Project is a great reward for us as Open Source software believers. We are proud of this milestone, grateful to the Apache Software Foundation and our mentors, and we know it's only the beginning of a new –hopefully long and successful– journey."
"Apache Unomi is the perfect solution to implement a user profile platform," said Jean-Baptiste Onofré, Fellow at Talend. "It fully addresses the user trust and privacy needs, allowing to easily create user profile and Web marketing features. As Unomi is powered by Apache Karaf, it's also a great platform for several use cases, such as digital marketing in Web applications, managing user profiles on IoT devices, and more."
"Apache Unomi enables Al-Monitor readers to be driven towards additional personalized content that corresponds, via content tags profiling and related automated segmentations, to what they have already accessed," said Valerie Voci, Head of Digital Strategy and Marketing at Al-Monitor. "This data follows our customers where they go, so it's a consistent experience whether they are getting these recommendations in their inbox or on the Website or both. And if a change takes place on one, that change is immediately reflected on the other. It helps us create a very cohesive marketing message and a great overall digital experience."
"As we were developing a progressive web app (PWA) for a client, we were looking for a Customer Data Platform (CDP) to store customer insights, such as behavioral and explicit customer data," said Lars Petersen, Co-Founder at Altola. "Privacy was table stake for us, along with the flexibility to customize data schema and open API. We selected Apache Unomi based on these parameters, we had it up and running on AWS in less than 30 min. and are very impressed with the maturity of the platform, its privacy by design and how easy it was to work with."
Posted at 12:21PM Mar 21, 2019
by Sally in General |
|
The Apache Way to Sustainable Open Source Success
The Apache Way defines Open Source in terms of both a legal and a social framework for collaboration. It helps others understand what makes Open Source powerful and how participants are expected to behave. In this post we will examine The Apache Way in the context of the Foundation's mission:
"The mission of the Apache Software Foundation (ASF) is to provide software for the public good. We do this by providing services and support for many like-minded software project communities consisting of individuals who choose to participate in ASF activities."
Let's dissect this mission statement.
"Provide Software for the Public Good"
Key points in this section:
- We produce software that is non-excludable and non-rivalrous
- Use of the software in any context does not reduce its availability to others
- Users and contributors have no committed responsibility to the foundation, our projects or our communities
- Use of a license that conforms to the Open Source Definition is necessary but not sufficient to deliver on our mission
Investopedia defines a public good as "a product that one individual can consume without reducing its availability to another individual, and from which no one is excluded." On the surface, this is a good definition for our use of the term. However, there is a nuance in our use. Our mission is not to produce "public goods" but to "provide software for the public good".
To understand why this is important, one needs to think about what motivates the ASF to produce software that is a public good.
Open Source software can be digitally copied and reused in an unlimited number of ways. Every user can modify it for their specific needs. They can combine it with other software. They can design innovative new products and services using it and can make a living from the proceeds. This is all possible without impacting other people's use of the software. As such, the ASF produces software that can be used for the public good in many different ways.
To allow us to deliver on this part of the mission, it is critical that we adopt a license that uses the law to protect the software curated here at the Foundation. For us that license is the Apache License, Version 2. In addition, we adopt an inbound licensing policy that defines which licenses are allowable on software reused within Apache projects. This policy can be summarized as:
- The license must meet the Open Source Definition (OSD).
- The license, as applied in practice, must not impose significant restrictions beyond those imposed by the Apache License 2.0.
This means that you can be assured that software curated by projects within The Apache Software Foundation is both a public good and for the public good. You can use Apache software for any purpose and you have no responsibility to the Foundation or the project to contribute back (though as addressed in the next section, it is often in your interests to do so).
It is important to recognize that there are software projects out there that adopt our license but do not adopt our inbound licensing policy. Such projects may bring restrictions that are not covered by our license; therefore, it is important to carefully examine the licensing policies of these projects. Using the Apache License alone may not provide you with the same options a Foundation project provides.
Apache projects are successful, in large part, because of our diligence with respect to clearly-defined licensing policies. Such diligence makes it much easier for downstream users to understand what they can and cannot do with Apache software. The Apache License is deliberately permissive to ensure that everyone has an opportunity to participate in Open Source within the ASF or elsewhere. Modifications of our license are allowed, but modified licenses are neither the Apache License nor affiliated with or endorsed by The Apache Software Foundation. No modified license can be represented as such. Modified licenses that use the Apache name are strictly disallowed, as they are both confusing to users and undermine the Apache brand.
While we recognize that there are many ways to license software, whether Open Source or otherwise, we assert that only projects that use both our license (unmodified) and our inbound licensing policy truly follow and adhere to The Apache Way.
While an OSD-approved license and associated policies are necessary for successful Open Source production, they are not sufficient. They provide a legal framework for the production of Open Source, but they do not provide a social framework, which brings us to the second sentence of our mission:
"The mission of the Apache Software Foundation is to provide software for the public good. We do this by providing services and support for many like-minded software project communities of individuals who choose to contribute to Apache projects."
"Like-Minded Software Project Communities of Individuals"
Key points in this section:
- The Apache Way provides a governance model designed to create a social framework for collaboration
- The Apache Software Foundation develops communities, and those communities develop software
- ASF project communities develop and reuse software components that in turn may be reused in products
- Users of ASF software often build products and services using our software components
- Our model, and others like it, have produced some of the largest and longest-lived Open Source projects that have literally revolutionized the industry
There is a lot packed into these few words. It is an understanding of these words that makes the difference between software that is under an Open Source license and software that reaches sustainability through The Apache Way. These words underscore the fact that the Foundation does not directly produce software. That's right, The Apache Software Foundation, with upwards of $8Bn of software code, does not directly produce software. Rather than focus on software, we focus on the creation of and support of collaborative communities; the software is an intentional by-product.
Our like-minded project communities come together because they share common problems that can be addressed in software. As the saying goes, "a problem shared is a problem halved". By bringing together individuals with their unique ideas and skills, we break down barriers to collaboration.
The Apache Way is carefully crafted to create a social structure for collaboration, which complements the legal framework discussed above. Where the legal framework ensures an equal right to use the software, The Apache Way ensures an equal ability to contribute to the software. This is critically important to the long term sustainability of Open Source software projects. This social structure for collaboration is missing from many non-Apache projects, yet a robust social structure is invariably a key component in long-term successful projects outside of the ASF.
The Apache Way is fully inclusive, open, transparent and consensus-based. It promotes vendor neutrality to prevent undue influence (or control) from a single company. It ensures that any individual with a valuable contribution is empowered, and it seeks to assure that a project remains sustainable despite inevitable changes in community membership over time.
Apache projects typically produce software components that can be combined with other software (of any license) in different ways to solve different problems. This provides plenty of opportunity for participants to collaborate within a given software project independent of their relationship outside the Foundation. This is very different from the idea of licensing your product as a whole under an Open Source license. Our model offers more opportunities for reuse which, in turn, increase the pool of individuals likely to contribute to the project.
In addition, our merit-based system seeks to ensure that as people come and go, for whatever reason, there is always someone to take their place. As a result, some ubiquitous Apache projects have existed for over 20 years and helped commercialize the World Wide Web; while dozens of newer projects have defined industry segments such as Big Data and IoT (Internet of Things).
A core tenet of The Apache Way is "Community Over Code", which encapsulates our deep belief that a healthy community is a far higher priority than good code. A strong community can always rectify a problem with the code, whereas an unhealthy community will likely struggle to maintain a codebase in a sustainable manner. Healthy communities ensure the Foundation has the stability to thrive for the next 20 years and beyond. Apache projects do not have the problem of scaling that others, who focus only on the legal frameworks of Open Source, suffer from. If you look around at projects that have grown up alongside the Apache projects, you will see a similar focus on scaling the governance model. This is no accident.
Why this is Important
Software is a critical part of any modern economy. It touches every part of every life in the developed world, and is increasingly transforming everyday life, from womb to grave, everywhere.
At The Apache Software Foundation, we believe that every developer has their personal motivations for building software. We celebrate their right to choose when and how they build their software, including their right to use a non-open license.
We will not dictate what is best for developers or for the software industry.
We care about the provision of software that enables our users, our contributors, and the general public to decide what is best for them.
We welcome you to use our software and contribute to our projects -- or not. It's up to you.
We ask that you leave commercial interests at the door.
Countless organizations are proving that their team members who collaborate in a vendor-neutral environment often apply Open Innovation processes (such as The Apache Way) to their work. This helps create internal efficiencies and lays the groundwork for new external opportunities that may provide additional added benefits.
Bringing only your intention of contributing what best serves the greater Apache community reinforces trust in the people and projects behind the Apache brand, and helps us realize our mission of providing software for the public good.
We learn together and work together to deliver the best software we can.
Apache software is available for all.
The freedom to choose is what makes the Foundation and Apache projects so strong.
Summary
The software industry has changed and continues to change. The ways software is delivered to end users have changed. Some of the leaders in our industry have retired and new leaders have emerged. But some things have not changed. Our model of collaborative software development, through a combination of a licensing and social framework, remains one of the most successful models of software production.
Increasing the number of users, even those who do not contribute to code, should be seen as a benefit, not a problem, in Open Source. More users present an opportunity. At Apache, more users means more success since they are our future contributors.
As a US 501(c)(3) public charitable organization, The Apache Software Foundation helps individuals and organizations understand how Open Source at scale works in a highly competitive market. For more than two decades our focus has not been on producing software, but rather mentoring communities who produce software. The Apache Way advances sustainable Open Source communities: everything we do is Open Source so all kinds of users can benefit from our experience. Apache is for everyone.
# # #
Posted at 01:18PM Mar 19, 2019
by Sally in General |
|
The Apache® Software Foundation Announces Apache Arrow™ Momentum
- C# Library
- Gandiva LLVM-based Expression Compiler
- Go Library
- Javascript Library
- Plasma Shared Memory Object Store
- Ruby Libraries (Apache Arrow and Apache Parquet)
- Rust Libraries (Parquet and DataFusion Query Engine)
Posted at 11:00AM Feb 19, 2019
by Sally in General |
|
The Apache Software Foundation Announces Apache® Hadoop® v3.2.0
- ABFS Filesystem connector —supports the latest Azure Datalake Gen2 Storage;
- Enhanced S3A connector —including better resilience to throttled AWS S3 and DynamoDB IO;
- Node Attributes Support in YARN —helps to tag multiple labels on the nodes based on its attributes and supports placing the containers based on expression of these labels;
- Storage Policy Satisfier —supports HDFS (Hadoop Distributed File System) applications to move the blocks between storage types as they set the storage policies on files/directories;
- Hadoop Submarine —enables data engineers to easily develop, train and deploy deep learning models (in TensorFlow) on very same Hadoop YARN cluster;
- C++ HDFS client —helps to do async IO to HDFS which helps downstream projects such as Apache ORC;
- Upgrades for long running services —supports in-place seamless upgrades of long running containers via YARN Native Service API (application program interface) and CLI (command-line interface).
Posted at 11:00AM Jan 23, 2019
by Sally in General |
|