Entries tagged [framework]
The Apache Software Foundation Announces Apache® Hadoop® v3.2.0
- ABFS Filesystem connector —supports the latest Azure Datalake Gen2 Storage;
- Enhanced S3A connector —including better resilience to throttled AWS S3 and DynamoDB IO;
- Node Attributes Support in YARN —helps to tag multiple labels on the nodes based on its attributes and supports placing the containers based on expression of these labels;
- Storage Policy Satisfier —supports HDFS (Hadoop Distributed File System) applications to move the blocks between storage types as they set the storage policies on files/directories;
- Hadoop Submarine —enables data engineers to easily develop, train and deploy deep learning models (in TensorFlow) on very same Hadoop YARN cluster;
- C++ HDFS client —helps to do async IO to HDFS which helps downstream projects such as Apache ORC;
- Upgrades for long running services —supports in-place seamless upgrades of long running containers via YARN Native Service API (application program interface) and CLI (command-line interface).
Posted at 11:00AM Jan 23, 2019 by Sally in General | |
The Apache Software Foundation Announces Apache® ServiceComb™ as a Top-Level Project
- Java-Chassis - an out-of-the-box Java microservices SDK that includes four parts: service contract, programming model, running model and communication model, with a complete set of microservices governance abilities such as load balancing, fallback, rate limiting, and call stack tracing. Microservices governance and business logic are isolated.
- Service-Center - a high-performance, highly available, stateless Golang implementation of the Service Discovery and Registration Center based on Etcd, which provides real-time service instance registration, real-time service instance notification, and inter-service testing based on contract.
- Saga - provides an eventual consistency solution for distributed transactions which could be a pain point of microservices.
- Asynchronous kernel - both synchronous and asynchronous programming models based on VertX effectively ensures high performance and low latency, both in traditional enterprise applications or in emerging businesses such as e-commerce, Internet, and IoT, to avoid avalanche effect when reaching peak loads.
- Out-of-the-box experience - developers using the scaffolding website start.servicecomb.io, can launch microservices-based projects with integrated service registration, discovery, communication and microservices governance capabilities, and centralized configuration by default.
- Open API - automatic code generation and isolating logic code from governance streamlines the DevOps pipeline, enabling different teams to efficiently and independently develop and manage code, test, and document using bidirectional contract files and OpenAPI.
"We use Apache ServiceComb to build our 'intelligent brain' for drone control. ServiceComb is an out-of-the-box microservices solution, which provides the microservices governance abilities without any coding," said Zhou Sujian, Chief Architect of CeeWa Intelligent Technology. "Compared with using or implementing a traditional RPC framework, a lot of development resources are saved. With ServiceComb, both the team development and the node deployment efficiency are doubled, which are very exciting. We are also very happy to see ServiceCombs work on integrating Open Source distributed tracing systems such as Apache Zipkin, Apache SkyWalking and Prometheus, which greatly improved our cross-node chain tracing ability, and the team's efficiency to locate and solve problems."
"Graduating as an Apache Top-Level Project demonstrates that all contributors have a place with Apache ServiceComb, whether they were part of the project before it arrived at the Apache Incubator or joined the community during the incubation process," added Jiang. "It is a pleasure to collaborate with volunteers in this open, equal, and diverse environment. We welcome new ServiceComb contributors to help with code development, evangelizing on innovations in microservices, promoting community development 'the Apache Way', and other ways of participating."
Posted at 10:00AM Oct 24, 2018 by Sally in General | |
The Apache Software Foundation Announces Apache® Wicket(TM) v8.0.0
- Java 8 or newer is required
- Servlet 3.1 is required
- Component databinding (IModel) is now a Functional Interface
- Native JSR 310 Date/Time support
- Java's Optional type is used in the right places
- Removal of many deprecated features from previous versions
Posted at 10:00AM May 23, 2018 by Sally in General | |
The Apache Software Foundation Announces Apache® Hadoop® v3.0.0 General Availability
- HDFS erasure coding —halves the storage cost of HDFS while also improving data durability;
- YARN Timeline Service v.2 (preview) —improves the scalability, reliability, and usability of the Timeline Service;
- YARN resource types —enables scheduling of additional resources, such as disks and GPUs, for better integration with machine learning and container workloads;
- Federation of YARN and HDFS subclusters transparently scales Hadoop to tens of thousands of machines;
- Opportunistic container execution improves resource utilization and increases task throughput for short-lived containers. In addition to its traditional, central scheduler, YARN also supports distributed scheduling of opportunistic containers; and
- Improved capabilities and performance improvements for cloud storage systems such as Amazon S3 (S3Guard), Microsoft Azure Data Lake, and Aliyun Object Storage System.
Apache Hadoop is widely deployed at numerous enterprises and institutions worldwide, such as Adobe, Alibaba, Amazon Web Services, AOL, Apple, Capital One, Cloudera, Cornell University, eBay, ESA Calvalus satellite mission, Facebook, foursquare, Google, Hortonworks, HP, Hulu, IBM, Intel, LinkedIn, Microsoft, Netflix, The New York Times, Rackspace, Rakuten, SAP, Tencent, Teradata, Tesla Motors, Twitter, Uber, and Yahoo. The project maintains a list of known users at https://wiki.apache.org/hadoop/PoweredBy
"It's tremendous to see this significant progress, from the raw tool of eleven years ago, to the mature software in today's release," said Doug Cutting, original co-creator of Apache Hadoop. "With this milestone, Hadoop better meets the requirements of its growing role in enterprise data systems. The Open Source community continues to respond to industrial demands."
Apache Hadoop's diverse community enjoys continued growth amongst the ASF's most active projects, and remains at the forefront of more than three dozen Apache Big Data projects.
Apache Hadoop has received countless awards, including top prizes at the Media Guardian Innovation Awards and Duke's Choice Awards, and has been hailed by industry analysts:
"...the lifeblood of organizational analytics…" —Gartner
"Hadoop Is Here To Stay" —Forrester
"...today Hadoop is the only cost-sensible and scalable open source alternative to commercially available Big Data management packages. It also becomes an integral part of almost any commercially available Big Data solution and de-facto industry standard for business intelligence (BI)." —MarketAnalysis.com/Market Research Media
"...commanding half of big data’s $100 billion annual market value...Hadoop is the go-to big data framework." —BigDataWeek.com
"Hadoop, and its associated tools, is currently the 'big beast' of the big data world and the Hadoop environment is undergoing rapid development..." —Bloor Research
"The opportunity to effect meaningful, even fundamental change in the Apache Hadoop project remains open," added Douglas. "Our new contributors uprooted the project from its historical strength in Web-scale analytics by introducing powerful, proven abstractions for data management, security, containerization, and isolation. Apache Hadoop drives innovation in Big Data by growing its community. We hope this latest release continues to draw developers, operators, and users to the ASF."
Catch Apache Hadoop in action at the Strata Data Conference in San Jose, CA, 5-8 March 2018, and at dozens of Hadoop Meetups held around the world.
Availability and Oversight
Apache Hadoop software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Hadoop, visit http://hadoop.apache.org/
About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server —the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 680 individual Members and 6,300 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Alibaba Cloud Computing, ARM, Bloomberg, Budget Direct, Capital One, Cash Store, Cerner, Cloudera, Comcast, Facebook, Google, Hortonworks, Huawei, IBM, Inspur, iSIGMA, ODPi, LeaseWeb, Microsoft, PhoenixNAP, Pivotal, Private Internet Access, Red Hat, Serenata Flowers, Target, Union Investment, WANdisco, and Yahoo. For more information, visit http://www.apache.org/ and https://twitter.com/TheASF
© The Apache Software Foundation. "Apache", "Hadoop", "Apache Hadoop", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 11:00AM Dec 14, 2017 by Sally in General | |
The Apache Software Foundation Announces Apache® Juneau™ as a Top-Level Project
Open Source framework for quickly and easily creating Java-based REST microservices and APIs in use at IBM, The Open Group, and Salesforce, among others.
Forest Hill, MD –31 October 2017– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache® Juneau™ has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.
Apache Juneau is a cohesive framework that allows developers to marshal POJOs (Plain Old Java Objects) and develop REST (Representational State Transfer) microservices and APIs. Marshalling is used to transform an object’s memory representation to a data format suitable for moving between different parts of a computer program (or across programs), and to simplify communications to remote objects with an object.
Apache Juneau consists of:
- A universal toolkit for marshalling POJOs to a wide variety of content types using a common cohesive framework;
- A universal REST server API for creating self-documenting REST interfaces using POJOs, simply deployed as one or more top-level servlets in any Servlet 3.1.0+ container;
- A universal REST client API for interacting with Juneau or 3rd-party REST interfaces using POJOs and proxy interfaces; and
- A REST microservice API that combines all the features above with a simple configurable Jetty server for creating lightweight standalone REST interfaces that start up in milliseconds.
Apache Juneau is in use at IBM, The Open Group, and Salesforce, among others. The Apache Streams project began incorporating Apache Juneau libraries in late 2016.
Availability and Oversight
Apache Juneau software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Juneau, visit http://juneau.apache.org/ and https://twitter.com/ApacheJuneau
About the Apache Incubator
The Apache Incubator is the entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects wishing to join the ASF enter through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit http://incubator.apache.org/
About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 680 individual Members and 6,300 Committers across six continents successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Alibaba Cloud Computing, ARM, Bloomberg, Budget Direct, Capital One, Cash Store, Cerner, Cloudera, Comcast, Facebook, Google, Hewlett Packard, Hortonworks, Huawei, IBM, Inspur, iSIGMA, ODPi, LeaseWeb, Microsoft, PhoenixNAP, Pivotal, Private Internet Access, Red Hat, Serenata Flowers, Target, WANdisco, and Yahoo. For more information, visit http://apache.org/ and https://twitter.com/TheASF
© The Apache Software Foundation. "Apache", "Juneau", "Apache Juneau", "Streams", "Apache Streams", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.
# # #
Posted at 10:00AM Oct 31, 2017 by Sally in General | |
Apache Struts Statement on Equifax Security Breach
The Apache Struts Project Management Committee (PMC) would like to comment on the Equifax security breach, its relation to the Apache Struts Web Framework and associated media coverage.
Our general advice to businesses and individuals utilizing Apache Struts as well as any other open or closed source supporting library in their software products and services is as follows:
1. Understand which supporting frameworks and libraries are used in your software products and in which versions. Keep track of security announcements affecting this products and versions.
3. Any complex software contains flaws. Don't build your security policy on the assumption that supporting software products are flawless, especially in terms of security vulnerabilities.
5. Establish monitoring for unusual access patterns to your public Web resources. Nowadays there are a lot of open source and commercial products available to detect such patterns and give alerts. We recommend such monitoring as good operations practice for business critical Web-based services.
For the Apache Struts Project Management Committee,
Vice President, Apache Struts
The Apache Software Foundation Announces Apache® Samza™ v0.13
- A higher level API that developers can use this to express complex processing pipelines on streams more concisely;
- Support for running Samza applications as a lightweight embedded library without relying on YARN;
- Support for flexible deployment options;
- Support for rolling upgrade of running Samza applications;
- Improved monitoring and failure detection using a built-in heart beating mechanism;
- Enabling better integrations with other cluster-manager frameworks and environments; and
- Several bug-fixes that improve reliability, stability and robustness of data processing,
Posted at 11:00AM May 15, 2017 by Sally in General | |
The Apache Software Foundation Announces Apache® Ranger™ as a Top-Level Project
- Ranger Key Management Service (compatible with Hadoop’s native KMS API to store and manage encryption keys for HDFS Transparent Data Encryption);
- Dynamic column masking and row filtering;
- Dynamic policy conditions (such as prohibition of toxic joins);
- User context enrichers (such as geo-location and time of day mappings); and
- Classification or tag based policies for Hadoop ecosystem components via integration with Apache Atlas.
# # #
Posted at 11:00AM Feb 08, 2017 by Sally in General | |
The Apache Software Foundation Announces Apache® OODT™ v1.0
- Data ingestion and processing;
- Automatic data discovery and metadata extraction;
- Metadata management;
- Workflow processing and support; and
- Resource management
Posted at 10:00AM Jun 29, 2016 by Sally in General | |
The Apache Software Foundation Announces Apache® TinkerPop™ as a Top-Level Project
The central component to Apache TinkerPop is Gremlin, a graph traversal machine and language, which makes it possible to write complex queries (called traversals) that can execute either as real-time OLTP queries, analytic OLAP queries, or a hybrid of the two.
Apache TinkerPop is in use at organizations such as DataStax and IBM, among many others. Amazon.com is currently using TinkerPop and Gremlin to process its order fullfillment graph which contains approximately one trillion edges.
The core Apache TinkerPop release provides production-ready, reference implementations of a number of different data systems including Neo4j (OLTP), Apache Giraph (OLAP), Apache Spark (OLAP), and Apache Hadoop (OLAP). However, the bulk of the implementations are maintained within the larger TinkerPop ecosystem. These implementations include commercial and Open Source graph databases and processors, Gremlin language variants for various programming languages on and off the Java Virtual Machine, visualization applications for graph analysis and many other tools and libraries. The TinkerPop ecosystem is richly supported with many options for developers to choose from.
TinkerPop originated in 2009 at the Los Alamos National Laboratory. After two major releases (TinkerPop1 in 2011 and TinkerPop2 in 2012), the project was submitted to the Apache Incubator in January 2015.
"Following in a long line of Apache projects that revolutionized entire industries, starting with with the Apache HTTP Server, continuing with Web Services, search, and Big Data technologies, Apache TinkerPop will no doubt reshape the Graph Computing landscape," said Hadrian Zbarcea, co-Vice President of ASF Fundraising and Incubator Mentor of Apache TinkerPop. "While TinkerPop has just graduated as an ASF Top Level Project, it is already seven years old, a mature technology, backed by a number of vendors, a vibrant community, and absolutely brilliant developers."
The project welcomes those interested in contributing to Apache TinkerPop. For more information, visit http://tinkerpop.apache.org/docs/3.2.0-incubating/dev/developer/#_contributing
Posted at 10:00AM May 23, 2016 by Sally in General | |
The Apache® Software Foundation announces Apache Flink™ v1.0
- initiating backwards compatibility of public APIs against all 1.x.y versions;
- introducing functionality for complex event processing (CEP);
- supporting large state beyond memory limits;
- supporting state versioning and savepoints; and
- improving the system's monitoring functionality
- QCon (London, 7-9 March 2016)
- Strata/Hadoop World (San Jose, 28-31 March 2016)
- Hadoop Summit (Dublin, 13-14 April 2016)
- Kafka Summit (San Francisco, 26 April 2016)
- Apache: Big Data (Vancouver, 9-12 May 2016)
- OSCON (Austin, TX, 18-19 May 2016)
- Strata/Hadoop World (London, 31 May - 3 June 2016)
- Berlin Buzzwords (Berlin, 5-7 June 2016)
- Flink Forward (Berlin, 12-14 September 2016)
# # #
Posted at 12:00PM Mar 08, 2016 by Sally in General | |
The Apache Software Foundation Announces Apache™ Brooklyn™ as a Top-Level Project
Posted at 11:00AM Nov 23, 2015 by Sally in General | |
The Apache Software Foundation Announces Apache™ Samza™ as a Top-Level Project
- DoubleDutch, developers of mobile apps for events and conferences, uses Samza to power their analytics platform and stream data live into an event dashboard for real-time insights;
- Forstcales' Big Data security analytics solutions use Samza to processes security events log as part of the data ingestion pipelines and on-line machine learning models creation process;
- Happy Pancake, Northern Europe's largest internet dating service, uses Samza for all event handlers and data replication;
- Advertising technology provider Improve Digital uses Samza as the foundation of a realtime processing capability performing data analytics and as the basis for an alerting system;
- Jack Henry & Associates uses Samza to process user activity data across its Banno suite of products for financial institutions;
- MobileAware uses Samza as a foundation for two mobile network products: real time analytics and multi channel notification (push, text message and HTML5);
- Technology startup Project Florida uses Samza for real-time monitoring of data streams from wearable sensors, for preventative healthcare purposes;
- Quantiply, providers of Cloud-based micro-applications, uses Samza to bring together user event, system performance, and business operational data for real-time visibility and decision support; and
- Social media business intelligence solution VinTank uses Samza to power their analysis and natural language processing (NLP) pipeline.
Posted at 10:21AM Jan 27, 2015 by Sally in General | |
The Apache Software Foundation Announces Apache™ MetaModel™ as a Top-Level Project
Posted at 10:00AM Dec 09, 2014 by Sally in General | |
The Apache Software Foundation Announces Apache™ Tez™ as a Top-Level Project