Apache Ignite 2.0: Redesigned Off-heap Memory, DDL and Machine Learning
We released the long-awaited Apache Ignite version 2.0 on May 5. The community spent almost a year incorporating tremendous changes to the legacy Apache Ignite 1.x architecture. And all of that effort paid off. Our collective blood, sweat (and perhaps even a few tears) opened up new and exciting opportunities for the Apache Ignite project.
Have I piqued your interest about this new release yet? Let's walk through some of the main new features that have appeared under the hood of Apache Ignite 2.0.
Reengineered Off-Heap Memory Architecture.
The platform’s entire memory architecture was reengineered from scratch. In a nutshell, all of the data and indexes are now stored in a completely new manageable off-heap memory that has no issues with memory fragmentation, accelerates SQL Grid significantly and helps your application easily tolerate Java GC pauses.
Take a peek at the illustration below and try to guess what’s changed. Afterward, please read this documentation to see if your eye caught everything that’s new.
Here’s something extremely noteworthy: the architecture now integrates seamlessly with disk drives. Why do we care about this? Stay tuned!
Data Definition Language.
This release introduces support for Data Definition Language (DDL) as a part of its SQL Grid functionality. Now you can define -- and, what’s more important, alter -- indexes in runtime without the need to restart your cluster. Apache Ignite users have long awaited this feature! Even more exciting news: users can leverage this with standard SQL commands like CREATE or DROP index. This is only the beginning! Go to this page to learn more about current DDL support.
Machine Learning Grid Beta - Distributed Algebra.
Apache Ignite is about more than in-memory storage. And it’s not just one more product for distributed computations or real-time streaming. It's much, much more than that. It's a hot blend of well-integrated distributed and highly concurrent modules that turned Apache Ignite into what is today: A robust data-fabric and framework with the goal of making your application thrive and outperform even the best of expectations.
But there was one thing missing until now. Drumroll, please: machine-learning support!
With Apache Ignite 2.0 you can check project’s own distributed algebra implementation. The distributed algebra is the foundation of the entire component. And soon you can expect to get distributed versions of widely used regression algorithms, decision trees and more.
Spring Data Integration.
Spring Data integration allows the interaction of an Apache Ignite cluster using the well-known and highly adopted Spring Data Framework. You can connect to the cluster by means of Spring Data repositories and start executing distributed SQL queries as well as simple CRUD operations.
Are you using Rocket MQ in your project and need to push data from the Rocket to Ignite? Here is an easy solution.
Hibernate L2 cache users have been anticipating support of Hibernate 5 on Apache Ignite for quite a long time. Apache Ignite 2.0 grants this desire. The integration now supports Hibernate 5 and contains a number of bug fixes and improvements.
Ignite.NET has been enhanced with an addition of a plugin system that allows the writing and embedding 3rd party .NET components into Ignite.NET.
The Ignite.C++ part of the community finally came up with a way to execute arbitrary C++ code on remote cluster machines.
This approach was initially tested for continuous queries. You can now register continuous queries' remote filters on any cluster node you like. Going forward you can expect support for the Ignite.C++ compute grid and more.
Want to learn more? Please join me June 7 for a webinar titled, “Apache® Ignite™: What’s New in Version 2.0.” I hope to see you there!
P.S. Just in case you can’t wait until June… here's a full list of the changes inside Apache Ignite 2.0.
Presenting Apache Ignite SQL Grid at Big Data Bootcamp
Apache Ignite community welcomes you to attend Big Data Bootcamp on March 27th, 28th and 29th 2017 in Santa Clara, USA.
The conference gathers experts and vendors from Big Data realm in sunny California who will be covering a variety of Big Data products and technologies, including, but not limited to, Hadoop, Spark, NoSQL, Data Science, Machine Learning, Artificial Intelligence & Deep Learning.
Apache Ignite will be introduced at the conference by its PMC chair and committer - Denis Magda.
As all we know, in-memory data grids bring exceptional performance and scalability gains to applications built on top of them. The applications truly achieve 10x more performance improvement and become easily scalable and fault-tolerant thanks to the unique data grids architecture. However, because of this particular architecture, a majority of data grids have to sacrifice traditional SQL support requiring application developers to completely rewrite their SQL-based code to support data grid specific APIs. This, however, is not true for Apache Ignite.
In this presentation, Denis will introduce Apache Ignite SQL Grid component that combines the best of two worlds - performance and scalability of data grids and traditional ANSI-99 SQL support of relational databases. Moreover, Denis will take an existing application that works with a relational database and will show how to run it on top of Apache Ignite with minimum efforts.
The talk is called "Apache Ignite SQL Grid: Hot Blend of Traditional SQL and Swift Data Grid" and takes place at 1:00 PM - 1:40PM on March 28. Refer to Big Data Bootcamp's agenda for more details.
Finally, use promotional code SPEAKER to receive $200 discount on or before March 15th by registering at the conference site.
See you at the conference!
Apache Ignite 1.9 Released
Apache Ignite community is pleased to announce Apache Ignite 1.9 - the next minor release of a well-known in-memory data fabric. The release, as usual, encompasses many bug fixes, performance improvements and fresh features. Below you can see a description of the most significant updates.
Apache Ignite was integrated with Kubernetes which is a modern open source container cluster manager. The integration helps to simplify a deployment of an Apache Ignite cluster in environments managed by Kubernetes and let the latter care of resources management, cluster's scalability and lifecycle.
For instance, you're no longer need to monitor a cluster state constantly to be sure that the number of cluster nodes doesn't go, let's say, below 4. If Kubernetes sees that one cluster node is disconnected and only 3 are left then it will start one more automatically to meet the deployment requirements.
Refer to Kubernetes Deployment Getting Started if this is the feature of interest for you.
Performance Optimizations and Benchmarks Automation
Apache Ignite 1.9 can boast of much better performance for core cache operations and SQL queries in compare to the previous Apache Ignite 1.8 release. In general, we observe up to 40% performance increase for particular operations.
It's no longer a challenge to reproduce the performance numbers. Starting with Apache Ignite 1.9 release all the benchmarks are delivered in every Apache Ignite distribution and can be easily executed in your own environment.
Data Modification Language and Queries Parallelism
The community keeps spending significant time improving SQL Grid component that empowers Apache Ignite users with in-memory database capabilities.
In this release, DML (Data Modification Language) support was expanded to the level of Ignite.NET and Ignite.C++ APIs. Plus, a streaming mode was introduced for DML allowing to execute DML operations even faster for specific scenarios like initial data preloading.
One more SQL Grid related optimization makes it possible to parallelize a query execution on every Ignite node where the query has been mapped. By default, a query is executed in a single thread on every participating node. However, for a variety of OLAP use cases it might be a bottleneck and this is where the query parallelism can help out.
Apache Ignite implemented .NET TransactionScope API allowing to work with distributed Apache Ignite transactions fully relaying on standard interfaces available in .NET Framework. Refer to this documentation page for more information.
Ignite.C++ introduced support for well-known continuous queries API. Now, you can listen to data modifications happened on Apache Ignite's distributed caches side from your C++ applications.
Ignite’s spark integration was upgraded to the latest Spark version. Presently, you can leverage from Ignite Shared RDDs in applications using latest Spark version.
Give a Try
Go and grab the latest 1.9 release from our main site. Looking forward to your feedback!