The Apache Usergrid community is happy to announce that Usergrid 2.1 is now available for download.
Apache Usergrid 2.1 is a major revision of the Usergrid code-base that includes a new, improved and streamlined persistence and query engine, as well as support for iOS and Android Push Notifications. Usergrid 2.1 supports the very same REST API as the Usergrid 1.x releases, but this new release is incompatible with the Usergrid 1.x database format.
The new "core persistence" engine in Usegrid 2.1 has been under development for two years now. Internally, this new engine provides a set of new modules for Entity Collections, Graph Database, Maps, Query-Index, Queues and Caching. The core persistence design favors simple and composable operations that can be made asynchronous via RxJava, uses Multi-Version Concurrency Control (MVCC) to avoid use of locks and the Query-Index module relies on ElasticSearch. Version 2.1 is the first official release to include the new core persistence engine, but members of the Usergrid team have been using Usergrid 2.x in production environments for about a year now.
If you're looking to deploy Usergrid, then 2.1 is where you should start. As we say on the download page, it's the best available version.You can download Usergrid 2.1 from our Releases page. In addition to the official sourcec-code release, we now offer a pre-build binary with the Usergrid Stack, Admin Portal and the HTML5/JavaScript and Java SDKs.
Whether you're a developer who wants to write applications that use Usergrid or a sys-admin who wants to know how to deploy Usegrid, we've got docs for you. Refer to our documentation page for Getting Started with Usergrid or jump straight to the Usergrid Deployment Guide.
Posted at 11:21AM Feb 18, 2016
by Dave Johnson in Releases |
|
By Michael Russo
Based on recent testing at Apigee, the upcoming Apache Usergrid 2 release is set to be the most scalable open-source Backend as a Service available. We were able to drive Usergrid to 10,000 transactions per second and, more importantly, found that Usergrid can scale horizontally. Here's the story of how we got there.
Apache Usergrid is a software stack that enables you to run a (Backend-as-a-Service) BaaS that can store, index, and query JSON objects. It also enables you to manage assets and provide authentication, push notifications, and a host of other features useful to developers—especially those working on mobile apps.
The project recently graduated from the Apache Incubator and is now a top-level project of the Apache Software Foundation (ASF). Usergrid is new at Apache, but Apigee has been using it in production for three years now as the foundation for Apigee's API BaaS product.
Usergrid 2 features the same REST API as Usergrid 1, but under the hood just about everything has changed. Usergrid 2 includes a completely new persistence engine backed by Apache Cassandra and a query/indexing service backed by ElasticSearch. We brought ElasticSearch into Usergrid because the query/index service in Usergrid 1 was not performing well and was complex and difficult to maintain. ElasticSearch does a much better job of query/index than we could have done ourselves. Additionally, separating key-value persistence from index/query allows us to scale each concern separately.
As the architecture of Usergrid changed drastically, we needed to have a new baseline performance benchmark to make sure the system scaled as well as, if not better than, it did before. Let's talk about how we tested.
The Usergrid team has invested a lot of time building repeatable test cases using the Gatling load-testing framework. Performance is a high priority for us and we need a way to validate performance metrics for every release candidate.
As Usergrid is open source, so are our Usergrid-specific Gatling scenarios, which you can find here: stack/loadtests (on Github).
One of our goals was to prove that we had the ability to scale more requests per second with more hardware, so we started small and worked our way up.
As the first in our series of new benchmarking for Usergrid, we wanted to start with a trivial use case to establish a solid baseline for the application. All testing scenarios use the HTTP API and test the concurrency and performance of the requests. We inserted a few million entities that we could later read from the system. The test case itself was simple. Each entity has a UUID (universally unique identifier) property. For all the entities we had inserted, we randomly read them out by their UUID:
GET /organization/application/collection/:entityUUID
First, we tried scaling the Usergrid application by its configuration. We configured a higher number of connections to use for Cassandra and a higher number of threads for Tomcat to use. This actually yielded higher latencies and system resource usage for marginally the same throughput. We saw better throughput when there was less concurrency allowed. This made sense, but we needed more, and immediately added more Usergrid servers to verify horizontal scalability. What will it take to get to 10,000 RPS?
| # Usergrid Servers | # Cassandra Nodes | Peak Requests Per Second |
|---|---|---|
| 6 | 4 | 1420 |
| 6 | 6 | 2248 |
| 10 | 6 | 3324 |
| 20 | 6 | 3820 |
| Switch to nine c3.2xlarge instances for Cassandra | ||
| 20 | 9 | 6321 |
| Switch to nine c3.4xlarge instances for Cassandra | ||
| 20 | 9 | 7237 |
| 30 | 9 | 9120 |
| 35 | 9 | 10268 |
It was time to see if Cassandra was keeping up. As we scaled up the load we found Cassandra read operation latencies were also increasing. Shouldn't Cassandra handle more, though? We observed a single Usergrid read by UUID was translating to about 10 read operations to cassandra. Optimization #1: reduce the number of read operations from Cassandra on our most trivial use case. Given what we know, we still decided to test up to a peak 10,000 RPS in the current state.
The cluster was scaled horizontally (more nodes) until we needed to vertically scale (bigger nodes) Cassandra due to high CPU usage. We stopped at 10,268 Requests Per Second with 35 c3.xlarge Usergrid servers and 9 c3.4xlarge Cassandra nodes. By this point numerous opportunities for improvement were identified in the codebase, and we had already executed on some. We fully expect to reach the same throughput with much less infrastructure in the coming weeks. In fact, we've already reached ~7,800 RPS with only 15 Usergrid servers since our benchmarking.
Here are the components that we used in our Usergrid performance testing:
As part of benchmarking, we wanted to ensure that all configurations and deployment scenarios exactly matched how we would run a production cluster. The main configurations that are recommended for production use of Usergrid are:
| Usergrid (Application) | Tomcat (Container) | Cassandra (Database) | |
|---|---|---|---|
| 1 | LOCAL QUORUM read and write consistency set for Cassandra operations | Blocking IO connector ( required in Usergrid) | 6+ node cluster size |
| 2 | Configure separate Keyspace used for Locks vs. Main Usergrid application | Use HTTP 1.1 and ensure keepAlive is configured | |
| 3 | Configure max # of connections used per Cassandra node to be 15 | Non-SSL connector (SSL typically handled by a load balancer) | Replication Factor = 3 |
As part of this testing, not only did we identify code optimizations that we can quickly fix for huge performance gains, we also learned more about tuning our infrastructure to handle high concurrency. Having this baseline gives us the motivation to continually improve performance of the Usergrid application, reducing the cost for operating a BaaS platform at huge scale.
This post is just the start of our performance series. Stay tuned, as we’ll be publishing more results in the future for the following Usergrid scenarios:
See you next time!
Posted at 06:06PM Sep 16, 2015
by Dave Johnson in Technology |
|
We are happy and proud to announce that Apache Usergrid has graduated from the Apache Incubator and we are now an official “top-level” project of the Apache Software Foundation.
Usergrid has been an open source project since 2011 and has been used in production for many customers over the years since then, but now, we’ve gone through the Apache Incubator, learned the Apache Way, grown our community, made three Incubator approved releases and made huge progress on development of a new persistence & indexing engine for the yet-to-be released Usergrid 2 — more about that later.
We're ready to stand on our own an ASF project and and like every Apache project, we welcome all contributors. We’ve got lots of work to do. If you’d like to help out, please join us on our mailing lists and in our HipChat room.
Usergrid, over and out...
Posted at 11:43AM Aug 28, 2015
by Dave Johnson in News |
|
Apache Usergrid 1.0.2 is now officially released and available for DOWNLOAD HERE.
Here's the release announcement that was posted to the Usergrid dev mailing list last week:
Apache Usergrid (incubating) is a multi-tenant Backend-as-a-Service (BaaS) stack for web and mobile applications, based on RESTful APIs. Usergrid consists of a Java-based web application that persists data to Apache Cassandra, an HTML5/JavaScript based management portal and a set of SDKs for JavaScript, iOS, Android, PHP, Java and more.
Usergrid 1.0.2 is a minor release with bug fixes and small improvements.
Bugs fixed
Here's the full list of JIRA issues resolved:
https://issues.apache.org/jira/issues/?jql=project%3Dusergrid%20and%20fixVersion%3D1.0.2
The release is available at:
http://www.apache.org/dyn/closer.cgi/incubator/usergrid/usergrid-1/v1.0.2
The tag used to create the release with is 1.0.2:
https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.git;a=commit;h=1.0.2
The release's Git commit ID is d52427b279306e19c163e89e9d5025760fc0b50a
The MD5 checksum of the release can be found at:
https://dist.apache.org/repos/dist/release/incubator/usergrid/usergrid-1/v1.0.2/apache-usergrid-1.0.2-incubating.tar.gz.md5
The signature of the release can be found at:
https://dist.apache.org/repos/dist/release/incubator/usergrid/usergrid-1/v1.0.2/apache-usergrid-1.0.2-incubating.tar.gz.asc
The GPG key used to sign the release are available at:
https://dist.apache.org/repos/dist/release/incubator/usergrid/KEYS
Posted at 09:33AM Jul 20, 2015
by Dave Johnson in Releases |
|
Welcome and pardon the mess, we're just getting started here with this blog. We've got a layout, based on the Apache Usergrid website, we've got some categories and we've got a FIRST POST, hopefully to be followed by lots of interesting articles, release announcements and happy news about Usergrid. Stay tuned.
Posted at 05:17PM Jul 19, 2015
by Dave Johnson in News |
|