Apache Bigtop

Wednesday Nov 06, 2013

Release of Apache Bigtop 0.7.0

Exciting times for the Apache Bigtop community and the Apache Hadoop ecosystem at large with the release of the brand new Apache Bigtop 0.7.0!

This new release brings tons of new features and fixes for the beloved 100% community and open source driven big data distribution.

Among the new features:

  • Addition of Phoenix, the Apache HBase SQL layer project

  • Addition of Apache Spark (incubating), the general-purpose cluster computing system

  • A brand new SolrCloud 4.5 integration with HDFS

  • Improvement to our puppet recipes so setting up your Apache Bigtop toolchain will be easier than ever

  • Addition of a new script init-hdfs.sh to initialize Apache HDFS filesystem structure so you can get up and running without manually setting up a complex directory structure

  • Add flexibility in the way Apache Oozie manages its Apache Tomcat application

  • Add a new standard location to install SQL connectors and other plug-ins


Also this releases fixes:

  • Libsnappy is now included in Apache Hadoop packages

  • Clean up some dependencies throughout our packages

  • Update to our documentation


Of course this release brings its set of upgrades:

  • Apache Hadoop 2.0.6-alpha

  • Apache Giraph 1.0.0

  • Apache Flume 1.4.0

  • Apache Pig datafu 1.0.0

  • Apache Crunch 0.7.0

  • Apache HBbase 0.94.12

  • Apache Hive 0.11.0

  • Hue 2.5.1

  • Apache Mahout 0.7.5

  • Apache Solr 4.5.0


And as usual Apache Bigtop supports and provide convenience artifacts for a wide array of GNU/Linux distributions:

  • Centos 5

  • Centos 6

  • Fedora 17

  • Fedora 18

  • Ubuntu Lucid

  • Ubuntu Precise

  • Ubuntu Quetzal

  • OpenSUSE 12

  • SLES11

With convenience repositories available: from http://www.apache.org/dist/bigtop/bigtop-0.7.0/repos/


Overall Apache Bigtop 0.7.0 is a great release and is in the continuity of what the previous ones brought to the field:

  • Stability

  • Reliability

  • Feature rich

The first release candidate uncovered a few issues which were promptly fixed for the following and last release candidate RC1.

As a developer of the distribution and a user, everything felt right and worked out of the box. I was even able to set up in a very short time an Apache Hadoop cluster shipping its logs to Apache Flume and aggregating them into an ElasticSearch cluster. Everything being visualized with Kibana.


I would like to take the opportunity to thank all the members Apache Bigtop community for all the effort put into such a great community and distribution.

I would also like to encourage everyone to give a try to our latest release and to not hesitate to come in, participate and give any feedback on our mailing lists: http://bigtop.apache.org/mail-lists.html  .

Saturday Jun 22, 2013

Apache Bigtop 0.6.0 has been released

Just in time for Hadoop Summit 2013, Apache Bigtop team is very pleased to announce the release of Apache Bigtop 0.6.0. The very first release of a fully integrated Big data management distribution built on the currently most advanced Hadoop 2.x -- Hadoop 2.0.5-alpha.

Apache Bigtop, as many of you might already know, is a project aimed at creating 100% open source and community driven Big data management distribution based on Apache Hadoop. You can learn more about it by reading one of our earlier blog posts on Apache Blogs.

The very astute readers of this blog would notice that given our quarterly release schedule Bigtop 0.6.0 should have been called Bigtop 0.7.0. It is true that we skipped a quarter. Our excuse is that we spent all this extra time on helping Hadoop community stabilize the Hadoop 2.x code line and making it a robust kernel for all the applications that are now part of the Bigtop distribution. And speaking of applications, we haven’t forgotten to grow the Bigtop family. Bigtop 0.6.0 adds Apache HCatalog and Apache Giraph to the mix. The full list of Hadoop applications available as part of Bigtop 0.6.0 release is now:

  • Apache Zookeeper 3.4.5Apache Flume 1.3.1

  • Apache HBase 0.94.5

  • Apache Pig 0.11.1

  • Apache Hive 0.10.0

  • Apache Sqoop 2 (AKA 1.99.2)

  • Apache Oozie 3.3.2

  • Apache Whirr 0.8.2

  • Apache Mahout 0.7

  • Apache Solr (SolrCloud) 4.2.1

  • Apache Crunch (incubating) 0.5.0

  • Apache HCatalog 0.5.0

  • Apache Giraph 1.0.0

  • LinkedIn DataFu 0.0.6

  • Cloudera Hue 2.3.0

The list of supported Linux platforms has expanded to include:

  • CentOS/RHEL 5 and 6

  • Fedora 17 and 18

  • SuSE Linux Enterprise 11

  • OpenSUSE 12.2

  • Ubuntu LTS Lucid (10.04) and Precise (12.04)

  • Ubuntu Quantal (12.10)

We would like to invite everybody to give the Bigtop 0.6.0 binary distribution a try. All you have to do is to pick your favorite Linux distribution, follow our wiki instructions and you will have your first pseudo-distributed cluster computing Pi in no time.

If you’re thinking about deploying Bigtop to a fully-distributed cluster you might find our Puppet code to be useful — after all we use it all the time ourselves to test Bigtop. There is brief documentation on how to run our Puppet recipes in a master-less puppet configuration, but a typical Puppet master setup should work as well. Bigtop plays an important role in CDH which leverages all its packaging code from Bigtop.

Finally, Apache Bigtop would not have been possible without the tireless work of all the volunteer developers. This is an amazing community to be part of, and if you would like to join us, now is the time. In fact, we decided to take advantage of Hadoop Summit drawing a lot of Hadoop developers to the San Francisco Bay Area and have our first meeting of the Apache Bigtop Working Group on Thu, Jun 27 2013. Come join us! It is a lot of fun to build the future of bigdata management together!

Happy Big Data discoveries,
Your faithful and tireless Bigtop development team!

Friday Apr 19, 2013

BigTop: the way to grow open Hadoop stack acceptance

BigTop is stepping up in its role as the foundation of a standard Hadoop-based data analytics stack, essentially bringing most of the commercial offering to the standard footing.
[Read More]

Thursday Dec 27, 2012

Apache Bigtop 0.5.0 has been released

Straight from the Santa's bag of toys Apache Bigtop 0.5.0 is now available. This is out first release as an Apache TPL (top level project) and it is based on Hadoop 2.0.2-alpha with even greater selection of fully integrated Hadoop ecosystem components (Solr, Crunch, etc):

  • Apache Zookeeper 3.4.5
  • Apache Flume 1.3.0
  • Apache HBase 0.94.2
  • Apache Pig 0.10.0
  • Apache Hive 0.9.0
  • Apache Sqoop 1.4.2
  • Apache Oozie 3.3.0
  • Apache Whirr 0.8.1
  • Apache Mahout 0.7
  • Apache Solr (SolrCloud) 4.0.0
  • Apache Crunch (incubating) 0.4.0
  • DataFu 0.0.4
  • Hue 2.1.0

 

The list of supported Linux platforms has expanded to:
  • CentOS/RHEL 5 and 6
  • Fedora 16 and 17
  • SuSE Linux Enterprise 11
  • OpenSUSE 12.2
  • Ubuntu LTS Lucid and Precise
  • Ubuntu Quantal

There is a lot of goodies in this release: SolrCloud, brand new Hue Oozie UI, latest and greatest Flume just to name few. We would like to invite everybody to join the developer community or simply give the binary Bigtop distribution a try. All you have to do is pick your favorite Linux Distro, follow our wiki instructions and you will have your first pseudo distributed cluster computing Pi in no time (on a pseudo distributed cluster the value of Pi is approximately 3.4).

If you're thinking about deploying Bigtop to a fully-distributed cluster you might find our Puppet code to be useful -- after all we use it all the time ourselves to test Bigtop. There is a brief documentation on how to run our Puppet recipes in a master-less puppet configuration, but a typical Puppet master setup should work as well.

Whatever you do, don’t forget to check us out at Apache and consider getting involved. Bigtop is a community-driven effort and we need your help. Of course, above all we need you to use Bigtop and give us your the feedback.

Happy Big Data discoveries,
Your faithful and tireless Bigtop development team!

Sunday Jul 08, 2012

What is Bigtop, and Why Should You Care?

Ever since Apache Bigtop entered an incubation, we've been answering a very basic question: what exactly is Bigtop and why should you or anyone in the Apache (or Hadoop) community care. The earliest and the most succinct answer (the one used for the Apache Incubator proposal) simply stated that "Bigtop is a project for the development of packaging and tests of the Hadoop ecosystem". That was a nice explanation of how Bigtop relates to the rest of the Apache Software Foundation's (ASF) Hadoop ecosystem projects, yet it doesn't really help you understand the aspirations of Bigtop that go beyond what the ASF has traditionally done.

[Read More]

Monday Apr 02, 2012

Bigtop presents full stack based on Apache Hadoop 1.0

First ever full stack of Hadoop 1.0 has been just released. It includes all data analytics components like Hive, HBase, Pig, Mahout and my more. The release is available for immediate download from all ASF mirrors for all major Linux distributions: Ubuntu, Fedora, CentOS, Suse.
[Read More]

Thursday Feb 09, 2012

All you wanted to know about Hadoop, but were too afraid to ask: genealogy of elephants.

Lining up versions of Hadoop and making sense of all of them and their relations can be quit difficult. 

This article attempts to address the moot points and help you understand the "bigger picture" - literally.

[Read More]

Wednesday Dec 28, 2011

Conception and validation of Hadoop BigData stack.

What is BigTop project? What are the goals and how it is getting to achieve it? What are the roots and founding ideas of the project?

I think you'll find the answers for these questions in what hopefully became a series of helpful posts helping IT professionals with Hadoop stack deployment and adoption.

[Read More]

Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation