Apache Sqoop

Wednesday November 07, 2012

What's new in Apache Sqoop 1.4.2

Sqoop 1.4.2 was released in August 2012. As this was an extremely important release for the Sqoop community - our first release as an Apache Top Level project - I would like to highlight the key features and fixes of this release. The entire change log can be viewed on our JIRA and actual bits can be downloaded from the usual place.

Hadoop 2.0.0 Support

One of the main the goals of the previous 1.4.1-incubating release was to make sure that Sqoop works on various Hadoop versions (that it's not locked to one specific version). We've made sure that Sqoop works on Hadoop 0.20, 1.0 and 0.23. We're excited to announce that in Sqoop 1.4.2 we've added support for Hadoop 2.0.x (SQOOP-574) and so now Sqoop will work on four different major Hadoop releases!

Even though we made sure that the same codebase will work across all different Hadoop versions, we couldn't ensure the same for the generated binary artifacts. Different Hadoop versions are not binary compatible with each other and there is nothing that Sqoop can do about that. You need to make sure that the Sqoop version that you're using has been compiled against the appropriate Hadoop version that your cluster is using. To help users pick the correct Sqoop binary, all artifacts in the release have a suffix that encodes the supported Hadoop version.
Compatibility with Old Connectors
One of our goals during incubation was to move the entire code base from the com.cloudera.sqoop namespace to the org.apache.sqoop namespace. The change would have broken backward compatibility for connectors compiled against Sqoop 1.3.x. Incompatibility was a showstopper for us and thus we developed this solution. Unfortunately during the code migration we overlooked one issue that affected seamless usage of Microsoft, Oracle and other special connectors. With that resolved, there are no outstanding compatibility issues and all special connectors that were built against Sqoop 1.3.x should work on 1.4.2 without any issues or required modifications.
Incremental Imports of Free Form Queries
One of many interesting and very useful features of Sqoop is incremental import: only deltas of new data are imported on an incremental basis to Hadoop (HDFS, Hive, HBase). We've extended this feature to support free form queries (SQOOP-444) as well.
Implicit and Explicit Connector Pickup Improvements
Logic that leads to connector selection in Sqoop was fragile and very hard to predict. Sqoop supports the parameters --driver and --connection-manager that are supposed to provide a reliable way to specify what connector and JDBC driver will be picked up. Unfortunately, by specifying the --driver option, the built-in connection manager selection defaults to the generic JDBC connector, ignoring the --connection-manager option and any installed third party connectors. To address those nasty issues we've improved the connector selection (SQOOP-529) process, so that the entire logic is now far more predictable and reliable.
Exporting Only a Subset of Columns
The Export feature allows you to move large data volumes from the Hadoop ecosystem (HDFS, Hive) to relational database systems and data warehouses for additional processing or reporting. Until this release, the table that Sqoop was performing export into had to contain exactly the same number of columns as the exported data. This is highly undesirable in some use cases, so we've made sure (SQOOP-503) that the parameter --columns that was working in import will also work for export. However exporting data to only a subset of columns brings additional requirements to the target table: other columns must have either a default value or be nullable. The export job will fail otherwise.
Verbose Logging
Verbose mode wasn't correctly working in Sqoop 1.4.x due to the aforementioned namespace migration. We've made sure to fix that (SQOOP-460,  SQOOP-488). In addition we've extended the reach of the --verbose parameter to generated mapreduce jobs (SQOOP-436), so we now provide much better logging when needed during the entire import or export process!
Hive Imports
Sqoop team has improved Hive imports in several ways. The first improvement was that the user no longer needs to remove the empty temporary directory after Sqoop moves data to Hive (SQOOP-443). The main benefit is that the user now can perform multiple Hive imports with the same temporary directory without the need for extra manual steps. Until this release, each Hive import was using a temporary directory with the same name as the importing table stored in the user's home directory. The second notable improvement was to support arbitrary temporary directories when doing a Hive import (SQOOP-483).

I believe that 1.4.2 is a great release with many improvements and interesting new features. All users are highly encouraged to upgrade to this latest release.


I would like to say that this blog really convinced me to do it ! Thanks, very good. We are really happy just for this post in this website.

Posted by acompanhantes sao paulo on August 13, 2013 at 04:01 PM PDT #

Users of apache solutions can get useful information about new features with read of this share that is very helpful. Readers of reviews get help from https://www.bestbritishwriter.com/ here without waste of time.

Posted by Phoebe Glass on August 30, 2018 at 04:26 AM PDT #

Creative Destruction is a sandbox survival mobile game. https://www.creativedestructionpc.com/

Posted by Babbu Mann on October 29, 2018 at 04:37 AM PDT #

<a href="https://www.google.com/">good article</a>

Posted by nis on January 13, 2019 at 10:00 PM PST #


Posted by ni on January 13, 2019 at 10:01 PM PST #

The latest version Apache Sqoop 1.4.2 details are shared in this article. These kinds of articles are much helpful to understand all the new updates of these versions. http://emeraldcutengagmentrings.com/ Here the way of presentation makes easy to understand and thanks for this article.

Posted by Terrin on January 29, 2019 at 01:19 AM PST #

In this article detailly shared about the application version Apache Sqoop 1.4.2. Here highlighted all the key features of this version. http://www.beautifuldiamondsonline.com/ It makes easy for me to understand and keep sharing more informative topics in this article.

Posted by Terrin on January 30, 2019 at 01:26 AM PST #

On the off chance that apache upheld mobile gadgets that would have been incredible. You will locate a tremendous determination of recreations for Android and ios(https://ucngame.com/) telephones in the Google Play Store and App Store, however, it very well may challenge recognize the genuine pearls. Here, we've assembled the absolute best Android recreations that are as of now accessible. Some are free and others cost a couple of bucks, however, every one of them gives an engaging background in a hurry.

Posted by Branden on July 05, 2019 at 12:15 PM PDT #

me encanta! Love the minimalist look and feel.

Posted by store on July 19, 2019 at 03:38 PM PDT #

Bra-vo! Beautiful work and amazing presentation. Congrats!

Posted by woman on July 19, 2019 at 04:22 PM PDT #

great work, especially love the packaging So nice!

Posted by scurmomata1988 on July 20, 2019 at 09:05 AM PDT #

Color palette is surprising calming @Guilherme Ramos Broliato Acho que tu vai curtir esse estilo

Posted by exinimcol1983 on July 20, 2019 at 10:19 AM PDT #

Very interesting effect! Brilliant job!!! Thanks everyone! I read all the comments and I am thankful for all of them. Always such a joy when people like my free time projects I simply do because I love doing them. (Also, I am happy that insects, a rather overlooked subject, are getting more exposure lately).

Posted by elisguedrog1986 on July 20, 2019 at 10:41 AM PDT #

Wow.........Very Nice. Epic use of colors! I love it ;)

Posted by membstepacse1981 on July 20, 2019 at 11:04 AM PDT #

omg!! i'm in love with it!! Wow love the color combination.

Posted by nedigentle1978 on July 20, 2019 at 11:32 AM PDT #

Cool '! Love how everything fits with its style, stunning!

Posted by lelederde1986 on July 20, 2019 at 03:50 PM PDT #


Posted by nonleipresun1974 on July 20, 2019 at 04:55 PM PDT #

Refined and modern in a pure french style. Magnificent ! Truly awesome colors and images

Posted by talosusthirs1983 on July 20, 2019 at 05:44 PM PDT #

发到站酷会炸~ This is sick!

Posted by flapdicaha1983 on July 20, 2019 at 07:09 PM PDT #

love it, nice animation Very powerful message executed extremely well. Amazing!

Posted by poventosig1983 on July 20, 2019 at 10:05 PM PDT #

Great stuff mr. dudinsky dude!! I'm impressed. Nice technique and a poetic view.

Posted by baperpbuchsvol1985 on July 20, 2019 at 10:41 PM PDT #

Your all work is very nice so I am waiting for your next project. These particles are amazing!

Posted by chrischargama1987 on July 20, 2019 at 11:19 PM PDT #

lol, million dollar logo right here) slap a black circle on some text and then make a 2 hour presentation about it)) Wow I love the intricate details, and fun colors.

Posted by ciadinipag1979 on July 20, 2019 at 11:52 PM PDT #

This is unreal! Fantastic!!!!

Posted by laypypatzchu1977 on July 21, 2019 at 12:24 AM PDT #

Nice identity, good work! Muito bons!

Posted by brunceusaco1978 on July 21, 2019 at 12:56 AM PDT #

Cool idea. Nice work! Bardzo fajne!

Posted by imcepwinpli1974 on July 21, 2019 at 01:28 AM PDT #

Thank you, guys! Your support means a lot for me! what a great shapes and ideas in illustrations

Posted by bareretmi1983 on July 21, 2019 at 02:33 AM PDT #

Glitter is gold this time. Futuristic excellence. Love it. Hilarious! Well done!

Posted by counbestbattcho1971 on July 21, 2019 at 03:06 AM PDT #

This is a great work! Great job! The mix between image and illustration is fabulous! :D

Posted by sapactica1989 on July 21, 2019 at 03:39 AM PDT #

nice work!!! Beautiful use of light and shadows!

Posted by smilroamathharp1983 on July 21, 2019 at 04:12 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed



Hot Blogs (today's hits)

Tag Cloud