Apache Sqoop

Monday November 21, 2011

Inaugural Sqoop Meetup

Over 30 people attended the inaugural Sqoop Meetup on the eve of Hadoop World in NYC. Faces were put to names, troubleshooting tips were swapped, and stories were topped - with the table-to-end-all-tables weighing in at 28 billion rows.

I started off the scheduled talks by discussing "Habits of Effective Sqoop Users." One tip to make your next debugging session more effective was to provide more information up front on the mailing list such as versions used and running with the --verbose flag enabled. Also, I pointed out workarounds to common MySQL and Oracle errors.

Next up was Eric Hernandez's "Sqooping 50 Million Rows a Day from MySQL," where he displayed battle scars from creating a single data source for analysts to mine. Key lessons learned were: (1.) Develop an incremental import when sqooping in large active tables. (2.) Limit the amount of parts that data will be stored in HDFS. (3.) Compress data in HDFS.

The final talk of the night was given by Joey Echeverria on "Scratching Your Own Itch." Joey methodically stepped future Sqoop committers through the science from finding a Sqoop bug, filing a jira, coding a patch, submitting it for review, revising accordingly, and finally to ship it '+1' approval.

With the conclusion of the scheduled talks, the hallway talks commenced and went well into the night. Sqoop Committer Aaron Kimball was even rumored to have shed a tear over the healthy turnout and impending momentum barreling towards the next Sqoop Meetup on the Left Coast. See you there!

Aaron Kimball

Kate Ting

Eric Hernandez

Joey Echeverria

Guest post by Kate Ting.
Photos from Masatake Iwasaki and Kate Ting.



Posted by Sara Taylor on August 29, 2018 at 01:17 AM PDT #

It has shared many entries on inagural sqoop and relevant https://www.collegessaywriter.com/coursework/ websites list. Then all the people we could manage their informative resources and essay online book stories.

Posted by Volksw on September 24, 2018 at 04:43 AM PDT #

Sites that offer coupons and deals online are very popular. Running a coupon website involves listing coupons and sales; you make money from paid. https://www.couponmall.net/in/store/flipkart/

Posted by emma roy on April 03, 2019 at 10:01 PM PDT #

Articles that are very useful and very helpful. Also visit my website at https://www.xiaomidev.com/daftar-harga-xiaomi-terbaru/

Posted by Alexander on July 13, 2019 at 09:23 PM PDT #

Keep up the great work ! Милота. Крутой стиль

Posted by Clomid on July 20, 2019 at 08:48 AM PDT #

Aeiforia Architects on of the best interior designing company in India. They are completed lots of very big project. So if you are finding any Interior designing company so you can go with Aeiforia Architects.

Posted by Aeiforia Architects on August 06, 2019 at 10:58 PM PDT #

When would held a meetup on the mobile phone security? I have some new info here https://celltrackingapps.com/hack-cell-phone/ to discuss! Thanks!

Posted by BrunoA on August 23, 2019 at 01:51 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed



Hot Blogs (today's hits)

Tag Cloud