Apache Samza

Wednesday February 22, 2017

Announcing the release of Apache Samza 0.12.0

We are excited to announce that the Apache Samza 0.12.0 has been released.

Samza has been powering real-time applications in production across several large companies (including LinkedIn, Netflix, Uber) for a few years now. Samza provides leading support for large-scale stateful stream processing with features such as:

  • First class support for local state (with RocksDB store). This allows a stateful application to scale up to 1.1 Million events/sec on a single SSD based machine.
  • Support for incremental checkpointing of state instead of full snapshots. This enables Samza to scale to applications with very large state.
  • Minimal impact during application maintenance.
In addition to general stream processing capabilities, Samza also supports:
  • A fully pluggable model for input sources (e.g. Kafka, Kinesis, DynamoDB streams etc.) and outputs (HDFS, Kafka, ElastiCache etc.). This allows applications to directly process data from various event sources without mandating that the data should be moved into Kafka.
  • A fully async programming model. This allows applications that make remote calls to increase parallelism very efficiently.
  • Features like canaries, upgrades and rollbacks that support extremely large deployments.
This 0.12.0 release adds several features to Samza to improve stability, performance and ease of use. Here are some highlights of this release.

Convergence of Batch and Real-time processing in Samza:
End of Stream support: Samza has always supported streaming input sources like Kafka. In such sources, it is assumed that the incoming stream of data is infinite. Samza will now have an ‘end-of-stream’ notion to support consuming from input sources that are finite (for example, on-disk files). This enables the Samza job to shut-down gracefully when it has finished consuming all data.

HDFS Consumer: Samza now provides first-class support for consuming data from HDFS files. This enables developers to define their processing logic once, and run it in both batch and streaming environments. This feature also allows for rapid experimentation with ETL’d HDFS data using Samza without the need to write a separate Hadoop job. (SAMZA-967)

Checkpoint Notifications:
Samza can now notify the SystemConsumer when performing a checkpoint. This can enable Samza to support consumers such as: Amazon Kinesis, Amazon SQS, Azure ServiceBus Queues/Topics, Google Cloud Pub-Sub, ActiveMQ, etc., which each manage checkpointing on their own. This also enables consumers to implement smart retention policies (such as deleting data once it has been consumed). (SAMZA-1042)

Support for Yarn Node Labels:
Often Samza YARN clusters have machines that are not homogenous. For example, nodes could have different memory hardware, CPUs, spinning disks or SSDs. With this feature, users can assign “labels” to nodes in their YARN cluster and use them to specify the where their Samza job should run. This feature allows flexibility in scheduling jobs based on trade-offs in resource requirements, performance and hardware costs. For example, stateful jobs can be configured to run on nodes with SSDs while stateless jobs can be configured to run on nodes with spinning disks. (SAMZA-1013)

Bug fixes:
This release also includes several critical bug-fixes and improvements for operational stability. Some notable ones include:
  • HttpFileSystem timeout for blocking reads when localizing containers (SAMZA-1079).
  • SamzaContainer should catch all Throwables instead of only exceptions (SAMZA-1077).
  • Deadlock between KafkaSystemProducer and KafkaProducer from kafka-clients lib (SAMZA-1069).
  • Change the commit order to support at least once processing when deduping with local store (SAMZA-1065).
Upgrades:
  • Upgraded Kafka version to 0.10. This enables us to take advantage of the critical fixes and improvements in Kafka.
  • Upgraded to Jetty 9 from Jetty 8.
  • Full support for Scala 2.11. All Samza jars will now have the scala version as 2.11 as a part of their file name. For example, samza-yarn_2.11-0.12.jar.
  • Samza is now source compatible with JDK 8 and above. Older JDKs are no longer supported.
Community Developments:
We made great community progress since the last release. We had two successful meetups where we presented Samza’s roadmap, and how Optimizely uses Samza. Several Samza use-cases in Uber and LinkedIn were featured in QCon 2016. Future:
There are a lot of exciting features to expect in our future release. Here are some highlights:
  • Support for Disk quota enforcement and throttling (SAMZA-956)
  • Support for high-level programming API for stream processing (SAMZA-1073)
  • Support for running Samza in stand-alone mode (SAMZA-516)
It’s a great time to get involved. You can start by reviewing the hello-samza tutorial, signing up for the mailing list, and grabbing some newbie JIRAs. I'd like to close by thanking everyone who's been involved in the project. It's been a great experience to be involved in this community, and I look forward to its continued growth.

Comments:

Awesome update. Loved it.

Posted by Appeven on December 04, 2017 at 10:08 AM GMT #

It is very important to me. Thanks for the information you shared. Awaiting for more posts like this.!

Posted by clickerheroes.co on December 07, 2017 at 07:19 AM GMT #

I Genuinely Appreciated Understanding It. Sitting Tight For Some More Incredible Articles Like This From You In The Nearing Days

Posted by 写作服务 on September 17, 2018 at 07:18 AM GMT #

Here you will be glad to play free online spider solitaire games as these games are one of the best gaming website http://onlinesolitaire.wikidot.com/about which will help you to enjoy these games in the best quality and will also let you remove your all stress.

Posted by Hard Work on September 24, 2018 at 07:32 AM GMT #

Awesome update. Loved it.

Posted by Wortlo on January 28, 2019 at 01:16 PM GMT #

As far as I believe, this is a very nice update just like fmwhatsapp. It is now one of the finest whatsapp mods ever in the history since it has thousands of features already and few more to be added very soon. Download it from https://fmwhatsappapk.com/download/

Posted by fmwhatsapp on April 01, 2019 at 04:13 AM GMT #

thank you so much

Posted by iphongthuy net on April 03, 2019 at 01:54 AM GMT #

It is very important to me. Thanks for the information you shared. Awaiting for more posts like this.!

Posted by Malveen on April 19, 2019 at 03:46 AM GMT #

It is very important to me. Thanks for the information you shared. Awaiting for more posts like this.!

Posted by www.megadede.ws on May 09, 2019 at 12:33 PM GMT #

Gue bisa konfirmasi nama2 itu sering muncul di transfertalk forum luar Termasuk juga Mahrez yang sekarang sudah melayang ke kleb lain Pilihan makin terbatas, makanya MU harus gerak cepet sebelum incerannya diserobot kleb lain http://nobartv.net/ http://www.elangnews.com/ http://mikangift.com/ https://www.bolagpsport.com/ http://nontontvbola.space/

Posted by siaran langsung bola on May 10, 2019 at 06:46 AM GMT #

E and send you the best of luck to all those years of this message and it's down and send you a p please don't hesitate in advance and best wishes and thanks again for the first one to do the Rom to be able and I have a good time for you and I have been trying and trying and I have been a long term care I will send the money for this calendar to share it with the new account approved for you and the other side I have been a long term care executive assistant at last year's event bloggin to be in a p https://www.trickywricky.com/ Hd wallpapers HD video recording of ROM visit to share this with a href to the same and revert to us in USD to the Rom to share this with a span and the other hand

Posted by Ram on May 26, 2019 at 06:50 AM GMT #

Sj I will pay the following link to download the latest in USD u I will upload tomorrow and the Rom I am going down today I am just wondering I have to do the same to share Universal theme of ROM visit the following URL to the Rom to share with me to get you for the first one I have to go with a good time and the other side of this.<a href="https://www.trickywricky.com/netflix-mod-apk-download-100-works/">netflix mod apk</a>

Posted by Adminbro on May 30, 2019 at 11:11 AM GMT #

The article helped me a lot. Thank you

Posted by avinahome on June 03, 2019 at 06:37 AM GMT #

Its very true and I just not bad to walk on the day, dear good if https://allenstanford.id will be a good site please back to the top. thanks for thats information and i will be share post in my social media and https://carasadap.com

Posted by carasadap.com on June 05, 2019 at 03:56 PM GMT #

Awesome update. Loved it, thanks

Posted by Streamog on June 14, 2019 at 12:31 AM GMT #

While there are free ways to use the internet for entertainment, there are other ways that require you to pay a small fee.

Posted by megadede.com on June 25, 2019 at 07:29 PM GMT #

So these are ideas for garage or basement tinkerers. Coming up with a prototype for most of these will cost no more than the money in your pocket.

Posted by filmy online on July 03, 2019 at 06:12 PM GMT #

Thanks, i like it

Posted by avinahome.com on July 09, 2019 at 01:57 AM GMT #

Increible trabajo ! keep going always cool

Posted by montrecondocs1988 on July 20, 2019 at 04:47 PM GMT #

If you got the time check out some of my stuff - would be very appreciated. Interesting object, great shots!

Posted by baisuscege1980 on July 20, 2019 at 05:29 PM GMT #

@Irakli Gharibashvili Gharibashvili awesome project!

Posted by manrynone1983 on July 20, 2019 at 05:50 PM GMT #

love these! great work Wooow!!! Wonderful characters !! Well done

Posted by umennalna1980 on July 20, 2019 at 11:29 PM GMT #

Nice colors This is so creative and unique! Absolutely adore this!

Posted by linkdowndolen1974 on July 21, 2019 at 12:19 AM GMT #

Suuuuper nice! :) Great work! I'm art intern journalist from CURATED London. With our journalist's recommendation, you will get submission fee waiver for“TCP Curators & Editors' Choice Awards” . If you want,you can contact our editorial department. @CURATED: Magazine EDITOR

Posted by rentayprepres1986 on July 21, 2019 at 02:57 AM GMT #

Excellent design. Looks really professional. Which execution is your personal favorite ?

Posted by peepevabo1986 on July 21, 2019 at 04:49 AM GMT #

As always the devil is in details! Amazing work Ash! Wow these are great! Love your style!

Posted by chaitxakmuestar1984 on July 21, 2019 at 05:21 AM GMT #

Classic and beautiful. Always on top! Wow! Look at these Colors...

Posted by grounuswomea1972 on July 21, 2019 at 06:01 AM GMT #

Amazing use of color! Incrivel ❤️

Posted by outapcurla1974 on July 21, 2019 at 06:34 AM GMT #

GOD Thanks!! @Konstantin Tischenko @zhou pengfei @Elbord Design

Posted by cuscatihyb1980 on July 21, 2019 at 08:11 AM GMT #

... Simply Amazing!

Posted by phanvilide1977 on July 21, 2019 at 08:43 AM GMT #

Cheers, Chris Looks cool for me. Good job :)

Posted by niesingcehard1978 on July 21, 2019 at 09:16 AM GMT #

its very simple, but very well done, the yellow color adds a strong visual identity. I I agree that the logo looks like one. pl, the identity that unfolds from this main concept maybe dont look unique, but very cohesive when we think about bank, and also I like de flat/minimal vibes this visual identity has ;) Your art has such beautiful flowing lines.

Posted by aruninpheo1972 on July 21, 2019 at 09:50 AM GMT #

可以说,很震撼了。 Nice personalities.

Posted by precdargitua1985 on July 21, 2019 at 10:22 AM GMT #

In love with the vibrant orange!!!! great work what a great shapes and ideas in illustrations

Posted by fobulpeten1986 on July 21, 2019 at 10:55 AM GMT #

thanks https://www.trickywricky.com/netflix-mod-apk/ this is <a href="https://www.trickywricky.com/netflix-mod-apk/">netflix mod apk</a>

Posted by yo yo on July 25, 2019 at 10:29 AM GMT #

nice https://www.trickywricky.com/

Posted by bravo on July 25, 2019 at 01:07 PM GMT #

Thanks https://aptoideapkdownload.co/aptoide-minecraft-free/

Posted by aptoide ios on August 09, 2019 at 04:54 AM GMT #

I think this is the best release of apache after the previous release. I am using apache for my server for my live nettv app

Posted by Livenettv APK on August 29, 2019 at 03:49 AM GMT #

A distributed stream processing framework built upon Apache Kafka and Apache Hadoop YARN. I'am very to see this article. The selection of your article is too good. Which is very interesting article and nice explanation on topic. You're doing great work. see: https://morningcredit.com/under-the-table-jobs/

Posted by mithvan on September 03, 2019 at 04:11 AM GMT #

hi thnx <a href="https://www.trickywricky.com/droid-vpn-setting-for-airtel-latest-2019/">thanks for sharing</a>

Posted by ramko on September 03, 2019 at 01:06 PM GMT #

<a href="https://www.trickywricky.com/droid-vpn-setting-for-airtel-latest-2019/">thanks for sharing</a>

Posted by droid vpn setting for airtel on September 03, 2019 at 01:07 PM GMT #

Thanks https://www.howelldumpsterrentals.com

Posted by howell dumpsters on September 03, 2019 at 04:20 PM GMT #

Good my web on http://www.folderhp.com/

Posted by FolderHP on September 27, 2019 at 03:06 AM GMT #

wanted to post a brief comment to be able to say thanks to you for some of the amazing tips and hints you are giving here.

Posted by love balls pc on September 30, 2019 at 07:43 AM GMT #

This article helps me a lot. Thank you

Posted by hoc nghe spa on October 01, 2019 at 01:21 AM GMT #

Nice article. Very informative info. Thanks for sharing.

Posted by VideoWatch Vidmate App on October 04, 2019 at 04:23 AM GMT #

Apache Samza is still amazing as a stream processing framework.

Posted by cgarchitects on October 09, 2019 at 04:31 AM GMT #

Nice article thank you for sharing also check <a href="https://spotifygeek.com/spotify-premium-apk/">https://spotifygeek.com/spotify-premium-apk/</a>

Posted by Spotify Geek on October 09, 2019 at 07:57 AM GMT #

Download Spotify Premium APK: https://spotifygeek.com/spotify-premium-apk/

Posted by Spotify Geek on October 09, 2019 at 07:58 AM GMT #

Giải mã giấc mơ lô đề, sổ mơ lô đề đầy đủ và chính xác nhất

Posted by giai ma giac mo on October 10, 2019 at 02:13 PM GMT #

I have гead so many posts about the blogger lovers however this parɑgraph is really a nice

Posted by raybaby.us on October 17, 2019 at 11:29 AM GMT #

It is very important to me. Thanks for the information you shared. Awaiting for more posts like this.!

Posted by mejor torrent on October 19, 2019 at 09:54 PM GMT #

Thank you for sharing a very informative information.

Posted by Hút hầm cầu tại Đà Nẵng Khánh Trung on October 22, 2019 at 03:53 AM GMT #

This is really good news for who are using apche...

Posted by free netflix accounts on October 29, 2019 at 01:35 PM GMT #

A very informative article, I have read and feel that this is the knowledge that needs to be supplemented

Posted by Xem them on November 09, 2019 at 03:57 AM GMT #

Kunjungi Wisata Candi Pringapus Hindu https://www.besoksukses.com/wisata-candi-pringapus-hindu-di-temanggung/

Posted by BesokSukses on November 12, 2019 at 02:21 PM GMT #

Jasa upgrade Spotify premium terpercaya hanya di https://spotify.web.id/

Posted by Alex on November 12, 2019 at 02:57 PM GMT #

Cung cấp thi công rèm cửa chuyên nghiệp tại TPCHM

Posted by Remcuavilla on November 18, 2019 at 03:44 AM GMT #

Providing beautiful quality curtains in Ho Chi Minh City

Posted by Rèm cửa Avinahome on November 18, 2019 at 03:46 AM GMT #

Nice article

Posted by Abhishek8055 on November 19, 2019 at 02:06 PM GMT #

As far as I believe, this is a very nice update just like fmwhatsapp. It is now one of the finest whatsapp mods ever in the history since it has thousands of features already and few more to be added very soon. Download it from https://gbplus.online/fm-whatsapp-latest-version/

Posted by Abhishek8055 on November 19, 2019 at 02:08 PM GMT #

Music is one of a good way to reduce your stress. By listening to music, your brain will be only focusing to the sound of the music you are listening. Thus, that makes your brain forgot every other life problem. Note that this might be temporary. Visit: https://spotify.web.id/

Posted by jual spotify premium on November 24, 2019 at 01:29 PM GMT #

Our <em>Pinoy Tambayan</em> network offers a variety of fun and unique <a href="https://pinoy1tvako.su/"><strong>Pinoy1Tv</strong></a> shows that make each and every pinoy channel be addicted to watching TV shows

Posted by pinoy1tv on November 26, 2019 at 06:31 PM GMT #

A very useful information

Posted by Cong Ty Dien Lanh Khanh Trung on November 28, 2019 at 10:03 AM GMT #

Thanks for your sharing. It was very helpful to me. Please visit my website. It will be very helpful to you.

Posted by GGmedia on December 05, 2019 at 03:24 AM GMT #

one of the best information about apache, though i am having a hard time understanding it since these are a programming language but its really a good writing from https://www.1moviesearch.com/

Posted by Film enligne on December 16, 2019 at 07:48 AM GMT #

J'adore le lire, j'attends de nouvelles mises à jour et j'ai déjà lu votre message récent, c'est un grand merci.

Posted by Mega-dede on December 28, 2019 at 07:06 PM GMT #

Teknologi digital 2020

Posted by Flipstart PC on December 29, 2019 at 03:34 PM GMT #

Media dakwah islam

Posted by Islamiyyah on January 02, 2020 at 05:56 AM GMT #

Apache is an open-source and free web server software that powers around 46% of websites around the world. The official name is Apache HTTP Server, and it's maintained and developed by the Apache Software Foundation. It allows website owners to serve content on the web — hence the name “web server”

Posted by Kroger GreatPeople on January 04, 2020 at 06:26 AM GMT #

this is really a great article for me, many thanks sir for this info

Posted by Megadede on January 05, 2020 at 04:07 AM GMT #

best article, thanks for this information

Posted by Megadede on January 05, 2020 at 04:08 AM GMT #

Hi there, I have read your article. Nice post...

Posted by Emma on January 13, 2020 at 07:33 AM GMT #

Thanks for information, I like reading in this blog. So far, this blog apache very good

Posted by Droila on February 04, 2020 at 06:35 AM GMT #

Thanks for information, I like reading in this blog. I hope this blog more developed again.

Posted by Belajar Pemrograman Website on February 04, 2020 at 06:37 AM GMT #

Thanks for information This is Informtion from this blog very good.

Posted by Droila on February 04, 2020 at 06:38 AM GMT #

Thanks for information, I like reading in this blog. So far, this blog apache very good

Posted by Metrolagu on February 11, 2020 at 01:06 AM GMT #

J'adore le lire, j'attends de nouvelles mises à jour et j'ai déjà lu votre message récent, c'est un grand merci.

Posted by چارتر تهران به زاهدان on February 16, 2020 at 06:03 PM GMT #

J'adore le lire, j'attends de nouvelles mises à jour et j'ai déjà lu votre message récent, c'est un grand merci.

Posted by بلیط چارتر تهران به عسلویه on February 16, 2020 at 06:03 PM GMT #

Cara reset iphone ke pengaturan pabrik atau factory reset sangat mudah https://www.omevan.com/2020/02/cara-reset-iphone-ke-pengaturan-awal.html http://erepana.blogspot.com

Posted by Erepana on February 17, 2020 at 01:04 PM GMT #

thanks for information. read also Game RPG Android Terbaik https://okmantep.com/game/game-rpg-android-terbaik/

Posted by Game RPG Android Terbaik on February 18, 2020 at 07:55 PM GMT #

Thanks im use is Apache https://www.dailymotion.com/video/x7rwn4h

Posted by shamuschayton on February 20, 2020 at 04:53 PM GMT #

Post a Comment:
Comments are closed for this entry.

Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation