Apache Samza

Wednesday February 22, 2017

Announcing the release of Apache Samza 0.12.0

We are excited to announce that the Apache Samza 0.12.0 has been released.

Samza has been powering real-time applications in production across several large companies (including LinkedIn, Netflix, Uber) for a few years now. Samza provides leading support for large-scale stateful stream processing with features such as:

  • First class support for local state (with RocksDB store). This allows a stateful application to scale up to 1.1 Million events/sec on a single SSD based machine.
  • Support for incremental checkpointing of state instead of full snapshots. This enables Samza to scale to applications with very large state.
  • Minimal impact during application maintenance.
In addition to general stream processing capabilities, Samza also supports:
  • A fully pluggable model for input sources (e.g. Kafka, Kinesis, DynamoDB streams etc.) and outputs (HDFS, Kafka, ElastiCache etc.). This allows applications to directly process data from various event sources without mandating that the data should be moved into Kafka.
  • A fully async programming model. This allows applications that make remote calls to increase parallelism very efficiently.
  • Features like canaries, upgrades and rollbacks that support extremely large deployments.
This 0.12.0 release adds several features to Samza to improve stability, performance and ease of use. Here are some highlights of this release.

Convergence of Batch and Real-time processing in Samza:
End of Stream support: Samza has always supported streaming input sources like Kafka. In such sources, it is assumed that the incoming stream of data is infinite. Samza will now have an ‘end-of-stream’ notion to support consuming from input sources that are finite (for example, on-disk files). This enables the Samza job to shut-down gracefully when it has finished consuming all data.

HDFS Consumer: Samza now provides first-class support for consuming data from HDFS files. This enables developers to define their processing logic once, and run it in both batch and streaming environments. This feature also allows for rapid experimentation with ETL’d HDFS data using Samza without the need to write a separate Hadoop job. (SAMZA-967)

Checkpoint Notifications:
Samza can now notify the SystemConsumer when performing a checkpoint. This can enable Samza to support consumers such as: Amazon Kinesis, Amazon SQS, Azure ServiceBus Queues/Topics, Google Cloud Pub-Sub, ActiveMQ, etc., which each manage checkpointing on their own. This also enables consumers to implement smart retention policies (such as deleting data once it has been consumed). (SAMZA-1042)

Support for Yarn Node Labels:
Often Samza YARN clusters have machines that are not homogenous. For example, nodes could have different memory hardware, CPUs, spinning disks or SSDs. With this feature, users can assign “labels” to nodes in their YARN cluster and use them to specify the where their Samza job should run. This feature allows flexibility in scheduling jobs based on trade-offs in resource requirements, performance and hardware costs. For example, stateful jobs can be configured to run on nodes with SSDs while stateless jobs can be configured to run on nodes with spinning disks. (SAMZA-1013)

Bug fixes:
This release also includes several critical bug-fixes and improvements for operational stability. Some notable ones include:
  • HttpFileSystem timeout for blocking reads when localizing containers (SAMZA-1079).
  • SamzaContainer should catch all Throwables instead of only exceptions (SAMZA-1077).
  • Deadlock between KafkaSystemProducer and KafkaProducer from kafka-clients lib (SAMZA-1069).
  • Change the commit order to support at least once processing when deduping with local store (SAMZA-1065).
Upgrades:
  • Upgraded Kafka version to 0.10. This enables us to take advantage of the critical fixes and improvements in Kafka.
  • Upgraded to Jetty 9 from Jetty 8.
  • Full support for Scala 2.11. All Samza jars will now have the scala version as 2.11 as a part of their file name. For example, samza-yarn_2.11-0.12.jar.
  • Samza is now source compatible with JDK 8 and above. Older JDKs are no longer supported.
Community Developments:
We made great community progress since the last release. We had two successful meetups where we presented Samza’s roadmap, and how Optimizely uses Samza. Several Samza use-cases in Uber and LinkedIn were featured in QCon 2016. Future:
There are a lot of exciting features to expect in our future release. Here are some highlights:
  • Support for Disk quota enforcement and throttling (SAMZA-956)
  • Support for high-level programming API for stream processing (SAMZA-1073)
  • Support for running Samza in stand-alone mode (SAMZA-516)
It’s a great time to get involved. You can start by reviewing the hello-samza tutorial, signing up for the mailing list, and grabbing some newbie JIRAs. I'd like to close by thanking everyone who's been involved in the project. It's been a great experience to be involved in this community, and I look forward to its continued growth.

Comments:

Awesome update. Loved it.

Posted by Appeven on December 04, 2017 at 10:08 AM GMT #

It is very important to me. Thanks for the information you shared. Awaiting for more posts like this.!

Posted by clickerheroes.co on December 07, 2017 at 07:19 AM GMT #

I Genuinely Appreciated Understanding It. Sitting Tight For Some More Incredible Articles Like This From You In The Nearing Days

Posted by 写作服务 on September 17, 2018 at 07:18 AM GMT #

Here you will be glad to play free online spider solitaire games as these games are one of the best gaming website http://onlinesolitaire.wikidot.com/about which will help you to enjoy these games in the best quality and will also let you remove your all stress.

Posted by Hard Work on September 24, 2018 at 07:32 AM GMT #

Awesome update. Loved it.

Posted by Wortlo on January 28, 2019 at 01:16 PM GMT #

As far as I believe, this is a very nice update just like fmwhatsapp. It is now one of the finest whatsapp mods ever in the history since it has thousands of features already and few more to be added very soon. Download it from https://fmwhatsappapk.com/download/

Posted by fmwhatsapp on April 01, 2019 at 04:13 AM GMT #

thank you so much

Posted by iphongthuy net on April 03, 2019 at 01:54 AM GMT #

It is very important to me. Thanks for the information you shared. Awaiting for more posts like this.!

Posted by Malveen on April 19, 2019 at 03:46 AM GMT #

It is very important to me. Thanks for the information you shared. Awaiting for more posts like this.!

Posted by www.megadede.ws on May 09, 2019 at 12:33 PM GMT #

Gue bisa konfirmasi nama2 itu sering muncul di transfertalk forum luar Termasuk juga Mahrez yang sekarang sudah melayang ke kleb lain Pilihan makin terbatas, makanya MU harus gerak cepet sebelum incerannya diserobot kleb lain http://nobartv.net/ http://www.elangnews.com/ http://mikangift.com/ https://www.bolagpsport.com/ http://nontontvbola.space/

Posted by siaran langsung bola on May 10, 2019 at 06:46 AM GMT #

E and send you the best of luck to all those years of this message and it's down and send you a p please don't hesitate in advance and best wishes and thanks again for the first one to do the Rom to be able and I have a good time for you and I have been trying and trying and I have been a long term care I will send the money for this calendar to share it with the new account approved for you and the other side I have been a long term care executive assistant at last year's event bloggin to be in a p https://www.trickywricky.com/ Hd wallpapers HD video recording of ROM visit to share this with a href to the same and revert to us in USD to the Rom to share this with a span and the other hand

Posted by Ram on May 26, 2019 at 06:50 AM GMT #

Sj I will pay the following link to download the latest in USD u I will upload tomorrow and the Rom I am going down today I am just wondering I have to do the same to share Universal theme of ROM visit the following URL to the Rom to share with me to get you for the first one I have to go with a good time and the other side of this.<a href="https://www.trickywricky.com/netflix-mod-apk-download-100-works/">netflix mod apk</a>

Posted by Adminbro on May 30, 2019 at 11:11 AM GMT #

The article helped me a lot. Thank you

Posted by avinahome on June 03, 2019 at 06:37 AM GMT #

Its very true and I just not bad to walk on the day, dear good if https://allenstanford.id will be a good site please back to the top. thanks for thats information and i will be share post in my social media and https://carasadap.com

Posted by carasadap.com on June 05, 2019 at 03:56 PM GMT #

Awesome update. Loved it, thanks

Posted by Streamog on June 14, 2019 at 12:31 AM GMT #

While there are free ways to use the internet for entertainment, there are other ways that require you to pay a small fee.

Posted by megadede.com on June 25, 2019 at 07:29 PM GMT #

So these are ideas for garage or basement tinkerers. Coming up with a prototype for most of these will cost no more than the money in your pocket.

Posted by filmy online on July 03, 2019 at 06:12 PM GMT #

Thanks, i like it

Posted by avinahome.com on July 09, 2019 at 01:57 AM GMT #

Increible trabajo ! keep going always cool

Posted by montrecondocs1988 on July 20, 2019 at 04:47 PM GMT #

If you got the time check out some of my stuff - would be very appreciated. Interesting object, great shots!

Posted by baisuscege1980 on July 20, 2019 at 05:29 PM GMT #

@Irakli Gharibashvili Gharibashvili awesome project!

Posted by manrynone1983 on July 20, 2019 at 05:50 PM GMT #

love these! great work Wooow!!! Wonderful characters !! Well done

Posted by umennalna1980 on July 20, 2019 at 11:29 PM GMT #

Nice colors This is so creative and unique! Absolutely adore this!

Posted by linkdowndolen1974 on July 21, 2019 at 12:19 AM GMT #

Suuuuper nice! :) Great work! I'm art intern journalist from CURATED London. With our journalist's recommendation, you will get submission fee waiver for“TCP Curators & Editors' Choice Awards” . If you want,you can contact our editorial department. @CURATED: Magazine EDITOR

Posted by rentayprepres1986 on July 21, 2019 at 02:57 AM GMT #

Excellent design. Looks really professional. Which execution is your personal favorite ?

Posted by peepevabo1986 on July 21, 2019 at 04:49 AM GMT #

As always the devil is in details! Amazing work Ash! Wow these are great! Love your style!

Posted by chaitxakmuestar1984 on July 21, 2019 at 05:21 AM GMT #

Classic and beautiful. Always on top! Wow! Look at these Colors...

Posted by grounuswomea1972 on July 21, 2019 at 06:01 AM GMT #

Amazing use of color! Incrivel ❤️

Posted by outapcurla1974 on July 21, 2019 at 06:34 AM GMT #

GOD Thanks!! @Konstantin Tischenko @zhou pengfei @Elbord Design

Posted by cuscatihyb1980 on July 21, 2019 at 08:11 AM GMT #

... Simply Amazing!

Posted by phanvilide1977 on July 21, 2019 at 08:43 AM GMT #

Cheers, Chris Looks cool for me. Good job :)

Posted by niesingcehard1978 on July 21, 2019 at 09:16 AM GMT #

its very simple, but very well done, the yellow color adds a strong visual identity. I I agree that the logo looks like one. pl, the identity that unfolds from this main concept maybe dont look unique, but very cohesive when we think about bank, and also I like de flat/minimal vibes this visual identity has ;) Your art has such beautiful flowing lines.

Posted by aruninpheo1972 on July 21, 2019 at 09:50 AM GMT #

可以说,很震撼了。 Nice personalities.

Posted by precdargitua1985 on July 21, 2019 at 10:22 AM GMT #

In love with the vibrant orange!!!! great work what a great shapes and ideas in illustrations

Posted by fobulpeten1986 on July 21, 2019 at 10:55 AM GMT #

thanks https://www.trickywricky.com/netflix-mod-apk/ this is <a href="https://www.trickywricky.com/netflix-mod-apk/">netflix mod apk</a>

Posted by yo yo on July 25, 2019 at 10:29 AM GMT #

nice https://www.trickywricky.com/

Posted by bravo on July 25, 2019 at 01:07 PM GMT #

Thanks https://aptoideapkdownload.co/aptoide-minecraft-free/

Posted by aptoide ios on August 09, 2019 at 04:54 AM GMT #

I think this is the best release of apache after the previous release. I am using apache for my server for my live nettv app

Posted by Livenettv APK on August 29, 2019 at 03:49 AM GMT #

A distributed stream processing framework built upon Apache Kafka and Apache Hadoop YARN. I'am very to see this article. The selection of your article is too good. Which is very interesting article and nice explanation on topic. You're doing great work. see: https://morningcredit.com/under-the-table-jobs/

Posted by mithvan on September 03, 2019 at 04:11 AM GMT #

hi thnx <a href="https://www.trickywricky.com/droid-vpn-setting-for-airtel-latest-2019/">thanks for sharing</a>

Posted by ramko on September 03, 2019 at 01:06 PM GMT #

<a href="https://www.trickywricky.com/droid-vpn-setting-for-airtel-latest-2019/">thanks for sharing</a>

Posted by droid vpn setting for airtel on September 03, 2019 at 01:07 PM GMT #

Thanks https://www.howelldumpsterrentals.com

Posted by howell dumpsters on September 03, 2019 at 04:20 PM GMT #

Good my web on http://www.folderhp.com/

Posted by FolderHP on September 27, 2019 at 03:06 AM GMT #

wanted to post a brief comment to be able to say thanks to you for some of the amazing tips and hints you are giving here.

Posted by love balls pc on September 30, 2019 at 07:43 AM GMT #

This article helps me a lot. Thank you

Posted by hoc nghe spa on October 01, 2019 at 01:21 AM GMT #

Nice article. Very informative info. Thanks for sharing.

Posted by VideoWatch Vidmate App on October 04, 2019 at 04:23 AM GMT #

Apache Samza is still amazing as a stream processing framework.

Posted by cgarchitects on October 09, 2019 at 04:31 AM GMT #

Nice article thank you for sharing also check <a href="https://spotifygeek.com/spotify-premium-apk/">https://spotifygeek.com/spotify-premium-apk/</a>

Posted by Spotify Geek on October 09, 2019 at 07:57 AM GMT #

Download Spotify Premium APK: https://spotifygeek.com/spotify-premium-apk/

Posted by Spotify Geek on October 09, 2019 at 07:58 AM GMT #

Giải mã giấc mơ lô đề, sổ mơ lô đề đầy đủ và chính xác nhất

Posted by giai ma giac mo on October 10, 2019 at 02:13 PM GMT #

I have гead so many posts about the blogger lovers however this parɑgraph is really a nice

Posted by raybaby.us on October 17, 2019 at 11:29 AM GMT #

It is very important to me. Thanks for the information you shared. Awaiting for more posts like this.!

Posted by mejor torrent on October 19, 2019 at 09:54 PM GMT #

Post a Comment:
  • HTML Syntax: NOT allowed

Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation