Apache MXNet (incubating)

Monday September 17, 2018

Announcing Apache MXNet (incubating) 1.3.0 Release

Today the Apache MXNet community is pleased to announce the 1.3 release of the Apache MXNet deep learning framework. We would like to thank the Apache MXNet community for all their valuable contributions towards the MXNet 1.3 release.

With this release, MXNet has Gluon package enhancements, ONNX export, experimental Clojure bindings, TensorRT integration, and many more features, enhancements and usability improvements! In this blog post, we briefly summarize some of the high-level features and improvements. For a comprehensive list of major features and bug fixes, read the Apache MXNet 1.3.0 release notes.

mxnet-1.3.0.png

Gluon package enhancements

Gluon RNN layers are now hybridizable: With this feature, Gluon RNN layers such as gluon.rnn.RNN, gluon.rnn.LSTM and gluon.rnn.GRU can be converted to HybridBlocks. Now, many dynamic networks that are based on Gluon RNN layers can be completely hybridized, exported and used in the inference APIs in other language bindings such as C/C++, Scala, R, etc.

Support for sparse tensor: Gluon HybridBlocks now support hybridization with sparse operators. To enable sparse gradients in gluon.nn.Embedding, simply set sparse_grad=True. Furthermore, gluon.contrib.nn.SparseEmbedding provides an example of leveraging sparse parameters to reduce communication cost and memory consumption for multi-GPU training with large embeddings.

Support for Synchronized Cross-GPU Batch Norm: Gluon now supports Synchronized Batch Normalization, available as gluon.contrib.nn.SyncBatchNorm. This enables stable training on large-scale networks with high memory consumption such as FCN for image segmentation.

Updated Gluon model zoo: Gluon Vision Model Zoo now provides MobileNetV2 pre-trained models. Updated existing pre-trained models to provide state-of-the-art performance on all ResNet v1, ResNet v2, and vgg16, vgg19, vgg16_bn, vgg19_bn models.

Introducing new Clojure bindings with MXNet

MXNet now has experimental support for the Clojure programming language. The MXNet Clojure package brings state-of-the-art deep learning to the Clojure community. It enables Clojure developers to code and to execute tensor computation on multiple CPUs or GPUs. It also enables users to write seamless tensor/matrix computations with multiple GPUs in Clojure. Now users can construct and customize state-of-art deep learning models in Clojure, and apply them to tasks such as image classification and data science challenges. To start using Clojure package in MXNet, check out the Clojure tutorials and Clojure API documentation.

Introducing control flow operators

This is the first step towards optimizing dynamic neural networks with variable computation graphs. This release adds symbolic and imperative control flow operators such as foreach, while_loop and cond. To learn more about how to use these operators, check out the Control Flow Operators tutorial.

Performance improvements

TensorRT runtime integration: TensorRT provides significant acceleration of model inference on NVIDIA GPUs compared to running the full graph in MXNet using unfused GPU operators. In addition to faster fp32 inference, TensorRT optimizes fp16 inference and is capable of int8 inference (provided the quantization steps are performed). Besides increasing throughput, TensorRT significantly reduces inference latency, especially for small batches. With 1.3 release, MXNet introduces the runtime integration of TensorRT (experimental), in order to accelerate inference. Follow the MXNet-TensorRT article on the MXNet developer wiki to learn more about how to use this feature.

MKL-DNN enhancements: MKL-DNN is an open source library from Intel that contains a set of CPU-optimized deep learning operators. In the previous release, MXNet introduced integration with MKL-DNN to accelerate training and inference execution on CPU. With 1.3 release, we have increased support for these activation functions: sigmoid, tanh and softrelu.

ONNX export support

Export MXNet models to ONNX format: MXNet 1.2 provided users a way to import ONNX models into MXNet for inference. More details are available in this ONNX blog post. With the latest 1.3 release, users can now export MXNet models into ONNX format and import those models into other deep learning frameworks for inference! Check out the MXNet to ONNX exporter tutorial to learn more about how to use the mxnet.contrib.onnx API.

Other experimental features

  1. Apart from what we have covered above, MXNet now has support for:

  2. A new memory pool type for GPU memory which is more suitable for all the workloads with dynamic-shape inputs and outputs. Set an environment variable as MXNET_GPU_MEM_POOL_TYPE=Round to enable this feature. Topology-aware Allreduce approach for single-machine GPU training. Train up to 6.6x and 5.9x faster on AlexNet and VGG compared to MXNet 1.2. Activate this feature using the “control the data communication” environmental variables.

  3. Improved Scala APIs that focus on providing type safety and a better user experience. Symbol.api and NDArray.api bring a new set of functions that have a complete signature. The documentation for all of the arguments also integrates directly with IntelliJ IDEA. The new and improved Scala examples demonstrate usage of these new APIs.

Check out further details on these features in full release notes.

Maintenance improvements

In addition to adding and extending new functionalities, the release also focusses on stability and refinements.

The community fixed 130 unstable tests improving MXNet’s stability and reliability. The MXNet Model Backwards Compatibility Checker was introduced. This is an automated test on MXNet’s continuous integration platform that verifies saved models’ backward compatibility. This helps ensure that models created with older versions of MXNet can be loaded and used with the newer versions.

Getting started with MXNet

Getting started with MXNet is simple, visit the install page to get started. PyPI packages are available to install for Windows, Linux, and Mac.

To learn more about MXNet Gluon package and deep learning, you can follow our 60-minute crash course, and then later complete this comprehensive set of tutorials, which covers everything from an introduction to deep learning to how to implement cutting-edge neural network models. You can also check out lots of material on MXNet tutorials, MXNet blog posts (中文), MXNet YouTube channel (中文). Have fun with MXNet 1.3.0!

Acknowledgments

We would like to thank everyone who contributed to the 1.3.0 release:

Aaron Markham, Abhinav Sharma, access2rohit, Alex Li, Alexander Alexandrov, Alexander Zai, Amol Lele, Andrew Ayres, Anirudh Acharya, Anirudh Subramanian, Ankit Khedia, Anton Chernov, aplikaplik, Arunkumar V Ramanan, Asmus Hetzel, Aston Zhang, bl0, Ben Kamphaus, brli, Burin Choomnuan, Burness Duan, Caenorst, Cliff Woolley, Carin Meier, cclauss, Carl Tsai, Chance Bair, chinakook, Chudong Tian, ciyong, ctcyang, Da Zheng, Dang Trung Kien, Deokjae Lee, Dick Carter, Didier A., Eric Junyuan Xie, Faldict, Felix Hieber, Francisco Facioni, Frank Liu, Gnanesh, Hagay Lupesko, Haibin Lin, Hang Zhang, Hao Jin, Hao Li, Haozhi Qi, hasanmua, Hu Shiwen, Huilin Qu, Indhu Bharathi, Istvan Fehervari, JackieWu, Jake Lee, James MacGlashan, jeremiedb, Jerry Zhang, Jian Guo, Jin Huang, jimdunn, Jingbei Li, Jun Wu, Kalyanee Chendke, Kellen Sunderland, Kovas Boguta, kpmurali, Kurman Karabukaev, Lai Wei, Leonard Lausen, luobao-intel, Junru Shao, Lianmin Zheng, Lin Yuan, lufenamazon, Marco de Abreu, Marek Kolodziej, Manu Seth, Matthew Brookhart, Milan Desai, Mingkun Huang, miteshyh, Mu Li, Nan Zhu, Naveen Swamy, Nehal J Wani, PatricZhao, Paul Stadig, Pedro Larroy, perdasilva, Philip Hyunsu Cho, Pishen Tsai, Piyush Ghai, Pracheer Gupta, Przemyslaw Tredak, Qiang Kou, Qing Lan, qiuhan, Rahul Huilgol, Rakesh Vasudevan, Ray Zhang, Robert Stone, Roshani Nagmote, Sam Skalicky, Sandeep Krishnamurthy, Sebastian Bodenstein, Sergey Kolychev, Sergey Sokolov, Sheng Zha, Shen Zhu, Sheng-Ying, Shuai Zheng, slitsey, Simon, Sina Afrooze, Soji Adeshina, solin319, Soonhwan-Kwon, starimpact, Steffen Rochel, Taliesin Beynon, Tao Lv, Thom Lane, Thomas Delteil, Tianqi Chen, Todd Sundsted, Tong He, Vandana Kannan, vdantu, Vishaal Kapoor, wangzhe, xcgoner, Wei Wu, Wen-Yang Chu, Xingjian Shi, Xinyu Chen, yifeim, Yizhi Liu, YouRancestor, Yuelin Zhang, Yu-Xiang Wang, Yuan Tang, Yuntao Chen, Zach Kimberg, Zhennan Qin, Zhi Zhang, zhiyuan-huang, Ziyue Huang, Ziyi Mu, Zhuo Zhang.

… and thanks to all of the Apache MXNet community supporters, spreading knowledge and helping to grow the community!

Comments:

Nice release... Thanks! Jack G. https://www.gunaydinmesajlari.org

Posted by Jack on September 28, 2018 at 01:21 PM UTC #

good http://www.gmail.com

Posted by aa on October 26, 2018 at 06:31 AM UTC #

The Apache software foundation software MXNet now has experimental support for the Clojure programming language. The MXNet Clojure package brings state-of-the-art deep learning to the Clojure community. It has many more features, enhancements, and usability improvements! In this blog post.http://pariscitytourguide.com/private-tours-paris/paris-city-tour/

Posted by hendry jullius on October 26, 2018 at 07:26 AM UTC #

Radio astronomie antenne tuto http://www.free-sky.fr programmation

Posted by Free sky on January 18, 2019 at 03:08 PM UTC #

I have watched movies and tv shows on my android device using the beetv app (https://beetvapp.com/) which is one of the finest apps out there.

Posted by beetv on February 07, 2019 at 06:09 AM UTC #

Thanks for the great content admin, I like to see more quality content on your website. you explained everything nicely. The Cisco Certifications exam is one of the toughest exams for IT professionals. Cisco focuses on network hardware and devices such as routers and network switches. This is why its certification program is geared toward the information technology field. KillerDumps is a reliable and trusty website that provides the most reliable Cisco dumps. Our Cisco Dump issue will help you pass the exam in first attempt. All Cisco exams are regularly updated and approved by our experts’ professionals. If you want to pass Cisco exam You can get <a href="https://www.killerdumps.com/cisco-exams">Cisco Exams Dumps</a>.

Posted by david cameron on February 07, 2019 at 07:41 AM UTC #

An incredibly useful improvements. As for me one of the best updates for the last time. This is going to be useful for my work at https://payforessayz.com/lab-report-writing-services/. Great!

Posted by Gregory on April 18, 2019 at 07:47 PM UTC #

Watching movies in mediabox hd is a great experience.

Posted by mediabox hd for IOS on July 11, 2019 at 05:42 AM UTC #

This is some great work. The choice of colors is very nice and helps bring a certain dark vibe to it. So cool!! Love the colors and light.

Posted by tiozyscomi1970 on July 20, 2019 at 05:16 PM UTC #

Beautiful work! And love that you show your sketches and study drawing. Wow, so good

Posted by xyediskesearch1988 on July 20, 2019 at 05:38 PM UTC #

awesome packages, simple and beautiful Great design!:)

Posted by kholalingan1975 on July 20, 2019 at 06:02 PM UTC #

u've made really usefull product for that business, great! love how your design dont took attract from main product, but all together its looks like something solid, great!! Vavavooo....... cool!

Posted by taisneakcarcewr1976 on July 20, 2019 at 06:29 PM UTC #

Your work is beautiful. I'm a fan now! Thanks @Quentin Li ! It's indeed an spectacular place, I really enjoyed shooting it!

Posted by zidemoosul1976 on July 20, 2019 at 08:31 PM UTC #

Just beautiful! Your works look very professional! I think we have a lot to learn from each other :)

Posted by exvetlawhitt1981 on July 20, 2019 at 11:49 PM UTC #

Dope!! Great job, I'm obsessed with those illustrations!

Posted by rmenobfeesou1989 on July 21, 2019 at 12:37 AM UTC #

Herzlichen Dank Birgit, es freut mich wirklich sehr zu hören! @Birgit Schlosser Awesome work!

Posted by bsolazerlie1980 on July 21, 2019 at 01:56 AM UTC #

Amazing idea and concepts! Great job here! Feel free to check out my last project.

Posted by vesmaraser1980 on July 21, 2019 at 03:34 AM UTC #

A haunting, creepy and sad commentary on the relationship between humans and their world !!! Incredible vision and execution, Florian !!! Thank you for this incredible work !!! very high tech!

Posted by erosopka1988 on July 21, 2019 at 05:01 AM UTC #

Big papa at it again Great job with the illustrations!

Posted by fikungcosseo1970 on July 21, 2019 at 05:34 AM UTC #

Amazing work! love the messages as well as the fantastic typography touch. Your project is so nice !

Posted by silvakenkgan1975 on July 21, 2019 at 06:14 AM UTC #

sooo coool! love it! thank you @Uttam Lilani @DINO 3AVR @arianna correia fiallo

Posted by rerogecert1981 on July 21, 2019 at 06:47 AM UTC #

这个神态太勾魂了~ Incredible :)

Posted by rackrerethe1984 on July 21, 2019 at 07:20 AM UTC #

damn so good! the red and blue work so well together. Love it

Posted by freesulatder1985 on July 21, 2019 at 07:52 AM UTC #

What a beautiful project. Strong but poetic pics also. Your technique is awesome but I think you photograph with your heart. Thanks for sharing your art. Gr8 job!

Posted by acinsorre1984 on July 21, 2019 at 08:24 AM UTC #

Sympa! Nice creative & execution Charis

Posted by tersprenostoh1989 on July 21, 2019 at 08:56 AM UTC #

Super! Awesome!! The minnie is pretty dope.

Posted by ganiledo1978 on July 21, 2019 at 09:28 AM UTC #

Great! Good Job and Best wishes for your future projects. Kindly give attention to my projects. Good Job and Best wishes for your future projects. Kindly give attention to my projects))

Posted by evondioni1982 on July 21, 2019 at 10:02 AM UTC #

I like the art direction here Such an interesting insight of what the future might hold for you. Really isn't something that the mainstream media and film producers have in mind. This collection shows how creative people can get.

Posted by linkmivimis1977 on July 21, 2019 at 10:35 AM UTC #

muy limpio y clasico a la vez! bien! NICE JOB.

Posted by vorstheviperg1982 on July 21, 2019 at 11:08 AM UTC #

Post a Comment:
  • HTML Syntax: NOT allowed

Calendar

Search

Hot Blogs (today's hits)

Tag Cloud

Categories

Feeds

Links

Navigation