Deep Learning Reference Stack v4.0 Now Available

20 Aug, 2019

Artificial Intelligence (AI) continues to represent one of the biggest transformations underway, promising to impact everything from the devices we use to cloud technologies, and reshape infrastructure, even entire industries. Intel is committed to advancing the Deep Learning (DL) workloads that power AI by accelerating enterprise and ecosystem development.

From our extensive work developing AI solutions, Intel understands how complex it is to create and deploy applications for deep learning workloads. That’s why we developed an integrated Deep Learning Reference Stack, optimized for Intel® Xeon® Scalable processor and released the companion Data Analytics Reference Stack.

Today, we’re proud to announce the next Deep Learning Reference Stack release, incorporating customer feedback and delivering an enhanced user experience with support for expanded use cases.

With this update, Intel further enables developers to quickly prototype and deploy DL workloads, reducing complexity while maintaining the flexibility for developers to customize solutions. Among the features added in this release:

  • TensorFlow 1.14, an end-to-end open source platform for machine learning (ML) that helps researchers push state-of-the-art solutions and developers easily build and deploy ML-powered applications more easily.
  • Intel® OpenVINO™ model server version 2019_R1.1, delivering improved neural network performance on a variety of Intel processors, helping unlock cost-effective, real-time vision applications.
  • Intel Deep Learning Boost (DL Boost) with AVX-512 Vector Neural Network Instruction (Intel AVX-512 VNNI), an x86 extension that’s part of the Advanced Vector Extensions 512 (AVX-512) platform, designed to accelerate deep neural network-based algorithms.
  • Deep Learning Compilers (TVM* 0.6), an end-to-end compiler stack.

The Deep Learning Reference Stack can be used in either a single or multi-node architecture (with Horovod), providing choice for development and deployment of DL workloads.

This release also incorporates the latest versions of developer tools and frameworks:

  • Operating System: Clear Linux* OS, customized to individual development needs and optimized for Intel platforms, including specific use cases like Deep Learning.*
  • Orchestration: Kubernetes to manage and orchestrate containerized applications for multi-node clusters with Intel platform awareness.
  • Containers: Docker Containers and Kata Containers with Intel® VT Technology for enhanced protection.
  • Libraries: Intel® Math Kernel Library for Deep Neural Networks (MKL DNN), highly-optimized for mathematical function performance.
  • Runtimes: Python application and service execution support.
  • Deployment: Kubeflow Seldon*, an open platform for deploying machine learning models on Kubernetes.
  • User Experience: Jupyter Hub*, a multi-user Hub that spawns, manages, and proxies multiple instances of the single-user Jupyter notebook server.

Each layer of the Deep Learning Reference Stack has been performance-tuned for Intel architecture, enabling impressive performance compared to non-optimized stacks.

Performance gains for the Deep Learning Reference Stack with Intel® OpenVINO™ and ResNet50 as follows:

Second Generation Intel® Xeon® Scalable Platform – 2x Intel® Xeon® Platinum 8280 Processor (2.7GHz, 28-core), HT On, Turbo On, 384 GB memory (12 x 32 GB DDR4 @ 2933 MHz), 7TB NVMe SSD SSDPE2KE076T8, Clear Linux 30700, BIOS SE5C620.86B.0D.01.0271.120720180605, ucode (0x4000013), Linux 4.19.65-69.LTS 2018, Intel® OpenVINO™ 2019_R1.1, AIXPRT CP2 (Community Preview), Benchmark: https://www.principledtechnologies.com/benchmarkxprt/aixprt/, Workload: ResNet-50, Compiler: GCC v9.1.1, Intel® MKL DNN v0.19

Intel® Xeon® Scalable Platform – 2x Intel® Xeon® Platinum 8180 Processor (2.5GHz, 28-core), HT On, Turbo On, 384 GB memory (12 x 32 GB DDR4 @ 2633 MHz), 1TB NVMe SSD SSDPE2KX010T7, Clear Linux 30700, BIOS SE5C620.86B.02.01.0008.031920191559, ucode (0x200005e), Linux 4.19.65-69.LTS 2018, Intel® OpenVINO™ 2019_R1.1, AIXPRT CP2 (Community Preview), Benchmark: https://www.principledtechnologies.com/benchmarkxprt/aixprt/, Workload: ResNet-50, Compiler: GCC v9.1.1, Intel® MKL DNN v0.19

Intel will continue working to help ensure popular frameworks and topologies run best on Intel architecture, giving customers a choice in the right solution for their needs. We are using this stack to innovate on our current Intel® Xeon® Scalable processors and plan to continue performance optimizations for coming generations.

Visit the Clear Linux* Stacks page to learn more and download the Deep Learning Reference Stack code, and contribute feedback. As always, we welcome ideas for further enhancements through the stacks mailing list.

 


Notices and Disclaimers
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors.
Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information, visit www.intel.com/benchmarks.
Performance results are based on testing as of 8/11/2019 and may not reflect all publicly available security updates. No product or component can be absolutely secure.

System configuration
Second Generation Intel® Xeon® Scalable Platform – 2x Intel® Xeon® Platinum 8280 Processor (2.7GHz, 28-core), HT On, Turbo On, 384 GB memory (12 x 32 GB DDR4 @ 2933 MHz), 7TB NVMe SSD SSDPE2KE076T8, Clear Linux 30700, BIOS SE5C620.86B.0D.01.0271.120720180605, ucode (0x4000013), Linux 4.19.65-69.LTS 2018, OpenVINO™ 2019_R1.1, AIXPRT CP2 (Community Preview), Benchmark: https://www.principledtechnologies.com/benchmarkxprt/aixprt/, Workload: ResNet-50, Compiler: GCC v9.1.1, Intel® MKL DNN v0.19
Intel® Xeon® Scalable Platform – 2x Intel® Xeon® Platinum 8180 Processor (2.5GHz, 28-core), HT On, Turbo On, 384 GB memory (12 x 32 GB DDR4 @ 2633 MHz), 1TB NVMe SSD SSDPE2KX010T7, Clear Linux 30700, BIOS SE5C620.86B.02.01.0008.031920191559, ucode (0x200005e), Linux 4.19.65-69.LTS 2018, OpenVINO™ 2019_R1.1, AIXPRT CP2 (Community Preview), Benchmark: https://www.principledtechnologies.com/benchmarkxprt/aixprt/, Workload: ResNet-50, Compiler: GCC v9.1.1, Intel® MKL DNN v0.19
Optimization Notice: Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice Revision #20110804
Intel, the Intel logo, and Intel Xeon® are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others.
© Intel Corporation