The Unified Communication Framework (UCF), a collaboration of industry, laboratories, and academia to create production-grade communication frameworks and open standards for data-centric and high-performance applications, today announced the formation of the Open Smart Network Application Programming Interface (OpenSNAPI) project.
New technologies continue to be developed to support the migration of the data center architecture from the old CPU-centric concept to the data-centric concept. An important part of this transition has involved the creation of new compute options, including smart and programmable interconnect solutions, also referred to as Smart Networking Adapters/SmartNICs or Data Processing Units (DPUs). DPUs are interconnect elements that include specialized in-Network Computing engines, which are engines that can participate in the application runtime and analyze application data as it is being transferred within the data center, or generic compute cores. DPUs can be used to enhance supercomputing performance, offload security or virtualization functions, increase storage performance, and more. Through collaboration between industry, laboratories, and academia within the UCF consortium, the goal of OpenSNAPI is to define and create a standard application programming interface (API) for accessing the compute engines on the network – specifically on a smart network adapter.
“The new world of data opens the door for higher degrees of scientific simulations which enable solving problems previously considered intractable, and for developing advanced deep learning engines,” said Steve Poole, distinguished senior scientist and chief architect at Los Alamos National Laboratory and UCF board member. “Our goal with OpenSNAPI is to further enable application developers to leverage the network compute cores in parallel to the host compute cores for accelerating application runtime, and to perform operations and processing much closer to the data. We invite the industry and academia to join us in this important endeavor.”
“With more devices coming online each day come new data sets that need to be processed, driving the need for compute efficiency,” said Brent Gorda, senior director of HPC, Infrastructure Line of Business, Arm. “The power constraints of SmartNIC devices reinforce the importance of the flexible, efficient IP solutions we are providing to our ecosystem. The OpenSNAPI unified API is helping to expand the applicability of emerging use-cases, such as in-network computing, and ultimately enabling more efficient data processing through broader deployments of SmartNICs.”
The UCF consortium manages several open-source development projects, including UCX and UCX for Apache Spark™. UCX provides an open unified communication software framework that enables users to exploit the capabilities of new high-performance computing systems, meet demands for scalability on millions of cores and support applications with critical functionality. UCX for Apache Spark is a high-performance, scalable and efficient ShuffleManager plugin for Apache Spark that utilizes RDMA and other high-performance transports to reduce CPU cycles needed for Shuffle data transfers.
