Qualcomm introduced energy-efficient AI microservers AI On-Prem Appliance Solution

Qualcomm Technologies announced the Qualcomm AI On-Prem Appliance Solution, a compact, energy-efficient hardware solution for on-premises processing of inference and computer vision workloads. The company also presented a ready-to-use set of AI applications, libraries, models and agents, the Qualcomm Cloud AI Inference Suite, capable of working at the edge in the clouds.

According to the press release, the combination of new products enables SMBs and industrial organizations to run custom and pre-built AI applications on their premises, including generative AI workloads. Qualcomm noted that in-house inference will significantly reduce operating costs and total total cost of ownership (TCO) compared to renting third-party AI infrastructure.

Image Source: Qualcomm

With the AI ​​On-Prem Appliance Solution coupled with the AI ​​Inference Suite, customers will be able to use proprietary data-driven generative AI, fine-tuned models, and technology infrastructure to automate processes and applications in virtually any environment, such as retail stores, restaurants, point-of-sales, dealerships, hospitals, factories and workshops where work processes are well-established, repeatable and ready for automation.

«The AI ​​On-Prem Appliance Solution and Cloud AI Inference Suite change the TCO of AI by enabling generative AI workloads to be processed locally rather than in the cloud,” the company said, highlighting that the AI ​​On-Prem Appliance Solution can significantly reduce application operating costs AI for enterprise and industrial needs in a wide range of areas. In addition, local deployment provides protection against sensitive data leaks.

The Qualcomm AI On-Prem Appliance Solution platform is powered by the Qualcomm Cloud AI family of accelerators. It is reported that the new product supports a wide range of capabilities, including:

• Scalable from a single desktop device to a wall-mounted cluster;
• Serving a variety of on-premises AI services, including out-of-the-box voice agents in a box, small language models (SLMs), large language models (LLMs), multimodal models (LMMs), RAGs, agent AI, AI automation platforms, and more;
• Supports a wide range of generative AI, natural language processing, and computer vision models;
• Stream AI processing of camera images and video for computer vision applications focused on safety, security and monitoring.

In turn, the Qualcomm Cloud AI Inference Suite offers a complete set of tools and libraries for developing or migrating generative AI applications to the AI ​​On-Prem Appliance Solution or other platforms based on Qualcomm Cloud AI accelerators. The set offers many APIs for user management and administration, for chats, for generating images, audio and video. Claims compatibility with OpenAI API and RAG support. In addition, integration with popular generative AI models and frameworks is available. Deployment using Kubernetes and bare metal is possible.