NVIDIA Corporation and the French company Mistral AI announced the Mistral NeMo 12B large language model (LLM), specially designed to solve various enterprise-level tasks – chatbots, data summarization, working with program code, etc.
Mistral NeMo 12B has 12 billion parameters and uses a context window of 128 thousand tokens. The inference uses the FP8 data format, which is said to reduce memory requirements and speed up deployment without any reduction in response accuracy.
Image Source: Pixabay.com
When training the model, the Megatron-LM library, which is part of the NVIDIA NeMo platform, was used. In this case, 3072 NVIDIA H100 accelerators based on DGX Cloud were used. It is claimed that Mistral NeMo 12B copes well with multi-pass dialogues, mathematical problems, programming, etc. The model has “common sense” and “world knowledge”. Overall, it reports accurate and reliable performance across a wide range of applications.
The model is released under the Apache 2.0 license and is offered as a NIM container. The implementation of LLM, according to the creators, takes a matter of minutes, not days. To run the model, one NVIDIA L40S accelerator, GeForce RTX 4090 or RTX 4500 is enough. Among the key advantages of deployment via NIM are high efficiency, low computational cost, security and privacy.
Alibaba Cloud presented at its annual Apsara conference a modular data center architecture called “CUBE…
The original Resident Evil 3: Nemesis turned 25 years old yesterday, and the digital distribution…
The United States and India have reached an agreement under which a new semiconductor manufacturing…
For more than 25 years since the release of the original Half-Life, players have tried…
Image Source: Mediatonic Among the available formats are team deathmatch, every man for himself, and…
Seasonic has released the PRIME PX-2200 2200 W power supply. The new product was first…