YandexART 2.0 AI model with support for generating text on images has been introduced

«Yandex has released YandexART 2.0, a new generation image generator. The neural network has learned to create inscriptions on an image and maintain several styles at once in one picture; objects in space and relative to each other are now located more naturally; and when creating images, more query details are taken into account.

Image source: Yandex

A distinctive feature of YandexART 2.0 is its hybrid neural network architecture, combining features of convolutional and transformer models. The convolutional model works like the human eye, identifying key features of an object, such as its shape, texture and edges, but it is limited in the length of the context, so it is assisted by a transformer for long queries. This architecture helps YandexART 2.0 handle multiple genres in a single image—for example, it can depict an anime label on a photorealistic lemonade bottle.

To train the YandexART 2.0 neural network, several hundred million pairs of images and text descriptions for them were used; a more accurate relationship was provided by an additional VLM model, with the help of which the pictures were analyzed and accompanied by detailed text descriptions. The array of training data was expanded to include several hundred thousand images with text – this helped YandexART 2.0 to supplement pictures with inscriptions in Latin letters.

«Yandex also developed its own system for assessing the quality of work for the image generator: the new model beat Midjourney v6.1 in terms of complexity and aesthetics in 66% and 58% of cases, respectively, and also came closer to it in terms of relevance to queries.

Business users can work with YandexART 2.0 on the Yandex Cloud platform – using the API, you can integrate the image generator into any application; It is possible to test its operation in demo mode to select the optimal queries. Corporate clients can generate logos, illustrations for articles, presentations or social networks.

The visual neural network is also available to private users in the web version of Alice and its own virtual assistant application; owners of free accounts can request up to five images per day, and subscribers of the Alice Pro option do not have such a limitation. With YandexART 2.0 you can create an avatar for social networks, an application icon, a print for a T-shirt, a postcard for a friend or an illustration for publication.