daniel@home:~$

AI World. Generative Artificial Intelligence News - Week 4/52 2024

Image Editing with LLMs by Automatically Enhancing Prompts

Human instructions are sometimes too brief for current methods to understand and follow. Multimodal Large Language Models (MLLMs) show promising capabilities in cross-modal understanding and visually-aware response generation through LMs.

Source: https://github.com/apple/ml-mgie

LLaVA-NeXT

New version of LLaVA-NeXT with improved reasoning, OCR, and world knowledge. LLaVA-NeXT even outperforms Gemini Pro on several benchmarks.

Compared to LLaVA-1.5, LLaVA-NeXT introduces several improvements:

- Increasing the input image resolution to 4 times more pixels. This allows it to capture more visual details.
- Supports three aspect ratios, up to resolutions of 672x672, 336x1344, 1344x336.
- Better visual reasoning and OCR capability with an improved visual instruction tuning data mixture.
- Better visual conversation for more scenarios, covering different applications. Better world knowledge and logical reasoning.
- Efficient deployment and inference with SGLang.

Along with performance improvements, LLaVA-NeXT maintains the minimalist design and data efficiency of LLaVA-1.5. It reuses the pretrained connector from LLaVA-1.5 and continues to use fewer than 1M visual instruction tuning samples. The largest 34B variant finishes training in ~1 day with 32 A100s. The code, data, and model will be made publicly available.

Source: https://llava-vl.github.io/blog/2024-01-30-llava-1-6/


Automatically translated from the Spanish original.

Publicado en: Mundo IA

Tags: LLaVA-NeXT Generación imagenes LLM

Want to subscribe to new posts?