Cohere releases Embed 4: a multimodal AI model designed for agentic search

Artificial intelligence startup Cohere Inc. today launched Embed 4, its latest AI model designed to provide embeddings for search and retrieval for AI applications such as assistants and agents.

Enterprise businesses rely heavily on retrieval-augmented generation, a technique that allows generative AI models to retrieve and incorporate fresh, accurate information in real time so that large language models respond to user queries with the best possible data.

Embedding models, such as Embed take data from documents and turn it into mathematical representations called vectors that can be used to represent the dynamic, multifaceted context of the information. In the case of Embed 4, this can include anything from text-based documents or images to tables, graphs, code and diagrams.

Cohere said the new model includes an extremely large context length – up to 128,000 tokens, or about a 200-page document – allowing it to ingest a lengthy annual financial report, product manual or detailed legal contract. It’s also multilingual, with more than 100 languages, featuring key business languages such as Arabic, Japanese, Korean and French proficiencies in addition to English.

Embed 4 enables organizations to search their unstructured documents, where most of their important data resides, Cohere said, adding that the unique part is that it can generate high-quality representations of complex mixed-modality documents, all within a unified vector.

The AI startup said it designed the new model to excel in regulated industries such as finance, healthcare and manufacturing with domain-specific understanding of these industries. These include capabilities for searching investor presentations, annual financial reports, medical records, procedural charts, product specification documents, repair guides and supply chain documents.

Cohere also noted that Embed 4 can deal with noisy real-world data by handling fuzzy images and poorly oriented documents. The model was trained against numerous documents of scanned documents, handwriting and other distressed documents, the company said. These are the types of complex data that many businesses will encounter in day-to-day multimodal data pre-processing, part of the manual pipeline.

Agora, an AI-driven search engine for 35,000 online stores and a customer of Cohere used the model to assist with its business, said it was able to build a better search using its advanced multimodal embedding features.

“E-commerce data is complex, containing images and multifaceted text descriptions,” said founder Param Jaggi. “Being able to represent our products in a unified embedding makes our search faster and our internal tooling more efficient,”

Embed’s capabilities are essential for accurate search and retrieval, which power generative AI models such as Cohere’s Command A, a low-cost model the company released last month. Models such as Command A power conversational assistants and AI agents, but rely heavily on search engines, which are connected to secure, proprietary company information to source relevant information to user questions. This is necessary to speed up responses, increase accuracy and reduce hallucinations.

Cohere said the new Embed 4 model is integrated with North, the company’s secure AI agent productivity platform where it powers its semantic search capability in its Compass product.

The Embed 4 model is also available starting today from Microsoft Azure AI Foundry, on Amazon SageMaker and for private deployments.

Images: Pixabay; Cohere