Indian AI startup Sarvam AI has released OpenHathi-Hi-v0.1, the first Hindi LLM in the OpenHathi series. Developed on a budget-friendly platform, the model, an extension of Llama2-7B, boasts GPT-3.5-like performance for Indic languages.

OpenHathi, featuring a 48K-token extension of Llama2-7B’s tokenizer, undergoes a two-phase training process. The initial phase focuses on embedding alignment, aligning randomly initialized Hindi embeddings, followed by bilingual language modeling, teaching the model cross-lingual attention across tokens.

The model demonstrates robust performance across various Hindi tasks, comparable to, if not surpassing, GPT-3.5, while maintaining English proficiency. Sarvam AI’s evaluation includes non-academic, real-world tasks alongside standard Natural Language Generation (NLG) tasks. Evaluations against GPT-3.5 generation with GPT-4 as the judge revealed superior performance in Hindi, both in native and Romanised scripts.

Developed in collaboration with academic partners at AI4Bharat, who contributed language resources and benchmarks, and fine-tuned in partnership with KissanAI, the model leverages conversational data from a bot interacting with farmers in multiple languages. 

KissanAI recently announced the launch of Dhenu 1.0, a groundbreaking Agriculture Large Language Model. Tailored specifically for Indian agricultural practices, this bilingual model comprehends English, Hindi, and Hinglish queries, a notable feature catering directly to farmers’ linguistic needs.

Pratyush Kumar and Vivek Raghavan, co-founders of Sarvam AI, launched the startup in July 2023, backed by $41 million in Series A funding led by Lightspeed, with participation from Peak XV Partners and Khosla Ventures. 

Stemming from the founders’ background in AI research and digital infrastructure development, the startup aims to cater to India’s unique needs, prioritising Generative AI integration for diverse Indian languages and fostering collaborations with enterprises for domain-specific AI model development using their data.

You can find the base model here.