As foundational models become more capable, a parallel trend is emerging in open-source AI: developers and entrepreneurs are building smaller, more specialised models that can be deployed locally. For many developers, the journey started with Meta’s Llama 2.
This conversation unfolded on an episode of AI Talks by AIM, powered by Meta. Sunil Abraham, public policy director of data economy and emerging tech at Meta India, led a deep dive into how open-source generative AI is driving real-world impact well beyond Silicon Valley.
He was joined by Pratik Desai, founder and CEO of KissanAI, and Adithya S Kolavi, AI researcher at CognitiveLab. Both are practitioners building frontier models for India’s unique requirements.
“Llama 2 was a very revolutionary step in the open source journey, especially in mine,” Kolavi said.
Kolavi said that Llama was more than just a model; it came with an ecosystem of frameworks that made it easier to run and adapt. “Inferencing was hard initially, but tools like VLM and SGLang came along and integrated with Llama early on, which made things easier,” he said.
This openness enabled Kolavi to transition from a full-stack developer to a researcher and model developer. “These models can be adapted for different languages and modalities,” he said. “Just having them openly available, without any restrictions, helped a lot in my research journey.”
As the ecosystem matured, fine-tuning support also improved. Hugging Face added integration, and other libraries such as Llama Factory followed suit, making the model even more accessible to developers.
Desai, who also launched the agricultural AI platform, Dhenu, echoed the sentiment. He credited Llama 2 for powering one of India’s earliest domain-specific language models in agriculture. “We used Llama 2 in collaboration with Sarvam AI to train the Dhenu model.”
He added that until that point, most efforts had focused on fine-tuning models for style, but he and his team chose to experiment with enriching the models by injecting more domain-specific knowledge.
Desai, who transitioned from academia to entrepreneurship, said that open source was the only viable route. He explained that as a bootstrap startup, they relied heavily on open-source tools like PyTorch and TensorFlow to get started.
Reflecting on his decision to move away from academia and focus on applied AI, Desai said, “If your work is actually not impactful or it’s not going to be used by folks, then it’s just a waste of time. There are thousands of papers getting published every year in every conference now, and most of them are not even getting read nowadays.”
On-Device Models and Local Impact
Kolavi pointed out that open models are crucial for enterprise AI and personal privacy. “Most of the time, enterprises want to host models on their own infrastructure,” he said.
He said that with smaller models being released now—less than 2 billion parameters—users can run them locally with good inference speeds. “You can do document processing or chat across your laptop or phone.”
Desai agreed and took it further. He sees the benefit of open and small models for deployment even in regions with poor internet connectivity. He revealed that he ported the KissanAI assistant onto Android phones.
The goal was to get the entire assistant working locally, without needing an internet connection. “This is useful where bandwidth is limited or expensive. Even embedded devices in the field can use local inference.”
Desai added that on-premise hosting, particularly for India’s digital public infrastructure, is critical. “If you’re building for smallholder farmers, you can fine-tune the model for that. If you’re building for B2B agri applications, you can tune it differently,” Desai explained. “You don’t need extension workers to physically visit villages. You can have small devices with the same knowledge always available.”
Kolavi addressed the concern of bias. “Every model reflects the data it’s trained on. Western data is overrepresented. But if you have data from your own context, like farming in India, you can fine-tune the model to reduce bias,” he said.
Desai explained the importance of local context with an example, noting that a question about corn should refer to Indian corn rather than its US counterpart. He pointed out that terms like ‘whitefly’ and ‘kernel’ vary across regions and a general-purpose model may not grasp these distinctions.
Where Open Source Goes Next
Both Kolavi and Desai believe open source is here to stay, but not without challenges.
Talking about Chinese models and benchmarks, Kolavi said, “Most benchmarks are taken with a grain of salt because models are trained on them directly. Leaderboards on static datasets don’t scale anymore. Human evaluations like LMSYS’s Arena are better.”
Desai expressed concern over AI models being released under restrictive licences, such as Creative Commons BY-NC, which limit commercial use. “I really do not like to work with those models,” he said, adding that if open weights are being released, they should come with permissive licences that allow repackaging and building startups on top of them.
For Desai, the goal of releasing open weight models should be to “foster an ecosystem of a new startup”.
“We’re working with IndiaAI, Agri Stack, and DPI, helping with open source contributions. We’re now working with Fortune 500 companies across India and the US. And we’re still bootstrapped,” he concluded.
As AI expands into domains like agriculture, the power of open source, from community-driven innovation to local deployments, continues to shape both the market and the mission.
The post Indian AI Startups are Obsessed with Open-Source Small Models appeared first on Analytics India Magazine.
