When it comes to AI services, predicting what will emerge tomorrow and how it will surpass current tools is impossible. Since its launch, Anthropic’s Claude has left a positive impression on users. Developers utilise it extensively, particularly when compared to other coding and task tools.
And then, Google dropped Gemini 2.5, which claimed to outperform its competitors like Claude Sonnet 3.7, OpenAI GPT-4.5, and others.
In such a scenario, the questions on everyone’s minds are: Is Google silently upping its game? Did the latest Gemini model update encourage Anthropic Claude’s user base to cancel subscriptions and move on? And how good is Google Gemini 2.5 over Anthropic’s AI model?
Google Gemini vs Claude: Benchmarks & Features
The latest from Anthropic’s house is Claude Sonnet 3.7, and Google’s newest addition is its Gemini 2.5 series of models. When their latest AI models are compared with some ecosystem-specific features, a winner emerges.
On LMArena, the Gemini Pro 2.5 model tops the charts, with a 1437 score compared to Claude’s 1303 score.
For writing and editing code, the Aider Polygot leaderboard claims that the Gemini 2.5 Pro model is 72.9% correct, and the Claude 3.7 Sonnet (thinking) is 64.9% correct.
“Gemini 2.5 Pro is now easily the best model for code,” Mckay Wrigley, a developer, said. He highlighted how the model doesn’t just agree with the user all the time, and demonstrated “flashes of genuine brilliance”.
Impressions from similar developers made users ponder that Google Gemini could be the best AI model for coding. The context window size for Gemini 2.5 Pro is one million, which is way bigger than Claude 3.7 Sonnet’s 2,00,000 limit.
On SWE-Bench Verified, Google’s Gemini 2.5 Pro scored 63.8% while Claude achieved a 62.3% accuracy.
When looking at the model’s IQ with reference to tests by Tracking AI, Gemini 2.5 Pro took the lead compared to Claude 3.7 Sonnet.
A report by Artificial Analysis compiling benchmark scores across multiple evaluation methods like MMLU-Pro, GPQA Diamond, and Humanity’s Last Exam puts Google Gemini 2.5 Pro on top of other models.
At the time of publishing this, Gemini supported image generation capabilities with Imagen 3, even without a paid subscription. However, Claude Sonnet 3.7 did not feature any image generation capabilities.
Gemini 2.5 is designed to be a multimodal model that can take inputs including text, images, audio, video, and code.
Similarly, Anthropic’s Claude does not feature any deep research abilities to scour the internet for resources on a topic one needs to research. Google’s Gemini does have this feature, even with the free version, for a limited number of times in a month.
Claude’s Losing Its Footing
Anthropic’s Claude is not in a great position at the moment. Many users complain that Anthropic has reduced the rate limits after promoting a discounted yearly subscription plan. Claude’s Subreddit has been flooded with such posts for a few months.
A Reddit user, who cancelled Claude’s subscription, told AIM that although he was paying $20 a month, there were issues with Claude. “I would open a chat, and I would want to code on a side project. Gemini 2.5 Pro or ChatGPT allowed me to ask for numerous little iterations in the code and other complex stuff for hours on end without getting any context window limitations.”
The user continued, “But with Claude, I could only use one chat for 10 to 15 messages, sometimes even 3 or 5 depending on the rate limits that day, which was very annoying. It made me open a new chat every time. I had to ask a new question, and feed in all the context again. It was irritating as most of my time went into explaining what I wanted, rather than being productive.”
A developer who wished to remain anonymous told AIM, “Trust is hard to earn, but easy to lose. Especially in the AI era, for a company like Anthropic. The developer remarked that the discounted Pro plan for Claude left a bad taste for many, considering the limits were reduced.
Many Reddit threads explained why users cancelled their subscriptions to switch to Gemini 2.5 Pro and other alternatives. Some said that Claude lagged behind the competition and did not have an edge any more, and some believed that the context window was too short with a short session duration.
Furthermore, certain users believed that the rate limit for the Pro plan on Claude could be to encourage upgrading to the MAX subscription plan. Marc Lou, a developer-turned-solopreneur, took to X and said that he had stopped using Claude Sonnet and would be moving to Gemini.
Will this be a trend moving forward? Does Anthropic Claude have any tricks up their sleeves to change this? The future developments in Claude will give a clearer picture of that.
The post Is Google Gemini Gaining Momentum Over Claude? appeared first on Analytics India Magazine.
