Elon Musk’s up to date Grok AI claims to be higher at coding and math

Elon Musk’s answer to ChatGPT is getting an replace to make it higher at math, coding and extra. Musk’s xAI has launched Grok-1.5 to early testers with “improved capabilities and reasoning” and the flexibility to course of longer contexts. The corporate claims it now stacks up in opposition to GPT-4, Gemini Professional 1.5 and Claude 3 Opus in a number of areas.

Going by xAI’s numbers, Grok-1.5 seems to be a big enchancment over Grok-1. It shot as much as 50.6 p.c within the MATH benchmark, over double the earlier rating. It additionally climbed to 90 p.c and 74.1 p.c in GSM8K (math phrase issues) and HumanEval (coding), respectively, in comparison with 62.9 p.c and 63.2 p.c earlier than. These numbers are inside shouting distance of Gemini Professional 1.5, GPT-4 and Claude 3 Opus — actually, the HumanEval coding rating beats all rivals besides Claude 3 Opus.

It may possibly additionally course of lengthy contexts of as much as 128K tokens inside its context window, that means it will possibly amalgamate knowledge from extra sources to know a state of affairs. “This permits Grok to have an elevated reminiscence capability of as much as 16 instances the earlier context size, enabling it to make the most of info from considerably longer paperwork,” the corporate mentioned.

xAI did not element Grok’s progress in different areas, although, the place it nonetheless could also be lagging (tutorial scores, multimodal and others). And Grok-1.5 might not preserve its place for lengthy. ChatGPT 5 is ready to reach someday this summer season, promising a function set that “makes it really feel like you’re speaking with an individual relatively than a machine,” based on OpenAI.

At the moment, Grok is simply obtainable for customers of the Premium+ tier on X (previously Twitter), although Elon Musk not too long ago promised to open it as much as X’s common Premium customers. The corporate additionally not too long ago open sourced its Grok chatbot, after Musk sued OpenAI and Sam Altman for allegedly abandoning its non-profit mission.