Anthropic’s Claude 2.1 LLM turbocharges performance, offers beta tool use

Anthropic has upped the ante for how much information a large language model (LLM) can consume at once, announcing on Tuesday that its just-released Claude 2.1 has a context window of 200,000 tokens. That’s roughly the equivalent of 500,000 words or more than 500 printed pages of information, Anthropic said.

The latest Claude version also is more accurate than its predecessor, has a lower price, and includes beta tool use, the company said in its announcement.

The new model powers Anthropic’s Claude generative AI chatbot, so both free and paying users can take advantage of most of Claude 2.1’s improvements. However, the 200,000 token context window is for paying Pro users, while free users still have a 100,000 token limit — significantly higher than GPT-3.5’s 16,000.

Claude 2’s beta tool feature will allow developers to integrate APIs and defined functions with the Claude model, similar to what’s been available in OpenAI’s models.

Claude’s previous 100,000 token context window had been significantly ahead of OpenAI in that metric until last month, when OpenAI announced a preview version of GPT-4 Turbo with a 128,000 token context windows. However, only ChatGPT Plus customers with $20/month subscriptions can access that model in chatbot form. (Developers can pay per usage for access to the GPT-4 API.)

While a large context window — the amount of data it can process at a time — looks compelling if you have a large document or other information, it’s not clear that LLMs can process large amounts of data as well as info in a smaller chunk. Greg Kamradt, an AI practitioner and entrepreneur who’s been tracking this issue, has run what he calls “needle in a haystack” analysis to see if tiny pieces of info within a large document are actually found when the LLM is queried. He repeats the tests putting a random statement in various portions of a large document that’s fed into the LLM and queried.

Copyright © 2023 IDG Communications, Inc.

Source :

Leave a Comment

SMM Panel PDF Kitap indir