Meta introduces Llama 2 as a successor to Llama AI models: Here’s what we know

Meta has announced a new AI model, Llama 2. It is the successor to Llama models with improved performance over the previous generation.

Llama 2 is a set of AI models that can produce text and code based on prompts, similar to other conversational systems.

Llama 2 is free to use for research and business purposes and can be fine-tuned on AWS, Azure and Hugging Face’s AI model hosting platform in pre-trained form. But Llama was not open to everyone as Meta chose to limit access to the models due to concerns about abuse.

Meta has highlighted features of Llama 2 on a whitepaper here are some basic features and differences between Llama 2 and Llama. 

How does Llama 2 different from Llama? 

Llama 2 has two versions, Llama 2 and Llama 2-Chat. Llama 2-Chat was optimised for interactive dialogues. Llama 2 and Llama 2-Chat have different levels of complexity: 7 billion parameters, 13 billion parameters and 70 billion parameters. 

“Parameters” are the components of a model learned from training data and essentially determine the ability of the model on a task, in this case producing text.

Llama 2 was trained on two trillion tokens, which are the basic units of text. For instance, “fan,” “tas” and “tic” for the word “fantastic. Llama 2 used twice as many tokens as Llama did (1.4 trillion). More tokens usually lead to better outcomes.

Sources for training data

Meta claims that it relied on online sources that are open to the public for training data but does not disclose the exact sources. It says that it does not utilise the data from the company’s own products or services and focuses on text that is “factual” in nature.

Llama 2 overall performance

Meta states that in a range of benchmarks, Llama 2 models are slightly inferior to the most prominent closed-source competitors, GPT-4 and PaLM 2, with Llama 2 lagging considerably behind GPT-4 in coding.

“Llama 2 outperforms other open source language models on many external benchmarks, including reasoning, coding, proficiency, and knowledge tests,” Meta said in a blog post.

Meta claims that Llama 2 answered on par across a set of roughly 4,000 prompts designed to probe for “helpfulness” and “safety.”

Meta also acknowledges that Llama 2, like all creative AI models, has prejudices along certain dimensions. For instance, it tends to produce “he” pronouns more often than “she” pronouns. Due to toxic text in the training data, it doesn’t surpass other models on toxicity tests.

The Llama 2-Chat models outperform the Llama 2 models on Meta’s own “helpfulness” and toxicity tests. But they also tend to be too careful, with the models preferring to reject some requests or replying with too much safety information.

The post Meta introduces Llama 2 as a successor to Llama AI models: Here’s what we know appeared first on Techlusive.