LLMs

pre-training phase, the language model is trained on a large and diverse dataset, typically consisting of text from the internet or other publicly available sources

• learn general language patterns, grammar, and semantic relationships between words and phrases
• the model is trained to predict the next word in a sentence given the previous words.

improved through reinforcement learning

• reinforcement learning is that it’s an additional step of fine-tuning that makes the base model more useful Reinforcement learning can be done via human feedback, like ChatGPT and Instruct GPT. Or with another AI model’s feedback, like Claude.

LLMs follow our instructions via prompts

Prompt engineering

prompt engineering is a short term thing

We won’t be interacting directly with LLMs in the near future. Instead we’ll have more user-friendly and intuitive AI interfaces could reduce the need for manual prompt engineering, allowing non-expert users to interact with models naturally. fine-tuned models for everything to where the gains from intricate details provided in high quality prompts are eroded.

Reasons for it prompting to stay

• simply a form of communication, and being good at prompting means you’ll have more precision and control over your output.
• Prompting is the most effective way to get a general purpose LLM like ChatGPT to be good at a specific task. It is unrealistic that we will fine tune a model for every specific task we’d want an LLM to help us with.

LLM Capabilities

• Generation: LLMs excel at generating coherent and contextually appropriate text - content creation, creative writing, dialogue generation for chatbots, and generating responses to user queries
• Translation: LLMs are capable of translating text between different languages with high proficiency.
• Summarization: LLMs can produce concise summaries of longer text documents while retaining key information.
• Performing Narrow AI Tasks with Ease: LLMs can be fine-tuned for specific narrow AI tasks, such as sentiment analysis, named entity recognition, and text classification.
• Reasoning: While LLMs have limitations in reasoning compared to humans, they can perform certain reasoning tasks by “thinking step by step” based on patterns observed in training data. For example, LLMs can answer questions that require logical reasoning or inference based on the information provided in a text passage.
• Understanding Language Across Modalities: language in the context of other modalities, such as images - describe images, generate captions, and answer questions about visual content.

What not (yet)

• Math: LLMs are not inherently skilled at solving complex mathematical problems, and their proficiency is generally limited to basic arithmetic and the patterns they have observed in training data.
• Being 100% Factual: LLMs may generate text that is factually incorrect, as they are not aware of real-world truths and rely solely on patterns in training data. Users should verify information generated by LLMs using reliable sources.

pre-trained open source base model, such as GPT-J or LLaMA

Challenges

• high training cost
• fine-tuning not a well-understood science
• data shortage for training
• limited context windows (text length and further context access)
• hallucinating information
• latency in responses
• not up-2-date
  • lack knowledge of recent events, trends, or technological advancements

Linking

• Let’s build GPT from Scratch
• Access to Claude 2 via Poe

Enjoy this post?