Develop Application with LLM and LangChain

Why LangChain

An application might necessitate prompting an LLM multiple times and parsing its output, leading to the requirement of writing a substantial amount of glue code. However, LangChain, developed by Harrison Chase, significantly simplifies this development process.

Langchain Value Adds:

1. Modular components
2. Common ways to combine components
3. Many processes have been converted to components(like prompts templates, parsers, Conversation chains, coversation memories, chainsing components, Document Loaders, Evaluation and Agents)

Langchain Components:

1. Models : Pre trained LLM models(llama,Falcon, Flant5, GPT)
2. Prompts : Style of creating inputs to pass in to models(eg. write a code snippent that creates a function that takes list input from user and retruns sorted result.)
3. Parsers: Taking output of models and parsing them to more structured format(convert llm output list to python list object)
4. Chains : We can add different work streams to form a chain.
5. Agents: It is kind of a manger who knows user inputs and then disperse tasks need to be performed from different tools

why use prompt templates :

1. Prompts can be long and detailed
2. Reuse good prompts when you can
3. Langchain also provides prompts for some common operations
4. One Important aspects of LangChain Prompt libraries is that it support output pausing. In React Framework using COT reasoning and giving LLM space to think in form of thought and then act and get observation.

Parsers:

Language models generate text outputs, but there’s often a need for more organized information. This is where output parsers play a role.

Output parsers are classes that structure language model outputs. They have two key methods:

1. Get format instructions: This method provides instructions for formatting the language model’s output.
2. Parse: This method takes the model’s response (a string) and organizes it into a structured format.

Parser uses LangChain’s ResponseSchema and StructureOutputParser to convert your output to python object

Memory

It uses LangChain’s Conversation buffer memory to store past interactions between llm and user. It also have feature of

1. window where you can specify how many past conversations to remember
2. Token where you can specify number of past tokens to remember
3. Summery : It creates summery of conversation so far and summery length depends upon number of max token provided in memory buffer function
4. Different type of memory:

a. Vector Data Memory

b. Entity Memory

c. We can use combination of memories

Chains:

It helps you run them on many inputs at a time, Consider there are many components in an application and each component have some input and some output. The output of one component is input to other component then we can forms chains using those components and run them sequentially or non sequentially.

1. sequential chains :

a. Simple sequential chains : It works well when there is a single input and single output in each chain

Simple sequential chains

b. Sequential chains: In this one chain can take multiple inputs from different chains or sources. Just have to remember the correct output keys

Sequential chains

2. Router Chains:

You have different chains and when you get user input you have to route to chain which is more fit for user input.

There will be different prompts for different chains and we will use multiprompt and LLM router chains and destination chain for routing to perticular prompt/chain.

Retrieval-Augmented Generation: Question Answering based on Custom Dataset

Different Components

1. Document Loader : There are different loaders available for different type of documents in langchain like Text, Csv, Pdf loaders
2. Embeddings : using a transformer nlp model

Embeddings create numerical representations for pieces of text. This numerical representation captures the semantic meaning of the piece of text that it’s been run over. Pieces of text with similar content will have similar vectors. This lets us compare pieces of text in the vector space

3. Vector database : Used to store chunks of documents in form of embedding vector.

a. Create chunk and then use a model to embed the chunks

b. Create Index: Store it to a vector database in form of index of chunk vectors

c. Query index : pick the user question and convert it to embedding and then query it to the whole index and take top k semantic similar chunks.

These chunks are then used by LLM for answering the question

4. Stuff

Other methods in place of stuff:

Problem with QNA :

Since the model is generative and asking complex question may result in wrong answers. Reasons may be there are many steps involved in finding the answers or direct answer to question is not present but all other information is available which can help to reach final output. ​

• Example :​
• Question asked with simple prompt : How would I get from Singapore to San Francisco?​
• Answer by model : The most direct route from Singapore to San Francisco is to fly. There are several airlines that offer direct flights from Singapore to San Francisco, including Singapore Airlines, United Airlines, and American Airlines. The flight time is approximately 15 hours.(Answer First Justify Later, reason of junk results)​

React:

To solve QNA problem we use this method.

ReAct combines chain of thought reasoning with action planning.​

Idea is reason first and then answer​

ReAct Components​

ReAct prompts

• guide LLMs through reasoning and action steps.​
• Prompts include: Question, Thought (reasoning step), Action (pre-defined choices), Observation (external data context).​

ReAct Actions​

• ReAct employs pre-defined actions: search, lookup, finish.​
• Small Python API interacts with external sources like Wikipedia.​
• Actions guide LLMs toward the solution by following a structured path.​

Tools

These enable language models to interact with resources like Wikipedia, LLM-math, and Search. Each tool has a straightforward interface with text input and output.

Agents

These are language models that drive decisions. They take user input, decide on an “action” to perform, and provide the necessary input for that action.

For deep understanding of Agent and React check this link:

Example​

question = “What is the first film that russel crowe won an oscar for, and who directed that movie?”​

Answer:​

Evaluations :

Challages:

Two challenges in evaluating chains/agents built with Large Language Models (LLMs):

1. Lack of Data:
— Projects often start with limited data for evaluation due to LLMs’ ability to excel with minimal examples.
— LLMs can perform tasks without extensive datasets, in contrast to traditional machine learning approaches that required significant data collection before model usage.

2. Lack of Metrics:
— Chains/agents often tackle tasks with inadequate metrics for assessing performance.
— Generating text, a common task, is more complex to evaluate compared to numeric predictions or classifications.

Solutions:

Two key strategies to address challenges in evaluating LangChainDatasets and agents:

1. Lack of Data:
— Introduced QAGenerationChain, a tool generating question-answer pairs from documents for evaluating question-answering tasks.

2. Lack of Metrics:
— Utilize Language Models to evaluate outputs, offering various chains and prompts for this purpose.

LangChain offers various types of evaluators to help you measure performance and integrity on diverse data. It useslangchain.evaluation component to perform this task.

• String Evaluators: These evaluators assess the predicted string for a given input, usually comparing it against a reference string.
• Trajectory Evaluators: These are used to evaluate the entire trajectory of agent actions.
• Comparison Evaluators: These evaluators are designed to compare predictions from two runs on a common input.

References:

Introduction | 🦜️🔗 Langchain

https://learn.deeplearning.ai/langchain/lesson/1/introduction