However, the course example completed the entire steps of chain of thought when using " Vector Search Index" tool:
Entering new AgentExecutor chain...
{
"action": "Vector Search Index",
"action_input": "toy story"
}
Observation: {'question': 'toy story', 'answer': 'Based on the information provided, "Toy Story" is a movie about a cowboy doll who feels threatened and jealous when a new spaceman figure becomes the top toy in a boy's room. It is a heartwarming animated film that explores themes of friendship, loyalty, and acceptance. }
Thought:{
"action": "Final Answer",
"action_input": "Based on the information provided, "Toy Story" has a similar plot to movies like "NeverEnding Story III", "E.T. the Extra-Terrestrial"...."
}
Finished chain.
Is there any problem with my agent_prompt that lead to the break of chain of thought?
Here is my agent_prompt for create_react_agent:
agent_prompt = PromptTemplate.from_template("""
You are an assistant providing information about the database.
Be as helpful as possible and return as much information as possible.
Do not answer any questions that do not relate to database.
Do not answer any questions using your pre-trained knowledge, only use the information provided in the context.
TOOLS:
------
You have access to the following tools:
{tools}
To use a tool, please use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: Observe the result of the action input, and check the remaining question to take action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here]
Begin!
Previous conversation history:
{chat_history}
New input: {input}
{agent_scratchpad}
""")
First off, I do not consider myself to be an expert in prompt engineering... From my limited experience though and certainly with langchain 0.1.x I have found that the agent prompt has a significant impact.
My first thought regarding the prompt you shared is that is "database" to vague.
You are an assistant providing information about the database.
Does this give too wide a scope?! This is where I would start with iterating the prompt and seeing what has an impact.
hi Martin,
even i use the default prompt
prompt = hub.pull("hwchase17/react-chat") which is used in the course example, it still fail to complete the chain of thought. It always stopped at "Action Input" then > Finished chain
May I know the script used in course? how can it finish the chain of thought loop? Many thanks.
prompt from "hwchase17/react-chat":
Assistant is a large language model trained by OpenAI.
Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.
Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.
Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.
TOOLS:
------
Assistant has access to the following tools:
{tools}
To use a tool, please use the following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here]
Begin!
Previous conversation history:
{chat_history}
New input: {input}
{agent_scratchpad}
agent_prompt = PromptTemplate.from_template("""
You are a movie expert providing information about movies.
Be as helpful as possible and return as much information as possible.
Do not answer any questions that do not relate to movies, actors or directors.
Do not answer any questions using your pre-trained knowledge, only use the information provided in the context.
TOOLS:
------
You have access to the following tools:
{tools}
To use a tool, please use the following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here]
Begin!
Previous conversation history:
{chat_history}
New input: {input}
{agent_scratchpad}
""")
I just discovered that the tool function should be set as "return_direct = false" instead of "return_direct = true" as shown in course example. This can solve the broken chain problem. As it won't stop the chain after using tools. It works for my case
tools = [
Tool.from_function(
name="General Chat",
description="For general chat not covered by other tools",
func=llm.invoke,
return_direct=False
),
Tool.from_function(
name="Vector Search Index", # (1)
description="Provides information about movie plots using Vector Search", # (2)
func = kg_qa, # (3)
return_direct=False
)
]
