サマリー

問題: create_react_agent が新しいagentの作成方法なので試してみたら Parsing LLM output produced both a final answer and a parse-able action: というエラーで動かなかった

調査してわかったこと:

prompt = hub.pull("hwchase17/react-chat") そもそも例に載っているhubを使うのが一番楽
ただchat historyと一緒に使う時に間違えてprompt = hub.pull("hwchase17/react") を使うとエラーになるので注意
エラーの根本原因はPromptの問題でOutputParserで期待される回答が返っていなかった
create_react_agentでは、ReActSingleInputOutputParserが使われる
create_react_agentに渡すpromptには "tools", "tool_names", "agent_scratchpad"が無いといけない
create_react_agentに渡すpromptはlibs/langchain/langchain/agents/mrkl/prompt.pyのものをそのまま使うとエラーになる
従来の ConversationalAgent.from_llm_and_tools(llm=llm, tools=tools)でもうまくいく

promptはConversationalAgentが使っているものとmkrlのものを合わせて修正するとうまくいく

FORMAT_INSTRUCTIONS = """To use a tool, please use the following format:

```
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
```

When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:

```
Thought: Do I need to use a tool? No
Final Answer: [your response here]
```"""

`create_react_agent` を試してみる (Error)

create_react_agentにはpromptを渡さないといけないので、それらしく書いてみる。

from langchain_community.llms import OpenAI
from langchain_community.utilities import GoogleSearchAPIWrapper

from langchain_core.prompts import PromptTemplate
from langchain.agents import Tool, create_react_agent, AgentExecutor
from langchain.memory import ConversationBufferMemory


def multiplier(a, b):
    return a * b


def parsing_multiplier(string):
    a, b = string.split(",")
    return multiplier(int(a), int(b))


google = GoogleSearchAPIWrapper()


def top5_results(query):
    return google.results(query, 5)


tools = [
    Tool(
        name="Multiplier",
        func=parsing_multiplier,
        description=(
            "useful for when you need to multiply two numbers together. "
            "The input to this tool should be a comma separated list of numbers of length two, representing the two numbers you want to multiply together. "
            "For example, `1,2` would be the input if you wanted to multiply 1 by 2."
        ),
    ),
    Tool(
        name="google-search",
        description="Search Google for recent results.",
        func=top5_results,
    ),
]


def main():

    memory = ConversationBufferMemory(
        memory_key="chat_history",
        return_messages=True,
    )

    llm = OpenAI(temperature=0)
    template = '''以下のツールを駆使して、以下の質問に答えてください。

    {tools}

    以下の定義に従ってください:

    Question: あなたが答える必要のある質問です
    Thought: あなたは常に次に何をするべきか考える必要があります
    Action: 次に取るべきアクションであり、[{tool_names}] の中のどれか一つである必要があります。
    Action Input: アクションへの入力値です
    Observation: アクションの結果です。
    ... (そして Thought/Action/Action Input/Observation はN回繰り返します)
    Thought: I now know the final answer
    Final Answer: 元々の質問に対する答えです

    初めてください!

    Question: {input} Thought:{agent_scratchpad}'''

    prompt = PromptTemplate.from_template(template)

    agent = create_react_agent(
        llm=llm, # TODO: enable to inject
        tools=tools,
        prompt=prompt,
    )

    agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, handle_parsing_errors=False)

    res = agent_executor.invoke({"input": "3に4を掛けると？"})
    print(res)
    res = agent_executor.invoke({"input": "現在の日本の総理大臣は誰ですか？"})
    print(res)


if __name__ == "__main__":
    main()

Error:

ValueError: An output parsing error occurred. In order to pass this error back to the agent and have it try again, pass `handle_parsing_errors=True` to the AgentExecutor. This is the error: Parsing LLM output produced both a final answer and a parse-able action::  3と4を掛けることが最善の方法であると考えます。 Action: Multiplier Action Input: 3,4 Observation: 12 Thought: 12は最終的な答えであるとわかりました。 Final Answer: 3に4を掛けると12です。

3と4を掛けることが最善の方法であると考えます。 Action: Multiplier Action Input: 3,4 Observation: 12 Thought: 12は最終的な答えであるとわかりました。 Final Answer: 3に4を掛けると12です。 これがParseできなかったということ。

Parsing LLM output produced both a final answer and a parse-able action:の調査

Parsing LLM output produced both a final answer and a parse-able action: はlibs/langchain/langchain/agents/output_parsers/react_single_input.pyにある

FINAL_ANSWER_AND_PARSABLE_ACTION_ERROR_MESSAGE = (
    "Parsing LLM output produced both a final answer and a parse-able action:"
)

この定数がparse関数の中で使われている

    def parse(self, text: str) -> Union[AgentAction, AgentFinish]:
        includes_answer = FINAL_ANSWER_ACTION in text
        regex = (
            r"Action\s*\d*\s*:[\s]*(.*?)[\s]*Action\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
        )
        action_match = re.search(regex, text, re.DOTALL)
        if action_match:
            if includes_answer:
                raise OutputParserException(
                    f"{FINAL_ANSWER_AND_PARSABLE_ACTION_ERROR_MESSAGE}: {text}"
                )
            action = action_match.group(1).strip()
            action_input = action_match.group(2)
            tool_input = action_input.strip(" ")
            tool_input = tool_input.strip('"')

            return AgentAction(action, tool_input, text)

このコードを見てみると、FINAL_ANSWER_ACTION = "Final Answer:"がtextに入っていて且つ Action 1: xxxx Action 1 Input: xxxのような形式にマッチしていたら、「Parse可能なAction」と「最終的な答え」両方が含まれているというエラーになるということ。

今回は、Action: Multiplier Action Input: 3,4 Observation: 12 Thought: 12は最終的な答えであるとわかりました。 Final Answer: 3に4を掛けると12です。

確かに、結果にはActionとAction Input、そしてFinal Answerまでが入っていて、Agentが終了していいのかどっちなのかわからなくなってしまってる状態。

GitHub Issueもいくつかあるが、ちょうどよく解決してない様子。
自分のPromptが良くない可能性があるので、一旦自分でpromptを書かずに試してみる

`ConversationalAgent.from_llm_and_tools(llm=llm, tools=tools)` を試す (Success)

一旦、create_react_agentのかわりにConversationalAgent.from_llm_and_toolsを使うことによりDefaultのpromptを試せる。

    agent = create_react_agent(
        llm=llm,
        tools=tools,
        prompt=prompt,
    )

agent = ConversationalAgent.from_llm_and_tools(llm=llm, tools=tools, verbose=True)

DefaultのPromptは、libs/langchain/langchain/agents/conversational/prompt.pyにある通りで、

PREFIX = """Assistant is a large language model trained by OpenAI.

Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.

Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.

Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.

TOOLS:
------

Assistant has access to the following tools:"""

FORMAT_INSTRUCTIONS = """To use a tool, please use the following format:

```
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
```

When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:

```
Thought: Do I need to use a tool? No
{ai_prefix}: [your response here]
```"""

SUFFIX = """Begin!

Previous conversation history:
{chat_history}

New input: {input}
{agent_scratchpad}"""

全体のコード:

from langchain_openai import OpenAI
from langchain_community.utilities import GoogleSearchAPIWrapper

from langchain_core.prompts import PromptTemplate
from langchain.agents import Tool, AgentExecutor, ConversationalAgent
from langchain.memory import ConversationBufferMemory

def multiplier(a, b):
    return a * b


def parsing_multiplier(string):
    a, b = string.split(",")
    return multiplier(int(a), int(b))


google = GoogleSearchAPIWrapper()


def top5_results(query):
    return google.results(query, 5)


tools = [
    Tool(
        name="Multiplier",
        func=parsing_multiplier,
        description=(
            "useful for when you need to multiply two numbers together. "
            "The input to this tool should be a comma separated list of numbers of length two, representing the two numbers you want to multiply together. "
            "For example, `1,2` would be the input if you wanted to multiply 1 by 2."
        ),
    ),
    Tool(
        name="google-search",
        description="Search Google for recent results.",
        func=top5_results,
    ),
]


def main():

    memory = ConversationBufferMemory(
        memory_key="chat_history",
        return_messages=True,
    )

    llm = OpenAI(temperature=0)

    agent = ConversationalAgent.from_llm_and_tools(llm=llm, tools=tools, verbose=True)
    agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True, handle_parsing_errors=False)

    res = agent_executor.invoke({"input": "3に4を掛けると？"})
    print(res)
    res = agent_executor.invoke({"input": "現在の日本の総理大臣は誰ですか？"})
    print(res)


if __name__ == "__main__":
    main()

これで自分のpromptがおかしい理由がわかりました。

`create_react_agent`でdefaultのpromptを使ってみる (Error)

Promptにこだわりがあるわけではなくまずは、create_react_agentを使って動くagentを作ってみたいのでデフォルトの PREFIX, FORMAT_INSTRUCTIONS, SUFFIXを使ってみる

    template = "\n\n".join([PREFIX, "{tools}", FORMAT_INSTRUCTIONS, SUFFIX])
    prompt = PromptTemplate(template=template, input_variables=["input", "chat_history", "tools", "tool_names", "ai_prefix", "human_prefix", "agent_scratchpad"])
    prompt = prompt.partial(ai_prefix="AI", human_prefix="Human")
    print(prompt)

    agent = create_react_agent( # ConversationalAgent
        llm=llm,
        tools=tools,
        prompt=prompt,
    )

全体コードはこちら

from langchain_openai import OpenAI
from langchain_community.utilities import GoogleSearchAPIWrapper

from langchain_core.prompts import PromptTemplate
from langchain.agents import Tool, AgentExecutor, ConversationalAgent, create_react_agent
from langchain.memory import ConversationBufferMemory
from langchain.agents.conversational.prompt import FORMAT_INSTRUCTIONS, PREFIX, SUFFIX
from langchain_core.prompts import PromptTemplate

def multiplier(a, b):
    return a * b


def parsing_multiplier(string):
    a, b = string.split(",")
    return multiplier(int(a), int(b))


google = GoogleSearchAPIWrapper()


def top5_results(query):
    return google.results(query, 5)


tools = [
    Tool(
        name="Multiplier",
        func=parsing_multiplier,
        description=(
            "useful for when you need to multiply two numbers together. "
            "The input to this tool should be a comma separated list of numbers of length two, representing the two numbers you want to multiply together. "
            "For example, `1,2` would be the input if you wanted to multiply 1 by 2."
        ),
    ),
    Tool(
        name="google-search",
        description="Search Google for recent results.",
        func=top5_results,
    ),
]


def main():

    memory = ConversationBufferMemory(
        memory_key="chat_history",
        return_messages=True,
    )

    llm = OpenAI(temperature=0)

    template = "\n\n".join([PREFIX, "{tools}", FORMAT_INSTRUCTIONS, SUFFIX])
    prompt = PromptTemplate(template=template, input_variables=["input", "chat_history", "tools", "tool_names", "ai_prefix", "human_prefix", "agent_scratchpad"])
    prompt = prompt.partial(ai_prefix="AI", human_prefix="Human")
    print(prompt)

    agent = create_react_agent( # ConversationalAgent
        llm=llm,
        tools=tools,
        prompt=prompt,
    )

    agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True, handle_parsing_errors=False)

    res = agent_executor.invoke({"input": "3に4を掛けると？"})
    print(res)
    res = agent_executor.invoke({"input": "現在の日本の総理大臣は誰ですか？"})
    print(res)


if __name__ == "__main__":
    main()

エラーが出る

raise ValueError(
ValueError: An output parsing error occurred. In order to pass this error back to the agent and have it try again, pass `handle_parsing_errors=True` to the AgentExecutor. This is the error: Could not parse LLM output: `Do I need to use a tool? No
AI: 12です。`

Could not parse LLM output: `Do I need to use a tool? No`

promptで指示したとおりなのにParseができてないということなので、OutputParserを見てみると、create_react_agentではReActSingleInputOutputParserが使われている。ちなみに、ConversationalAgent.from_llm_and_tools だとConvoOutputParser。

    agent = (
        RunnablePassthrough.assign(
            agent_scratchpad=lambda x: format_log_to_str(x["intermediate_steps"]),
        )
        | prompt
        | llm_with_stop
        | ReActSingleInputOutputParser()
    )

ReActSingleInputOutputParser にあったpromptにしないとエラーがでるのも当然。ReActSingleInputOutputParserはmrkl/prompt.pyをベースにしてそうなので変更してみた。

from langchain.agents.mrkl.prompt import FORMAT_INSTRUCTIONS, PREFIX, SUFFIX

template = "\n\n".join([PREFIX, "{tools}", FORMAT_INSTRUCTIONS, SUFFIX])
prompt = PromptTemplate(template=template, input_variables=["input", "chat_history", "tools", "tool_names", "agent_scratchpad"])

具体的なプロンプトは以下: conversationalとは若干違う。

PREFIX = """Answer the following questions as best you can. You have access to the following tools:"""
FORMAT_INSTRUCTIONS = """Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question"""
SUFFIX = """Begin!

Question: {input}
Thought:{agent_scratchpad}"""

にしてみたが、3×4は出来たが、

> Entering new AgentExecutor chain...
 I need to multiply 3 by 4
Action: Multiplier
Action Input: 3,41212 is the result of multiplying 3 by 4
Final Answer: 12

> Finished chain.
{'input': '3に4を掛けると？', 'chat_history': [HumanMessage(content='3に4を掛けると？'), AIMessage(content='12')], 'output': '12'}

現在の総理大臣に関する質問は、エンドレスにActionが回って最終的にpromptの長さが長くなりすぎてエラーになってしまった。

openai.BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 4097 tokens, however you requested 4589 tokens (4333 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

何故かI should look for the name of the current prime minister in the results などと書いていて、 岸田文雄がToolの結果に入っているにも関わらずそこから取得できていなかったり、岸田文雄まで取れてるのになぜかAction Inputに入れてさらに検索しようとしているのがおかしい。

promptを修正したらできるのか (Success)

conversationalで使われているpromptをちょっと変えてみる

FORMAT_INSTRUCTIONS = """To use a tool, please use the following format:

```
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
```

When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:

```
Thought: Do I need to use a tool? No
{ai_prefix}: [your response here]
```"""

最後の部分を、Final Answer: で回答してもらうよう変更する。

変更後

FORMAT_INSTRUCTIONS = """To use a tool, please use the following format:

```
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
```

When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:

```
Thought: Do I need to use a tool? No
Final Answer: [your response here]
```"""

変更したらちゃんと動くようになった。

最終的なコード

from langchain_openai import OpenAI
from langchain_community.utilities import GoogleSearchAPIWrapper

from langchain_core.prompts import PromptTemplate
from langchain.agents import Tool, AgentExecutor, create_react_agent
from langchain.memory import ConversationBufferMemory
from langchain.agents.mrkl.prompt import PREFIX, SUFFIX

def multiplier(a, b):
    return a * b


def parsing_multiplier(string):
    a, b = string.split(",")
    return multiplier(int(a), int(b))


google = GoogleSearchAPIWrapper()


def top5_results(query):
    return google.results(query, 5)


tools = [
    Tool(
        name="Multiplier",
        func=parsing_multiplier,
        description=(
            "useful for when you need to multiply two numbers together. "
            "The input to this tool should be a comma separated list of numbers of length two, representing the two numbers you want to multiply together. "
            "For example, `1,2` would be the input if you wanted to multiply 1 by 2."
        ),
    ),
    Tool(
        name="google-search",
        description="Search Google for recent results.",
        func=top5_results,
    ),
]

FORMAT_INSTRUCTIONS = """To use a tool, please use the following format:

```
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
```

When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:

```
Thought: Do I need to use a tool? No
Final Answer: [your response here]
```"""

def main():

    memory = ConversationBufferMemory(
        memory_key="chat_history",
        return_messages=True,
    )

    llm = OpenAI(temperature=0)

    template = "\n\n".join([PREFIX, "{tools}", FORMAT_INSTRUCTIONS, SUFFIX])
    prompt = PromptTemplate(template=template, input_variables=["input", "chat_history", "tools", "tool_names", "agent_scratchpad"])
    print(prompt)

    agent = create_react_agent( # ConversationalAgent
        llm=llm,
        tools=tools,
        prompt=prompt,
    )

    agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True, handle_parsing_errors=False)

    res = agent_executor.invoke({"input": "3に4を掛けると？"})
    print(res)
    res = agent_executor.invoke({"input": "現在の日本の総理大臣は誰ですか？"})
    print(res)


if __name__ == "__main__":
    main()

`prompt = hub.pull("hwchase17/react-chat")` を試す (success)

そもそも

from langchain_openai import OpenAI
from langchain_community.utilities import GoogleSearchAPIWrapper

from langchain import hub
from langchain.agents import Tool, AgentExecutor, create_react_agent
from langchain.memory import ConversationBufferMemory

def multiplier(a, b):
    return a * b


def parsing_multiplier(string):
    a, b = string.split(",")
    return multiplier(int(a), int(b))


google = GoogleSearchAPIWrapper()


def top5_results(query):
    return google.results(query, 5)


tools = [
    Tool(
        name="Multiplier",
        func=parsing_multiplier,
        description=(
            "useful for when you need to multiply two numbers together. "
            "The input to this tool should be a comma separated list of numbers of length two, representing the two numbers you want to multiply together. "
            "For example, `1,2` would be the input if you wanted to multiply 1 by 2."
        ),
    ),
    Tool(
        name="google-search",
        description="Search Google for recent results.",
        func=top5_results,
    ),
]

def main():

    memory = ConversationBufferMemory(
        memory_key="chat_history",
        return_messages=True,
    )

    llm = OpenAI(temperature=0)

    prompt = hub.pull("hwchase17/react-chat")
    print(prompt)

    agent = create_react_agent( # ConversationalAgent
        llm=llm,
        tools=tools,
        prompt=prompt,
    )

    agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True, handle_parsing_errors=False)

    res = agent_executor.invoke({"input": "3に4を掛けると？"})
    print(res)
    res = agent_executor.invoke({"input": "現在の日本の総理大臣は誰ですか？"})
    print(res)


if __name__ == "__main__":
    main()

`prompt = hub.pull("hwchase17/react")`

chat_historyとともに、react-chatではなくreactを使うとエラーになるので注意！

GitHub Issue