By: Samuel Chan · August 1, 2024
Table of Content
llama3-groq-70b-8192-tool-use-preview
, which is state-of-the-art and finetuned specifically for tool use.Llama-3-Groq-70B-Tool-Use is the highest performing model on the Berkeley Function Calling Leaderboard (BFCL), outperforming all other open source and proprietary models. Our models have achieved remarkable results, setting new benchmarks for Large Language Models with tool use capabilities:
- Llama-3-Groq-70B-Tool-Use: 90.76% overall accuracy (#1 on BFCL at the time of publishing)
- Llama-3-Groq-8B-Tool-Use: 89.06% overall accuracy (#3 on BFCL at the time of publishing)
Tip: Other tool-use models
dotenv
to load my keys from a .env
file. You can install it by running pip install python-dotenv
and create a .env
file in the same directory as your Python script with the following content:
retrieve_from_endpoint
is just a convenience function
that we can use to retrieve data from the Sectors API. We are creating
an abstraction layer so we can re-use this function throughout our script.
langchain
provides a Tool
class as well as a tool
decorator that we can use to
wrap any Python function and turn it into a tool. Here’s how we use it to create a grand
total of three tools:
llama3-groq-70b-8192-tool-use-preview
model from Groq. Not only
is this model state-of-the-art and specializes in tool use, it is also open source and
more easily accessible than other proprietary models.
In the following code, we instantiate a ChatGroq
object with the model name and the Groq API key.
This provides us with the llm
object that we’ll pass, along with tools
created earlier, to our
orchestrator agent. What’s missing is the a prompt template that we’ll use to
specify some system constraints and provide some overall guidance to the LLM.
ChatPromptTemplate
object that contains a message for the system and a message for the user. The system message provides some guidance to the LLM on how to respond to the user’s input.llm
, tools
, and prompt
into an agent
object. This agent is responsible for orchestrating the interaction between the user, the LLM, and the tools.AgentExecutor
object that will execute the agent and handle the interaction between the user and the system. Setting verbose
to
True
will print out the system’s responses to the user’s queries and additional messages that the system might generate.query_4
and query_5
, you will need to implement additional tools that can retrieve historical performance of a stock
since its IPO listing. This section: Company’s Performance since IPO of the API Documentation
should provide you with the necessary information to extend your RAG system with this capability.
query_1
, we asked “What are the top 3 companies by transaction volume over the last 7 days?” and the system responded with:
get_top_companies_by_tx_volume
requires a start and end date to retrieve the data. It then correctly
retrieves the data and have to perform a reduction operation to find the top 3 companies by transaction volume.
As a bonus, you may also want to provide a calculator tool that your RAG can use to perform the necessary summation
and sorting operations to find the top 3 companies by transaction volume. In cases where it fail to interpret the intent
correctly, your RAG system might instead return the top 3 entries for the last 7 days, which is probably a good approximation
but not fully correct.
On query_2
, we asked “Based on the closing prices of BBCA between 1st and 30th of June 2024, are we seeing an uptrend or downtrend? Try to explain why.”
Our RAG model return with the following response:
query_3
is considerably trickier as it requires the system to retrieve the market cap for both BBCA and BREN, compare them, and then retrieve additional information for the company with the largest market cap.
Our original query is “What is the company with the largest market cap between BBCA and BREN? For said company, retrieve the email, phone number, listing date and website for further research.” and
our RAG system responded with:
query_4
and query_5
, you should expect a response similar to the one below: