🪛 Features as Tools

This tutorial guides you through creating an LLM generated restaurant recommendation function by using Features as Tools.

This is an example of how an LLM can make use of feature views as data retrieval tools. At development time, you provide the set of feature views that the LLM agent is allowed to call. The LLM uses the feature view’s description field to determine what tools are relevant for answering the user question. It makes the corresponding feature retrieval call(s) using the Tecton entity keys provided in the context of the call.

Install Packages

_ = !pip install 'tecton-gen-ai[tecton,langchain,llama-index,dev]' langchain-openai llama-index-llms-openai

Setup

from tecton_gen_ai.testing import set_dev_mode

set_dev_mode()

Features as Tools

In the following cell you’ll create a Tecton Agent with a simple prompt and two different feature views for the LLM as tools. The system prompt in this example, instructs the LLM to address the user by name and to only suggest restaurants that in near the location. In order to address the user by name the LLM will need to use a tool. The location is configured to be provided as part of the context in the request by using a RequestSource object.

from tecton_gen_ai.fco import prompt, AgentService, AgentClient
from tecton_gen_ai.utils.tecton_utils import make_request_source

def restaurant_recommender_agent( user_info_fv, recent_visit_fv):    

    location_request = make_request_source(location = str)
    
    @prompt(sources=[ location_request])
    def sys_prompt(location_request ):
        location = location_request["location"]
        return f"""
        Address the user by name.
        You are a consierge service that recommends restaurants.
        Respond to the user's question. 
        If the user asks for a restaurant recommendation respond with a specific restaurant that you know and suggested menu items. 
        Only suggest restaurants that are in or near {location}. 
        """
            
    return AgentService(
        name="restaurant_recommender",
        prompts=[sys_prompt],
        tools=[ user_info_fv, recent_visit_fv],  # feature views used as tools
    )

The sys_prompt instructions above require the LLM to “address the user by name” which requires the use of a tool.

In the next cell we create a couple of mock feature views that provide basic user information and the list of their recently visited restaurants respectively.

In practice, these feature views would be implemented as stream feature views that would provide the latest user information and recent visits within seconds of the corresponding user activity.

from tecton_gen_ai.testing import make_local_batch_feature_view


# mock user info data
user_info = [
    {"user_id": "user1", "name": "Jim",  "age": 30, "food_preference": "American"},
    {"user_id": "user2", "name": "John", "age": 40, "food_preference": "Italian"},
    {"user_id": "user3", "name": "Jane", "age": 50, "food_preference": "Chinese"},
]
user_info_fv = make_local_batch_feature_view(
    "user_info_fv",
    user_info,
    ["user_id"],
    description="User's basic information."
)

# mock user's recent visits 
recent_eats = [
    {"user_id": "user1", "last_3_visits":str(["Mama Ricotta's", "The Capital Grille", "Firebirds Wood Fired Grill"])},
    {"user_id": "user2", "last_3_visits":str(["Mama Ricotta's", "Villa Antonio", "Viva Chicken"])},
    {"user_id": "user3", "last_3_visits":str(["Wan Fu", "Wan Fu Quality Chinese Cuisine", "Ru San's"])},
]
recent_eats_fv = make_local_batch_feature_view(
    "recent_eats_fv",
    recent_eats,
    entity_keys=["user_id"],
    description="User's recent restaurant visits."
)

The feature views entity key in both cases is user_id, the key will need to be provided in the context when invoking the LLM agent.

In the following cell, you create a Tecton AgentClient with uses the mock feature views defined above.

# create the Tecton Agent
recommender_agent = restaurant_recommender_agent( user_info_fv, recent_eats_fv )

# create a client to invoke the agent
client = AgentClient.from_local( recommender_agent )

Put it all together

The Tecton AgentClient created above can be used to create a LangChain or LlamaIndex runnable agent.

In the cell below you will instantiate an LLM model using OpenAI’s gpt-4o-mini model and create a LangChain agent that is ready to use the tools we’ve provided above. In the following section, you’ll do the same using a LlamaIndex.

Obtain an OpenAI API key and replace “your-openai-key” in the following cell.

import os
from langchain_openai import ChatOpenAI
from tecton_gen_ai.testing.utils import print_md


# replace with your key
# os.environ["OPENAI_API_KEY"] = "your-openai-key"

# instantiate LLM model for  LangChain 
langchain_llm = ChatOpenAI(model="gpt-4o-mini")

# create invocable agent for LangChain 
langchain_agent = client.make_agent(langchain_llm, system_prompt="sys_prompt")

Testing with different context

In the following cells you’ll provide different context and invoke the LLM agent to test how features as tools provide user specific personalization.

The first test uses the langchain_agent, it’s input and output are a dictionaries with “input” and “output” keys:

print("Context - user 1 in Charlotte \n\nResponse:")
with client.set_context({"user_id": "user1", "location":"Ballantyne, Charlotte, NC"}):
    print_md(langchain_agent.invoke({"input":"what do you know about me and what would you recommend"})["output"])

Context - user 1 in Charlotte 

Response:

Hi Jim! Based on your recent visits, it looks like you enjoy American cuisine. I recommend trying The Cheesecake   
Factory in Ballantyne. They have a diverse menu, but here are some specific items you might enjoy:                 
 • Bang Bang Chicken and Shrimp: A delicious, spicy dish with crispy chicken and shrimp.                           
 • Factory Burrito Grande: A massive burrito filled with your choice of protein and topped with sour cream and     
   guacamole.                                                                                                      
 • Godiva® Cheesecake: For dessert, you can't go wrong with this rich and indulgent cheesecake.                    
Enjoy your meal!                                                                                                   

Different Context for Different Users

In the following cells you can see how the response changes based on the user_id and the location provided resulting in a personalized response for each user and based on their current location.

The following test uses the llamaindex_agent with its chat method that receiving a query string as input and returns output as a string as well:

print( "Context - user 2 in Charlotte \n\nResponse:")
with client.set_context({"user_id": "user2", "location":"Ballantyne, Charlotte, NC"}):
    print_md(langchain_agent.invoke({"input":"I want to try a place tonight that I've never been to that also fits my preference"})["output"])

Context - user 2 in Charlotte 

Response:

Hi John! Since you're in the mood for Italian and want to try something new, I recommend checking out Bistro La Bon
in Ballantyne.                                                                                                     
You might enjoy their Pasta Primavera or the Osso Buco, and don’t forget to try their delicious Tiramisu for       
dessert! It’s a cozy place with a great atmosphere. Enjoy your dinner!                                             

print( "Context - user 3 in Charlotte \n\nResponse:")
with client.set_context({"user_id": "user3", "location":"Charlotte, NC"}):
    print_md(langchain_agent.invoke({"input":"Recommend one of my regular spots for dinner"})["output"])

Context - user 3 in Charlotte 

Response:

Hi Jane! I see that you've been visiting some great spots. Since you enjoy Chinese cuisine, I recommend trying Wan 
Fu Quality Chinese Cuisine if you haven't been there recently. Their Kung Pao Chicken and Peking Duck are          
particularly delicious. Enjoy your dinner!                                                                         

With LlamaIndex

Tecton also integrates with LlamaIndex in the same way it does with LangChain in order to provide enriched prompts and features as tools.

The only difference is in the LangChain.invoke vs LlamaIndex.chat methods and their signatures. Tecton delivers fresh context in the same way for both:

from llama_index.llms.openai import OpenAI

# instantiate LLM model for with LlamaIndex integration 
llamaindex_llm = OpenAI(model="gpt-4o-mini-2024-07-18")

# create invocable agent for LlamaIndex 
llamaindex_agent = client.make_agent(llamaindex_llm, system_prompt="sys_prompt")

print( "Context - user 3 in Charlotte \n\nResponse:")
with client.set_context({"user_id": "user3", "location":"Charlotte, NC"}):
    print_md(llamaindex_agent.chat("Recommend one of my regular spots for dinner").response)

Context - user 3 in Charlotte 

Response:

It looks like you've recently visited Wan Fu and Ru San's. If you're in the mood for some delicious sushi, I       
recommend heading back to Ru San's. Their spicy tuna roll and the salmon sashimi are particularly popular! Enjoy   
your dinner!                                                                                                       

Notice that in the LlamaIndex case, given that it is stateful, a new instance of the agent is needed if the context is different between calls.

# create new instance of agent for LlamaIndex 
llamaindex_agent = client.make_agent(llamaindex_llm, system_prompt="sys_prompt")

print( "Context - user 2 in New York \n\nResponse:")
with client.set_context({"user_id": "user2", "location":"New York, NY"}):
    print_md(llamaindex_agent.chat("something romantic for dinner").response)

Context - user 2 in New York 

Response:

John, for a romantic dinner in New York, I recommend Carbone. It's an upscale Italian restaurant known for its     
classic dishes and intimate atmosphere.                                                                            
You might enjoy their Spicy Rigatoni Vodka, which is a fan favorite, and the Veal Parmesan is a must-try as well.  
Don't forget to finish off with their delicious Tiramisu for dessert. Enjoy your evening!                          

Conclusion

Tecton delivers real-time, streaming and batch feature pipelines in production. Enhancing an LLMs context through features as tools, brings the power of fresh context to generative AI applications. By using features as tools for the LLM, you give the LLM the ability to retrieve relevant information only when the user question needs requires it.