📚 Using Prompts, Features as Tools and Knowledge
By combining the Tecton GenAI functions, you can provide much better context in for your LLM applications. - The knowledge function provides RAG functionality by taking unstructured text and calculating embedding vectors for them and storing them in a vector database that provides similarity search capabilities to better respond to user queries. - The prompt provides instructions for the LLM and is enriched with context from feature pipelines or directly from the real-time context. - Features as tools give the LLM the ability to retrieve additional data as needed to respond to the user question.
In this notebook, you will combine all three tools to create an example restaurant recommendation chatbot.
Install Packages
!pip install 'tecton-gen-ai[tecton,langchain,llama-index,dev]' langchain-openai llama-index-llms-openai -q
Log in to Tecton
Make sure to hit enter after pasting in your authentication token.
import tecton
tecton.login("explore.tecton.ai")
In this example you will use an OpenAI gpt-4o LLM model to test.
You will need to provide an Open AI API key for this purpose.
Replace “your-openai-key” with your own key.
import os
# replace with your key
os.environ["OPENAI_API_KEY"] = "your-openai-key"
from tecton_gen_ai.fco import prompt
from tecton_gen_ai.agent import AgentClient, AgentService
# prompt without any user context
@prompt()
def sys_prompt( ):
return f"""
You are a consierge service that recommends restaurants.
Always provide an address.
Always suggested menu items.
"""
#define the Tecton Agent Service specifying the prompts, sources of knowledge and feature tools
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
)
from langchain_openai import ChatOpenAI
def support_chatbot(service, user_query, context=None):
from tecton_gen_ai.testing.utils import print_md
#create an agent client that provides context for LLM workflows
client = AgentClient.from_local(service)
# instantiate LLM model for LangChain
langchain_llm = ChatOpenAI(model="gpt-4o-mini")
# create invocable agent for LangChain
langchain_agent = client.make_agent(langchain_llm, system_prompt="sys_prompt")
with client.set_context(context):
print_md(langchain_agent.invoke({"input":user_query})["output"])
support_chatbot(service, "recommend a restaurant for tonight")
Sure! How about trying Carmine's Italian Restaurant? It's a great spot for delicious Italian cuisine. Address: 200 W 44th St, New York, NY 10036 Suggested Menu Items: • Start with the Bruschetta as an appetizer. • For the main course, you can't go wrong with their Famous Lasagna or the Chicken Parmigiana. • Finish off with a slice of their classic Tiramisu for dessert. Enjoy your dinner!
A Feature Enriched Prompt
import pandas as pd
from tecton_gen_ai.testing import make_local_batch_feature_view
from tecton_gen_ai.utils.tecton import make_request_source
# what if we had more up to date information about the user, like their name, age and cuisines preference
user_info = pd.DataFrame(
[
{"user_id": "user1", "name": "Jim", "age": 30, "food_preference": "American"},
{"user_id": "user2", "name": "John", "age": 40, "food_preference": "Italian"},
{"user_id": "user3", "name": "Jane", "age": 50, "food_preference": "Chinese"},
]
)
user_info_fv = make_local_batch_feature_view( "user_info_fv", user_info, entity_keys=["user_id"], description="User's basic information.")
# and what if we knew their current location
location_request = make_request_source(location = str)
#we could create a prompt like this
@prompt(sources=[ location_request, user_info_fv]) # specifies sources of data for the prompt
def sys_prompt(location_request, user_info_fv ):
location = location_request["location"]
name = user_info_fv["name"]
return f"""
Address the user by their name. Their name is {name}.
You are a consierge service that recommends restaurants.
Only suggest restaurants that are in or near {location}.
Always provide an address.
Always suggested menu items.
"""
#define the Tecton Agent Service specifying the prompts, sources of knowledge and feature tools
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
)
mobile_app_context={"user_id":"user3", "location":"Charlotte, NC"}
support_chatbot(service, "recommend a restaurant for tonight", mobile_app_context)
Sure, Jane! I recommend trying The Capital Grille. It's a fantastic place for a fine dining experience. Address: 201 N Tryon St, Charlotte, NC 28202 Suggested Menu Items: • Dry Aged Porterhouse • Seared Tenderloin with Butter Poached Lobster • Lobster Mac 'n' Cheese • Vanilla Bean Crème Brûlée for dessert Enjoy your evening!
# but now what if the question requires more data
mobile_app_context={"user_id":"user1", "location":"Charlotte, NC"}
support_chatbot(service, "recommend a restaurant for tonight based on my preference", mobile_app_context)
Of course, Jim! Could you please share your preferred cuisine or any specific dishes you enjoy? That way, I can recommend a restaurant that best fits your taste!
Features as Tools
#define the Tecton Agent Service specifying the prompts, sources of knowledge and feature tools
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
tools=[user_info_fv] # user_info_fv provides food preference and recent_eats_fv provides recent dining
)
# we should get a much better answer
support_chatbot(service, "recommend a restaurant for tonight based on my preference", {"user_id":"user1", "location":"Charlotte, NC"})
Here’s a great restaurant for you tonight, Jim:
The Capital Grille
• Address: 201 N Tryon St, Charlotte, NC 28202
• Recommended Menu Items:
• Dry Aged Porterhouse Steak
• Pan-Seared Tenderloin
• Lobster Mac 'n' Cheese
• Freshly Made Cheesecake
Enjoy your meal!
# user's recent visits
recent_eats = pd.DataFrame(
[
{"user_id": "user1", "last_3_visits":str(["Mama Ricotta's", "The Capital Grille", "Firebirds Wood Fired Grill"])},
{"user_id": "user2", "last_3_visits":str(["Mama Ricotta's", "Villa Antonio", "Viva Chicken"])},
{"user_id": "user3", "last_3_visits":str(["Wan Fu", "Wan Fu Quality Chinese Cuisine", "Ru San's"])},
]
)
recent_eats_fv = make_local_batch_feature_view( "recent_eats_fv", recent_eats, entity_keys=["user_id"], description="User's recent restaurant visits.")
#define the Tecton Agent Service specifying the prompts, sources of knowledge and feature tools
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
tools=[user_info_fv, recent_eats_fv] # recent_eats_fv provides recent dining
)
# if the user asks a more complex question...
mobile_app_context = {"user_id":"user3", "location":"Charlotte, NC"}
support_chatbot(service, "I need a new restaurant for tonight", mobile_app_context)
Hi Jane! Since you're looking for a new restaurant, how about trying Szechuan Palace? It's a great spot for Chinese cuisine. Address: 2200 W Trade St, Charlotte, NC 28208 Recommended menu items: • Kung Pao Chicken • Mapo Tofu • Szechuan Beef Enjoy your dinner!
Knowledge
Adding knowledge to an LLM provides domain specific information that extends beyond its base training. With it, the LLM is able to select relevant text that helps respond to user queries and avoid hallucination. Under the covers, Tecton calculates embedding vectors and stores the data in a vector search database which the LLM workflow agent can use to do semantic search. With Tecton knowledge, other fields can be used to filter the dataset prior to executing the semantic search which provides even more relevance to the results.
# Let's say we only want to use restaurants that have signed up for the service,
# and not those that the general LLM training knows about
#change the prompt instructions
@prompt(sources=[ location_request, user_info_fv]) # specifies sources of data for the prompt
def sys_prompt(location_request, user_info_fv ):
location = location_request["location"]
name = user_info_fv["name"]
return f"""
Address the user by their name. Their name is {name}.
You are a consierge service that recommends restaurants.
Only suggest restaurants that are in or near {location}.
Always provide an address.
Always suggested menu items.
Only select restaurants that have signed up on the service.
""" # notice that we've added an instruction
#example knowledge about restaurants that have signed up
restaurants_signed_up = pd.DataFrame([
{
"restaurant_name": "Mama Ricotta's",
"zipcode": "28203",
"cuisine": "Italian",
"description": "A Charlotte staple since 1992, Mama Ricotta's offers authentic Italian cuisine in a warm, family-friendly atmosphere. Known for their hand-tossed pizzas, homemade pasta, and signature chicken parmesan."
},
{
"restaurant_name": "The Capital Grille",
"zipcode": "28202",
"cuisine": "American Steakhouse",
"description": "An upscale steakhouse chain featuring dry-aged steaks, fresh seafood, and an extensive wine list. The Charlotte location offers a refined dining experience with impeccable service and a sophisticated ambiance."
},
{
"restaurant_name": "Firebirds Wood Fired Grill",
"zipcode": "28277",
"cuisine": "American",
"description": "A polished casual restaurant known for its classic American cuisine cooked over an open wood fire. Specialties include hand-cut steaks, fresh seafood, and signature cocktails in a warm, contemporary setting."
},
{
"restaurant_name": "Villa Antonio",
"zipcode": "28210",
"cuisine": "Italian",
"description": "An elegant Italian restaurant offering a romantic atmosphere and authentic cuisine. Known for its homemade pasta, extensive wine selection, and attentive service, perfect for special occasions."
},
{
"restaurant_name": "Viva Chicken",
"zipcode": "28203",
"cuisine": "Peruvian",
"description": "A fast-casual eatery specializing in Peruvian-style rotisserie chicken. Offers fresh, flavorful dishes with a modern twist on traditional Peruvian cuisine, including quinoa stuffed avocados and yuca fries."
},
{
"restaurant_name": "Wan Fu",
"zipcode": "28226",
"cuisine": "Chinese",
"description": "A local favorite for Chinese cuisine, Wan Fu offers a wide array of traditional and Americanized Chinese dishes. Known for its generous portions, friendly service, and comfortable dining atmosphere."
},
{
"restaurant_name": "Wan Fu Quality Chinese Cuisine",
"zipcode": "28277",
"cuisine": "Chinese",
"description": "An upscale Chinese restaurant focusing on authentic flavors and high-quality ingredients. Offers a more refined dining experience with a menu featuring both classic and innovative Chinese dishes."
},
{
"restaurant_name": "Ru San's",
"zipcode": "28203",
"cuisine": "Japanese",
"description": "A popular sushi restaurant known for its extensive menu and all-you-can-eat option. Offers a wide variety of sushi rolls, sashimi, and other Japanese dishes in a casual, vibrant atmosphere."
},
{
"restaurant_name": "Le Bernardin",
"zipcode": "10019",
"cuisine": "French Seafood",
"description": "A world-renowned, Michelin three-star restaurant specializing in exquisite seafood. Chef Eric Ripert's menu features innovative preparations of the finest global seafood in an elegant setting."
},
{
"restaurant_name": "Katz's Delicatessen",
"zipcode": "10002",
"cuisine": "Jewish Deli",
"description": "An iconic New York institution since 1888, famous for its hand-carved pastrami and corned beef sandwiches. The bustling, no-frills atmosphere is part of its enduring charm."
},
{
"restaurant_name": "Eleven Madison Park",
"zipcode": "10010",
"cuisine": "Contemporary American",
"description": "A three-Michelin-starred restaurant offering an innovative tasting menu focusing on locally sourced, plant-based ingredients. Known for its impeccable service and artistic presentation."
},
{
"restaurant_name": "Peter Luger Steak House",
"zipcode": "11211",
"cuisine": "Steakhouse",
"description": "A Brooklyn institution since 1887, Peter Luger is famous for its dry-aged steaks and old-school, cash-only policy. The no-frills atmosphere focuses attention on the exceptional quality of the meat."
},
{
"restaurant_name": "Di Fara Pizza",
"zipcode": "11230",
"cuisine": "Pizza",
"description": "A legendary Brooklyn pizzeria, Di Fara is known for its handcrafted pies made by founder Dom DeMarco. Each pizza is a work of art, featuring high-quality ingredients and meticulous preparation."
},
{
"restaurant_name": "Balthazar",
"zipcode": "10012",
"cuisine": "French Brasserie",
"description": "A SoHo classic, Balthazar offers authentic French brasserie fare in a vibrant, bustling atmosphere. Known for its fresh seafood, classic French dishes, and popular weekend brunch."
},
{
"restaurant_name": "Momofuku Ko",
"zipcode": "10003",
"cuisine": "Contemporary Asian",
"description": "Chef David Chang's two-Michelin-starred restaurant offers an ever-changing tasting menu blending Asian and Western influences. The intimate counter seating provides a unique, interactive dining experience."
},
{
"restaurant_name": "The Halal Guys",
"zipcode": "10019",
"cuisine": "Middle Eastern",
"description": "Starting as a food cart, The Halal Guys has become a New York institution. Famous for its chicken and gyro over rice, topped with their legendary white sauce. Now with brick-and-mortar locations."
},
{
"restaurant_name": "Russ & Daughters",
"zipcode": "10002",
"cuisine": "Jewish Appetizing",
"description": "A New York classic since 1914, specializing in traditional Jewish appetizing foods. Famous for its hand-sliced smoked salmon, bagels, and other Jewish delicacies. Now includes a sit-down cafe."
},
{
"restaurant_name": "Lombardi's",
"zipcode": "10012",
"cuisine": "Pizza",
"description": "America's first pizzeria, established in 1905. Lombardi's continues to serve classic New York-style pizza from its coal-fired oven. Known for its simple, high-quality ingredients and historic charm."
},
{
"restaurant_name": "Joe's Shanghai",
"zipcode": "10013",
"cuisine": "Chinese",
"description": "Famous for introducing soup dumplings to New York, Joe's Shanghai offers authentic Shanghai-style cuisine. The bustling, no-frills atmosphere adds to the authentic experience."
}
]
)
from tecton_gen_ai.testing import make_local_source
from tecton_gen_ai.testing.utils import create_testing_vector_db_config
from tecton_gen_ai.fco import source_as_knowledge
#provide a vector db config
conf = create_testing_vector_db_config("/tmp/test.db", remove_if_exists=True)
#create embeddings of the restaurant descriptions in the vector DB
src = make_local_source("restaurants_signed_up", restaurants_signed_up, auto_timestamp=True, description="Restaurants that signed up for the service")
restaurant_knowledge = source_as_knowledge(
src,
vector_db_config=conf,
vectorize_column="description",
filter = [("zipcode", str, "the zip code for the restaurant"), ("cuisine", str, "the cuisine that they serve")]
)
#rebuild the service with restaurant knowledge included
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
knowledge=[restaurant_knowledge], # added a source of knowledge
tools=[user_info_fv, recent_eats_fv]
)
# test the app
mobile_app_context = {"user_id":"user3", "location":"28277"}
support_chatbot(service, "recommend a restaurant I have not visited with dry-aged steaks and tiramisu", mobile_app_context)
I found a great restaurant that offers dry-aged steaks and tiramisu, and it looks like you haven't visited it yet:
Firebirds Wood Fired Grill
• Address: 10805 S Tryon St, Charlotte, NC 28273
• Description: A polished casual restaurant known for its classic American cuisine cooked over an open wood fire.
Specialties include hand-cut steaks, fresh seafood, and signature cocktails in a warm, contemporary setting.
• Suggested Menu Items:
• Dry-Aged Ribeye
• Tiramisu
Enjoy your dining experience, Jane!
# test for a zipcode that is not close to any in the apps knowledge
mobile_app_context = {"user_id":"user1", "location":"10200"}
support_chatbot(service, "recommend a new restaurant with dry-aged steaks and Tiramisu", mobile_app_context)
It looks like there are currently no restaurants in or near 10200 that serve dry-aged steaks or Tiramisu. If you have other preferences or types of cuisine in mind, please let me know, and I'll be happy to help you find a suitable option!
# test for recent visits with item they do not serve
mobile_app_context = {"user_id":"user2", "location":"28277"}
support_chatbot(service, "recommend a one of my recent visits that has Tiramisu", mobile_app_context)
Here are your recent visits with Tiramisu options: 1 Mama Ricotta's • Cuisine: Italian • Address: 28203 (specific address not provided) • Description: A Charlotte staple since 1992, Mama Ricotta's offers authentic Italian cuisine in a warm, family-friendly atmosphere. Known for their hand-tossed pizzas, homemade pasta, and signature chicken parmesan. • Suggested Menu Item: Tiramisu 2 Villa Antonio • Cuisine: Italian • Address: 28210 (specific address not provided) • Description: An elegant Italian restaurant offering a romantic atmosphere and authentic cuisine. Known for its homemade pasta, extensive wine selection, and attentive service, perfect for special occasions. • Suggested Menu Item: Tiramisu Unfortunately, Viva Chicken does not offer Tiramisu, as it specializes in Peruvian-style dishes. If you would like more details or assistance with reservations at either of these restaurants, just let me know!
