📚 Using Prompts, Features as Tools and Knowledge
By combining the Tecton GenAI functions, you can provide much better context in for your LLM applications. - The knowledge function provides RAG functionality by taking unstructured text and calculating embedding vectors for them and storing them in a vector database that provides similarity search capabilities to better respond to user queries. - The prompt provides instructions for the LLM and is enriched with context from feature pipelines or directly from the real-time context. - Features as tools give the LLM the ability to retrieve additional data as needed to respond to the user question.
In this notebook, you will combine all three tools to create an example restaurant recommendation chatbot.
Install Packages
_ = !pip install 'tecton-gen-ai[tecton,langchain,llama-index,dev]' langchain-openai llama-index-llms-openai -q
Setup
from tecton_gen_ai.testing import set_dev_mode
set_dev_mode()
In this example you will use an OpenAI gpt-4o LLM model to test.
You will need to provide an Open AI API key for this purpose.
Replace “your-openai-key” with your own key.
import os
# replace with your key
os.environ["OPENAI_API_KEY"] = "your-openai-key"
from tecton_gen_ai.fco import prompt
from tecton_gen_ai.agent import AgentClient, AgentService
# prompt without any user context
@prompt()
def sys_prompt( ):
return f"""
You are a consierge service that recommends restaurants.
Always provide an address.
Always suggested menu items.
"""
#define the Tecton Agent Service specifying the prompts, sources of knowledge and feature tools
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
)
from langchain_openai import ChatOpenAI
def support_chatbot(service, user_query, context=None):
from tecton_gen_ai.testing.utils import print_md
#create an agent client that provides context for LLM workflows
client = AgentClient.from_local(service)
# instantiate LLM model for LangChain
langchain_llm = ChatOpenAI(model="gpt-4o-mini-2024-07-18", temperature=0.5)
# create invocable agent for LangChain
langchain_agent = client.make_agent(langchain_llm, system_prompt="sys_prompt")
with client.set_context(context):
print_md(langchain_agent.invoke({"input":user_query})["output"])
support_chatbot(service, "recommend a restaurant for tonight")
Sure! I recommend trying "The Rustic" located at 3656 Howell St, Dallas, TX 75204. Menu highlights include: • Brisket Tacos: Tender brisket with fresh toppings. • Chicken & Waffles: A delightful combination of crispy fried chicken and fluffy waffles. • Pork Chop: Grilled to perfection, served with seasonal vegetables. • House-made S'mores: A sweet ending with toasted marshmallows and chocolate. Enjoy your meal!
A Feature Enriched Prompt
from tecton_gen_ai.testing import make_local_batch_feature_view
from tecton_gen_ai.utils.tecton_utils import make_request_source
# what if we had more up to date information about the user, like their name, age and cuisines preference
user_info = [
{"user_id": "user1", "name": "Jim", "age": 30, "food_preference": "American"},
{"user_id": "user2", "name": "John", "age": 40, "food_preference": "Italian"},
{"user_id": "user3", "name": "Jane", "age": 50, "food_preference": "Chinese"},
]
user_info_fv = make_local_batch_feature_view(
"user_info_fv",
user_info,
["user_id"],
description="User's basic information."
)
# and what if we knew their current location
location_request = make_request_source(location = str)
#we could create a prompt like this
@prompt(sources=[ location_request, user_info_fv]) # specifies sources of data for the prompt
def sys_prompt(location_request, user_info_fv ):
return f"""
Address the user by their name. Their name is {user_info_fv['name']}.
You are a consierge service that recommends restaurants.
Only suggest restaurants that are in or near {location_request['location']}.
Always provide an address.
Always suggested menu items.
"""
#define the Tecton Agent Service specifying the prompts, sources of knowledge and feature tools
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
)
mobile_app_context={"user_id":"user3", "location":"Charlotte, NC"}
support_chatbot(service, "recommend a restaurant for tonight", mobile_app_context)
Hi Jane! I recommend trying The Capital Grille. It's a fantastic steakhouse known for its dry-aged steaks and extensive wine list. Address: 201 N Tryon St, Charlotte, NC 28202 Suggested Menu Items: • Bone-in Ribeye • Filet Mignon • Lobster Mac 'n' Cheese • 24 oz. Porterhouse Enjoy your dinner!
# but now what if the question requires more data
mobile_app_context={"user_id":"user1", "location":"Charlotte, NC"}
support_chatbot(service, "recommend a restaurant for tonight based on my preference", mobile_app_context)
Sure thing, Jim! Could you please share your preferences? For example, do you have a specific cuisine in mind, or are you looking for something casual or more upscale?
Features as Tools
#define the Tecton Agent Service specifying the prompts, sources of knowledge and feature tools
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
tools=[user_info_fv] # user_info_fv provides food preference and recent_eats_fv provides recent dining
)
# we should get a much better answer
support_chatbot(service, "recommend a restaurant for tonight based on my preference", {"user_id":"user1", "location":"Charlotte, NC"})
Here are a couple of American restaurants in Charlotte, NC that you might enjoy, Jim: 1 The Capital Grille • Address: 201 N Tryon St, Charlotte, NC 28202 • Recommended Menu Items: Dry Aged Porterhouse, Lobster Mac 'N' Cheese, and the Flourless Chocolate Cake. 2 Midwood Smokehouse • Address: 1401 Central Ave, Charlotte, NC 28205 • Recommended Menu Items: Brisket, Pulled Pork, and the Mac & Cheese. Enjoy your dinner!
# user's recent visits
recent_eats = [
{"user_id": "user1", "last_3_visits":str(["Mama Ricotta's", "The Capital Grille", "Firebirds Wood Fired Grill"])},
{"user_id": "user2", "last_3_visits":str(["Mama Ricotta's", "Villa Antonio", "Viva Chicken"])},
{"user_id": "user3", "last_3_visits":str(["Wan Fu", "Wan Fu Quality Chinese Cuisine", "Ru San's"])},
]
recent_eats_fv = make_local_batch_feature_view( "recent_eats_fv", recent_eats, entity_keys=["user_id"], description="User's recent restaurant visits.")
#define the Tecton Agent Service specifying the prompts, sources of knowledge and feature tools
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
tools=[user_info_fv, recent_eats_fv] # recent_eats_fv provides recent dining
)
# if the user asks a more complex question...
mobile_app_context = {"user_id":"user3", "location":"Charlotte, NC"}
support_chatbot(service, "I need a new restaurant for tonight", mobile_app_context)
Here are a couple of restaurant recommendations for you, Jane, that you might enjoy in Charlotte, NC: 1 Dumpling Lady • Address: 2020 E 7th St, Charlotte, NC 28204 • Recommended Menu Items: • Pork Dumplings • Scallion Pancakes • Hot and Sour Soup 2 Szechuan Palace • Address: 11601 Carolina Pl Pkwy, Pineville, NC 28134 • Recommended Menu Items: • Kung Pao Chicken • Mapo Tofu • Spicy Szechuan Noodles Enjoy your dinner!
Knowledge
Adding knowledge to an LLM provides domain specific information that extends beyond its base training. With it, the LLM is able to select relevant text that helps respond to user queries and avoid hallucination. Under the covers, Tecton calculates embedding vectors and stores the data in a vector search database which the LLM workflow agent can use to do semantic search. With Tecton knowledge, other fields can be used to filter the dataset prior to executing the semantic search which provides even more relevance to the results.
# Let's say we only want to use restaurants that have signed up for the service,
# and not those that the general LLM training knows about
#change the prompt instructions
@prompt(sources=[ location_request, user_info_fv]) # specifies sources of data for the prompt
def sys_prompt(location_request, user_info_fv ):
location = location_request["location"]
name = user_info_fv["name"]
return f"""
Address the user by their name. Their name is {name}.
You are a consierge service that recommends restaurants.
Only suggest restaurants that are in or near {location}.
Always provide an address.
Always suggested menu items.
Only select restaurants that have signed up on the service.
""" # notice that we've added an instruction
#example knowledge about restaurants that have signed up
restaurants_signed_up = [
{
"restaurant_name": "Mama Ricotta's",
"zipcode": "28203",
"cuisine": "Italian",
"description": "A Charlotte staple since 1992, Mama Ricotta's offers authentic Italian cuisine in a warm, family-friendly atmosphere. Known for their hand-tossed pizzas, homemade pasta, and signature chicken parmesan."
},
{
"restaurant_name": "The Capital Grille",
"zipcode": "28202",
"cuisine": "American Steakhouse",
"description": "An upscale steakhouse chain featuring dry-aged steaks, fresh seafood, and an extensive wine list. The Charlotte location offers a refined dining experience with impeccable service and a sophisticated ambiance."
},
{
"restaurant_name": "Firebirds Wood Fired Grill",
"zipcode": "28277",
"cuisine": "American",
"description": "A polished casual restaurant known for its classic American cuisine cooked over an open wood fire. Specialties include hand-cut steaks, fresh seafood, and signature cocktails in a warm, contemporary setting."
},
{
"restaurant_name": "Villa Antonio",
"zipcode": "28210",
"cuisine": "Italian",
"description": "An elegant Italian restaurant offering a romantic atmosphere and authentic cuisine. Known for its homemade pasta, extensive wine selection, and attentive service, perfect for special occasions."
},
{
"restaurant_name": "Viva Chicken",
"zipcode": "28203",
"cuisine": "Peruvian",
"description": "A fast-casual eatery specializing in Peruvian-style rotisserie chicken. Offers fresh, flavorful dishes with a modern twist on traditional Peruvian cuisine, including quinoa stuffed avocados and yuca fries."
},
{
"restaurant_name": "Wan Fu",
"zipcode": "28226",
"cuisine": "Chinese",
"description": "A local favorite for Chinese cuisine, Wan Fu offers a wide array of traditional and Americanized Chinese dishes. Known for its generous portions, friendly service, and comfortable dining atmosphere."
},
{
"restaurant_name": "Wan Fu Quality Chinese Cuisine",
"zipcode": "28277",
"cuisine": "Chinese",
"description": "An upscale Chinese restaurant focusing on authentic flavors and high-quality ingredients. Offers a more refined dining experience with a menu featuring both classic and innovative Chinese dishes."
},
{
"restaurant_name": "Ru San's",
"zipcode": "28203",
"cuisine": "Japanese",
"description": "A popular sushi restaurant known for its extensive menu and all-you-can-eat option. Offers a wide variety of sushi rolls, sashimi, and other Japanese dishes in a casual, vibrant atmosphere."
},
{
"restaurant_name": "Le Bernardin",
"zipcode": "10019",
"cuisine": "French Seafood",
"description": "A world-renowned, Michelin three-star restaurant specializing in exquisite seafood. Chef Eric Ripert's menu features innovative preparations of the finest global seafood in an elegant setting."
},
{
"restaurant_name": "Katz's Delicatessen",
"zipcode": "10002",
"cuisine": "Jewish Deli",
"description": "An iconic New York institution since 1888, famous for its hand-carved pastrami and corned beef sandwiches. The bustling, no-frills atmosphere is part of its enduring charm."
},
{
"restaurant_name": "Eleven Madison Park",
"zipcode": "10010",
"cuisine": "Contemporary American",
"description": "A three-Michelin-starred restaurant offering an innovative tasting menu focusing on locally sourced, plant-based ingredients. Known for its impeccable service and artistic presentation."
},
{
"restaurant_name": "Peter Luger Steak House",
"zipcode": "11211",
"cuisine": "Steakhouse",
"description": "A Brooklyn institution since 1887, Peter Luger is famous for its dry-aged steaks and old-school, cash-only policy. The no-frills atmosphere focuses attention on the exceptional quality of the meat."
},
{
"restaurant_name": "Di Fara Pizza",
"zipcode": "11230",
"cuisine": "Pizza",
"description": "A legendary Brooklyn pizzeria, Di Fara is known for its handcrafted pies made by founder Dom DeMarco. Each pizza is a work of art, featuring high-quality ingredients and meticulous preparation."
},
{
"restaurant_name": "Balthazar",
"zipcode": "10012",
"cuisine": "French Brasserie",
"description": "A SoHo classic, Balthazar offers authentic French brasserie fare in a vibrant, bustling atmosphere. Known for its fresh seafood, classic French dishes, and popular weekend brunch."
},
{
"restaurant_name": "Momofuku Ko",
"zipcode": "10003",
"cuisine": "Contemporary Asian",
"description": "Chef David Chang's two-Michelin-starred restaurant offers an ever-changing tasting menu blending Asian and Western influences. The intimate counter seating provides a unique, interactive dining experience."
},
{
"restaurant_name": "The Halal Guys",
"zipcode": "10019",
"cuisine": "Middle Eastern",
"description": "Starting as a food cart, The Halal Guys has become a New York institution. Famous for its chicken and gyro over rice, topped with their legendary white sauce. Now with brick-and-mortar locations."
},
{
"restaurant_name": "Russ & Daughters",
"zipcode": "10002",
"cuisine": "Jewish Appetizing",
"description": "A New York classic since 1914, specializing in traditional Jewish appetizing foods. Famous for its hand-sliced smoked salmon, bagels, and other Jewish delicacies. Now includes a sit-down cafe."
},
{
"restaurant_name": "Lombardi's",
"zipcode": "10012",
"cuisine": "Pizza",
"description": "America's first pizzeria, established in 1905. Lombardi's continues to serve classic New York-style pizza from its coal-fired oven. Known for its simple, high-quality ingredients and historic charm."
},
{
"restaurant_name": "Joe's Shanghai",
"zipcode": "10013",
"cuisine": "Chinese",
"description": "Famous for introducing soup dumplings to New York, Joe's Shanghai offers authentic Shanghai-style cuisine. The bustling, no-frills atmosphere adds to the authentic experience."
}
]
from tecton_gen_ai.testing import make_local_source
from tecton_gen_ai.testing.utils import make_local_vector_db_config
from tecton_gen_ai.fco import source_as_knowledge
#provide a vector db config
conf = make_local_vector_db_config()
#create embeddings of the restaurant descriptions in the vector DB
src = make_local_source("restaurants_signed_up", restaurants_signed_up, description="Restaurants that signed up for the service")
restaurant_knowledge = source_as_knowledge(
src,
vector_db_config=conf,
vectorize_column="description",
filter = [("zipcode", str, "the zip code for the restaurant"), ("cuisine", str, "the cuisine that they serve")]
)
#rebuild the service with restaurant knowledge included
service = AgentService(
name="restaurant_recommender",
prompts=[sys_prompt],
knowledge=[restaurant_knowledge], # added a source of knowledge
tools=[user_info_fv, recent_eats_fv]
)
# test the app
mobile_app_context = {"user_id":"user3", "location":"28277"}
support_chatbot(service, "recommend a restaurant I have not visited with dry-aged steaks and tiramisu", mobile_app_context)
metadata.zipcode = '28277'
I found a great restaurant for you, Jane, that serves dry-aged steaks and tiramisu, and you haven't visited it yet:
Firebirds Wood Fired Grill
• Address: 9824 Rea Rd, Charlotte, NC 28277
• Cuisine: American
• Description: A polished casual restaurant known for its classic American cuisine cooked over an open wood fire.
Specialties include hand-cut steaks, fresh seafood, and signature cocktails in a warm, contemporary setting.
• Suggested Menu Items:
• Dry-Aged Ribeye Steak
• Tiramisu for dessert
Enjoy your meal!
# test for a zipcode that is not close to any in the apps knowledge
mobile_app_context = {"user_id":"user1", "location":"10200"}
support_chatbot(service, "recommend a new restaurant with dry-aged steaks and Tiramisu", mobile_app_context)
metadata.zipcode = '10200'
metadata.zipcode = '10200'
I'm sorry, Jim, but it seems there are currently no restaurants in or near 10200 that offer dry-aged steaks or Tiramisu. If you're open to other types of cuisine or specific dishes, please let me know, and I can help you find something else!
# test for recent visits with item they do not serve
mobile_app_context = {"user_id":"user2", "location":"28277"}
support_chatbot(service, "recommend a one of my recent visits that has Tiramisu", mobile_app_context)
metadata.zipcode = '28277'
metadata.zipcode = '28277'
metadata.zipcode = '28277'
metadata.zipcode = '28277'
Hi John! Unfortunately, it seems that none of your recent visits—Mama Ricotta's, Villa Antonio, or Viva Chicken—offer Tiramisu. However, I found a couple of restaurants in the 28277 area that do have Tiramisu on their menu: 1 Firebirds Wood Fired Grill • Address: 11325 N Community House Rd, Charlotte, NC 28277 • Cuisine: American • Description: A polished casual restaurant known for its classic American cuisine cooked over an open wood fire. Specialties include hand-cut steaks, fresh seafood, and signature cocktails in a warm, contemporary setting. • Suggested Menu Item: Tiramisu for dessert. 2 Wan Fu Quality Chinese Cuisine • Address: 10808 Providence Rd, Charlotte, NC 28277 • Cuisine: Chinese • Description: An upscale Chinese restaurant focusing on authentic flavors and high-quality ingredients, offering a refined dining experience with both classic and innovative dishes. • Suggested Menu Item: Tiramisu (if available as a fusion dessert). Enjoy your meal!
