Query History and Learning
The query history and learning module provides query history storage, pattern analysis, personalized ranking, and learning from user query patterns.
Overview
The history module enables:
- Query History Storage: Store queries with metadata, results, and user interactions
- Query Pattern Analysis: Analyze frequent queries, identify patterns, and detect trends
- Personalized Entity Ranking: Boost entities based on popularity and user preferences
- Query Statistics: Get insights into query patterns and usage
- History Management: View, clear, and manage query history
Usage
Storing Query History
from app.core.vectordb.history import store_query_history
# Store a query with results
results = await semantic_search("living room lights")
result = await store_query_history(
query="living room lights",
results=results,
selected_entity_id="light.living_room",
user_id="user123", # Optional
)
print(result["query_id"]) # Unique query ID
print(result["success"]) # True if stored successfully
Getting Query History
from app.core.vectordb.history import get_query_history
from datetime import datetime, timedelta, UTC
# Get recent queries
history = await get_query_history(limit=50)
# Get queries for a specific user
history = await get_query_history(limit=50, user_id="user123")
# Get queries within a date range
start_date = datetime.now(UTC) - timedelta(days=7)
end_date = datetime.now(UTC)
history = await get_query_history(
limit=50,
start_date=start_date,
end_date=end_date,
)
Getting Query Statistics
from app.core.vectordb.history import get_query_statistics
# Get statistics for the last 30 days
stats = await get_query_statistics(days=30)
print(stats["total_queries"]) # Total number of queries
print(stats["unique_queries"]) # Number of unique queries
print(stats["most_common_queries"]) # Most common queries
print(stats["most_common_intents"]) # Most common intents
print(stats["most_common_domains"]) # Most common domains
print(stats["most_selected_entities"]) # Most selected entities
print(stats["queries_by_time_of_day"]) # Distribution by time of day
Entity Popularity
from app.core.vectordb.history import get_entity_popularity
# Get entity popularity count
popularity = await get_entity_popularity("light.living_room")
print(popularity) # Number of times entity was selected
Personalized Entity Ranking
from app.core.vectordb.history import boost_entity_ranking
# Boost entity ranking based on popularity
results = await semantic_search("lights")
boosted_results = await boost_entity_ranking(
results,
boost_factor=0.1, # Boost factor (0.0-1.0)
)
# Results are re-sorted with popularity boost
for result in boosted_results:
print(result["entity_id"])
print(result["similarity_score"]) # May include popularity boost
if "popularity_boost" in result:
print(result["popularity_boost"]) # Boost amount
Clearing Query History
from app.core.vectordb.history import clear_query_history
from datetime import datetime, timedelta, UTC
# Clear all query history
result = await clear_query_history()
# Clear history for a specific user
result = await clear_query_history(user_id="user123")
# Clear history before a specific date
before_date = datetime.now(UTC) - timedelta(days=90)
result = await clear_query_history(before_date=before_date)
print(result["deleted_count"]) # Number of queries deleted
print(result["success"]) # True if successful
Query History Schema
Each stored query includes:
- query_id: Unique identifier for the query
- query_text: Original query text
- timestamp: When the query was stored (ISO format)
- intent: Query intent (SEARCH, CONTROL, STATUS, etc.)
- domain: Predicted entity domain (light, sensor, etc.)
- result_count: Number of results returned
- selected_entity_id: Entity ID that was selected/used (if any)
- user_id: Optional user identifier
- context: Context information (time_of_day, area, etc.)
- results: Summary of top results with rankings
Learning from Queries
The module learns from user queries and selections:
- Entity Popularity: Tracks how often entities are selected
- Query Patterns: Identifies common query patterns
- Personalization: Adapts to user preferences over time
Example: Learning from User Selection
# User searches for "living room lights"
results = await semantic_search("living room lights")
# User selects "light.living_room"
await store_query_history(
query="living room lights",
results=results,
selected_entity_id="light.living_room",
)
# Future searches will boost "light.living_room" based on popularity
future_results = await semantic_search("lights")
boosted_results = await boost_entity_ranking(future_results)
Integration with Semantic Search
The history module integrates seamlessly with semantic search:
from app.core.vectordb.history import store_query_history, boost_entity_ranking
from app.core.vectordb.search import semantic_search
# Perform semantic search
results = await semantic_search("living room lights")
# Boost results based on popularity
boosted_results = await boost_entity_ranking(results)
# Store query history
await store_query_history(
query="living room lights",
results=boosted_results,
selected_entity_id=boosted_results[0]["entity_id"] if boosted_results else None,
)
Query Statistics
The statistics module provides insights into query patterns:
Most Common Queries
stats = await get_query_statistics(days=30)
for query_info in stats["most_common_queries"]:
print(f"{query_info['query']}: {query_info['count']} times")
Most Common Intents
stats = await get_query_statistics(days=30)
for intent_info in stats["most_common_intents"]:
print(f"{intent_info['intent']}: {intent_info['count']} times")
Most Selected Entities
stats = await get_query_statistics(days=30)
for entity_info in stats["most_selected_entities"]:
print(f"{entity_info['entity_id']}: {entity_info['count']} times")
Time-Based Patterns
stats = await get_query_statistics(days=30)
print(stats["queries_by_time_of_day"])
# {
# "morning": 50,
# "afternoon": 30,
# "evening": 40,
# "night": 10
# }
Privacy Considerations
The history module supports privacy controls:
- User Identification: Optional user IDs for multi-user scenarios
- History Clearing: Clear history for specific users or time periods
- Data Retention: Control how long history is retained
Example: Privacy Controls
# Clear history for a specific user
await clear_query_history(user_id="user123")
# Clear old history (older than 90 days)
before_date = datetime.now(UTC) - timedelta(days=90)
await clear_query_history(before_date=before_date)
Best Practices
-
Store queries after user interaction: Store query history after the user selects an entity or performs an action.
-
Use popularity boosting judiciously: Don't over-boost popular entities; maintain a balance with semantic similarity.
-
Clear old history regularly: Implement retention policies to clear old history and maintain performance.
-
Respect user privacy: Provide users with controls to view and clear their query history.
-
Monitor statistics: Use query statistics to understand user patterns and improve search accuracy.
Limitations
-
Simplified popularity tracking: Current implementation uses a simplified popularity counter. Future versions may include more sophisticated learning algorithms.
-
No conversation context: Each query is stored independently. Future versions may support conversation context.
-
Storage limitations: Query history is stored in the vector database. Large histories may require additional storage management.
Future Enhancements
Future versions may include: - ML-based learning algorithms - Conversation context tracking - Advanced pattern recognition - Multi-user personalization - Query correction and refinement - Auto-complete suggestions