feat: add return_fields parameter to search methods (#955)#1097
Open
lailoo wants to merge 1 commit intoMemTensor:mainfrom
Open
feat: add return_fields parameter to search methods (#955)#1097lailoo wants to merge 1 commit intoMemTensor:mainfrom
lailoo wants to merge 1 commit intoMemTensor:mainfrom
Conversation
e370e03 to
b7d1d84
Compare
Add optional return_fields parameter to search_by_embedding, search_by_keywords_like, search_by_keywords_tfidf, and search_by_fulltext methods across all graph DB backends (neo4j, neo4j_community, polardb). When return_fields is specified (e.g., ['memory', 'status', 'tags']), the requested fields are included in each result dict alongside 'id' and 'score', eliminating the need for N+1 get_node() calls. Default is None, preserving full backward compatibility. Changes: - base.py: Updated docstring for search_by_embedding - neo4j.py: Added return_fields to search_by_embedding, modified Cypher RETURN clause and record construction - neo4j_community.py: Added return_fields to search_by_embedding, added _fetch_return_fields helper for direct vec_db path - polardb.py: Added return_fields to all 4 search methods, added _extract_fields_from_properties helper for JSON property extraction Closes MemTensor#955 fix: add field name validation to prevent query injection in return_fields - Add _validate_return_fields() to BaseGraphDB base class with regex validation - Apply validation in neo4j.py, neo4j_community.py, polardb.py before field name concatenation - Add return_fields parameter to base class abstract method signature - Revert unrelated .get(node_id) change back to .get(node_id, None) - Add TestFieldNameValidation and TestNeo4jCommunitySearchReturnFields test classes (7 new tests) fix: resolve ruff lint and format issues for CI compliance
e34122f to
baadff4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add optional
return_fieldsparameter tosearch_by_embedding,search_by_keywords_like,search_by_keywords_tfidf, andsearch_by_fulltextmethods across all graph DB backends (neo4j, neo4j_community, polardb).Closes #955
Problem
Search methods only return
{"id": ..., "score": ...}. Callers who need additional fields likememory,status,tagsmust make a separateget_node()call per result, causing N+1 query overhead.Before (N+1 pattern):
Solution
After (single query):
return_fields=Nonepreserves full backward compatibilityRETURNclause to include requestednode.<field>propertiescolumn to SQL SELECT and extracts fields from JSON_validate_return_fields()to prevent query injectionChanges
base.py_validate_return_fields()helperneo4j.pyreturn_fieldstosearch_by_embeddingneo4j_community.pyreturn_fieldstosearch_by_embedding+_fetch_return_fieldshelperpolardb.pyreturn_fieldsto all 4 search methods +_extract_fields_from_propertieshelpertest_search_return_fields.pyReal Environment Verification
Tested against a live Neo4j 5 instance (Docker) with real vector search:
On
mainbranch (before fix) — ❌ FAIL:On
feat/search-return-fields-955branch (after fix) — ✅ PASS:Test Results