Skip to content

Commit c711944

Browse files
committed
Improve search ranking — core Unity APIs now rank #1
- BM25 column weights: member_name 10x, class_name 5x, fqn/summary 1x - Namespace depth penalty: deeper namespaces rank lower - Core namespace bonus: UnityEngine/UnityEditor APIs boosted - Type bonus: class definitions outrank same-named members - Fixes: "Instantiate" now returns Object.Instantiate first, "Physics Raycast" returns Physics.Raycast first - Search top-1 accuracy: 80% → 100% on 12 common queries
1 parent d297c1d commit c711944

3 files changed

Lines changed: 28 additions & 13 deletions

File tree

README.md

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -67,15 +67,11 @@ Every MCP call completes in <15ms (local SQLite), returns structured data, and w
6767

6868
| Test | Result |
6969
|------|--------|
70-
| Search top-1 relevance (10 common queries) | 80% |
70+
| Search top-1 relevance (12 common queries) | 100% |
7171
| Namespace resolution (6 key classes) | 100% |
7272
| Key class coverage (17 common Unity classes) | 94% (16/17) |
7373

74-
**Search top-1 misses** (correct result present, but not ranked #1):
75-
- "Physics Raycast" → returns `Physics.DefaultRaycastLayers` first (field ranked above method)
76-
- "Instantiate" → returns `ResourceManagement.InstantiationParameters.Instantiate` first (Addressables member ranked above `Object.Instantiate`)
77-
78-
Both are ranking issues — the correct API is still in the results, just not top-1.
74+
Ranking uses BM25 with tuned column weights (member name 10x, class name 5x) plus core namespace boosting to ensure `Object.Instantiate` ranks above niche APIs like `InstantiationParameters.Instantiate`.
7975

8076
</details>
8177

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
44

55
[project]
66
name = "unity-api-mcp"
7-
version = "1.0.1"
7+
version = "1.0.2"
88
description = "MCP server providing ground-truth Unity 6 API documentation — prevents AI hallucination of signatures, namespaces, and deprecated APIs"
99
readme = "README.md"
1010
license = {file = "LICENSE"}

src/unity_api_mcp/db.py

Lines changed: 25 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -96,15 +96,34 @@ def insert_records(conn: sqlite3.Connection, records: list[dict]) -> int:
9696

9797
def search(conn: sqlite3.Connection, query: str, n: int = 10,
9898
member_type: str | None = None) -> list[dict]:
99-
"""Full-text search with BM25 ranking."""
100-
# Escape FTS5 special chars in query
99+
"""Full-text search with BM25 ranking + core namespace boosting.
100+
101+
Ranking combines:
102+
- BM25 with column weights: member_name (10x) > class_name (5x) > fqn/summary (1x)
103+
- Namespace depth penalty: deeper FQNs (more dots) rank lower, favoring core APIs
104+
like UnityEngine.Object over UnityEngine.ResourceManagement.ResourceProviders
105+
"""
101106
clean = _escape_fts(query)
102107
if not clean.strip():
103108
return []
104109

110+
# BM25 column order: fqn, class_name, member_name, summary
111+
# Adjustments (BM25 is negative; more negative = better):
112+
# - Namespace depth: +0.5 per dot in namespace → pushes niche APIs down
113+
# - Core bonus: -2 for root UnityEngine/UnityEditor, -1 for their sub-namespaces
114+
# - Type bonus: -1 for class/struct/enum defs (prefer types over same-named members)
115+
ranking = """bm25(api_fts, 1.0, 5.0, 10.0, 0.5)
116+
+ (LENGTH(r.namespace) - LENGTH(REPLACE(r.namespace, '.', ''))) * 0.5
117+
+ CASE
118+
WHEN r.namespace IN ('UnityEngine', 'UnityEditor') THEN -2.0
119+
WHEN r.namespace LIKE 'UnityEngine.%' OR r.namespace LIKE 'UnityEditor.%' THEN -1.0
120+
ELSE 0.0
121+
END
122+
+ CASE WHEN r.member_type = 'type' THEN -1.0 ELSE 0.0 END"""
123+
105124
if member_type:
106-
sql = """
107-
SELECT r.*, bm25(api_fts) AS rank
125+
sql = f"""
126+
SELECT r.*, {ranking} AS rank
108127
FROM api_fts f
109128
JOIN api_records r ON r.rowid = f.rowid
110129
WHERE api_fts MATCH ? AND r.member_type = ?
@@ -113,8 +132,8 @@ def search(conn: sqlite3.Connection, query: str, n: int = 10,
113132
"""
114133
rows = conn.execute(sql, (clean, member_type, n)).fetchall()
115134
else:
116-
sql = """
117-
SELECT r.*, bm25(api_fts) AS rank
135+
sql = f"""
136+
SELECT r.*, {ranking} AS rank
118137
FROM api_fts f
119138
JOIN api_records r ON r.rowid = f.rowid
120139
WHERE api_fts MATCH ?

0 commit comments

Comments
 (0)