FIX: Cursor.describe invalid data #355

dlevy-msft-sql · 2025-12-01T17:19:35Z

Work Item / Issue Reference

GitHub Issue: #352

Summary

This PR fixes cursor.description to properly return SQL type codes per DB-API 2.0 specification, while maintaining backwards compatibility with libraries like pandas that compare against Python types.

Key Changes

New SQLTypeCode class - A dual-compatible type code that compares equal to both:
- SQL type integers (e.g., desc[1] == -9) for DB-API 2.0 compliance
- Python types (e.g., desc[1] == str) for pandas/library compatibility
SQL Server spatial type support - Added handling for:
- geography (e.g., GPS coordinates, regions)
- geometry (e.g., shapes, planar coordinates)
- hierarchyid (e.g., org charts, tree structures)
ODBC 3.x date/time type codes - Added support for:
- SQL_TYPE_DATE (91)
- SQL_TYPE_TIME (92)
- SQL_TYPE_TIMESTAMP (93)
- SQL_TYPE_TIMESTAMP_WITH_TIMEZONE (95)
Hash/equality contract fix - Made SQLTypeCode unhashable since __eq__ compares to multiple types with different hash values

Files Changed

File	Changes
`mssql_python/cursor.py`	Added `SQLTypeCode` class, updated `_initialize_description()`
`mssql_python/constants.py`	Added `SQL_SS_UDT`, `SQL_SS_TIME2`, `SQL_SS_XML` constants
`mssql_python/pybind/ddbc_bindings.cpp`	Added C++ handling for spatial types
`mssql_python/__init__.py`	Exported `SQLTypeCode`
`mssql_python/mssql_python.pyi`	Added type stubs for `SQLTypeCode`
`tests/test_002_types.py`	Added `SQLTypeCode` unit tests
`tests/test_004_cursor.py`	Added spatial type tests, thread safety tests

Usage Examples

SQLTypeCode - Dual Compatibility

cursor.execute("SELECT id, name FROM users")
desc = cursor.description

# Style 1: Compare with Python types (pandas compatibility)
if desc[0][1] == int:
    print("Integer column")

# Style 2: Compare with SQL type codes (DB-API 2.0)
if desc[0][1] == 4:  # SQL_INTEGER
    print("Integer column")

# Get raw SQL type code
type_code = int(desc[0][1])  # Returns 4 for SQL_INTEGER

Spatial Types

# Geography (GPS coordinates)
cursor.execute("""
    INSERT INTO locations (geo) 
    VALUES (geography::STGeomFromText('POINT(-122.349 47.651)', 4326))
""")

# Fetch as binary
cursor.execute("SELECT geo FROM locations")
geo_bytes = cursor.fetchone()[0]  # Returns bytes

# Or fetch as WKT text
cursor.execute("SELECT geo.STAsText() FROM locations")
wkt = cursor.fetchone()[0]  # Returns "POINT (-122.349 47.651)"

Testing

All existing tests pass
Added 40+ new tests for spatial types (geography, geometry, hierarchyid)
Added thread safety tests for _column_metadata
Added SQLTypeCode unit tests
Coverage: 98% diff coverage, 76% overall

Copilot Review Comments Addressed

Comment	Resolution
Missing ODBC 3.x date/time codes	Added 91, 92, 93, 95 to `_type_map`
Hash/eq contract violation	Set `__hash__ = None` (unhashable)
Geography executemany test incomplete	Test now inserts and verifies `geo_col`

Breaking Changes

None. The SQLTypeCode class maintains full backwards compatibility:

cursor.description[i][1] == str still works ✅
cursor.description[i][1] == -9 also works ✅

…ins an LOB

Copilot

Pull request overview

This PR fixes issue #352 by implementing two main improvements: (1) correcting cursor.description to return SQL type codes (integers) instead of Python type objects, ensuring DB-API 2.0 specification compliance, and (2) adding support for SQL Server spatial data types (geography, geometry, hierarchyid) by handling the SQL_SS_UDT type code (-151).

Key changes:

Fixed cursor.description[i][1] to return SQL type integer codes (e.g., 4, -9) instead of Python types (int, str) per DB-API 2.0 spec
Added SQL_SS_UDT (-151) support for SQL Server User-Defined Types including spatial data types
Updated output converter lookup to use SQL type codes consistently throughout the codebase

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
mssql_python/cursor.py	Changed `cursor.description` to return SQL type codes instead of Python types; added `_column_metadata` storage; updated `_build_converter_map` to use SQL type codes; added mappings for UDT, XML, DATETIME2, SMALLDATETIME types
mssql_python/constants.py	Added `SQL_SS_UDT = -151` constant for SQL Server User-Defined Types
mssql_python/pybind/ddbc_bindings.cpp	Added C++ constants for SQL_SS_UDT, SQL_DATETIME2, SQL_SMALLDATETIME; implemented UDT handling in SQLGetData_wrap, FetchBatchData, FetchMany_wrap, and FetchAll_wrap for LOB streaming
tests/test_004_cursor.py	Added `lob_wvarchar_column` to test schema; updated `test_cursor_description` to expect SQL type codes; added comprehensive geography type test suite (14 new tests); separated LOB and non-LOB fetch tests; fixed output converter test for UTF-16LE encoding
tests/test_003_connection.py	Simplified converter integration tests to use SQL type constants directly instead of dynamic type detection

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/test_004_cursor.py

mssql_python/constants.py

mssql_python/cursor.py

…ins an LOB

…ython into Issue-352

github-actions · 2026-01-05T19:27:13Z

📊 Code Coverage Report

🔥 Diff Coverage 92%	🎯 Overall Coverage 77%	📈 Total Lines Covered: `5555` out of `7212` 📁 Project: `mssql-python`

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

mssql_python/init.py (100%)
mssql_python/connection.py (88.2%): Missing lines 50,1020
mssql_python/constants.py (100%)
mssql_python/cursor.py (90.4%): Missing lines 134,152,1126,1473,2541
mssql_python/pybind/ddbc_bindings.cpp (100%)

Summary

Total: 99 lines
Missing: 7 lines
Coverage: 92%

mssql_python/connection.py

Lines 46-54

Lines 1016-1024

mssql_python/cursor.py

Lines 130-138

Lines 148-156

Lines 1122-1130

Lines 1469-1477

Lines 2537-2545

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.hpp: 58.8%
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.cpp: 69.5%
mssql_python.pybind.ddbc_bindings.h: 69.7%
mssql_python.pybind.connection.connection.cpp: 75.3%
mssql_python.row.py: 79.5%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.__init__.py: 84.9%
mssql_python.cursor.py: 85.2%

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

mssql_python/cursor.py

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

mssql_python/connection.py:952

The add_output_converter docstring says non-NULL values passed to converters “will be a bytes object”, but the Row conversion logic only encodes str to UTF-16LE bytes; other types (e.g., int, decimal.Decimal, datetime) are passed through as their native Python objects. Please update the docstring to reflect the actual value types converters may receive (bytes for strings/binary, native objects otherwise).

        Args:
            sqltype (int, SQLTypeCode, or type): The integer SQL type value to convert, which can be
                          one of the defined standard constants (e.g. SQL_VARCHAR) or a
                          database-specific value (e.g. -151 for the SQL Server 2008
                          geometry data type). Also accepts SQLTypeCode objects (from
                          cursor.description) or Python types (e.g., str, int) for
                          backward compatibility.
            func (callable): The converter function which will be called with a single parameter,
                            the value, and should return the converted value. If the value is NULL
                            then the parameter passed to the function will be None, otherwise it
                            will be a bytes object.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/connection.py

mssql_python/pybind/ddbc_bindings.cpp

tests/test_002_types.py

…UDT against SQL_NO_TOTAL, update hash test

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mssql_python/mssql_python.pyi

mssql_python/cursor.py

tests/test_004_cursor.py

saurabh500

Looks pretty good to me.

Do we want to consider doing something about testing with the stable release of libraries which are using the driver?

saurabh500

Looks pretty good to me.

- Improve __hash__ error message with helpful guidance for dict key usage - Move spatial tests (geography/geometry/hierarchyid) to test_017_spatial_types.py - Add binary parameter round-trip tests for geography and geometry - Update type stub and unit test for new hash behavior

dlevy-msft-sql · 2026-02-02T21:44:03Z

Looks pretty good to me.

Do we want to consider doing something about testing with the stable release of libraries which are using the driver?

This PR is already way larger than it probably should have been. I'd do a PR for each library test once the libraries to test with are defined.

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

tests/test_004_cursor.py:1

Changed from catching generic Exception to specific mssql_python.Error. This is more precise and follows best practices for error handling in tests.

"""

tests/test_004_cursor.py:1

The date '2025-11-26' is in the future. Consider using a past date or current date to avoid potential timezone-related test failures.

"""

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/test_004_cursor.py

mssql_python/pybind/ddbc_bindings.cpp

mssql_python/cursor.py

- Improve __hash__ error message with helpful guidance for dict key usage - Move spatial tests (geography/geometry/hierarchyid) to test_017_spatial_types.py - Add binary parameter round-trip tests for geography and geometry - Update type stub and unit test for new hash behavior

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

tests/test_004_cursor.py:1

This change corrects the exception type from a generic Exception to mssql_python.Error, which is more specific and accurate for database errors. This is already fixed in the updated code.

"""

tests/test_004_cursor.py:13063

Simplified cleanup by removing unnecessary try-except block around DROP TABLE statement. This is already fixed in the updated code.

        cursor.execute(f"DROP TABLE IF EXISTS {table_name}")

tests/test_004_cursor.py:1

Added proper encoding handling for string comparison in output converter test. The driver passes string values as UTF-16LE encoded bytes to output converters, so the test now correctly encodes the comparison value. This is already fixed in the updated code.

"""

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

saurabh500 · 2026-02-02T22:53:49Z

SqlTypeCode should be used instead of SQLTypeCode. I am assuming this is a public type and the naming would matter more. PascalCase for Python classes is recommended and applicable to acronyms also.

dlevy-msft-sql added 4 commits November 24, 2025 20:05

Fix for FetchMany(number of rows) ignores batch size when table conta…

c403899

…ins an LOB

First round of code review fixes

f96d14c

Fix test_fetchone

e135e40

Fix for issue 352

38966d0

Copilot AI review requested due to automatic review settings December 1, 2025 17:19

Copilot started reviewing on behalf of dlevy-msft-sql December 1, 2025 17:20 View session

Copilot finished reviewing on behalf of dlevy-msft-sql December 1, 2025 17:23

Copilot AI reviewed Dec 1, 2025

View reviewed changes

tests/test_004_cursor.py Outdated Show resolved Hide resolved

tests/test_004_cursor.py Outdated Show resolved Hide resolved

mssql_python/constants.py Outdated Show resolved Hide resolved

mssql_python/cursor.py Outdated Show resolved Hide resolved

dlevy-msft-sql added bug Something isn't working pr-size: medium Moderate update size labels Dec 1, 2025

dlevy-msft-sql added 6 commits December 1, 2025 11:31

Apply fixes from copilot review

3f52dd7

Fix for FetchMany(number of rows) ignores batch size when table conta…

38b041b

…ins an LOB

First round of code review fixes

139994a

Fix test_fetchone

6fd823e

Fix for issue 352

a7d9b8e

Apply fixes from copilot review

646ef04

dlevy-msft-sql force-pushed the Issue-352 branch from 3f52dd7 to 646ef04 Compare December 9, 2025 20:56

dlevy-msft-sql added 2 commits January 5, 2026 13:01

Merge branch 'Issue-352' of https://github.com/dlevy-msft-sql/mssql-p…

8018872

…ython into Issue-352

Merge branch 'main' into Issue-352

67268e5

dlevy-msft-sql added 4 commits January 7, 2026 22:49

Merge branch 'main' into Issue-352

d335ed0

Merge branch 'main' into Issue-352

2dc93f3

Merge branch 'main' into Issue-352

eee5256

Merge branch 'microsoft:main' into Issue-352

d289a9b

dlevy-msft-sql added pr-size: small Minimal code update pr-size: medium Moderate update size and removed pr-size: medium Moderate update size labels Jan 16, 2026

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Outdated Show resolved Hide resolved

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Outdated Show resolved Hide resolved

sumitmsft reviewed Jan 19, 2026

View reviewed changes

mssql_python/cursor.py Show resolved Hide resolved

dlevy-msft-sql requested a review from Copilot February 2, 2026 03:24

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 03:25 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

docs: Fix add_output_converter docstring to reflect actual value types

496c3d1

dlevy-msft-sql requested a review from Copilot February 2, 2026 03:39

dlevy-msft-sql removed the pr-size: small Minimal code update label Feb 2, 2026

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 03:39 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

mssql_python/connection.py Outdated Show resolved Hide resolved

mssql_python/pybind/ddbc_bindings.cpp Outdated Show resolved Hide resolved

tests/test_002_types.py Outdated Show resolved Hide resolved

fix: Add python_type fallback for get_output_converter, guard SQL_SS_…

0dfa606

…UDT against SQL_NO_TOTAL, update hash test

dlevy-msft-sql requested a review from Copilot February 2, 2026 03:54

Copilot started reviewing on behalf of dlevy-msft-sql February 2, 2026 03:55 View session

Copilot AI reviewed Feb 2, 2026

View reviewed changes

dlevy-msft-sql marked this pull request as ready for review February 2, 2026 04:03

saurabh500 reviewed Feb 2, 2026

View reviewed changes

mssql_python/mssql_python.pyi Show resolved Hide resolved

mssql_python/cursor.py Outdated Show resolved Hide resolved

saurabh500 reviewed Feb 2, 2026

View reviewed changes

tests/test_004_cursor.py Outdated Show resolved Hide resolved

saurabh500 reviewed Feb 2, 2026

View reviewed changes

tests/test_004_cursor.py Outdated Show resolved Hide resolved

saurabh500 reviewed Feb 2, 2026

View reviewed changes

saurabh500 previously approved these changes Feb 2, 2026

View reviewed changes

Merge branch 'main' into Issue-352

dee3852

dlevy-msft-sql requested a review from Copilot February 2, 2026 21:44

Copilot AI reviewed Feb 2, 2026

View reviewed changes

tests/test_004_cursor.py Show resolved Hide resolved

mssql_python/pybind/ddbc_bindings.cpp Show resolved Hide resolved

mssql_python/cursor.py Show resolved Hide resolved

mssql_python/cursor.py Show resolved Hide resolved

dlevy-msft-sql added 2 commits February 2, 2026 16:22

fix: Define python_type before conditional use to prevent NameError

8f7516c

dlevy-msft-sql dismissed saurabh500’s stale review via 8f7516c February 2, 2026 22:22

dlevy-msft-sql requested a review from Copilot February 2, 2026 22:25

Copilot AI reviewed Feb 2, 2026

View reviewed changes

dlevy-msft-sql requested a review from saurabh500 February 2, 2026 22:27

FIX: Cursor.describe invalid data #355

Are you sure you want to change the base?

FIX: Cursor.describe invalid data #355

Uh oh!

Conversation

dlevy-msft-sql commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Work Item / Issue Reference

Summary

Key Changes

Files Changed

Usage Examples

SQLTypeCode - Dual Compatibility

Spatial Types

Testing

Copilot Review Comments Addressed

Breaking Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Code Coverage Report

🔥 Diff Coverage

92%

🎯 Overall Coverage

77%

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

Summary

mssql_python/connection.py

mssql_python/cursor.py

📋 Files Needing Attention

🔗 Quick Links

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

saurabh500 left a comment

Choose a reason for hiding this comment

Uh oh!

saurabh500 left a comment

Choose a reason for hiding this comment

Uh oh!

dlevy-msft-sql commented Feb 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dlevy-msft-sql commented Dec 1, 2025 •

edited

Loading

github-actions bot commented Jan 5, 2026 •

edited

Loading