Skip to content

Regression: decimal(M,D) with unsigned/zerofill or as dec/fixed is rejected, where numeric(M,D) unsigned passes #1528

Description

@dimitri-yatsenko

Summary

decimal(M,D) with any SQL modifier or alias is rejected outright under 2.x, where
0.14.x accepted it as a native passthrough. This is a regression, and the cause looks
like an oversight rather than a decision.

decimal(2,2)              -> DECIMAL   (core type)
decimal(2, 2) unsigned    -> DataJointError: Unsupported attribute type decimal(2, 2) unsigned
decimal(5)                -> DataJointError
decimal(2,2) zerofill     -> DataJointError

Note that the neighbouring native types all pass through with the usual portability
warning, including numeric — which is SQL's own alias for decimal, i.e. the same
type:

int unsigned              -> INTEGER   (native, warns)
float unsigned            -> FLOAT     (native, warns)
numeric(2,2) unsigned     -> NUMERIC   (native, warns)

So today the workaround is to spell the type numeric instead of decimal, which is
not a state we want to ship.

Cause

0.14.9 had one permissive pattern covering both spellings:

# 0.14.9 declare.py:29
DECIMAL=r"(decimal|numeric)(\s*\(.+\))?(\s+unsigned)?$",

2.x split this in two. decimal became a core type with a strict pattern
(declare.py:46):

"decimal": (r"decimal\s*\(\d+\s*,\s*\d+\)$", None),

and the old permissive branch survived only under the name NUMERIC
(declare.py:68), which kept numeric and dropped decimal:

NUMERIC=r"numeric(\s*\(.+\))?(\s+unsigned)?$",  # numeric is SQL alias, use decimal instead

Because there is no native fallback pattern for decimal, match_type finds no match
and raises at declare.py:110-113. Unlike a native type, there is no warning and no
passthrough — the table simply cannot be declared.

Impact

Found while migrating the elements: element-optogenetics declares four such
attributes (optogenetics.py:98,99,114,115, all decimal(2, 2) unsigned proportions),
and those tables cannot be declared at all under 2.x. Any pre-2.0 schema using an
unsigned or single-argument decimal hits the same wall.

Proposed fix

Restore the native passthrough by widening the pattern at declare.py:68:

NUMERIC=r"(decimal|numeric|dec|fixed)(\s*\(.+\))?(\s+unsigned)?(\s+zerofill)?$",

Core types are inserted into TYPE_PATTERN ahead of the native patterns and
match_type returns the first match, so decimal(2,2) still resolves to the core
DECIMAL type. Only the modified and aliased forms fall through to native-with-warning,
which is exactly how int unsigned and float unsigned already behave.

Dropping unsigned from a decimal costs only the non-negativity constraint, not
precision, so a warning is the right level of pushback. An error forces the user to make
a semantic change they never asked about, in the middle of a migration.

Documentation

Two doc changes belong with this:

  • docs/src/how-to/migrate-to-v20.md:1180 currently promises
    decimal(M,D) → decimal(M,D) # unchanged, which is what someone with an unsigned
    decimal reads immediately before hitting the error.
  • docs/src/explanation/type-system.md:93 covers unsigned integers but says nothing
    about modifiers on decimal, and nothing anywhere lists which native spellings are
    passed through versus rejected.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIndicates an unexpected problem or unintended behavior

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions