Expand range of definitions that can be moved into stdlib (#1259)

The actual definitions that got moved into the stdlib here are pretty few:

* `clip()`
* `cross()`
* `dxx()`, `ddy()` etc.
* `degrees()`
* `distance()`
* `dot()`
* `faceforward()`

The meat of the change is infrastructure changes required to support these new declarations

* Generic versions of the standard operators (e.g., `operator+`) were added that are generic for a type `T` that implements the matching `__Builtin`-prefixed interface. An open question is whether we can now drop the non-generic versions in favor of just having these generic operators.

* A `__BuiltinLogicalType` interface was added to capture the commonality between integers and `bool`

* `__BuiltinArithmeticType` was extended so that implementations must support initialization from an `int`

* `__BuiltinFloatingPointType` was extended to require an accessor that returns the value of pi for the given type, and the concrete floating-point types were extended to provide definitions of this value.

* It turns out that our logic for checking if two functions have the same signature (and should thus count as redeclarations/redefinitions) wasn't taking generic constraints into account at all. That was fixed with a stopgap solution that checks if the generic constraints are pairwise identical, but I didn't implement the more "correct" fix that would require canonicalizing the constraints.

* When doing overload resolution and considering potential callees, logic was added so that a non-generic candidate should always be selected over a generic one (generally the Right Thing to do), and also so that a generic candidate with fewer parameters will be selected over one with more (an approximation of the much more complicated rule we'd ideally have).

* The formatting of declarations/overloads for "ambiguous overload" errors was fleshed out a bit to include more context (the "kind" of declaration where appropriate, the return type for function declarations) and to properly space thing when outputting specialization of operator overloads that end with `<` (so that we print `func < <int>(int, int)` instead of just `func <<int,int>(int,int)`).

* The core lookup routines were heavily refactored and reorganized to try to make them bottleneck more effectively so that all paths handle all the nuances of inheritance, extensions, etc.

* Because of the refactoring to lookup logic, the semantic checking logic related to checking if a type conforms to an interface was updated to be driven based on the `Type` that is supposed to be conforming, rather than a `DeclRef` to the type's declaration. This allows it to use the type-based lookup entry point and eliminates one special-case entry point for lookup.

In addition to the various core changes, this change also refactors some of the existing stdlib code to favor writing more things in actual Slang syntax, and less in C++ code that uses `StringBuilder` to construct the Slang syntax. There is a lot more that could be done along those lines, but even pushing this far is showing that the current approach that `slang-generate` takes for how to separate meta-level C++ and Slang code isn't really ideal, so a revamp of the generator code is probably needed before I continue pushing.

One surprising casualty of the refactoring of lookup is that we no longer have the `lookedUpDecls` field in `LookupResult`. That field probably didn't belong there anyway, but the role it served was important. The idea of `lookedUpDecls` was to avoid looking up in the same interface more than once in cases where a type might have a "diamond" inheritance pattern. Removing that field doesn't appear to affect correctness of any of our existing tests, but by adding a specific test for "diamond" inheritance I could see that the refactoring introduced a regression and made looking up a member inherited along multiple paths ambiguous.

Rather than add back `lookedUpDecls` I went for a simpler (but arguably even hackier) solution where when ranking candidates from a `LookupResult` we check for identical `DeclRef`s and arbitrarily favor one over the other. One complication that arises here is that when comparing `DeclRef`s inherited along different paths they might have a `ThisTypeSubstitution` for the same type, but with different subtype witnesses (because different inheritance paths could lead to different transitive subtype witnesses: e.g., `A : B : D` and `A : C : D`).

1 files changed, 220 insertions, 72 deletions

diff --git a/source/slang/core.meta.slang b/source/slang/core.meta.slang
index 392922d38..2df6cb36b 100644
--- a/source/slang/core.meta.slang
+++ b/source/slang/core.meta.slang
@@ -21,7 +21,14 @@ syntax globallycoherent : GloballyCoherentModifier;
 interface __BuiltinType {}
 
 // A type that can be used for arithmetic operations
-interface __BuiltinArithmeticType : __BuiltinType {}
+interface __BuiltinArithmeticType : __BuiltinType
+{
+        /// Initialize from a 32-bit signed integer value.
+    __init(int value);
+}
+
+    /// A type that can be used for logical/bitwsie operations
+interface __BuiltinLogicalType : __BuiltinType {}
 
 // A type that logically has a sign (positive/negative/zero)
 interface __BuiltinSignedArithmeticType : __BuiltinArithmeticType {}
@@ -31,14 +38,16 @@ interface __BuiltinIntegerType : __BuiltinArithmeticType
 {}
 
 // A type that can represent non-integers
-interface __BuiltinRealType : __BuiltinArithmeticType {}
+interface __BuiltinRealType : __BuiltinSignedArithmeticType {}
 
 // A type that uses a floating-point representation
-interface __BuiltinFloatingPointType : __BuiltinRealType, __BuiltinSignedArithmeticType
+interface __BuiltinFloatingPointType : __BuiltinRealType
 {
-    // A builtin floating-point type must have an initializer that takes
-    // a floating-point value...
+        /// Initialize from a 32-bit floating-point value.
     __init(float value);
+
+        /// Get the value of the mathematical constant pi in this type.
+    static This getPi();
 }
 
 // A type resulting from an `enum` declaration.
@@ -116,52 +125,58 @@ __generic<T, let N : int> __intrinsic_op(select) vector<T,N> operator?:(vector<b
 ${{{{
 // We are going to use code generation to produce the
 // declarations for all of our base types.
-
 static const int kBaseTypeCount = sizeof(kBaseTypes) / sizeof(kBaseTypes[0]);
 for (int tt = 0; tt < kBaseTypeCount; ++tt)
 {
-    EMIT_LINE_DIRECTIVE();
-    sb << "__builtin_type(" << int(kBaseTypes[tt].tag) << ") struct " << kBaseTypes[tt].name;
-
-    // Declare interface conformances for this type
+}}}}
 
-    sb << "\n    : __BuiltinType\n";
+__builtin_type($(int(kBaseTypes[tt].tag)))
+struct $(kBaseTypes[tt].name)
+    : __BuiltinType
 
+${{{{
     switch (kBaseTypes[tt].tag)
     {
     case BaseType::Half:
     case BaseType::Float:
     case BaseType::Double:
-        sb << "\n    ,  __BuiltinFloatingPointType\n";
-        sb << "\n    ,  __BuiltinRealType\n";
-        sb << "\n    ,  __BuiltinSignedArithmeticType\n";
-        sb << "\n    ,  __BuiltinArithmeticType\n";
-        sb << "\n    ,  __BuiltinType\n";
+}}}}
+    ,  __BuiltinFloatingPointType
+    ,  __BuiltinRealType
+    ,  __BuiltinSignedArithmeticType
+    ,  __BuiltinArithmeticType
+${{{{
         break;
     case BaseType::Int8:
     case BaseType::Int16:
     case BaseType::Int:
     case BaseType::Int64:
-        sb << "\n    ,  __BuiltinSignedArithmeticType\n";
+}}}}
+    ,  __BuiltinSignedArithmeticType
+${{{{
         ; // fall through to:
     case BaseType::UInt8:
     case BaseType::UInt16:
     case BaseType::UInt:
     case BaseType::UInt64:
-        sb << "\n    ,  __BuiltinArithmeticType\n";
-        sb << "\n    ,  __BuiltinIntegerType\n";
+}}}}
+    ,  __BuiltinArithmeticType
+    ,  __BuiltinIntegerType
+${{{{
         ; // fall through to:
     case BaseType::Bool:
-        sb << "\n    ,  __BuiltinType\n";
+}}}}
+    ,  __BuiltinLogicalType
+${{{{
         break;
 
     default:
         break;
     }
+}}}}
+{
 
-    sb << "\n{\n";
-
-
+${{{{
     // Declare initializers to convert from various other types
     for (int ss = 0; ss < kBaseTypeCount; ++ss)
     {
@@ -175,12 +190,12 @@ for (int tt = 0; tt < kBaseTypeCount; ++tt)
         ConversionCost conversionCost = getBaseTypeConversionCost(
             kBaseTypes[tt],
             kBaseTypes[ss]);
+}}}}
 
-        EMIT_LINE_DIRECTIVE();
-        sb << "__implicit_conversion(" << conversionCost << ")\n";
+    __implicit_conversion($(conversionCost))
+    __init($(kBaseTypes[ss].name) value);
 
-        EMIT_LINE_DIRECTIVE();
-        sb << "__init(" << kBaseTypes[ss].name << " value);\n";
+${{{{
     }
 
     // If this is a basic integer type, then define explicit
@@ -206,13 +221,33 @@ ${{{{
         break;
     }
 
-    sb << "};\n";
+    // If this is a floating-point type, then we need to
+    // define the basic `getPi()` function that is used
+    // to implement generic versions of `degrees()` and
+    // `radians()`.
+    //
+    switch (kBaseTypes[tt].tag)
+    {
+    default:
+        break;
+    case BaseType::Half:
+    case BaseType::Float:
+    case BaseType::Double:
+}}}}
+        static $(kBaseTypes[tt].name) getPi() { return $(kBaseTypes[tt].name)(3.14159265358979323846264338328); }
+${{{{
+        break;
+    }
+}}}}
+
+}
+
+${{{{
 }
 
 // Declare built-in pointer type
 // (eventually we can have the traditional syntax sugar for this)
 }}}}
-
 __generic<T>
 __magic_type(PtrType)
 __intrinsic_type($(kIROp_PtrType))
@@ -242,26 +277,31 @@ __intrinsic_type($(kIROp_StringType))
 struct String
 {};
 
-${{{{
-// Declare vector and matrix types
-
-sb << "__generic<T = float, let N : int = 4> __magic_type(Vector) struct vector\n{\n";
-sb << "    typedef T Element;\n";
+    /// An `N` component vector with elements of type `T`.
+__generic<T = float, let N : int = 4>
+__magic_type(Vector)
+struct vector
+{
+        /// The element type of the vector
+    typedef T Element;
 
-// Declare initializer taking a single scalar of the elemnt type
-sb << "    __implicit_conversion(" << kConversionCost_ScalarToVector << ")\n";
-sb << "    __intrinsic_op(" << kIROp_constructVectorFromScalar << ")\n";
-sb << "    __init(T value);\n";
 
-// Allow initialization from same type
-sb << "    __init(vector<T,N> value);\n";
+        /// Initialize a vector where all elements have the same scalar `value`.
+    __implicit_conversion($(kConversionCost_ScalarToVector))
+    __intrinsic_op($(kIROp_constructVectorFromScalar))
+    __init(T value);
 
-sb << "};\n";
-}}}}
+        /// Initialize a vector from a value of the same type
+    // TODO: we should revise semantic checking so this kind of "identity" conversion is not required
+    __init(vector<T,N> value);
+}
 
+    /// A matrix with `R` rows and `C` columns, with elements of type `T`.
 __generic<T = float, let R : int = 4, let C : int = 4>
 __magic_type(Matrix)
-struct matrix {};
+struct matrix
+{
+}
 
 ${{{{
 static const struct {
@@ -310,20 +350,24 @@ for (int tt = 0; tt < kTypeCount; ++tt)
 }
 
 // Declare additional built-in generic types
-//        EMIT_LINE_DIRECTIVE();
+}}}}
 
+__generic<T>
+__intrinsic_type($(kIROp_ConstantBufferType))
+__magic_type(ConstantBuffer)
+struct ConstantBuffer {}
 
-sb << "__generic<T>\n";
-sb << "__intrinsic_type(" << kIROp_ConstantBufferType << ")\n";
-sb << "__magic_type(ConstantBuffer) struct ConstantBuffer {};\n";
+__generic<T>
+__intrinsic_type($(kIROp_TextureBufferType))
+__magic_type(TextureBuffer)
+struct TextureBuffer {}
 
-sb << "__generic<T>\n";
-sb << "__intrinsic_type(" << kIROp_TextureBufferType << ")\n";
-sb << "__magic_type(TextureBuffer) struct TextureBuffer {};\n";
+__generic<T>
+__intrinsic_type($(kIROp_ParameterBlockType))
+__magic_type(ParameterBlockType)
+struct ParameterBlock {}
 
-sb << "__generic<T>\n";
-sb << "__intrinsic_type(" << kIROp_ParameterBlockType << ")\n";
-sb << "__magic_type(ParameterBlockType) struct ParameterBlock {};\n";
+${{{{
 
 static const char* kComponentNames[]{ "x", "y", "z", "w" };
 static const char* kVectorNames[]{ "", "x", "xy", "xyz", "xyzw" };
@@ -454,20 +498,25 @@ for( int C = 2; C <= 4; ++C )
 
     sb << "}\n";
 }
-
-// Declare built-in texture and sampler types
+}}}}
 
 
+    /// Sampling state for filtered texture fetches.
+__magic_type(SamplerState, $(int(SamplerStateFlavor::SamplerState)))
+__intrinsic_type($(kIROp_SamplerStateType))
+struct SamplerState
+{
+}
 
-sb << "__magic_type(SamplerState," << int(SamplerStateFlavor::SamplerState) << ")\n";
-sb << "__intrinsic_type(" << kIROp_SamplerStateType << ")\n";
-sb << "struct SamplerState {};";
+    /// Sampling state for filtered texture fetches that include a comparison operation before filtering.
+__magic_type(SamplerState, $(int(SamplerStateFlavor::SamplerComparisonState)))
+__intrinsic_type($(kIROp_SamplerComparisonStateType))
+struct SamplerComparisonState
+{
+}
 
-sb << "__magic_type(SamplerState," << int(SamplerStateFlavor::SamplerComparisonState) << ")\n";
-sb << "__intrinsic_type(" << kIROp_SamplerComparisonStateType << ")\n";
-sb << "struct SamplerComparisonState {};";
+${{{{
 
-// TODO(tfoley): Need to handle `RW*` variants of texture types as well...
 static const struct {
     char const*			    name;
     TextureFlavor::Shape	baseShape;
@@ -1403,6 +1452,9 @@ for (int tt = 0; tt < kBaseTextureTypeCount; ++tt)
 
 for (auto op : unaryOps)
 {
+    char const* fixity = (op.flags & POSTFIX) != 0 ? "__postfix " : "__prefix ";
+    char const* qual = (op.flags & ASSIGNMENT) != 0 ? "in out " : "";
+
     for (auto type : kBaseTypes)
     {
         if ((type.flags & op.flags) == 0)
@@ -1411,9 +1463,6 @@ for (auto op : unaryOps)
         char const* resultType = type.name;
         if (op.flags & BOOL_RESULT) resultType = "bool";
 
-        char const* fixity = (op.flags & POSTFIX) != 0 ? "__postfix " : "__prefix ";
-        char const* qual = (op.flags & ASSIGNMENT) != 0 ? "in out " : "";
-
         // scalar version
         sb << fixity;
         sb << "__intrinsic_op(" << int(op.opCode) << ") " << resultType << " operator" << op.opName << "(" << qual << type.name << " value);\n";
@@ -1428,10 +1477,35 @@ for (auto op : unaryOps)
         sb << fixity;
         sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << qual << "matrix<" << type.name << ",N,M> value);\n";
     }
+
+    // Synthesize generic versions
+    if(op.interface)
+    {
+        char const* resultType = "T";
+        if (op.flags & BOOL_RESULT) resultType = "bool";
+
+        // scalar version
+        sb << fixity;
+        sb << "__generic<T : " << op.interface << ">\n";
+        sb << "__intrinsic_op(" << int(op.opCode) << ") " << resultType << " operator" << op.opName << "(" << qual << "T value);\n";
+
+        // vector version
+        sb << "__generic<T : " << op.interface << ", let N : int> ";
+        sb << fixity;
+        sb << "__intrinsic_op(" << int(op.opCode) << ") vector<" << resultType << ",N> operator" << op.opName << "(" << qual << "vector<T,N> value);\n";
+
+        // matrix version
+        sb << "__generic<T : " << op.interface << ", let N : int, let M : int> ";
+        sb << fixity;
+        sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << qual << "matrix<T,N,M> value);\n";
+    }
 }
 
 for (auto op : binaryOps)
 {
+    char const* leftQual = "";
+    if(op.flags & ASSIGNMENT) leftQual = "in out ";
+
     for (auto type : kBaseTypes)
     {
         if ((type.flags & op.flags) == 0)
@@ -1443,10 +1517,15 @@ for (auto op : binaryOps)
 
         if (op.flags & BOOL_RESULT) resultType = "bool";
 
-        char const* leftQual = "";
-        if(op.flags & ASSIGNMENT) leftQual = "in out ";
-
-        // TODO: handle `SHIFT`
+        // TODO: We should handle a `SHIFT` flag on the op
+        // by changing `rightType` to `int` in order to
+        // account for the fact that the shift amount should
+        // always have a fixed type independent of the LHS.
+        //
+        // (It is unclear why this change hadn't been made
+        // already, so it is possible that such a change
+        // breaks overload resolution or other parts of
+        // the compiler)
 
         // scalar version
         sb << "__intrinsic_op(" << int(op.opCode) << ") " << resultType << " operator" << op.opName << "(" << leftQual << leftType << " left, " << rightType << " right);\n";
@@ -1459,9 +1538,38 @@ for (auto op : binaryOps)
         sb << "__generic<let N : int, let M : int> ";
         sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << leftQual << "matrix<" << leftType << ",N,M> left, matrix<" << rightType << ",N,M> right);\n";
 
-        // We are going to go ahead and explicitly define combined
-        // operations for the scalar-op-vector, etc. cases, rather
-        // than rely on promotion rules.
+        // We currently synthesize addiitonal overloads
+        // for the case where one or the other operand
+        // is a scalar. This choice serves a few purposes:
+        //
+        // 1. It avoids introducing scalar-to-vector or
+        // scalar-to-matrix promotions before the operator,
+        // which might allow some back ends to produce
+        // more optimal code.
+        //
+        // 2. It avoids concerns about making overload resolution
+        // and the inference rules for `N` and `M` able to
+        // handle the mixed vector/scalar or matrix/scalar case.
+        //
+        // 3. Having explicit overloads for the matrix/scalar cases
+        // here means that we do *not* need to support a general
+        // implicit conversion from scalars to matrices, unless
+        // we decide we want to.
+        //
+        // Note: Case (2) of the motivation shouldn't really apply
+        // any more, because we end up having to support similar
+        // inteference for built-in binary math functions where
+        // vectors and scalars might be combined (and where defining
+        // additional overloads to cover all the combinations doesn't
+        // seem practical or desirable).
+        //
+        // TODO: We should consider whether dropping these extra
+        // overloads is possible and worth it. The optimization
+        // concern (1) could possibly be addressed in specific
+        // back-ends. The issue (3) about not wanting to support
+        // implicit scalar-to-matrix conversion may be moot if
+        // we end up needing to support mixed scalar/matrix input
+        // for builtin in non-operator functions anyway.
 
         // scalar-vector and scalar-matrix
         if (!(op.flags & ASSIGNMENT))
@@ -1480,6 +1588,46 @@ for (auto op : binaryOps)
         sb << "__generic<let N : int, let M : int> ";
         sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << leftQual << "matrix<" << leftType << ",N,M> left, " << rightType << " right);\n";
     }
+
+    // Synthesize generic versions
+    if(op.interface)
+    {
+        char const* leftType = "T";
+        char const* rightType = leftType;
+        char const* resultType = leftType;
+
+        if (op.flags & BOOL_RESULT) resultType = "bool";
+        // TODO: handle `SHIFT`
+
+        // scalar version
+        sb << "__generic<T : " << op.interface << ">\n";
+        sb << "__intrinsic_op(" << int(op.opCode) << ") " << resultType << " operator" << op.opName << "(" << leftQual << leftType << " left, " << rightType << " right);\n";
+
+        // vector version
+        sb << "__generic<T : " << op.interface << ", let N : int> ";
+        sb << "__intrinsic_op(" << int(op.opCode) << ") vector<" << resultType << ",N> operator" << op.opName << "(" << leftQual << "vector<" << leftType << ",N> left, vector<" << rightType << ",N> right);\n";
+
+        // matrix version
+        sb << "__generic<T : " << op.interface << ", let N : int, let M : int> ";
+        sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << leftQual << "matrix<" << leftType << ",N,M> left, matrix<" << rightType << ",N,M> right);\n";
+
+        // scalar-vector and scalar-matrix
+        if (!(op.flags & ASSIGNMENT))
+        {
+            sb << "__generic<T : " << op.interface << ", let N : int> ";
+            sb << "__intrinsic_op(" << int(op.opCode) << ") vector<" << resultType << ",N> operator" << op.opName << "(" << leftQual << leftType << " left, vector<" << rightType << ",N> right);\n";
+
+            sb << "__generic<T : " << op.interface << ", let N : int, let M : int> ";
+            sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << leftQual << leftType << " left, matrix<" << rightType << ",N,M> right);\n";
+        }
+
+        // vector-scalar and matrix-scalar
+        sb << "__generic<T : " << op.interface << ", let N : int> ";
+        sb << "__intrinsic_op(" << int(op.opCode) << ") vector<" << resultType << ",N> operator" << op.opName << "(" << leftQual << "vector<" << leftType << ",N> left, " << rightType << " right);\n";
+
+        sb << "__generic<T : " << op.interface << ", let N : int, let M : int> ";
+        sb << "__intrinsic_op(" << int(op.opCode) << ") matrix<" << resultType << ",N,M> operator" << op.opName << "(" << leftQual << "matrix<" << leftType << ",N,M> left, " << rightType << " right);\n";
+    }
 }
 }}}}