Create and cache flattened inheritance lists (#2740)

* Create and cache flattened inheritance lists

The basic change here is to have a cached lookup that can map a `Type`,
or a `DeclRef` that might refer to a type or `extension`, to a list of
the *facets* that comprise it.

The notion of a *facet* here is similar to what the C++ standard calls
"sub-objects".
A declared type like a `struct` has:

* a facet for its own direct members
* one facet for each of its (transitive) base `struct` types
* one facet for each `interface` it conforms to
* one facet for each `extension` that applies to that type

The set of facets for a type is de-duplicated (so that "diamond"
inheritance patterns don't cause issues) and deterministically ordered,
using a variation of the C3 linearization algorithm.

The creation of a linearized list of facets should help the compiler
implementation in two key places:

* Testing if a type implements an interface (or inherits from a base
  type) should now only take time linear in the number of (transitive)
  bases of that type. We can simply scan the linearized facet list to
  see if it contains a facet corresponding to the given base.

* Looking up the members of a type (or a value of a given type) should
  be greatly simplified, since all of the members can be found in a
  single linear scan of the facet list. In addition, those facets will
  be ordered so that facets for "more derived" types will precede those
  for "less derived" types, so that shadowing in the case of overrides
  should be easier to implement.

This change only implements the first of these two improvements, since
there is already a *lot* of churn involved.

Notes and caveats:

* The handling of conjunction types (e.g., `IFoo & IBar`) complicates
  the implementation, both because the simple approach to subtype
  testing alluded to above is no longer complete, and also because
  we need to be more careful about what forms of subtype witnesses
  we construct, so that we can maintain the currently-required invariant
  that two witnesses are only equal if they have matching structure.

* We don't implement the full/"proper" C3 algorithm here because it has
  some failure cases that we'd still like to support. In particular if
  we have both `IX : IA, IB` and `IY : IB, IA`, the C3 algorithm says it
  is illegal to have `IZ : IX, IY` because the two bases it inherits
  from disagree on the relative ordering of `IA` and `IB` in their
  own linearizations. Handling such cases may make our implementation
  less efficient, and it will also require testing of those corner
  caes.

* When it comes time to revamp the implementation of lookup, we will
  need to deal with the fact that a single linear list (seemingly)
  cannot give us sufficient information to decide which of two members
  of the same name should shadow the other, or if there is an ambiguity.
  Or rather, it *can* give us that information if we are willing to
  accept some very user-unfriendly behavior and simply say that
  declarations earlier in the linearization always shadow later
  declarations, even if the facets involved are not related by an
  inheritance relationship of any kind.

* In order to remove one kind of vicious circularity from the approach,
  the linearization that we are computing for `extension` declarations
  will not be sufficient for lookups in the body of such an `extension`.
  A future change may need to have support for creating and caching
  two distinct linearizations for each `extension`: one that is to be
  used when that `extension` is pulled into the linearization for a
  type that it applies to, and another for when lookup will be performed
  in the context of the `extension` itself.

* This change does *not* include the simple expedient of adding a direct
  cache for subtype tests to the `SharedSemanticsContext`, although
  adding such a cache would be a simple matter.

* This change introduces more deduplication for subtype witnesses,
  which should enable more deduplication for other `Val`s (including
  `Type`s), but it does not introduce any assumptions that equal
  `Val`s or `Type`s must have identical pointer representations.

* Eventually we may find that, similar to the situation with `Type`s,
  we will want to have a split between surface-level and canonicalized
  versions of other `Val`s, including subtype witnesses.

* Fix clang error.

* remove debugging code.

---------

Co-authored-by: Yong He <yonghe@outlook.com>

1 files changed, 235 insertions, 1 deletions

diff --git a/source/slang/slang-ast-builder.cpp b/source/slang/slang-ast-builder.cpp
index e6e1b5e75..7f1b90837 100644
--- a/source/slang/slang-ast-builder.cpp
+++ b/source/slang/slang-ast-builder.cpp
@@ -338,7 +338,7 @@ DifferentialPairType* ASTBuilder::getDifferentialPairType(
     return as<DifferentialPairType>(rsType);
 }
 
-DeclRef<InterfaceDecl> ASTBuilder::getDifferentiableInterface()
+DeclRef<InterfaceDecl> ASTBuilder::getDifferentiableInterfaceDecl()
 {
     DeclRef<InterfaceDecl> declRef = DeclRef<InterfaceDecl>(getBuiltinDeclRef("DifferentiableType", nullptr));
     return declRef;
@@ -378,6 +378,11 @@ MeshOutputType* ASTBuilder::getMeshOutputTypeFromModifier(
     return as<MeshOutputType>(rsType);
 }
 
+Type* ASTBuilder::getDifferentiableInterfaceType()
+{
+    return DeclRefType::create(this, getDifferentiableInterfaceDecl());
+}
+
 DeclRef<Decl> ASTBuilder::getBuiltinDeclRef(const char* builtinMagicTypeName, Val* genericArg)
 {
     auto decl = m_sharedASTBuilder->findMagicDecl(builtinMagicTypeName);
@@ -443,6 +448,235 @@ TypeType* ASTBuilder::getTypeType(Type* type)
     return getOrCreate<TypeType>(type);
 }
 
+TypeEqualityWitness* ASTBuilder::getTypeEqualityWitness(
+    Type* type)
+{
+    return getOrCreate<TypeEqualityWitness>(type);
+}
+
+
+SubtypeWitness* ASTBuilder::getDeclaredSubtypeWitness(
+    Type*                   subType,
+    Type*                   superType,
+    DeclRef<Decl> const&    declRef)
+{
+    auto witness = getOrCreate<DeclaredSubtypeWitness>(
+        subType, superType, declRef.declRefBase);
+    return witness;
+}
+
+SubtypeWitness* ASTBuilder::getTransitiveSubtypeWitness(
+    SubtypeWitness* aIsSubtypeOfBWitness,
+    SubtypeWitness* bIsSubtypeOfCWitness)
+{
+top:
+    // Our job is to take the witnesses that `a <: b` and `b <: c`
+    // and produce a valid witness that `a <: c`
+    //
+    // There are some special cases we want to handle, in order
+    // to simplify logic elesewhere in the compiler. For example,
+    // if either of the input witnesses is a type *equality* witness,
+    // then the other witness can be returned as-is.
+    //
+    // If `a == b`, then the `b <: c` witness is also a witness of `a <: c`.
+    //
+    if(as<TypeEqualityWitness>(aIsSubtypeOfBWitness))
+    {
+        return bIsSubtypeOfCWitness;
+    }
+
+    // Similarly, if `b == c`, then the `a <: b` witness is a witness for `a <: c`
+    //
+    if (as<TypeEqualityWitness>(bIsSubtypeOfCWitness))
+    {
+        return aIsSubtypeOfBWitness;
+    }
+
+    // HACK: There is downstream code generation logic that assumes that
+    // a `TransitiveSubtypeWitness` will never have a transitive witness
+    // as its `b <: c` witness. If we are at risk of creating such a witness here,
+    // we will shift things around to make that not be the case:
+    //
+    if (auto bIsTransitiveSubtypeOfCWitness = as<TransitiveSubtypeWitness>(bIsSubtypeOfCWitness))
+    {
+        // Let's call the intermediate type here `x`, we know that the `b <: c`
+        // witness is based on witnesses that `b <: x` and `x <: c`:
+        //
+        auto bIsSubtypeOfXWitness = bIsTransitiveSubtypeOfCWitness->subToMid;
+        auto xIsSubtypeOfCWitness = bIsTransitiveSubtypeOfCWitness->midToSup;
+
+        // We can recursively call this operation to produce a witness that
+        // `a <: x`, based on the witnesses we already have for `a <: b` and `b <: x`:
+        //
+        auto aIsSubtypeOfXWitness = getTransitiveSubtypeWitness(
+            aIsSubtypeOfBWitness,
+            bIsSubtypeOfXWitness);
+
+        // Now we can perform a "tail recursive" call to this function (via `goto`
+        // to combine the `a <: x` witness with our `x <: c` witness:
+        //
+        aIsSubtypeOfBWitness = aIsSubtypeOfXWitness;
+        bIsSubtypeOfCWitness = xIsSubtypeOfCWitness;
+        goto top;
+    }
+
+    auto aType = aIsSubtypeOfBWitness->sub;
+    auto cType = bIsSubtypeOfCWitness->sup;
+
+    // If the right-hand side is a conjunction witness for `B <: C`
+    // of the form `(B <: X)&(B <: Y)`, then we have it that `C = X&Y`
+    // and we'd rather form a conjunction witness for `A <: C`
+    // that is of the form `(A <: X)&(A <: Y)`.
+    //
+    if(auto bIsSubtypeOfXAndY = as<ConjunctionSubtypeWitness>(bIsSubtypeOfCWitness))
+    {
+        auto bIsSubtypeOfXWitness = bIsSubtypeOfXAndY->getLeftWitness();
+        auto bIsSubtypeOfYWitness = bIsSubtypeOfXAndY->getRightWitness();
+
+        return getConjunctionSubtypeWitness(
+            aType,
+            cType,
+            getTransitiveSubtypeWitness(aIsSubtypeOfBWitness, bIsSubtypeOfXWitness),
+            getTransitiveSubtypeWitness(aIsSubtypeOfBWitness, bIsSubtypeOfYWitness));
+    }
+
+    // If the right-hand witness `R` is of the form `extract(i, W)`, then
+    // `W` is a witness that `B <: X&Y&...` for some conjunction, where `C`
+    // is one component of that conjunction.
+    //
+    if(auto bIsSubtypeViaExtraction = as<ExtractFromConjunctionSubtypeWitness>(bIsSubtypeOfCWitness))
+    {
+        // We decompose the witness `extract(i, W)` to get both
+        // the witness `W` that `B <: X&Y&...` as well as the index
+        // `i` of `C` within the conjunction.
+        //
+        auto bIsSubtypeOfConjunction = bIsSubtypeViaExtraction->conjunctionWitness;
+        auto indexOfCInConjunction = bIsSubtypeViaExtraction->indexInConjunction;
+
+        // We lift the extraction to the outside of the composition, by
+        // forming a witness for `A <: C` that is of the form
+        // `extract(i, L . W )`, where `L` is the left-hand witnes (for `A <: B`).
+        // The composition `L . W` is a witness that `A <: X&Y&...`, and
+        // the `i`th component of it should be a witness that `A <: C`.
+        //
+        return getExtractFromConjunctionSubtypeWitness(
+            aType,
+            cType,
+            getTransitiveSubtypeWitness(aIsSubtypeOfBWitness, bIsSubtypeOfConjunction),
+            indexOfCInConjunction);
+    }
+
+    // If none of the above special cases applied, then we are just going to create
+    // a `TransitiveSubtypeWitness` directly.
+    //
+    // TODO: Identify other cases that we can potentially simplify.
+    // It is particularly notable that we do not have simplification rules that
+    // detect when the left-hand side of a composition has some particular
+    // structure. This may be fine, or it may not; we should write down a more
+    // formal set of rules for the allowed structure of our witnesses to
+    // guarantee that our simplifications are sufficient.
+
+    TransitiveSubtypeWitness* transitiveWitness = getOrCreateWithDefaultCtor<TransitiveSubtypeWitness>(
+        aType,
+        cType,
+        aIsSubtypeOfBWitness,
+        bIsSubtypeOfCWitness);
+    transitiveWitness->sub = aType;
+    transitiveWitness->sup = cType;
+    transitiveWitness->subToMid = aIsSubtypeOfBWitness;
+    transitiveWitness->midToSup = bIsSubtypeOfCWitness;
+
+    return transitiveWitness;
+}
+
+SubtypeWitness* ASTBuilder::getExtractFromConjunctionSubtypeWitness(
+    Type*           subType,
+    Type*           superType,
+    SubtypeWitness* conjunctionWitness,
+    int             indexOfSuperTypeInConjunction)
+{
+    // We are taking a witness `W` for `S <: L&R` and
+    // using it to produce a witness for `S <: L`
+    // or `S <: R`.
+
+    // If it turns out that the witness `W` is itself
+    // formed as a conjuction of witnesses: `(S <: L) & (S <: R)`,
+    // then we can simply re-use the appropriate sub-witness.
+    //
+    if (auto conjWitness = as<ConjunctionSubtypeWitness>(conjunctionWitness))
+    {
+        return conjWitness->getComponentWitness(indexOfSuperTypeInConjunction);
+    }
+
+    // TODO: Are there other simplification cases we should be paying attention
+    // to here? For example:
+    //
+    // * What if the original witness is transitive?
+
+    auto witness = getOrCreateWithDefaultCtor<ExtractFromConjunctionSubtypeWitness>(
+        subType,
+        superType,
+        conjunctionWitness,
+        indexOfSuperTypeInConjunction);
+
+    witness->sub = subType;
+    witness->sup = superType;
+    witness->conjunctionWitness = conjunctionWitness;
+    witness->indexInConjunction = indexOfSuperTypeInConjunction;
+    return witness;
+}
+
+SubtypeWitness* ASTBuilder::getConjunctionSubtypeWitness(
+    Type*           sub,
+    Type*           lAndR,
+    SubtypeWitness* subIsLWitness,
+    SubtypeWitness* subIsRWitness)
+{
+    // If a conjunction witness for `S <: L&R` is being formed,
+    // where the constituent witnesses for `S <: L` and `S <: R`
+    // are themselves extractions of the first and second
+    // components, respectively, of a single witness `W`, then
+    // we can simply use `W` as-is.
+    //
+    auto lExtract = as<ExtractFromConjunctionSubtypeWitness>(subIsLWitness);
+    auto rExtract = as<ExtractFromConjunctionSubtypeWitness>(subIsRWitness);
+    if(lExtract && rExtract)
+    {
+        if (lExtract->indexInConjunction == 0
+            && rExtract->indexInConjunction == 1)
+        {
+            auto lInner = lExtract->conjunctionWitness;
+            auto rInner = rExtract->conjunctionWitness;
+            if (lInner == rInner)
+            {
+                return lInner;
+            }
+        }
+    }
+
+    // TODO: Depending on how we decide our canonicalized witnesses
+    // should be structured, we could detect the case where the
+    // `S <: L` and `S <: R` witnesses are both transitive compositions
+    // of the form `X . A` and `X . B`, such that we *could* form
+    // a composition around a conjunction - that is, produce
+    // `X . (A & B)` rather than `(X . A) & (X . B)`.
+    //
+    // For now we are favoring putting the composition (transitive
+    // witness) deeper, so that we have more chances to expose a
+    // conjunction witness at higher levels.
+
+    auto witness = getOrCreateWithDefaultCtor<ConjunctionSubtypeWitness>(
+        sub,
+        lAndR,
+        subIsLWitness,
+        subIsRWitness);
+    witness->componentWitnesses[0] = subIsLWitness;
+    witness->componentWitnesses[1] = subIsRWitness;
+    witness->sub = sub;
+    witness->sup = lAndR;
+    return witness;
+}
+
 bool ASTBuilder::NodeDesc::operator==(NodeDesc const& that) const
 {
     if (hashCode != that.hashCode) return false;