diff options
| author | ArielG-NV <159081215+ArielG-NV@users.noreply.github.com> | 2024-05-16 00:04:12 -0400 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-16 00:04:12 -0400 |
| commit | 1b89f78cd1762aa08402bd656e807b66833b11d0 (patch) | |
| tree | 2be71c9d97af8d28d440981d0c5adc726d9eac56 /source/slang/slang-check-decl.cpp | |
| parent | 3b0de8b6ea484091146f61e663c63beeac5b4798 (diff) | |
Capabilities System, CapabilitySet Logic Overhaul (#4145)
* Capabilities System, Backing Logic Overhaul
Fixes #4015
Problems to address:
1. Currently the capabilities system spends anywhere from 25-50% of compile time on the CapabilityVisitor. Most of this time is spent on join logic: 1. Finding abstract atoms 2. Comparing list1<->list2. This should and can be made significantly faster.
2. Error system does not produce errors with auxiliary information. This will require a partial redesign to provide more useful semantic information for debugging.
What was addressed:
1. Array backed `CapabilityConjunctionSet` was replaced in-favor for a `UIntSet` backed `CapabilityTargetSets`. The design is described below.
Design:
* `CapabilityTargetSets` is a `Dictionary<targetAtom, CapabilityTargetSet>`. This is not an array for 2 reasons: 1. Easy to figure out which target is missing between two `CapabilityTargetSets` 2. To statically allocate an array requires the preprocessor to manually annotate which Capability is a target and link that Capability to an index. This means a dictionary is required for lookup regardless of implementation.
* `CapabilityTargetSet` is an intermediate representation of all capabilities for a singular `target` atom (`glsl`, `hlsl`, `metal`, ...). This structure contains a dictionary to all stage specific capability sets for fast lookup of stage capabilities supported by a `CapabilitySet` for a `target` atom. This reduces number of sets searched.
* `CapabilityStageSet` is an intermediate representation of all capabilities for a singular `stage` atom (`vertex`, `fragment`, ...). This structure holds all disjoint capability sets for a `stage`. A disjoint set is rare, but may exist in some scenarios (as an example): `{glsl, EXT_GL_FOO}{glsl, _GLSL_130, _GLSL_150}`. This reduces the number of sets searched.
* `UIntSet` is the main reason for the redesign for better performance and memory usage. All set operations only require a few operations, making all set logic trivial and with minimal cost to run. All algorithms were modified to focus around `UIntSet` operations.
2. Errors
* Semantic information are now better linked to the calling function to provide a connection of function<->function_body for when saving semantic information for errors.
* Missing targets now print errors much like other error code by finding code which could be a cause of incompatibility.
What is missing:
1. Add non naive support for non-stage specific capabilities such as `{hlsl, _sm_5_0}`. Currently non stage specific targets emulate the behavior through assigning such capabilities to every stage: `{hlsl, _sm_5_0, vertex} {hlsl, _sm_5_0, fragment}...`. Removal of this behavior would remove redundant shader stage sets being made at construction time (~80% of new implementation runtime). This is an addition, not an overhaul.
2. Optionally: `UIntSet` should be modified to support SIMD operations for significantly faster operations. This is not required immediately since `UIntSet` is already not a performance constraint.
Notes:
* UIntSet had implementation bugs which were fixed in this PR.
* The old capabilities system had bugs which were fixed in this PR when transforming to the new implementation.
* fix .natvis debug view
* Small optimizations I found while working on the addition
the AST building pass looks like so now:
1% = ~capabilitySet
2% = capabilitySet()
1.5% capabilitySet::unionWith()
0.8% capabilitySet::join()
1.5% auxillary info for debugging
~0.5-1% extra visitor overhead
~5% total for the visitor
~6.5% for total runtime costs
* fix caps which were wrong but worked
* push minor syntax fix (still looking for why other tests fail)
* perf & bug fixes
1. did not properly remake isBetterForTarget for this->empty case with that as Invalid. This is best case in this senario.
2. Remade seralizer for stdlib generation. Faster (more direct) & cleaner code.
NOTE: did not address review comments
* fix glsl.meta caps error
* fixing findBest logic again & UIntSet wrapper
findBest was not checking for 'more specialized' targets & was element counter was flawed
* faster getElements algorithm + natvis for UIntSet + wrong warning
* type incompatability of bitscanForward implementations
* try to fix warnings again
* remove ptr for clang intrinsic
* add missing header
* ifdef to allow clang compile
* compiler hackery to fix up platform/type independent operations
* bracket
* fix MSVC error
* missing template
* change types out again
* changes to fix compiling
* adjustment to parameter for Clang/GCC
* added iterator to delay processing all atomSets of a CapabilitySet
* add a few missing consts's
* ensure we never have more than 1 disjointSet
Added a wrapper + assert + union functionality to all possible disjoint sets. This was done in favor of a removal of the LinkedList for 2 reasons:
1. We still need 0-1 set functionality.
2. Might as well keep the code, just disallow the problematic functionality.
* address review comments
non linked-list refactor review comments addressed; add doc comments + remove redundant code
* comments + remove isValid for bool operator
* push removal of linkedlist for capabilities
* add missing break
* address review comments
minor adjustments of syntax
* push a fix to the `CapabilitySet({shader, missing target})` code
* quality + error
1. add iterator to UIntSet
2. do not specialize target_switch if profile is derived from case (GLSL_150 is not compatable with GLSL_400)
* fix target_switch erroring + temporarily remove UIntSet::Interator
temporarily remove UIntSet::Interator. It will be added after, testing code on CI first so I can multi-task fixing the UIntSet Iterator
* fix the UIntSet iterator
* Revert "fix the UIntSet iterator" temporarily to pull from master
* add metal error as per texture.slang
(took a while I realize this was why things were breaking, likely should adjust errors to reflect this)
* Rework UIntSet to have a template for output type
This is done so it is reasonable to debug the iterator output and not just dealing with messy int's
Fix problems with the iterators implemented + invalid capabilities handling
* removed incorrect `__target_switch` capability
barycentric was being used with anticipation of `profile glsl450`, this does not expand into `GL_EXT_fragment_shader_barycentric`, this instead caused an error which is hidden during cross-compile.
* remove some uses of getElements
* remove undeclared_stage for now
* remove redundant code associated with `undeclared_stage`
* remove unused variable
* address review
specifically to note removed static in a thread dangerous scope. Now using a `const static` for read only (thread safe) which precompile steps generate
* move GLSL_150 capdef change to sm_4_1 (more accurate)
* address most review comments
did not address: https://github.com/shader-slang/slang/pull/4145#discussion_r1602256776
* revert incorrect code review suggestion
* push changes for all code review suggestions
Diffstat (limited to 'source/slang/slang-check-decl.cpp')
| -rw-r--r-- | source/slang/slang-check-decl.cpp | 244 |
1 files changed, 81 insertions, 163 deletions
diff --git a/source/slang/slang-check-decl.cpp b/source/slang/slang-check-decl.cpp index 9a4ee9d71..fbf5332b8 100644 --- a/source/slang/slang-check-decl.cpp +++ b/source/slang/slang-check-decl.cpp @@ -744,7 +744,7 @@ namespace Slang void visitInheritanceDecl(InheritanceDecl* inheritanceDecl); - void diagnoseUndeclaredCapability(Decl* decl, const DiagnosticInfo& diagnosticInfo, const CapabilityConjunctionSet* failedAvailableSet); + void diagnoseUndeclaredCapability(Decl* decl, const DiagnosticInfo& diagnosticInfo, const CapabilityAtomSet& failedAtomsInsideAvailableSet); }; @@ -9932,11 +9932,17 @@ namespace Slang oldCaps); } } + + // if stmt inside parent, set the provenance tracker to the calling function + if(!decl) + decl = visitor->getParentFuncOfVisitor(); if (referencedDecl && decl) { - for (auto& capSet : nodeCaps.getExpandedAtoms()) + for (auto& capSet : nodeCaps.getAtomSets()) { - for (auto atom : capSet.getExpandedAtoms()) + auto elements = capSet.getElements<CapabilityAtom>(); + decl->capabilityRequirementProvenance.reserve(decl->capabilityRequirementProvenance.getCount()+elements.getCount()); + for (auto atom : elements) { decl->capabilityRequirementProvenance.addIfNotExists(atom, DeclReferenceWithLoc{ referencedDecl, referenceLoc }); } @@ -10008,9 +10014,9 @@ namespace Slang } if (!maybeRequireCapability) - targetCap = (CapabilitySet(CapabilityName::any_target).getTargetsThisIsMissingFromOther(set)); + targetCap = (CapabilitySet(CapabilityName::any_target).getTargetsThisHasButOtherDoesNot(set)); else - targetCap = (maybeRequireCapability->capabilitySet.getTargetsThisIsMissingFromOther(set)); + targetCap = (maybeRequireCapability->capabilitySet.getTargetsThisHasButOtherDoesNot(set)); } else { @@ -10024,10 +10030,8 @@ namespace Slang { diagnoseCapabilityErrors(Base::getSink(), outerContext.getOptionSet(), targetCase->body->loc, Diagnostics::conflictingCapabilityDueToStatement, bodyCap, "target_switch", oldCap); } - for (auto& conjunction : targetCap.getExpandedAtoms()) - set.unionWith(conjunction); + set.unionWith(targetCap); } - set.canonicalize(); handleReferenceFunc(stmt, set, stmt->loc); } @@ -10092,8 +10096,7 @@ namespace Slang { for (auto decoration : parent->getModifiersOfType<RequireCapabilityAttribute>()) { - for (auto& set : decoration->capabilitySet.getExpandedAtoms()) - localDeclaredCaps.unionWith(set); + localDeclaredCaps.unionWith(decoration->capabilitySet); } } else @@ -10102,13 +10105,8 @@ namespace Slang shouldBreak = true; } // Merge decl's capability declaration with the parent. - for (auto& localConjunction : localDeclaredCaps.getExpandedAtoms()) - { - if (declaredCaps.isIncompatibleWith(localConjunction)) - declaredCaps.unionWith(localConjunction); - else - declaredCaps.join(localDeclaredCaps); - } + declaredCaps.nonDestructiveJoin(localDeclaredCaps); + // If the parent already has inferred capability requirements, we should stop now // since that already covers transitive parents. if (shouldBreak) @@ -10127,27 +10125,37 @@ namespace Slang decl->inferredCapabilityRequirements = getDeclaredCapabilitySet(decl); } - void SemanticsDeclCapabilityVisitor::visitFunctionDeclBase(FunctionDeclBase* funcDecl) + template<typename ProcessFunc> + static inline void _dispatchCapabilitiesVisitorOfFunctionDecl(SemanticsVisitor* visitor, FunctionDeclBase* funcDecl, ProcessFunc propegateFuncForReferences) { + visitor->setParentFuncOfVisitor(funcDecl); + for (auto member : funcDecl->members) { - ensureDecl(member, DeclCheckState::CapabilityChecked); - _propagateRequirement(this, funcDecl->inferredCapabilityRequirements, funcDecl, member, member->inferredCapabilityRequirements, member->loc); + visitor->ensureDecl(member, DeclCheckState::CapabilityChecked); + _propagateRequirement(visitor, funcDecl->inferredCapabilityRequirements, funcDecl, member, member->inferredCapabilityRequirements, member->loc); } - visitReferencedDecls(*this, funcDecl->body, funcDecl->loc, funcDecl->findModifier<RequireCapabilityAttribute>(), [this, funcDecl](SyntaxNode* node, const CapabilitySet& nodeCaps, SourceLoc refLoc) - { - _propagateRequirement(this, funcDecl->inferredCapabilityRequirements, funcDecl, node, nodeCaps, refLoc); - }); + + visitReferencedDecls(*visitor, funcDecl->body, funcDecl->loc, funcDecl->findModifier<RequireCapabilityAttribute>(), propegateFuncForReferences); if (!isEffectivelyStatic(funcDecl)) { auto parentAggTypeDecl = getParentAggTypeDecl(funcDecl); if (parentAggTypeDecl) { - ensureDecl(parentAggTypeDecl, DeclCheckState::CapabilityChecked); - _propagateRequirement(this, funcDecl->inferredCapabilityRequirements, funcDecl, parentAggTypeDecl, parentAggTypeDecl->inferredCapabilityRequirements, funcDecl->loc); + visitor->ensureDecl(parentAggTypeDecl, DeclCheckState::CapabilityChecked); + _propagateRequirement(visitor, funcDecl->inferredCapabilityRequirements, funcDecl, parentAggTypeDecl, parentAggTypeDecl->inferredCapabilityRequirements, funcDecl->loc); } } + } + + void SemanticsDeclCapabilityVisitor::visitFunctionDeclBase(FunctionDeclBase* funcDecl) + { + _dispatchCapabilitiesVisitorOfFunctionDecl(this, funcDecl, + [this, funcDecl](SyntaxNode* node, const CapabilitySet& nodeCaps, SourceLoc refLoc) + { + _propagateRequirement(this, funcDecl->inferredCapabilityRequirements, funcDecl, node, nodeCaps, refLoc); + }); auto declaredCaps = getDeclaredCapabilitySet(funcDecl); @@ -10169,26 +10177,12 @@ namespace Slang } auto vis = getDeclVisibility(funcDecl); + + // If 0 capabilities were annotated on a function, capabilities are inferred from the function body if (declaredCaps.isEmpty()) { - // If the user has not declared any capabilities, - // we should diagnose a warning if any_target is not - // a super-set by exact atoms. - if (vis == DeclVisibility::Public && !funcDecl->inferredCapabilityRequirements.isEmpty()) - { - if (!getModuleDecl(funcDecl)->isInLegacyLanguage) - { - if (!funcDecl->inferredCapabilityRequirements.isExactSubset(getAnyPlatformCapabilitySet())) - { - diagnoseCapabilityErrors( - getSink(), - this->getOptionSet(), - funcDecl->loc, - Diagnostics::missingCapabilityRequirementOnPublicDecl, - funcDecl, funcDecl->inferredCapabilityRequirements); - } - } - } + declaredCaps = funcDecl->inferredCapabilityRequirements; + return; } else { @@ -10199,7 +10193,7 @@ namespace Slang // At a minimum we will propagate shader requirements to our // function from calling children in all cases so the parent // can enforce shader targets correctly and propagate to `main` - const CapabilityConjunctionSet* failedAvailableCapabilityConjunction = nullptr; + CapabilityAtomSet failedAvailableCapabilityConjunction; if (!CapabilitySet::checkCapabilityRequirement( declaredCaps, funcDecl->inferredCapabilityRequirements, @@ -10209,7 +10203,7 @@ namespace Slang funcDecl->inferredCapabilityRequirements = declaredCaps; } else - funcDecl->inferredCapabilityRequirements.simpleJoinWithSetMask(declaredCaps, CapabilityName::stage); + funcDecl->inferredCapabilityRequirements.nonDestructiveJoin(declaredCaps); } else { @@ -10241,7 +10235,7 @@ namespace Slang ensureDecl(requirementDecl, DeclCheckState::CapabilityChecked); ensureDecl(implDecl.declRefBase, DeclCheckState::CapabilityChecked); - const CapabilityConjunctionSet* failedAvailableCapabilityConjunction = nullptr; + CapabilityAtomSet failedAvailableCapabilityConjunction; if (!CapabilitySet::checkCapabilityRequirement( requirementDecl->inferredCapabilityRequirements, implDecl.getDecl()->inferredCapabilityRequirements, @@ -10303,7 +10297,7 @@ namespace Slang return defaultVis; } - void diagnoseCapabilityProvenance(CompilerOptionSet& optionSet, DiagnosticSink* sink, Decl* decl, CapabilityAtom missingAtom) + void diagnoseCapabilityProvenance(CompilerOptionSet& optionSet, DiagnosticSink* sink, Decl* decl, CapabilityAtom atomToFind, bool optionallyNeverPrintDecl) { HashSet<Decl*> printedDecls; auto thisModule = getModuleDecl(decl); @@ -10311,9 +10305,9 @@ namespace Slang while (declToPrint) { printedDecls.add(declToPrint); - if (auto provenance = declToPrint->capabilityRequirementProvenance.tryGetValue(missingAtom)) + if (auto provenance = declToPrint->capabilityRequirementProvenance.tryGetValue(atomToFind)) { - sink->diagnose(provenance->referenceLoc, Diagnostics::seeUsingOf, provenance->referencedDecl); + diagnoseCapabilityErrors(sink, optionSet, provenance->referenceLoc, Diagnostics::seeUsingOf, provenance->referencedDecl); declToPrint = provenance->referencedDecl; if (printedDecls.contains(declToPrint)) break; @@ -10332,54 +10326,17 @@ namespace Slang break; } } - if (declToPrint) + if (declToPrint && !optionallyNeverPrintDecl) { diagnoseCapabilityErrors(sink, optionSet, declToPrint->loc, Diagnostics::seeDefinitionOf, declToPrint); } } - // Print diagnostics tracing which referenced decls are not compatible with the given atom. - void diagnoseIncompatibleAtomProvenance(SemanticsVisitor* visitor, DiagnosticSink* sink, Decl* decl, CapabilityAtom incompatibleAtom, int traceLevels = 10) + void SemanticsDeclCapabilityVisitor::diagnoseUndeclaredCapability(Decl* decl, const DiagnosticInfo& diagnosticInfo, const CapabilityAtomSet& failedAtomsInsideAvailableSet) { - Decl* refDecl = nullptr; - SourceLoc loc; - HashSet<Decl*> printedDecls; - while (traceLevels > 0) - { - refDecl = nullptr; - visitReferencedDecls(*visitor, decl, decl->loc, decl->findModifier<RequireCapabilityAttribute>(), [&](SyntaxNode* node, const CapabilitySet& nodeCaps, SourceLoc refLoc) - { - if (nodeCaps.isIncompatibleWith(incompatibleAtom)) - { - if (auto referencedDecl = as<Decl>(node)) - { - refDecl = referencedDecl; - loc = refLoc; - } - else - diagnoseCapabilityErrors(sink, visitor->getOptionSet(), refLoc, Diagnostics::seeDefinitionOf, "statement"); - } - }); - if (!refDecl) - break; - if (printedDecls.add(refDecl)) - { - diagnoseCapabilityErrors(sink, visitor->getOptionSet(), loc, Diagnostics::seeUsingOf, refDecl); - decl = refDecl; - } - else - { - break; - } - traceLevels--; - } - } - - void SemanticsDeclCapabilityVisitor::diagnoseUndeclaredCapability(Decl* decl, const DiagnosticInfo& diagnosticInfo, const CapabilityConjunctionSet* failedAvailableSet) - { - if (decl->inferredCapabilityRequirements.getExpandedAtoms().getCount() == 0) + if (decl->inferredCapabilityRequirements.isEmpty()) return; - if(!failedAvailableSet) + if(failedAtomsInsideAvailableSet.isEmpty() || failedAtomsInsideAvailableSet.contains((UInt)CapabilityAtom::Invalid)) return; // There are two causes for why type checking failed on failedAvailableSet. @@ -10394,90 +10351,51 @@ namespace Slang // } // In this case we should diagnose error reporting printf isn't defined on a required target. // - // The second scenario is when the callee is using a capability that is not provided by the requirement. - // For example: - // [require(hlsl,b,c)] - // void caller() - // { - // useD(); // require capability (hlsl,d) - // } - // In this case we should report that useD() is using a capability that is not declared by caller. - // - // Now, we detect if we are case 1. - if (decl->inferredCapabilityRequirements.isIncompatibleWith(*failedAvailableSet)) + { - // Find the most derived atom that is leading to the incompatiblity. - for (Index i = failedAvailableSet->getExpandedAtoms().getCount() - 1; i >= 0; i--) + CapabilityAtom outFailedAtom{}; + if (hasTargetAtom(failedAtomsInsideAvailableSet, outFailedAtom)) { - auto atom = failedAvailableSet->getExpandedAtoms()[i]; - if (!isDirectChildOfAbstractAtom(atom)) - continue; - if (decl->inferredCapabilityRequirements.isIncompatibleWith(atom)) + diagnoseCapabilityErrors(getSink(), this->getOptionSet(), decl->loc, Diagnostics::declHasDependenciesNotCompatibleOnTarget, decl, outFailedAtom); + + // Anything defined on a non-failed target atom may be the culprit to why we fail having a target capability. + // Print out all possible culprits. + CapabilityAtomSet failedAtomSet; + failedAtomSet.add((UInt)outFailedAtom); + CapabilityAtomSet targetsNotUsedSet; + CapabilityAtomSet::calcSubtract(targetsNotUsedSet, getAtomSetOfTargets(), failedAtomSet); + + for (auto atom : targetsNotUsedSet) { - diagnoseCapabilityErrors(getSink(), this->getOptionSet(), decl->loc, Diagnostics::declHasDependenciesNotDefinedOnTarget, decl, atom); - diagnoseIncompatibleAtomProvenance(this, getSink(), decl, atom); - return; + CapabilityAtom formattedAtom = (CapabilityAtom)atom; + diagnoseCapabilityProvenance(this->getOptionSet(), getSink(), decl, formattedAtom, true); } + return; } - return; } - // If we reach here, we are case 2. + //// The second scenario is when the callee is using a capability that is not provided by the requirement. + //// For example: + //// [require(hlsl,b,c)] + //// void caller() + //// { + //// useD(); // require capability (hlsl,d) + //// } + //// In this case we should report that useD() is using a capability that is not declared by caller. + //// - CapabilityConjunctionSet* matchingRequirement = &decl->inferredCapabilityRequirements.getExpandedAtoms().getFirst(); - CapabilityAtom missingAtom = matchingRequirement->getExpandedAtoms().getFirst(); - if (missingAtom == CapabilityAtom::Invalid) - return; + //// If we reach here, we are case 2. - if (failedAvailableSet) + // We will produce all failed atoms. This is important since provenance of multiple atoms + // can come from multiple referenced items in a function body. + for (auto i : failedAtomsInsideAvailableSet) { - Int maxIntersectionCount = 0; - for (auto& usedSet : decl->inferredCapabilityRequirements.getExpandedAtoms()) - { - auto intersection = usedSet.countIntersectionWith(*failedAvailableSet); - if (intersection > maxIntersectionCount) - { - matchingRequirement = &usedSet; - maxIntersectionCount = intersection; - } - } - Index pos = 0; - for (Index i = 0; i < matchingRequirement->getExpandedAtoms().getCount(); i++) - { - auto atom = matchingRequirement->getExpandedAtoms()[i]; - while (pos < failedAvailableSet->getExpandedAtoms().getCount()) - { - if (failedAvailableSet->getExpandedAtoms()[pos] < atom) - pos++; - else - break; - } - - if (pos >= failedAvailableSet->getExpandedAtoms().getCount() || - failedAvailableSet->getExpandedAtoms()[pos] != atom) - { - missingAtom = atom; - break; - } - } - - // Select the most derived atom of `missingAtom`. - for (Index i = matchingRequirement->getExpandedAtoms().getCount() - 1; i >= 0 ; i--) - { - auto atom = matchingRequirement->getExpandedAtoms()[i]; - if (CapabilityConjunctionSet(atom).implies(missingAtom)) - { - missingAtom = atom; - break; - } - } + CapabilityAtom formattedAtom = (CapabilityAtom)i; + diagnoseCapabilityErrors(getSink(), this->getOptionSet(), decl->loc, diagnosticInfo, decl, formattedAtom); + // Print provenances. + diagnoseCapabilityProvenance(this->getOptionSet(), getSink(), decl, formattedAtom); } - - diagnoseCapabilityErrors(getSink(), this->getOptionSet(), decl->loc, diagnosticInfo, decl, missingAtom); - - // Print provenances. - diagnoseCapabilityProvenance(this->getOptionSet(), getSink(), decl, missingAtom); } } |
