diff options
| author | Jay Kwak <82421531+jkwak-work@users.noreply.github.com> | 2025-05-17 02:26:44 +0000 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-05-16 19:26:44 -0700 |
| commit | d58243d9041947c99f18b82385e62c082507decb (patch) | |
| tree | e51c9543dc39f269cea3f7b276edd8ebccddd14c | |
| parent | 0be6970c01d212aac7ea7db9b6c1556c530d6889 (diff) | |
Support Vulkan memory model (#7057)
The user can explicitly use Vulkan memory model, or it will be
automatically used when cooperative-matrix is used.
When vulkan memory model is used, two keywords, "Coherent" and
"Volatile", are not allowed.
There are many differences regarding atomic and texture but
this PR has changes limited to support `globallycoherent`
keyword. When variables with `globallycoherent` is used with `OpLoad`, it
will use additional options, `MakePointerAvailable|NonPrivatePointer`,
that will provide the same effect. For `OpStore`, it will use
`MakePointerVisible|NonPrivatePointer`.
45 files changed, 528 insertions, 164 deletions
diff --git a/docs/command-line-slangc-reference.md b/docs/command-line-slangc-reference.md index 296441a2e..89e7f0b4c 100644 --- a/docs/command-line-slangc-reference.md +++ b/docs/command-line-slangc-reference.md @@ -1162,6 +1162,8 @@ A capability describes an optional feature that a target may or may not support. * `spvTensorAddressingNV` * `spvMaximalReconvergenceKHR` * `spvQuadControlKHR` +* `spvVulkanMemoryModelKHR` +* `spvVulkanMemoryModelDeviceScopeKHR` * `metallib_latest` * `dxil_lib` * `any_target` @@ -1294,6 +1296,7 @@ A capability describes an optional feature that a target may or may not support. * `cooperative_matrix_block_load` * `tensor_addressing` * `cooperative_matrix_2` +* `vk_mem_model` * `pixel` * `tesscontrol` * `tesseval` diff --git a/docs/user-guide/a2-01-spirv-target-specific.md b/docs/user-guide/a2-01-spirv-target-specific.md index 4bdac1953..1ab143cb0 100644 --- a/docs/user-guide/a2-01-spirv-target-specific.md +++ b/docs/user-guide/a2-01-spirv-target-specific.md @@ -14,6 +14,21 @@ Experimental support for the older versions of SPIR-V Slang's SPIR-V backend is stable when emitting SPIR-V 1.3 and later, however, support for SPIR-V 1.0, 1.1 and 1.2 is still experimental. When targeting the older SPIR-V profiles, Slang may produce SPIR-V that uses the instructions and keywords that were introduced in the later versions of SPIR-V. +Memory model +------------ +By default, the Slang compiler produces the SPIRV code using GLSL450 memory model. The user can opt-in to Vulkan Memory Model by specifying `-capability vk_mem_model`. For using APIs, the following lines can be added to explicitly specify the capability. +``` + slang::CompilerOptionEntry entry; + entry.name = slang::CompilerOptionName::Capability; + entry.value.kind = slang::CompilerOptionValueKind::String; + entry.value.stringValue0 = "vk_mem_model"; + + slang::SessionDesc sessionDesc = {}; + sessionDesc.compilerOptionEntries = &entry; + sessionDesc.compilerOptionEntryCount = 1; +``` + +If the shader uses `CoopVec` or `CoopMat` intrinsics, then the Slang compiler will automatically use `vk_mem_model` capability. Combined texture sampler ------------------------ diff --git a/docs/user-guide/a3-02-reference-capability-atoms.md b/docs/user-guide/a3-02-reference-capability-atoms.md index 549d98ece..4596e8fed 100644 --- a/docs/user-guide/a3-02-reference-capability-atoms.md +++ b/docs/user-guide/a3-02-reference-capability-atoms.md @@ -606,6 +606,12 @@ Extensions `spvQuadControlKHR` > Represents the SPIR-V capability for quad group control. +`spvVulkanMemoryModelKHR` +> Represents the SPIR-V capability for vulkan memory model. + +`spvVulkanMemoryModelDeviceScopeKHR` +> Represents the SPIR-V capability for vulkan memory model. + `GL_EXT_buffer_reference` > Represents the GL_EXT_buffer_reference extension. @@ -972,6 +978,9 @@ Compound Capabilities `cooperative_vector_training` > Capabilities needed to train cooperative vectors +`cooperative_matrix` +> Capabilities needed to use cooperative matrices + `cooperative_matrix_reduction` > Capabilities needed to use reduction operations with cooperative matrix @@ -993,6 +1002,9 @@ Compound Capabilities `cooperative_matrix_2` > Capabilities needed to use tensor addressing +`vk_mem_model` +> Capabilities needed to use vulkan memory model + `any_stage` > Collection of all shader stages @@ -1301,9 +1313,6 @@ Other ---------------------- *Capabilities which may be deprecated* -`cooperative_matrix` -> Capabilities needed to use cooperative matrices - `SPIRV_1_0` > Use `spirv_1_0` instead diff --git a/docs/user-guide/toc.html b/docs/user-guide/toc.html index a60780e65..ace7da1d0 100644 --- a/docs/user-guide/toc.html +++ b/docs/user-guide/toc.html @@ -192,6 +192,7 @@ <li data-link="spirv-target-specific"><span>SPIR-V-Specific Functionalities</span> <ul class="toc_list"> <li data-link="spirv-target-specific#experimental-support-for-the-older-versions-of-spir-v"><span>Experimental support for the older versions of SPIR-V</span></li> +<li data-link="spirv-target-specific#memory-model"><span>Memory model</span></li> <li data-link="spirv-target-specific#combined-texture-sampler"><span>Combined texture sampler</span></li> <li data-link="spirv-target-specific#system-value-semantics"><span>System-Value semantics</span></li> <li data-link="spirv-target-specific#using-sv_instanceid-with-spir-v-target"><span>Using SV_InstanceID with SPIR-V target</span></li> diff --git a/source/slang/slang-capabilities.capdef b/source/slang/slang-capabilities.capdef index 9a1a89bfe..1799d4bfc 100644 --- a/source/slang/slang-capabilities.capdef +++ b/source/slang/slang-capabilities.capdef @@ -587,6 +587,7 @@ def SPV_NV_tensor_addressing : _spirv_1_6; /// [EXT] def SPV_NV_cooperative_matrix2 : SPV_NV_tensor_addressing + SPV_KHR_cooperative_matrix; + // SPIRV Capabilities. /// Represents the SPIR-V capability for atomic float 32 add operations. @@ -781,6 +782,14 @@ def spvMaximalReconvergenceKHR : SPV_KHR_maximal_reconvergence; /// [EXT] def spvQuadControlKHR : SPV_KHR_quad_control; +/// Represents the SPIR-V capability for vulkan memory model. +/// [EXT] +def spvVulkanMemoryModelKHR : SPV_KHR_vulkan_memory_model; + +/// Represents the SPIR-V capability for vulkan memory model. +/// [EXT] +def spvVulkanMemoryModelDeviceScopeKHR : SPV_KHR_vulkan_memory_model; + // The following capabilities all pertain to how ray tracing shaders are translated // to GLSL, where there are two different extensions that can provide the core // functionality of `TraceRay` and the related operations. @@ -1169,6 +1178,7 @@ alias cooperative_vector = _sm_6_9 | cpp | _cuda_sm_9_0 | spvCooperativeVectorNV alias cooperative_vector_training = spvCooperativeVectorTrainingNV; /// Capabilities needed to use cooperative matrices +/// [Compound] alias cooperative_matrix = spvCooperativeMatrixKHR; /// Capabilities needed to use reduction operations with cooperative matrix /// [Compound] @@ -1192,6 +1202,10 @@ alias tensor_addressing = spvTensorAddressingNV; /// [Compound] alias cooperative_matrix_2 = spvCooperativeMatrixKHR + spvCooperativeMatrixReductionsNV + spvCooperativeMatrixConversionsNV + spvCooperativeMatrixPerElementOperationsNV + spvCooperativeMatrixTensorAddressingNV + spvCooperativeMatrixBlockLoadsNV + spvTensorAddressingNV; +/// Capabilities needed to use vulkan memory model +/// [Compound] +alias vk_mem_model = spvVulkanMemoryModelKHR + spvVulkanMemoryModelDeviceScopeKHR; + // Non-internal shader stages // /// Pixel shader stage diff --git a/source/slang/slang-emit-spirv.cpp b/source/slang/slang-emit-spirv.cpp index 54417899c..5c1ccaf36 100644 --- a/source/slang/slang-emit-spirv.cpp +++ b/source/slang/slang-emit-spirv.cpp @@ -1507,7 +1507,104 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex getSection(SpvLogicalSectionID::MemoryModel), nullptr, m_addressingMode, - SpvMemoryModelGLSL450); + m_memoryModel); + + if (m_memoryModel == SpvMemoryModelVulkan) + { + requireSPIRVCapability(SpvCapabilityVulkanMemoryModel); + ensureExtensionDeclaration(UnownedStringSlice("SPV_KHR_vulkan_memory_model")); + + auto targetCaps = m_targetProgram->getTargetReq()->getTargetCaps(); + if (targetCaps.implies(CapabilityAtom::spvVulkanMemoryModelDeviceScopeKHR)) + { + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + } + } + } + + bool NeedToUseCoherentLoadOrStore(IRInst* pointer) + { + if (m_memoryModel != SpvMemoryModelVulkan) + return false; + + auto ptrType = as<IRPtrTypeBase>(pointer->getFullType()); + if (!ptrType) + return false; + + SpvStorageClass storageClass = SpvStorageClassFunction; + if (ptrType->hasAddressSpace()) + storageClass = addressSpaceToStorageClass(ptrType->getAddressSpace()); + + // "NonPrivatePointerKHR requires a pointer in Uniform, Workgroup, CrossWorkgroup, Generic, + // Image or StorageBuffer storage classes." + switch (storageClass) + { + case SpvStorageClassUniform: + case SpvStorageClassWorkgroup: + case SpvStorageClassCrossWorkgroup: + case SpvStorageClassGeneric: + case SpvStorageClassImage: + case SpvStorageClassStorageBuffer: + break; + default: + return false; + } + + IRInst* baseObj = pointer; + while (baseObj) + { + baseObj = getRootAddr(baseObj); + switch (baseObj->getOp()) + { + case kIROp_RWStructuredBufferGetElementPtr: + baseObj = baseObj->getOperand(0); + continue; + default: + break; + } + break; + } + + if (baseObj == nullptr) + return false; + + for (auto decoration : baseObj->getDecorations()) + { + if (decoration->getOp() == kIROp_MemoryQualifierSetDecoration) + { + auto collection = as<IRMemoryQualifierSetDecoration>(decoration); + IRIntegerValue flags = collection->getMemoryQualifierBit(); + if (flags & MemoryQualifierSetModifier::Flags::kCoherent) + { + return true; + } + } + } + return false; + } + + bool NeedToUseCoherentImageLoadOrStore(IRInst* image) + { + if (m_memoryModel != SpvMemoryModelVulkan) + return false; + + if (auto opLoad = as<IRLoad>(image->getOperand(0))) + { + auto texPtr = opLoad->getPtr(); + for (auto decoration : texPtr->getDecorations()) + { + if (decoration->getOp() == kIROp_MemoryQualifierSetDecoration) + { + auto collection = as<IRMemoryQualifierSetDecoration>(decoration); + IRIntegerValue flags = collection->getMemoryQualifierBit(); + if (flags & MemoryQualifierSetModifier::Flags::kCoherent) + { + return true; + } + } + } + } + return false; } IRInst* m_defaultDebugSource = nullptr; @@ -4208,6 +4305,9 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex break; case kIROp_AtomicInc: { + if (m_memoryModel == SpvMemoryModelVulkan) + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + IRBuilder builder{inst}; const auto memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); @@ -4225,6 +4325,9 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex break; case kIROp_AtomicDec: { + if (m_memoryModel == SpvMemoryModelVulkan) + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + IRBuilder builder{inst}; const auto memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); @@ -4245,6 +4348,9 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex IRBuilder builder{inst}; if (isAtomicableAddressSpace(inst->getOperand(0)->getDataType())) { + if (m_memoryModel == SpvMemoryModelVulkan) + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + const auto memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); const auto memorySemantics = @@ -4260,7 +4366,7 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex } else { - result = emitOpLoad(parent, inst, inst->getFullType(), inst->getOperand(0)); + result = emitLoadMaybeCoherent(parent, inst); } } break; @@ -4269,6 +4375,9 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex IRBuilder builder{inst}; if (isAtomicableAddressSpace(inst->getOperand(0)->getDataType())) { + if (m_memoryModel == SpvMemoryModelVulkan) + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + const auto memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); const auto memorySemantics = @@ -4284,7 +4393,7 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex } else { - result = emitOpStore(parent, inst, inst->getOperand(0), inst->getOperand(1)); + result = emitStoreMaybeCoherent(parent, inst); } } break; @@ -4293,6 +4402,9 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex IRBuilder builder{inst}; if (isAtomicableAddressSpace(inst->getOperand(0)->getDataType())) { + if (m_memoryModel == SpvMemoryModelVulkan) + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + const auto memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); const auto memorySemantics = @@ -4309,12 +4421,15 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex } else { - result = emitOpStore(parent, inst, inst->getOperand(0), inst->getOperand(1)); + result = emitStoreMaybeCoherent(parent, inst); } } break; case kIROp_AtomicCompareExchange: { + if (m_memoryModel == SpvMemoryModelVulkan) + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + IRBuilder builder{inst}; const auto memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); @@ -4343,6 +4458,9 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex case kIROp_AtomicOr: case kIROp_AtomicXor: { + if (m_memoryModel == SpvMemoryModelVulkan) + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + IRBuilder builder{inst}; const auto memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); @@ -4443,56 +4561,80 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex SpvInst* emitImageLoad(SpvInstParent* parent, IRImageLoad* load) { - if (load->hasAuxCoord1()) - { - return emitInst( - parent, - load, - SpvOpImageRead, - load->getDataType(), - kResultID, - load->getImage(), - load->getCoord(), - SpvImageOperandsSampleMask, - load->getAuxCoord1()); - } - else + IRBuilder builder(load); + builder.setInsertBefore(load); + + SpvInst* memoryScope = nullptr; + bool coherentImage = NeedToUseCoherentImageLoadOrStore(load); + if (coherentImage) { - return emitInst( - parent, - load, - SpvOpImageRead, - load->getDataType(), - kResultID, - load->getImage(), - load->getCoord()); + memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); } + + return emitInstCustomOperandFunc( + parent, + load, + SpvOpImageRead, + [&]() + { + emitOperand(load->getDataType()); + emitOperand(kResultID); + emitOperand(load->getImage()); + emitOperand(load->getCoord()); + + if (load->hasAuxCoord1()) + { + emitOperand(SpvImageOperandsSampleMask); + emitOperand(load->getAuxCoord1()); + } + + if (coherentImage) + { + emitOperand( + SpvImageOperandsMakeTexelVisibleMask | SpvImageOperandsNonPrivateTexelMask); + + emitOperand(memoryScope); + } + }); } SpvInst* emitImageStore(SpvInstParent* parent, IRImageStore* store) { - if (store->hasAuxCoord1()) - { - return emitInst( - parent, - store, - SpvOpImageWrite, - store->getImage(), - store->getCoord(), - store->getValue(), - SpvImageOperandsSampleMask, - store->getAuxCoord1()); - } - else + IRBuilder builder(store); + builder.setInsertBefore(store); + + SpvInst* memoryScope = nullptr; + bool coherentImage = NeedToUseCoherentImageLoadOrStore(store); + if (coherentImage) { - return emitInst( - parent, - store, - SpvOpImageWrite, - store->getImage(), - store->getCoord(), - store->getValue()); + memoryScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); } + + return emitInstCustomOperandFunc( + parent, + store, + SpvOpImageWrite, + [&]() + { + emitOperand(store->getImage()); + emitOperand(store->getCoord()); + emitOperand(store->getValue()); + + if (store->hasAuxCoord1()) + { + emitOperand(SpvImageOperandsSampleMask); + emitOperand(store->getAuxCoord1()); + } + + if (coherentImage) + { + emitOperand( + SpvImageOperandsMakeTexelAvailableMask | + SpvImageOperandsNonPrivateTexelMask); + + emitOperand(memoryScope); + } + }); } SpvInst* emitImageSubscript(SpvInstParent* parent, IRImageSubscript* subscript) @@ -5194,21 +5336,28 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex { auto collection = as<IRMemoryQualifierSetDecoration>(decoration); IRIntegerValue flags = collection->getMemoryQualifierBit(); - if (flags & MemoryQualifierSetModifier::Flags::kCoherent) - { - emitOpDecorate( - getSection(SpvLogicalSectionID::Annotations), - nullptr, - dstID, - SpvDecorationCoherent); - } - if (flags & MemoryQualifierSetModifier::Flags::kVolatile) + + // https://github.khronos.org/SPIRV-Registry/extensions/KHR/SPV_KHR_vulkan_memory_model.html#_modifications_to_the_spir_v_specification_version_1_3 + // "Coherent is not allowed when the declared memory model is VulkanKHR." + // "Volatile is not allowed when the declared memory model is VulkanKHR" + if (m_memoryModel != SpvMemoryModelVulkan) { - emitOpDecorate( - getSection(SpvLogicalSectionID::Annotations), - nullptr, - dstID, - SpvDecorationVolatile); + if (flags & MemoryQualifierSetModifier::Flags::kCoherent) + { + emitOpDecorate( + getSection(SpvLogicalSectionID::Annotations), + nullptr, + dstID, + SpvDecorationCoherent); + } + if (flags & MemoryQualifierSetModifier::Flags::kVolatile) + { + emitOpDecorate( + getSection(SpvLogicalSectionID::Annotations), + nullptr, + dstID, + SpvDecorationVolatile); + } } if (flags & MemoryQualifierSetModifier::Flags::kRestrict) { @@ -5389,23 +5538,29 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex else if (auto collection = as<IRMemoryQualifierSetDecoration>(decor)) { IRIntegerValue flags = collection->getMemoryQualifierBit(); - if (flags & MemoryQualifierSetModifier::Flags::kCoherent) - { - emitOpMemberDecorate( - getSection(SpvLogicalSectionID::Annotations), - nullptr, - spvStructID, - SpvLiteralInteger::from32(id), - SpvDecorationCoherent); - } - if (flags & MemoryQualifierSetModifier::Flags::kVolatile) + // https://github.khronos.org/SPIRV-Registry/extensions/KHR/SPV_KHR_vulkan_memory_model.html#_modifications_to_the_spir_v_specification_version_1_3 + // "Coherent is not allowed when the declared memory model is VulkanKHR." + // "Volatile is not allowed when the declared memory model is VulkanKHR" + if (m_memoryModel != SpvMemoryModelVulkan) { - emitOpMemberDecorate( - getSection(SpvLogicalSectionID::Annotations), - nullptr, - spvStructID, - SpvLiteralInteger::from32(id), - SpvDecorationVolatile); + if (flags & MemoryQualifierSetModifier::Flags::kCoherent) + { + emitOpMemberDecorate( + getSection(SpvLogicalSectionID::Annotations), + nullptr, + spvStructID, + SpvLiteralInteger::from32(id), + SpvDecorationCoherent); + } + if (flags & MemoryQualifierSetModifier::Flags::kVolatile) + { + emitOpMemberDecorate( + getSection(SpvLogicalSectionID::Annotations), + nullptr, + spvStructID, + SpvLiteralInteger::from32(id), + SpvDecorationVolatile); + } } if (flags & MemoryQualifierSetModifier::Flags::kRestrict) { @@ -6718,10 +6873,46 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex } else { - return emitOpLoad(parent, inst, inst->getDataType(), inst->getPtr()); + return emitLoadMaybeCoherent(parent, inst); } } + SpvInst* emitLoadMaybeCoherent(SpvInstParent* parent, IRInst* inst) + { + IRBuilder builder{inst}; + builder.setInsertBefore(inst); + + SpvInst* deviceScope = nullptr; + IRInst* pointer = inst->getOperand(0); + + bool coherentPointer = NeedToUseCoherentLoadOrStore(pointer); + if (coherentPointer) + { + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + deviceScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); + } + + return emitInstCustomOperandFunc( + parent, + inst, + SpvOpLoad, + [&]() + { + emitOperand(inst->getFullType()); + emitOperand(kResultID); + emitOperand(pointer); + + if (coherentPointer) + { + emitOperand( + SpvMemoryAccessMakePointerVisibleMask | + SpvMemoryAccessNonPrivatePointerMask); + + emitOperand(deviceScope); + } + }); + } + SpvInst* emitStore(SpvInstParent* parent, IRStore* inst) { auto ptrType = as<IRPtrTypeBase>(inst->getPtr()->getDataType()); @@ -6749,10 +6940,46 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex } else { - return emitOpStore(parent, inst, inst->getPtr(), inst->getVal()); + return emitStoreMaybeCoherent(parent, inst); } } + SpvInst* emitStoreMaybeCoherent(SpvInstParent* parent, IRInst* inst) + { + IRBuilder builder{inst}; + builder.setInsertBefore(inst); + + SpvInst* deviceScope = nullptr; + IRInst* pointer = inst->getOperand(0); + IRInst* object = inst->getOperand(1); + + bool coherentPointer = NeedToUseCoherentLoadOrStore(pointer); + if (coherentPointer) + { + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + deviceScope = emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); + } + + return emitInstCustomOperandFunc( + parent, + inst, + SpvOpStore, + [&]() + { + emitOperand(pointer); + emitOperand(object); + + if (coherentPointer) + { + emitOperand( + SpvMemoryAccessMakePointerAvailableMask | + SpvMemoryAccessNonPrivatePointerMask); + + emitOperand(deviceScope); + } + }); + } + SpvInst* emitSwizzledStore(SpvInstParent* parent, IRSwizzledStore* inst) { auto sourceVectorType = as<IRVectorType>(inst->getSource()->getDataType()); @@ -7886,8 +8113,7 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex // The ordinary case is the debug variable has a backing ordinary variable. // We can simply emit a store into the backing variable for the DebugValue operation. // - builder.setInsertBefore(debugValue); - return emitOpStore(parent, debugValue, debugValue->getDebugVar(), debugValue->getValue()); + return emitStoreMaybeCoherent(parent, debugValue); } IRInst* getName(IRInst* inst) @@ -8593,6 +8819,43 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex default: break; } + + bool needToUseCoherentLoadOrStore = false; + if (m_memoryModel == SpvMemoryModelVulkan) + { + switch (opcode) + { + case SpvOpControlBarrier: + case SpvOpMemoryBarrier: + case SpvOpAtomicLoad: + case SpvOpAtomicStore: + case SpvOpAtomicExchange: + case SpvOpAtomicCompareExchange: + case SpvOpAtomicCompareExchangeWeak: + case SpvOpAtomicIIncrement: + case SpvOpAtomicIDecrement: + case SpvOpAtomicIAdd: + case SpvOpAtomicISub: + case SpvOpAtomicSMin: + case SpvOpAtomicUMin: + case SpvOpAtomicSMax: + case SpvOpAtomicUMax: + case SpvOpAtomicAnd: + case SpvOpAtomicOr: + case SpvOpAtomicXor: + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + break; + case SpvOpImageRead: + needToUseCoherentLoadOrStore = + NeedToUseCoherentImageLoadOrStore(spvInst->getOperand(3)); + break; + case SpvOpImageWrite: + needToUseCoherentLoadOrStore = + NeedToUseCoherentImageLoadOrStore(spvInst->getOperand(1)); + break; + } + } + const auto opParent = parentForOpCode(opcode, parent); const auto opInfo = m_grammarInfo->opInfos.lookup(opcode); @@ -8642,6 +8905,15 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex } else { + IRBuilder builder(spvInst); + SpvInst* memoryScope = nullptr; + if (needToUseCoherentLoadOrStore) + { + requireSPIRVCapability(SpvCapabilityVulkanMemoryModelDeviceScope); + memoryScope = + emitIntConstant(IRIntegerValue{SpvScopeDevice}, builder.getUIntType()); + } + last = emitInstCustomOperandFunc( opParent, assignedInst, @@ -8650,6 +8922,36 @@ struct SPIRVEmitContext : public SourceEmitterBase, public SPIRVEmitSharedContex { for (const auto operand : spvInst->getSPIRVOperands()) emitSpvAsmOperand(operand); + + if (needToUseCoherentLoadOrStore) + { + // Check if user specified memory operand explicitly + uint32_t usedMask = 0; + for (auto operand : spvInst->getSPIRVOperands()) + { + if (operand->getOp() == kIROp_SPIRVAsmOperandLiteral) + { + if (auto valInst = as<IRIntLit>(operand->getOperand(0))) + { + usedMask |= uint32_t(valInst->getValue()); + } + } + } + + uint32_t requiredMask = SpvImageOperandsNonPrivateTexelMask; + if (opcode == SpvOpImageRead) + requiredMask |= SpvImageOperandsMakeTexelVisibleMask; + else + requiredMask |= SpvImageOperandsMakeTexelAvailableMask; + + // If user specified any of the required masks, we cannot specified + // anymore. + if (usedMask & requiredMask) + return; + + emitOperand(requiredMask); + emitOperand(memoryScope); + } }); } } diff --git a/source/slang/slang-ir-spirv-legalize.cpp b/source/slang/slang-ir-spirv-legalize.cpp index 3319aa89d..87a6cc4b9 100644 --- a/source/slang/slang-ir-spirv-legalize.cpp +++ b/source/slang/slang-ir-spirv-legalize.cpp @@ -1810,6 +1810,14 @@ struct SPIRVLegalizationContext : public SourceEmitterBase // Scan through the entry points and find the max version required. auto processInst = [&](IRInst* globalInst) { + switch (globalInst->getOp()) + { + case kIROp_CoopVectorType: + case kIROp_CoopMatrixType: + m_sharedContext->m_memoryModel = SpvMemoryModelVulkan; + break; + } + for (auto decor : globalInst->getDecorations()) { switch (decor->getOp()) @@ -1844,6 +1852,11 @@ struct SPIRVLegalizationContext : public SourceEmitterBase processInst(globalInst); } + if (targetCaps.implies(CapabilityAtom::SPV_KHR_vulkan_memory_model)) + { + m_sharedContext->m_memoryModel = SpvMemoryModelVulkan; + } + if (m_sharedContext->m_spvVersion < 0x10300) { // Direct SPIRV backend does not support generating SPIRV before 1.3, diff --git a/source/slang/slang-ir-spirv-legalize.h b/source/slang/slang-ir-spirv-legalize.h index 3c9bdf26a..d2258ffb8 100644 --- a/source/slang/slang-ir-spirv-legalize.h +++ b/source/slang/slang-ir-spirv-legalize.h @@ -31,6 +31,8 @@ struct SPIRVEmitSharedContext unsigned int m_spvVersion = 0x10000; bool m_useDemoteToHelperInvocationExtension = false; + SpvMemoryModel m_memoryModel = SpvMemoryModelGLSL450; + bool isSpirv14OrLater() { return m_spvVersion >= 0x10400; } bool isSpirv15OrLater() { return m_spvVersion >= 0x10500; } bool isSpirv16OrLater() { return m_spvVersion >= 0x10600; } diff --git a/tests/cooperative-matrix/add.slang b/tests/cooperative-matrix/add.slang index c904da0f7..79abedfd5 100644 --- a/tests/cooperative-matrix/add.slang +++ b/tests/cooperative-matrix/add.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: int32_t // CHECK-NEXT: 1 diff --git a/tests/cooperative-matrix/array.slang b/tests/cooperative-matrix/array.slang index eb26bacbf..ee9366f5f 100644 --- a/tests/cooperative-matrix/array.slang +++ b/tests/cooperative-matrix/array.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK: 1.000000 diff --git a/tests/cooperative-matrix/comparison.slang b/tests/cooperative-matrix/comparison.slang index 8b29876df..da4ac7a2e 100644 --- a/tests/cooperative-matrix/comparison.slang +++ b/tests/cooperative-matrix/comparison.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: uint32_t // CHECK-NEXT: 0 diff --git a/tests/cooperative-matrix/conversion.slang b/tests/cooperative-matrix/conversion.slang index 24fba3cc6..6cfdd0d9e 100644 --- a/tests/cooperative-matrix/conversion.slang +++ b/tests/cooperative-matrix/conversion.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-NEXT: 2.000000 diff --git a/tests/cooperative-matrix/copyFrom.slang b/tests/cooperative-matrix/copyFrom.slang index 9ab8acd4a..3551a2597 100644 --- a/tests/cooperative-matrix/copyFrom.slang +++ b/tests/cooperative-matrix/copyFrom.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: int32_t // CHECK-COUNT-256: 4 diff --git a/tests/cooperative-matrix/div.slang b/tests/cooperative-matrix/div.slang index 17b237280..da2db6206 100644 --- a/tests/cooperative-matrix/div.slang +++ b/tests/cooperative-matrix/div.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: int32_t // CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/fill.slang b/tests/cooperative-matrix/fill.slang index f9068577b..ba3588315 100644 --- a/tests/cooperative-matrix/fill.slang +++ b/tests/cooperative-matrix/fill.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: int32_t // CHECK-COUNT-256: 10 diff --git a/tests/cooperative-matrix/inout.slang b/tests/cooperative-matrix/inout.slang index 3ff9a16d6..5502a52bd 100644 --- a/tests/cooperative-matrix/inout.slang +++ b/tests/cooperative-matrix/inout.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-NEXT: 2.000000 diff --git a/tests/cooperative-matrix/length.slang b/tests/cooperative-matrix/length.slang index 580b713f3..f6dadfb0d 100644 --- a/tests/cooperative-matrix/length.slang +++ b/tests/cooperative-matrix/length.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK):-vk -output-using-type -emit-spirv-directly // Note the length is NOT row * column. // When the memory scope is set to subgroup, each thread gets 16 * 16 / 32 = 8 where 32 is the value used in `numthreads`. diff --git a/tests/cooperative-matrix/load-store-arbitrary-array-vec.slang b/tests/cooperative-matrix/load-store-arbitrary-array-vec.slang index d34494fd8..ba74821f0 100644 --- a/tests/cooperative-matrix/load-store-arbitrary-array-vec.slang +++ b/tests/cooperative-matrix/load-store-arbitrary-array-vec.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -emit-spirv-directly // CHECK: 1 // CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/load-store-arbitrary-array.slang b/tests/cooperative-matrix/load-store-arbitrary-array.slang index 8de4e84cc..2e8a67868 100644 --- a/tests/cooperative-matrix/load-store-arbitrary-array.slang +++ b/tests/cooperative-matrix/load-store-arbitrary-array.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -emit-spirv-directly // CHECK: 1 // CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/load-store-groupshared.slang b/tests/cooperative-matrix/load-store-groupshared.slang index db32e85fd..bd546353c 100644 --- a/tests/cooperative-matrix/load-store-groupshared.slang +++ b/tests/cooperative-matrix/load-store-groupshared.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -emit-spirv-directly // CHECK: 1 // CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/load-store-rwbyteaddressbuffer.slang b/tests/cooperative-matrix/load-store-rwbyteaddressbuffer.slang index 6894bdfe5..6eb86eb68 100644 --- a/tests/cooperative-matrix/load-store-rwbyteaddressbuffer.slang +++ b/tests/cooperative-matrix/load-store-rwbyteaddressbuffer.slang @@ -1,5 +1,5 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -Xslang -DRWBAB +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -Xslang -DRWBAB // CHECK: 1 // CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/load-store-rwstructuredbuffer.slang b/tests/cooperative-matrix/load-store-rwstructuredbuffer.slang index 6a94fd30e..65da7f8d5 100644 --- a/tests/cooperative-matrix/load-store-rwstructuredbuffer.slang +++ b/tests/cooperative-matrix/load-store-rwstructuredbuffer.slang @@ -1,5 +1,5 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -Xslang -DRWSB +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -Xslang -DRWSB // CHECK: type: int32_t // CHECK-NEXT: 1 diff --git a/tests/cooperative-matrix/load-store-tensorlayout.slang b/tests/cooperative-matrix/load-store-tensorlayout.slang index 849c85c0e..d20e08a82 100644 --- a/tests/cooperative-matrix/load-store-tensorlayout.slang +++ b/tests/cooperative-matrix/load-store-tensorlayout.slang @@ -1,11 +1,11 @@ -//TEST(compute):SIMPLE(filecheck=SPIRV):-target spirv-asm -entry computeMain -stage compute -skip-spirv-validation -//TEST(compute):SIMPLE(filecheck=SPIRV_BL):-target spirv-asm -entry computeMain -stage compute -skip-spirv-validation -DBLOCK_LOAD +//TEST(compute):SIMPLE(filecheck=SPIRV):-target spirv-asm -entry computeMain -stage compute +//TEST(compute):SIMPLE(filecheck=SPIRV_BL):-target spirv-asm -entry computeMain -stage compute -DBLOCK_LOAD -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -render-feature cooperative-matrix-tensor-addressing -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -render-feature cooperative-matrix-tensor-addressing -Xslang -DRW +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-tensor-addressing +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-tensor-addressing -Xslang -DRW -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK_BL):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -render-feature cooperative-matrix-block-loads -Xslang -DBLOCK_LOAD -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK_BL):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -render-feature cooperative-matrix-block-loads -Xslang -DBLOCK_LOAD -Xslang -DRW +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK_BL):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-block-loads -Xslang -DBLOCK_LOAD +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK_BL):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-block-loads -Xslang -DBLOCK_LOAD -Xslang -DRW //CHECK: 0 //CHECK-NEXT: 0 diff --git a/tests/cooperative-matrix/load-store-tensorview.slang b/tests/cooperative-matrix/load-store-tensorview.slang index 6757338be..44c1f2944 100644 --- a/tests/cooperative-matrix/load-store-tensorview.slang +++ b/tests/cooperative-matrix/load-store-tensorview.slang @@ -1,11 +1,11 @@ -//TEST(compute):SIMPLE(filecheck=SPIRV):-target spirv-asm -entry computeMain -stage compute -skip-spirv-validation -//TEST(compute):SIMPLE(filecheck=SPIRV_BL):-target spirv-asm -entry computeMain -stage compute -skip-spirv-validation -DBLOCK_LOAD +//TEST(compute):SIMPLE(filecheck=SPIRV):-target spirv-asm -entry computeMain -stage compute +//TEST(compute):SIMPLE(filecheck=SPIRV_BL):-target spirv-asm -entry computeMain -stage compute -DBLOCK_LOAD -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -render-feature cooperative-matrix-tensor-addressing -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -render-feature cooperative-matrix-tensor-addressing -Xslang -DRW +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-tensor-addressing +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-tensor-addressing -Xslang -DRW -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK_BL):-vk -output-using-type -emit-spirv-directly -Xslang -DBLOCK_LOAD -render-feature cooperative-matrix-block-loads -skip-spirv-validation -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK_BL):-vk -output-using-type -emit-spirv-directly -Xslang -DBLOCK_LOAD -render-feature cooperative-matrix-block-loads -skip-spirv-validation -Xslang -DRW +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK_BL):-vk -output-using-type -emit-spirv-directly -Xslang -DBLOCK_LOAD -render-feature cooperative-matrix-block-loads +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK_BL):-vk -output-using-type -emit-spirv-directly -Xslang -DBLOCK_LOAD -render-feature cooperative-matrix-block-loads -Xslang -DRW //CHECK: 2 //CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/map-element-single.slang b/tests/cooperative-matrix/map-element-single.slang index 583630d14..1661ee105 100644 --- a/tests/cooperative-matrix/map-element-single.slang +++ b/tests/cooperative-matrix/map-element-single.slang @@ -1,6 +1,6 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-per-element-operations -skip-spirv-validation -Xslang -DTEST_MODE=0 -//DISABLE_TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-per-element-operations -skip-spirv-validation -Xslang -DTEST_MODE=1 -//DISABLE_TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-per-element-operations -skip-spirv-validation -Xslang -DTEST_MODE=2 +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-per-element-operations -Xslang -DTEST_MODE=0 +//DISABLE_TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-per-element-operations -Xslang -DTEST_MODE=1 +//DISABLE_TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-per-element-operations -Xslang -DTEST_MODE=2 //CHECK: type: int32_t //CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/mat-mul-add-spirv-matrix-operands.slang b/tests/cooperative-matrix/mat-mul-add-spirv-matrix-operands.slang index a93efb4b9..92f11e2e2 100644 --- a/tests/cooperative-matrix/mat-mul-add-spirv-matrix-operands.slang +++ b/tests/cooperative-matrix/mat-mul-add-spirv-matrix-operands.slang @@ -1,4 +1,4 @@ -//TEST(compute):SIMPLE(filecheck=CHECK): -entry computeMain -stage compute -target spirv -skip-spirv-validation +//TEST(compute):SIMPLE(filecheck=CHECK): -entry computeMain -stage compute -target spirv // This test checks that the correct SPIRV Cooperative Matrix Operands are emitted for OpCooperativeMatrixMulAddKHR operaions RWStructuredBuffer<uint32_t> outputBuffer1; diff --git a/tests/cooperative-matrix/mat-mul-add.slang b/tests/cooperative-matrix/mat-mul-add.slang index 3dc472de7..6209b9612 100644 --- a/tests/cooperative-matrix/mat-mul-add.slang +++ b/tests/cooperative-matrix/mat-mul-add.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-COUNT-256: 241.0 diff --git a/tests/cooperative-matrix/mod.slang b/tests/cooperative-matrix/mod.slang index 015dd227a..fe87714b0 100644 --- a/tests/cooperative-matrix/mod.slang +++ b/tests/cooperative-matrix/mod.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -emit-spirv-directly // CHECK: 0 // CHECK-NEXT: 0 diff --git a/tests/cooperative-matrix/mul.slang b/tests/cooperative-matrix/mul.slang index 4f5751c6e..64f339c8f 100644 --- a/tests/cooperative-matrix/mul.slang +++ b/tests/cooperative-matrix/mul.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: int32_t // CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/out.slang b/tests/cooperative-matrix/out.slang index 7d4a83fc2..c65b71551 100644 --- a/tests/cooperative-matrix/out.slang +++ b/tests/cooperative-matrix/out.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-NEXT: 2.000000 diff --git a/tests/cooperative-matrix/parameter.slang b/tests/cooperative-matrix/parameter.slang index eb6823aa1..2e4a8fe57 100644 --- a/tests/cooperative-matrix/parameter.slang +++ b/tests/cooperative-matrix/parameter.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-NEXT: 3.000000 diff --git a/tests/cooperative-matrix/reduce.slang b/tests/cooperative-matrix/reduce.slang index bbba587ce..c01100940 100644 --- a/tests/cooperative-matrix/reduce.slang +++ b/tests/cooperative-matrix/reduce.slang @@ -1,7 +1,7 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK_ROW):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-reductions -skip-spirv-validation -Xslang -DTEST_MODE=0 -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK_COLUMN):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-reductions -skip-spirv-validation -Xslang -DTEST_MODE=1 -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK_ROW_AND_COLUMN):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-reductions -skip-spirv-validation -Xslang -DTEST_MODE=2 -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK_2X2):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-reductions -skip-spirv-validation -Xslang -DTEST_MODE=3 +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK_ROW):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-reductions -Xslang -DTEST_MODE=0 +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK_COLUMN):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-reductions -Xslang -DTEST_MODE=1 +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK_ROW_AND_COLUMN):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-reductions -Xslang -DTEST_MODE=2 +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHK_2X2):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-reductions -Xslang -DTEST_MODE=3 //CHK_ROW-COUNT-8: 36 diff --git a/tests/cooperative-matrix/return.slang b/tests/cooperative-matrix/return.slang index 2da0c81ad..456b8f292 100644 --- a/tests/cooperative-matrix/return.slang +++ b/tests/cooperative-matrix/return.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -skip-spirv-validation -emit-spirv-directly +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-NEXT: 3.000000 diff --git a/tests/cooperative-matrix/scalar-mul.slang b/tests/cooperative-matrix/scalar-mul.slang index 29d8f03f0..f1a370ce0 100644 --- a/tests/cooperative-matrix/scalar-mul.slang +++ b/tests/cooperative-matrix/scalar-mul.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-NEXT: 4.500000 diff --git a/tests/cooperative-matrix/struct.slang b/tests/cooperative-matrix/struct.slang index 592e553ae..586f958ca 100644 --- a/tests/cooperative-matrix/struct.slang +++ b/tests/cooperative-matrix/struct.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-NEXT: 1.000000 diff --git a/tests/cooperative-matrix/sub.slang b/tests/cooperative-matrix/sub.slang index a42fee9e8..b1b89a1e4 100644 --- a/tests/cooperative-matrix/sub.slang +++ b/tests/cooperative-matrix/sub.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: int32_t // CHECK-NEXT: 1 diff --git a/tests/cooperative-matrix/subscript-in-func.slang b/tests/cooperative-matrix/subscript-in-func.slang index 1bb8df433..445ae9f65 100644 --- a/tests/cooperative-matrix/subscript-in-func.slang +++ b/tests/cooperative-matrix/subscript-in-func.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: float // CHECK-NEXT: 1.000000 diff --git a/tests/cooperative-matrix/subscript.slang b/tests/cooperative-matrix/subscript.slang index 0d765b6b3..efa65de87 100644 --- a/tests/cooperative-matrix/subscript.slang +++ b/tests/cooperative-matrix/subscript.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: int32_t // CHECK-NEXT: 2 diff --git a/tests/cooperative-matrix/transpose.slang b/tests/cooperative-matrix/transpose.slang index cd317cac2..135a5ce23 100644 --- a/tests/cooperative-matrix/transpose.slang +++ b/tests/cooperative-matrix/transpose.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation -render-feature cooperative-matrix-conversions +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -render-feature cooperative-matrix-conversions //CHECK: type: int32_t //CHECK: 1 diff --git a/tests/cooperative-matrix/unary_neg.slang b/tests/cooperative-matrix/unary_neg.slang index be803d8c6..9e3ec5816 100644 --- a/tests/cooperative-matrix/unary_neg.slang +++ b/tests/cooperative-matrix/unary_neg.slang @@ -1,4 +1,4 @@ -//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly -skip-spirv-validation +//TEST(compute):COMPARE_COMPUTE(filecheck-buffer=CHECK):-vk -output-using-type -emit-spirv-directly // CHECK: type: int32_t // CHECK-NEXT: -1 diff --git a/tests/diagnostics/extension-full-name.slang.actual b/tests/diagnostics/extension-full-name.slang.actual deleted file mode 100644 index 0d53a90ce..000000000 --- a/tests/diagnostics/extension-full-name.slang.actual +++ /dev/null @@ -1,32 +0,0 @@ -result code = -1 -standard error = { -tests/diagnostics/extension-full-name.slang(179): error 30019: expected an expression of type 'GenericType<half>.State', got 'int' - state1 = 0; // Error: expected expr of type 'GenericType<half>.State', got 'int' - ^ -tests/diagnostics/extension-full-name.slang(180): error 30019: expected an expression of type 'GenericType<float>.InnerType.Options', got 'int' - options1 = 0; // Error: expected expr of type 'GenericType<float>.InnerType.Options', got 'int' - ^ -tests/diagnostics/extension-full-name.slang(181): error 30019: expected an expression of type 'NonGenericType.Config', got 'int' - config1 = 0; // Error: expected expr of type 'NonGenericType.Config', got 'int' - ^ -tests/diagnostics/extension-full-name.slang(182): error 30019: expected an expression of type 'Container.Nested.Settings', got 'int' - settings1 = 0; // Error: expected expr of type 'Container.Nested.Settings', got 'int' - ^ -tests/diagnostics/extension-full-name.slang(183): error 30019: expected an expression of type 'Container.Nested.DeepNested.Record', got 'int' - record1 = 0; // Error: expected expr of type 'Container.Nested.DeepNested.Record', got 'int' - ^ -tests/diagnostics/extension-full-name.slang(191): error 30027: 'getOptions' is not a member of 'GenericType<int>.InnerType'. - intInner.getOptions(); // This won't compile as intInner doesn't have getOptions - ^~~~~~~~~~ -tests/diagnostics/extension-full-name.slang(221): error 30019: expected an expression of type 'GenericType<float>.InnerType.Options', got 'GenericType<half>.State' - .createState(); // Error: expected expr of type 'GenericType<float>.InnerType.Options', - ^ -tests/diagnostics/extension-full-name.slang(223): error 30019: expected an expression of type 'GenericType<half>.State', got 'NonGenericType.Config' - state1 = nonGenType.createConfig(); // Error: expected expr of type 'GenericType<half>.State', - ^ -tests/diagnostics/extension-full-name.slang(225): error 30019: expected an expression of type 'NonGenericType.Config', got 'Container.Nested.Settings' - config1 = nested.createSettings(); // Error: expected expr of type 'NonGenericType.Config', got - ^ -} -standard output = { -} diff --git a/tests/hlsl/simple/globallycoherent.hlsl b/tests/hlsl/simple/globallycoherent.hlsl index 70b5cdb2b..07f0dd57f 100644 --- a/tests/hlsl/simple/globallycoherent.hlsl +++ b/tests/hlsl/simple/globallycoherent.hlsl @@ -1,7 +1,15 @@ //TEST:COMPARE_HLSL:-profile cs_5_0 +//TEST:SIMPLE(filecheck=SPIRV): -target spirv-asm +//TEST:SIMPLE(filecheck=VK_MEM_MODEL): -target spirv-asm -emit-spirv-directly -capability vk_mem_model // Check output for `globallycoherent` +//SPIRV: OpDecorate {{.*}} Coherent + +//VK_MEM_MODEL-NOT: OpDecorate {{.*}} Coherent +//VK_MEM_MODEL: OpLoad {{.*}} MakePointerVisible|NonPrivatePointer +//VK_MEM_MODEL: OpStore {{.*}} MakePointerAvailable|NonPrivatePointer + #ifndef __SLANG__ #define gBuffer gBuffer_0 #define SV_DispatchThreadID SV_DISPATCHTHREADID diff --git a/tests/spirv/coherent-2.slang b/tests/spirv/coherent-2.slang index 425ca0fe1..d68b38ad0 100644 --- a/tests/spirv/coherent-2.slang +++ b/tests/spirv/coherent-2.slang @@ -1,6 +1,7 @@ // Test that globallycoherent works on arrays of uavs. //TEST:SIMPLE(filecheck=CHECK): -target spirv -stage compute -entry main -emit-spirv-directly +//TEST:SIMPLE(filecheck=VK_MEM_MODEL): -target spirv -stage compute -entry main -emit-spirv-directly -capability vk_mem_model globallycoherent RWByteAddressBuffer buffer[]; @@ -14,4 +15,6 @@ void main(int tid : SV_DispatchThreadID) // CHECK-DAG: OpDecorate %buffer Coherent // CHECK-DAG: OpDecorate %[[VAR1:[a-zA-Z0-9_]+]] NonUniform // CHECK: %[[VAR1]] = OpAccessChain %{{.*}} %buffer + + //VK_MEM_MODEL: OpLoad {{.*}} MakePointerVisible|NonPrivatePointer } diff --git a/tests/spirv/coherent-texture.slang b/tests/spirv/coherent-texture.slang new file mode 100644 index 000000000..ab1ce260f --- /dev/null +++ b/tests/spirv/coherent-texture.slang @@ -0,0 +1,24 @@ +//TEST:SIMPLE(filecheck=SPIRV): -target spirv-asm -entry computeMain -stage compute +//TEST:SIMPLE(filecheck=VK_MEM_MODEL): -target spirv-asm -entry computeMain -stage compute -capability vk_mem_model + +//SPIRV: OpDecorate {{.*}} Coherent + +//VK_MEM_MODEL-NOT: OpDecorate {{.*}} Coherent +//VK_MEM_MODEL: OpImageRead %{{.*}} MakeTexelVisible|NonPrivateTexel % +//VK_MEM_MODEL: OpImageWrite %{{.*}} MakeTexelAvailable|NonPrivateTexel % +//VK_MEM_MODEL: OpImageRead %{{.*}} MakeTexelVisible|NonPrivateTexel % +//VK_MEM_MODEL: OpImageWrite %{{.*}} MakeTexelAvailable|NonPrivateTexel % + +globallycoherent +RWTexture2D<int4> outputTexture2D; + +[numthreads(1,1,1)] +void computeMain() +{ + // This line issues kIROp_ImageLoad/Store + outputTexture2D[0].xz = outputTexture2D[0].yw; + + // The current implementation uses spirv_asm for subscription, + // which goes through a different part of implementation in slang-emit-spirv.cpp. + outputTexture2D[uint2(0,0)] = outputTexture2D[uint2(1,1)]; +} diff --git a/tests/spirv/coherent.slang b/tests/spirv/coherent.slang index ba58b713b..33a559700 100644 --- a/tests/spirv/coherent.slang +++ b/tests/spirv/coherent.slang @@ -1,6 +1,7 @@ // Test that globallycoherent works. //TEST:SIMPLE(filecheck=CHECK): -target spirv -stage compute -entry main -emit-spirv-directly +//TEST:SIMPLE(filecheck=VK_MEM_MODEL): -target spirv -stage compute -entry main -emit-spirv-directly -capability vk_mem_model struct S { @@ -17,4 +18,5 @@ void main(int tid : SV_DispatchThreadID) output[tid] = buffer[0].member; // CHECK-DAG: OpMemberDecorate {{.*}} 0 Coherent // CHECK-DAG: OpDecorate %buffer Coherent + // VK_MEM_MODEL: OpLoad {{.*}} MakePointerVisible|NonPrivatePointer } |
