Add validation for destination of atomic operations (#6093)

* Reimplement the GLSL atomic* functions in terms of __intrinsic_op

Many of these functions map directly to atomic IR instructions.
The functions taking atomic_uint are left as they are.

This helps to address #5989, since the destination pointer type validation can then be
written only for the atomic IR instructions.

* Add validation for atomic operations

Diagnose an error if the destination of the atomic operation is not appropriate, where
appropriate means it's either:
- 'groupshared'
- from a device buffer

This closes #5989.

* Add tests for GLSL atomics destination validation

Attempting to use the GLSL atomic functions on destinations that are neither groupshared
nor from a device buffer should fail with the following error:

error 41403: cannot perform atomic operation because destination is neither groupshared
             nor from a device buffer.

* Validate atomic operations after address space specialization

Address space specialization for SPIR-V is not done as part of `linkAndOptimizeIR`, as it
is for e.g. Metal, so opt out and add a separate call for SPIR-V.

* Allow unchecked in/inout parameters for non-SPIRV targets

* Clean up callees left without uses during address space specialziation

* format code

---------

Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
Co-authored-by: Yong He <yonghe@outlook.com>

1 files changed, 10 insertions, 0 deletions

diff --git a/source/slang/slang-ir-specialize-address-space.cpp b/source/slang/slang-ir-specialize-address-space.cpp
index a3c45617d..ae0542734 100644
--- a/source/slang/slang-ir-specialize-address-space.cpp
+++ b/source/slang/slang-ir-specialize-address-space.cpp
@@ -13,6 +13,7 @@ struct AddressSpaceContext : public AddressSpaceSpecializationContext
 
     Dictionary<IRInst*, AddressSpace> mapInstToAddrSpace;
     InitialAddressSpaceAssigner* addrSpaceAssigner;
+    HashSet<IRFunc*> functionsToConsiderRemoving;
 
     AddressSpaceContext(IRModule* inModule, InitialAddressSpaceAssigner* inAddrSpaceAssigner)
         : module(inModule), addrSpaceAssigner(inAddrSpaceAssigner)
@@ -279,6 +280,8 @@ struct AddressSpaceContext : public AddressSpaceSpecializationContext
                                     callInst = as<IRCall>(builder.replaceOperand(
                                         callInst->getOperands(),
                                         specializedCallee));
+                                    // At this point, the original callee may be left without uses.
+                                    functionsToConsiderRemoving.add(callee);
                                 }
                                 auto callResultAddrSpace =
                                     getFuncResultAddrSpace(specializedCallee);
@@ -394,6 +397,13 @@ struct AddressSpaceContext : public AddressSpaceSpecializationContext
         }
 
         applyAddressSpaceToInstType();
+
+        for (IRFunc* func : functionsToConsiderRemoving)
+        {
+            SLANG_ASSERT(!func->findDecoration<IREntryPointDecoration>());
+            if (!func->hasUses())
+                func->removeAndDeallocate();
+        }
     }
 };