Add utility to trace creation of problematic IRInsts to assist LLM in debugging (#7820)

* Initial plan

* Add SLANG_DEBUG_IR_BREAK environment variable support

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Apply code formatting to SLANG_DEBUG_IR_BREAK implementation

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Improve stack trace debugging with -rdynamic flag and backtrace_symbols

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Address PR feedback: use PlatformUtil::getEnvironmentVariable, remove -rdynamic flag, and delete fallback branch

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Address PR feedback: simplify env var parsing, move backtrace to PlatformUtil, use #if for SLANG_LINUX_FAMILY

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Address PR feedback: remove unneeded include, make backtrace() more generic by removing uid parameter

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* Fix and clone source tracking.

* Add python script to dump traces.

* Update instructions.

* Batch calls to addr2line

* Cleanup claude instructions.

* update claude action.

* Remove duplicated build instructions from claude.yml workflow

Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>

* fix build error.

* Fix build errors

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: csyonghe <2652293+csyonghe@users.noreply.github.com>
Co-authored-by: Yong He <yonghe@outlook.com>
Co-authored-by: Gangzheng Tong <tonggangzheng@gmail.com>

11 files changed, 234 insertions, 17 deletions

diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
index 8475f70f5..00d8350fd 100644
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -23,6 +23,17 @@ Add the "pr: breaking" label to  your PR if you are introducing public API chang
 or you are introducing changes to the Slang language that will cause the compiler to error out on existing Slang code.
 It is rare for a PR to be a breaking change.
 
+## Debugging
+
+If you encounter a bug related to a problematic instruction, it is often useful to trace the location where the instruction is created.
+You can use the `extras/insttrace.py` script to do this. For example, during debugging you find that an instruction with `_debugUID=1234`
+is wrong, you can run the following command to trace the callstack where the instruction is created:
+
+```bash
+# From workspace root:
+python3 ./extras/insttrace.py 1234 ./build/Debug/bin/slangc tests/my-test.slang -target spirv
+```
+
 ## Testing
 
 Your PR should include a regression test for the bug you are fixing.
diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml
index dcc36e760..c2e0fb68e 100644
--- a/.github/workflows/claude.yml
+++ b/.github/workflows/claude.yml
@@ -189,19 +189,14 @@ jobs:
       # Run Claude Code Action with optimized environment variables
       - name: Execute Claude Code Action # Right now direct prompt to automatic PR Review
         id: claude-action
-        uses: anthropics/claude-code-action@v0.0.31
+        uses: anthropics/claude-code-action@v0.0.38
         with:
           # Direct Prompt is for testing. We shall use the triggers (on) which shall trigger this part on runtime
           custom_instructions: |
             # Build system information:
             - OS: Ubuntu Linux
-            - Build commands: 
-              * Configure: `cmake --preset default`
-              * Build: `cmake --build --preset debug`
-              * Test: `./build/Release/bin/slang-test ./tests/path/to/test.slang`
-              * Format code: `./extras/formatting.sh`
             - Project is pre-built and ready for development tasks
-            - Run formatting script before committing changes
+            - See CLAUDE.md for detailed build, test, and formatting instructions
 
             # CRITICAL: You have access to the mcp__deepwiki__ask_question tool for deep repository knowledge.
 
@@ -215,14 +210,6 @@ jobs:
               - Always follow existing code patterns and architectural decisions discovered through deepwiki
               - Consult the tool when you need context about unfamiliar parts of the codebase
 
-            # Debugging backend crashes or invalid downstream code:
-
-              Note that any issues in the generated target code could stem from IR passes or even the front-end type checking
-              early in the pipeline, and you need to focus on tracking the root cause that breaks the consistency/invariants/assumptions
-              of the IR instead of putting in band-aid fixes in the later passes or in the emit logic. The philosphy of the compiler is to
-              keep the target code emission logic as simple and direct as possible, and most of the heavy lifting code transform is done
-              in the IR passes.
-
           mcp_config: |
             {
               "mcpServers": {
diff --git a/CLAUDE.md b/CLAUDE.md
index a9a18ebf2..48fed1fee 100755
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -139,6 +139,22 @@ When checking the IR dump, look for type consistency or logical errors in the IR
 pass at fault. Focus on passes that makes significant and systematic changes to the IR, such as specialization, inlining,
 type legalization, and buffer lowering passes. You may iterate this process multiple times to narrow down the issue.
 
+#### InstTrace
+
+Note that any issues in the generated target code could stem from IR passes or even the front-end type checking
+early in the pipeline, and you need to focus on tracking the root cause that breaks the consistency/invariants/assumptions
+of the IR instead of putting in band-aid fixes in the later passes or in the emit logic. The philosphy of the compiler is to
+keep the target code emission logic as simple and direct as possible, and most of the heavy lifting code transform is done
+in the IR passes.
+
+If you encounter a bug related to a problematic instruction, it is often useful to trace the location where the instruction is created.
+You can use the `extras/insttrace.py` script to do this. For example, during debugging you find that an instruction with `_debugUID=1234`
+is wrong, you can run the following command to trace the callstack where the instruction is created:
+
+```bash
+# From workspace root:
+python3 ./extras/insttrace.py 1234 ./build/Debug/bin/slangc tests/my-test.slang -target spirv
+```
 
 #### slangc with `-target spirv-asm`
 slangc with `-target spirv-asm` is the most common way to see how the given slang shader is compiled into spirv code.
diff --git a/extras/insttrace.py b/extras/insttrace.py
new file mode 100755
index 000000000..180cc2ec4
--- /dev/null
+++ b/extras/insttrace.py
@@ -0,0 +1,115 @@
+# InstTrace Debugging Utility
+#
+# This script is used to trace the callstack at the creation of a specific IR instruction in the Slang compiler.
+# This is useful for debugging purposes, especially if you encounter a compiler bug related to an instruction
+# that appears to be incorrect, and you want to find out where it was created in the codebase.
+#
+# Usage: python3 ./extras/insttrace.py <inst UID> <commandline_to_slangc_or_slang-test>
+#
+# The script will print the callstack at the point where the specified instruction was created.
+# <inst UID> is the unique identifier of the instruction you want to trace, which can be found by inspecting
+# the _debugUID field of an IRInst object in the Slang compiler source code.
+# If the instruction is a clone of another instruction, it will also trace the creation of the original instruction
+# recursively.
+
+import sys
+import subprocess
+import re
+import os
+
+def traceInst(inst_uid, command):
+    # Run the command with the provided arguments
+    # Set the environment variable SLANG_IR_ALLOC_BREAK to the instruction UID
+    env = dict(os.environ, SLANG_DEBUG_IR_BREAK=inst_uid)
+    process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
+    stdout, stderr = process.communicate()
+
+    # Parse the output to find the string between "BEGIN IR Trace" and "END IR Trace"
+    ir_trace = re.search(r"BEGIN IR Trace(.*?)END IR Trace", stdout.decode(encoding='utf-8', errors='ignore'), re.DOTALL)
+    if not ir_trace:
+        print("No IR Trace found in the output.")
+        return
+
+    traceOutput = ir_trace.group(1)
+    regex = r"(\S+)\(\+(0x[0-9a-f]+)\) \[0x[0-9a-f]+\]"
+
+    lines = traceOutput.splitlines()
+    
+    # First, collect all addresses grouped by library file for batching
+    lib_addresses = {}  # libFile -> list of addresses
+    line_info = []  # (line, libFile, address) for each line
+    
+    for line in lines:
+        match = re.search(regex, line)
+        if match:
+            libFile = match.group(1)
+            address = match.group(2)
+            line_info.append((line, libFile, address))
+            
+            if libFile not in lib_addresses:
+                lib_addresses[libFile] = []
+            lib_addresses[libFile].append(address)
+        else:
+            line_info.append((line, None, None))
+    
+    # Batch call addr2line for each library file
+    address_to_symbol = {}  # (libFile, address) -> symbol info
+    
+    for libFile, addresses in lib_addresses.items():
+        if not addresses:
+            continue
+            
+        # Call addr2line once with all addresses for this library
+        addr2line_command = ["addr2line", "-e", libFile, "-f", "-C"] + addresses
+        try:
+            addr2line_process = subprocess.Popen(addr2line_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+            stdout, stderr = addr2line_process.communicate()
+            
+            # Parse the output - addr2line returns function name and location for each address
+            output_lines = stdout.decode(encoding='utf-8', errors='ignore').strip().split('\n')
+            
+            # Each address produces 2 lines: function name, then file:line
+            for i, address in enumerate(addresses):
+                if i * 2 + 1 < len(output_lines):
+                    function_name = output_lines[i * 2].strip()
+                    location = output_lines[i * 2 + 1].strip()
+                    symbol_info = f"{function_name} {location}"
+                    address_to_symbol[(libFile, address)] = symbol_info
+                else:
+                    address_to_symbol[(libFile, address)] = f"<unknown> {address}"
+        except Exception as e:
+            # Fallback if addr2line fails
+            for address in addresses:
+                address_to_symbol[(libFile, address)] = f"<addr2line failed> {address}"
+    
+    # Now print the results using the cached symbol information
+    for line, libFile, address in line_info:
+        if libFile and address:
+            symbol_info = address_to_symbol.get((libFile, address), f"<not found> {address}")
+            print(symbol_info)
+        else:
+            # print the line as is if it doesn't match the address format
+            print(line)
+
+    print("(end of stacktrace)")
+
+    # Find "Inst #%u is a clone of Inst #%u" in the trace output, and trace the original instruction
+    # if it exists.
+    clone_match = re.search(r"Inst #(\d+) is a clone of Inst #(\d+)", traceOutput)
+    if clone_match:
+        clone_inst_uid = clone_match.group(1)
+        original_inst_uid = clone_match.group(2)
+        traceInst(original_inst_uid, command)
+
+def main():
+    if len(sys.argv) < 3:
+        print("InstTrace Debugging Utility")
+        print("Usage: python insttrace.py <inst UID> <commandline_to_slangc_or_slang-test>")
+        sys.exit(1)
+
+    inst_uid = sys.argv[1]
+    command = sys.argv[2:]
+    traceInst(inst_uid, command)
+
+if __name__ == "__main__":
+    main()
\ No newline at end of file
diff --git a/source/core/slang-platform.cpp b/source/core/slang-platform.cpp
index 2c2bdd25e..aab1f3044 100644
--- a/source/core/slang-platform.cpp
+++ b/source/core/slang-platform.cpp
@@ -15,6 +15,11 @@
 #include <dlfcn.h>
 #endif
 
+
+#if SLANG_LINUX_FAMILY
+#include <execinfo.h>
+#endif
+
 namespace Slang
 {
 // SharedLibrary
@@ -331,4 +336,25 @@ static const PlatformFlags s_familyFlags[int(PlatformFamily::CountOf)] = {
 #endif
 }
 
+/* static */ void PlatformUtil::backtrace()
+{
+#if SLANG_LINUX_FAMILY
+    // Print stack trace for debugging assistance
+    void* stackTrace[64];
+    int stackDepth = ::backtrace(stackTrace, 64);
+    char** symbols = ::backtrace_symbols(stackTrace, stackDepth);
+    if (symbols)
+    {
+        for (int i = 0; i < stackDepth; ++i)
+        {
+            fprintf(stdout, "%s\n", symbols[i]);
+        }
+        free(symbols);
+    }
+    fprintf(stdout, "\n");
+#else
+    fprintf(stdout, "Stack trace not available on this platform.\n");
+#endif
+}
+
 } // namespace Slang
diff --git a/source/core/slang-platform.h b/source/core/slang-platform.h
index 0b97aca6d..04559cbcf 100644
--- a/source/core/slang-platform.h
+++ b/source/core/slang-platform.h
@@ -150,6 +150,10 @@ struct PlatformUtil
     /// @param text Text to be displayed in 'debugger output'
     /// @return SLANG_E_NOT_AVAILABLE if not on this platform, and potentially other errors
     static SlangResult outputDebugMessage(const char* text);
+
+    /// Print a stack trace to stderr for debugging purposes.
+    /// Only available on Linux family platforms.
+    static void backtrace();
 };
 
 #ifndef _MSC_VER
diff --git a/source/slang/slang-ir-clone.cpp b/source/slang/slang-ir-clone.cpp
index 74a972c1d..a7b68efe2 100644
--- a/source/slang/slang-ir-clone.cpp
+++ b/source/slang/slang-ir-clone.cpp
@@ -54,6 +54,10 @@ IRInst* cloneInstAndOperands(IRCloneEnv* env, IRBuilder* builder, IRInst* oldIns
     SLANG_ASSERT(builder);
     SLANG_ASSERT(oldInst);
 
+#if SLANG_ENABLE_IR_BREAK_ALLOC
+    _debugSetInstBeingCloned(oldInst->_debugUID);
+    SLANG_DEFER(_debugResetInstBeingCloned());
+#endif
     // We start by mapping the type of the orignal instruction
     // to its replacement value, if any.
     //
diff --git a/source/slang/slang-ir-link.cpp b/source/slang/slang-ir-link.cpp
index b874b9f28..a3466c8c7 100644
--- a/source/slang/slang-ir-link.cpp
+++ b/source/slang/slang-ir-link.cpp
@@ -1332,6 +1332,11 @@ IRInst* cloneInst(
     IRInst* originalInst,
     IROriginalValuesForClone const& originalValues)
 {
+#if SLANG_ENABLE_IR_BREAK_ALLOC
+    _debugSetInstBeingCloned(originalInst->_debugUID);
+    SLANG_DEFER(_debugResetInstBeingCloned());
+#endif
+
     switch (originalInst->getOp())
     {
         // We need to special-case any instruction that is not
@@ -1427,7 +1432,6 @@ IRInst* cloneInst(
     }
     auto funcType = cloneType(context, originalInst->getFullType());
     context->builder = oldBuilder;
-
     IRInst* clonedInst = builder->createIntrinsicInst(
         funcType,
         originalInst->getOp(),
diff --git a/source/slang/slang-ir.cpp b/source/slang/slang-ir.cpp
index 3c2d5d2d1..63d3766ab 100644
--- a/source/slang/slang-ir.cpp
+++ b/source/slang/slang-ir.cpp
@@ -2,6 +2,7 @@
 #include "slang-ir.h"
 
 #include "../core/slang-basic.h"
+#include "../core/slang-platform.h"
 #include "../core/slang-writer.h"
 #include "slang-ir-dominators.h"
 #include "slang-ir-insts.h"
@@ -1738,8 +1739,21 @@ void IRBuilder::_maybeSetSourceLoc(IRInst* inst)
 }
 
 #if SLANG_ENABLE_IR_BREAK_ALLOC
-SLANG_API uint32_t _slangIRAllocBreak = 0xFFFFFFFF;
+uint32_t _slangIRAllocBreak = 0xFFFFFFFF;
+bool _slangIRPrintStackAtBreak = false;
 static bool _slangIRAllocBreakFirst = true;
+static uint32_t _slangInstBeingCloned = 0xFFFFFFFF;
+
+void _debugSetInstBeingCloned(uint32_t uid)
+{
+    _slangInstBeingCloned = uid;
+}
+
+void _debugResetInstBeingCloned()
+{
+    _slangInstBeingCloned = 0xFFFFFFFF;
+}
+
 uint32_t& _debugGetIRAllocCounter()
 {
     static uint32_t counter = 0;
@@ -1758,6 +1772,20 @@ uint32_t _debugGetAndIncreaseInstCounter()
 #if _WIN32 && defined(_MSC_VER)
         __debugbreak();
 #endif
+        if (_slangIRPrintStackAtBreak)
+        {
+            fprintf(stdout, "BEGIN IR Trace\nInstruction #%u created at:\n", _slangIRAllocBreak);
+            PlatformUtil::backtrace();
+            if (_slangInstBeingCloned != 0xFFFFFFFF)
+            {
+                fprintf(
+                    stdout,
+                    "Inst #%u is a clone of Inst #%u.\n",
+                    _slangIRAllocBreak,
+                    _slangInstBeingCloned);
+            }
+            fprintf(stdout, "END IR Trace\n");
+        }
     }
     return _debugGetIRAllocCounter()++;
 }
diff --git a/source/slang/slang-ir.h b/source/slang/slang-ir.h
index 54fc3d8de..0f4da4f0d 100644
--- a/source/slang/slang-ir.h
+++ b/source/slang/slang-ir.h
@@ -2640,6 +2640,10 @@ bool isMovableInst(IRInst* inst);
 
 #if SLANG_ENABLE_IR_BREAK_ALLOC
 uint32_t& _debugGetIRAllocCounter();
+extern uint32_t _slangIRAllocBreak;
+extern bool _slangIRPrintStackAtBreak;
+void _debugSetInstBeingCloned(uint32_t uid);
+void _debugResetInstBeingCloned();
 #endif
 
 // TODO: Ellie, comment and move somewhere more appropriate?
diff --git a/source/slang/slang.cpp b/source/slang/slang.cpp
index f65681e4b..29ca2328b 100644
--- a/source/slang/slang.cpp
+++ b/source/slang/slang.cpp
@@ -4,8 +4,10 @@
 #include "../core/slang-castable.h"
 #include "../core/slang-io.h"
 #include "../core/slang-performance-profiler.h"
+#include "../core/slang-platform.h"
 #include "../core/slang-shared-library.h"
 #include "../core/slang-string-util.h"
+#include "../core/slang-string.h"
 #include "../core/slang-type-convert-util.h"
 #include "../core/slang-type-text-util.h"
 // Artifact
@@ -24,6 +26,7 @@
 #include "slang-check.h"
 #include "slang-doc-ast.h"
 #include "slang-doc-markdown-writer.h"
+#include "slang-ir.h"
 #include "slang-lookup.h"
 #include "slang-lower-to-ir.h"
 #include "slang-mangle.h"
@@ -160,6 +163,21 @@ void Session::init()
 {
     SLANG_ASSERT(BaseTypeInfo::check());
 
+#if SLANG_ENABLE_IR_BREAK_ALLOC
+    // Read environment variable for IR debugging
+    StringBuilder irBreakEnv;
+    if (SLANG_SUCCEEDED(PlatformUtil::getEnvironmentVariable(
+            UnownedStringSlice("SLANG_DEBUG_IR_BREAK"),
+            irBreakEnv)))
+    {
+        String envValue = irBreakEnv.produceString();
+        if (envValue.getLength())
+        {
+            _slangIRAllocBreak = stringToInt(envValue);
+            _slangIRPrintStackAtBreak = true;
+        }
+    }
+#endif
 
     _initCodeGenTransitionMap();