Sort test list to be deterministic (#7432)

On Linux/MacOS, the test file lists were not in a sorted order and the order was non-deterministic.
It causes intermittent CI failures.

This commit is to make the list sorted and CI more reliable.

1 files changed, 10 insertions, 0 deletions

diff --git a/tools/slang-test/slang-test-main.cpp b/tools/slang-test/slang-test-main.cpp
index 742300d22..f03ae8292 100644
--- a/tools/slang-test/slang-test-main.cpp
+++ b/tools/slang-test/slang-test-main.cpp
@@ -4579,6 +4579,16 @@ void runTestsInDirectory(TestContext* context)
 {
     List<String> files;
     getFilesInDirectory(context->options.testDir, files);
+
+    // NTFS on Windows stores files in sorted order but not on Linux/Macos.
+    // Because of that, the testing on Linux/Macos were randomly failing, which
+    // is a good thing because it reveals problems. But it is useless
+    // if we cannot reproduce the failures deterministrically.
+    // https://github.com/shader-slang/slang/issues/7388
+    //
+    // TODO: We need a way to shuffle the list in a deterministic manner.
+    files.sort();
+
     auto processFile = [&](String file)
     {
         if (shouldRunTest(context, file))