Avoid crash when bad argument given to [instance(...)] attribute (#464)

Fixes #463

Some of the attributes were failing to check for a `null` result from `checkConstantIntVal`, and so they crashed when a bad expression was used in an attribute. The particular way this had been triggered was that a user put an HLSL geometry shader in the same file with other code, using an entry point like:

```hlsl
[instance(COUNT)]
void myGeometryShader(...) {...}
```

They then defined `COUNT` as a preprocessor macro when compiling using the GS, but left it undefined otherwise. The result was that the argument to the `instance` attribute would fail to type check, and thus wouldn't count as a constant integer value, so that `checkConstantIntVal` returns `null` and results in the crash.

The workaround for the user is to always define `COUNT`, even when not compiling the GS.

The fix in the compiler is to guard against `null` in these cases and bail out of attribute checking. I also implemented logic so that `CheckIntegerConstantExpression` (which is invoked by `checkConstantIntVal`) will not produce an additional error message if the underlying expression failed to type check. In this casem the user will get an `undefined identifier: COUNT` error message, and we don't need to waste their time by also telling them that this isn't a compile-time constant expression.

2 files changed, 17 insertions, 0 deletions

diff --git a/tests/bugs/gh-463.slang b/tests/bugs/gh-463.slang
new file mode 100644
index 000000000..95890eba5
--- /dev/null
+++ b/tests/bugs/gh-463.slang
@@ -0,0 +1,11 @@
+//TEST:SIMPLE:
+
+// undefined identifier used in `[instance(...)]` attribute
+
+struct V { float4 u : U; };
+
+[instance(COUNT)]
+void myGS(triangle V input[3], TriangleStream<V> output)
+{
+	
+}
diff --git a/tests/bugs/gh-463.slang.expected b/tests/bugs/gh-463.slang.expected
new file mode 100644
index 000000000..d40a845a3
--- /dev/null
+++ b/tests/bugs/gh-463.slang.expected
@@ -0,0 +1,6 @@
+result code = -1
+standard error = {
+tests/bugs/gh-463.slang(7): error 30015: undefined identifier 'COUNT'.
+}
+standard output = {
+}