{"id":3185,"date":"2010-11-11T14:40:00","date_gmt":"2010-11-11T14:40:00","guid":{"rendered":"https:\/\/test.simple-talk.com\/uncategorized\/subterranean-il-generics-and-array-covariance\/"},"modified":"2021-04-29T15:30:55","modified_gmt":"2021-04-29T15:30:55","slug":"subterranean-il-generics-and-array-covariance","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/blogs\/subterranean-il-generics-and-array-covariance\/","title":{"rendered":"Subterranean IL: Generics and array covariance"},"content":{"rendered":"<p>Arrays in .NET are curious beasts. They are the only built-in collection types in the CLR, and SZ-arrays (single dimension, zero-indexed) have their own commands and IL syntax. One of their stranger properties is they have a kind of built-in covariance long before generic variance was added in .NET 4. However, this causes a subtle but important problem with generics. First of all, we need to briefly recap on array covariance.<\/p>\n<p><b>SZ-array covariance<\/b><\/p>\n<p>To demonstrate, I&#8217;ll tweak the classes I introduced in my previous posts: <\/p>\n<pre>public class IncrementableClass {\n    public int Value;\n    \n    public virtual void Increment(int incrementBy) {\n        Value += incrementBy;\n    }\n}\n\npublic class IncrementableClassx2 : IncrementableClass {\n    public override void Increment(int incrementBy) {\n        base.Increment(incrementBy);\n        base.Increment(incrementBy);\n    }\n}<\/pre>\n<\/p>\n<p>In the CLR, SZ-arrays of reference types are implicitly convertible to arrays of the element&#8217;s supertypes, all the way up to <code>object<\/code> (note that this does not apply to value types). That is, an instance of <code>IncrementableClassx2[]<\/code> can be used wherever a <code>IncrementableClass[]<\/code> or <code>object[]<\/code> is required. When an SZ-array could be used in this fashion, a run-time type check is performed when you try to insert an object into the array to make sure you&#8217;re not trying to insert an instance of <code>IncrementableClass<\/code> into an <code>IncrementableClassx2[]<\/code>.<\/p>\n<p>This check means that the following code will compile fine but will fail at run-time: <\/p>\n<pre>IncrementableClass[] array = new IncrementableClassx2[1];\narray[0] = new IncrementableClass();    \/\/ throws ArrayTypeMismatchException<\/pre>\n<p> These checks are enforced by the various <code>stelem*<\/code> and <code>ldelem*<\/code> il instructions in such a way as to ensure you can&#8217;t insert a <code>IncrementableClass<\/code> into a <code>IncrementableClassx2[]<\/code>. For the rest of this post, however, I&#8217;m going to concentrate on the <code>ldelema<\/code> instruction.  <\/p>\n<p><b>ldelema<\/b><\/p>\n<p>This instruction pops the array index (<code>int32<\/code>) and array reference (<code>O<\/code>) off the stack, and pushes a pointer (<code>&amp;<\/code>) to the corresponding array element. However, unlike the <code>ldelem<\/code> instruction, the instruction&#8217;s type argument must match the run-time array type <em>exactly<\/em>. This is because, once you&#8217;ve got a managed pointer, you can use that pointer to both load and store values in that array element using the <code>ldind*<\/code> and <code>stind*<\/code> (load\/store indirect) instructions. As the same pointer can be used for both input and output to the array, the type argument to <code>ldelema<\/code> must be <em>in<\/em>variant. At the time, this was a perfectly reasonable restriction, and maintained array type-safety within managed code.<\/p>\n<p>However, along came generics, and with it the <code>constrained callvirt<\/code> instruction. So, what happens when we combine array covariance and <code>constrained callvirt<\/code>?<\/p>\n<pre>.method public static void CallIncrementArrayValue() {\n\n    \/\/ IncrementableClassx2[] arr = new IncrementableClassx2[1]\n    ldc.i4.1\n    newarr IncrementableClassx2\n    \n    \/\/ arr[0] = new IncrementableClassx2();\n    dup\n    newobj instance void IncrementableClassx2::.ctor()\n    ldc.i4.0\n    stelem.ref\n    \n    \/\/ IncrementArrayValue&lt;IncrementableClass&gt;(arr, 0)\n    \/\/ here, we're treating an IncrementableClassx2[] as IncrementableClass[]\n    dup\n    ldc.i4.0\n    call void IncrementArrayValue&lt;class IncrementableClass&gt;(!!0[],int32)\n    \n    \/\/ ...\n    ret\n}\n\n.method public static void IncrementArrayValue&lt;(IncrementableClass) T&gt;(\n        !!T[] arr, int32 index) {\n\n    \/\/ arr[index].Increment(1)\n    ldarg.0\n    ldarg.1\n    ldelema !!T\n    ldc.i4.1\n    constrained. !!T\n    callvirt instance void IIncrementable::Increment(int32)\n    \n    ret\n}<\/pre>\n<p> And the result: <\/p>\n<pre>Unhandled Exception: System.ArrayTypeMismatchException:\n       Attempted to access an element as a type incompatible with the array.\n   at IncrementArrayValue[T](T[] arr, Int32 index)\n   at CallIncrementArrayValue()<\/pre>\n<\/p>\n<p>Hmm. We&#8217;re instantiating the generic method as <code>IncrementArrayValue&lt;IncrementableClass&gt;<\/code>, but passing in an <code>IncrementableClassx2[]<\/code>, hence the <code>ldelema<\/code> instruction is failing as it&#8217;s expecting an <code>IncrementableClass[]<\/code>.<\/p>\n<p><b>On features and feature conflicts<\/b><\/p>\n<p>What we&#8217;ve got here is a conflict between existing behaviour (<code>ldelema<\/code> ensuring type safety on covariant arrays) and new behaviour (managed pointers to object references used for every <code>constrained callvirt<\/code> on generic type instances). And, although this is an edge case, there is no general workaround. The generic method could be hidden behind several layers of assemblies, wrappers and interfaces that make it a requirement to use array covariance when calling the generic method. Furthermore, this will only fail at runtime, whereas compile-time safety is what generics were designed for!<\/p>\n<p>The solution is the <code>readonly.<\/code> prefix instruction. This modifies the <code>ldelema<\/code> instruction to ignore the exact type check for arrays of reference types, and so it lets us take the address of array elements using a covariant type to the actual run-time type of the array: <\/p>\n<pre>.method public static void IncrementArrayValue&lt;(IncrementableClass) T&gt;(\n        !!T[] arr, int32 index) {\n\n    \/\/ arr[index].Increment(1)\n    ldarg.0\n    ldarg.1\n    readonly.\n    ldelema !!T\n    ldc.i4.1\n    constrained. !!T\n    callvirt instance void IIncrementable::Increment(int32)\n    \n    ret\n}<\/pre>\n<p> But what about type safety? In return for ignoring the type check, the resulting <em>controlled mutability<\/em> pointer can only be used in the following situations: <\/p>\n<ul>\n<li>As the object parameter to <code>ldfld<\/code>, <code>ldflda<\/code>, <code>stfld<\/code>, <code>call<\/code> and <code>constrained callvirt<\/code> instructions<\/li>\n<li>As the pointer parameter to <code>ldobj<\/code> or <code>ldind*<\/code><\/li>\n<li>As the source parameter to <code>cpobj<\/code><\/li>\n<\/ul>\n<p> In other words, the only operations allowed are those that read from the pointer; <code>stind*<\/code> and similar that alter the pointer itself are banned. This ensures that the array element we&#8217;re pointing to won&#8217;t be changed to anything untoward, and so type safety within the array is maintained.<\/p>\n<p>This is a typical example of the maxim that whenever you add a feature to a program, you have to consider how that feature interacts with every single one of the existing features. Although an edge case, the <code>readonly.<\/code> prefix instruction ensures that generics and array covariance work together and that compile-time type safety is maintained.<\/p>\n<p><a href=\"https:\/\/www.simple-talk.com\/community\/blogs\/simonc\/archive\/2010\/11\/17\/95700.aspx\">Tune in next time<\/a> for a look at the <code>.ctor<\/code> generic type constraint, and what it means.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Arrays in .NET are curious beasts. They are the only built-in collection types in the CLR, and SZ-arrays (single dimension, zero-indexed) have their own commands and IL syntax. One of their stranger properties is they have a kind of built-in covariance long before generic variance was added in .NET 4. However, this causes a subtle&#8230;&hellip;<\/p>\n","protected":false},"author":186659,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[143538,2],"tags":[],"coauthors":[],"class_list":["post-3185","post","type-post","status-publish","format-standard","hentry","category-dotnet-development","category-blogs"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/3185","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/186659"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=3185"}],"version-history":[{"count":2,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/3185\/revisions"}],"predecessor-version":[{"id":75611,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/3185\/revisions\/75611"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=3185"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=3185"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=3185"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=3185"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}