Elvis operator / Null-Coalescing operator

Null-conditional / Null-Coalescing operators ( '?.' and '??' ) cannot currently be used on serialized fields.

When deserializing a MonoBehaviour, all unreferenced fields are not set to null, instead it references a object with a == operator overload, so it results true when testing for null (when in fact it is not a null reference).

This leads to some unexpected behaviors like:

Code (CSharp):

1.  
2. public class TestBehaviour : MonoBehaviour
3. {
4.     UnityEngine.GameObject reference;
5.  
6.     void Start() {
7.         if (reference?.transform == null) {
8.             Debug.Log("It works!"); // This line is printed as expected
9.         }
10.     }
11. }
12.  

But changing the reference to public changes the execution path

Code (CSharp):

1.  
2. public class TestBehaviour : MonoBehaviour
3. {
4.     public UnityEngine.GameObject reference;
5.  
6.     void Start() {
7.         if (reference?.transform == null) { // throws an exception
8.             Debug.Log("Not executed"); // This line is not reached
9.         }
10.     }
11. }
12.  

Exception:

UnassignedReferenceException: The variable reference of TestBehaviour has not been assigned.
You probably need to assign the reference variable of the TestBehaviour script in the inspector.​

This null reference behaviour was probably created to help developers when trying to access unassigned references, where it would print the attribute name inside the exception, but this 'helper' is now causing an UnassignedReferenceException where it should not exist.


The ?? operator also does not work as expected on that scenario.

Code (CSharp):

1.  
2. var validReference = optionalReference ?? referencedAtribute;
3.  

The code above would always store 'optionalReference' on 'validReference', regardless if 'optionalReference' is set or not.

-

My opinion:
this pretty UnassignedReferenceException only works when trying to access a Serialized Object attribute, so the user is not safe from NullReferenceExceptions.

- It tries to solve a 'problem' that is not Unity's fault (out of its domain)
- It helps only on some specific scenarios
- It creates other problems with it

Basically not worth it.


What should companies do to avoid such weird bugs? Recommend its developers to not use '?.' and '??' at all? (yes I've heard that).


At least give me the option to read attributes as null, when they are not referenced on Unity editor.

 

I think many would agree. This will become quite a lot worse when C# 6 is rolled out and ?. becomes available as well.

 

Could you do a quick check for what the second version does in builds? I believe Unity does not replace the null with a bullshit "let me help you will your nullref" object in builds.

If that's the case, maybe the difference between editor time and runtime is enough to get rid of it! It really only helps the people who are struggling through Roll-A-Ball, and is an ugly gotcha for everyone else.


The ?? error has probably always been there, I just haven't run into it because ?? doesn't really work with the whole == override thing.

 

I'm wondering what unity is to do here. I agree that all the bogus null objects should be removed (for serialization and also for GetComponent returns) - we just want to write standard C# (and get rid of useless allocations while at it.

However, you'd still have this problem for Destroy() and your own variables. The nullref overload there is well... it is convenient and at this point impossible to remove really

Maybe now that they're in the .net steering group they could ask microsoft for some mechanism to have ? and ?? compiles to using the actual Equals or == operator instead of hardcoded compare. I guess that might impact performance so it would need to only compile to that for certain types.

 

This was discussed in some length on the blog a couple of years ago.

Basically, the custom operator is there to avoid the need to double up every single null check with an isAlive check or similar, since the object could have been destroyed on the C++ side of things. In my opinion, the custom operator should be removed and the implicit bool cast that is already in place should be used for that purpose instead.

 

While I agree for if (blah == null) - I think with C# 6 the situation is different now. It means that we still couldn't write m_comp?.Var defeating the now common ?. operator.

I really don't like the operator either but at the same time unity has always set the goal that you can just write standard C# . I think atm I agree, this is broken, but maybe with their influence in .net now they can do something about the underlying problem.

 

Are you talking about the special handling of null fields in the editor, or the == null override? Or both?

 

So the special handling of null fields is easier to handle - just remove it. It's causing inconsistencies between the editor and the runtime and it's causing the profiler to show memory allocations that's not there at runtime. The help it gives to new users who doesn't know what a nullref is doesn't seem to help much, from what I'm seeing in the scripting forums.

The == null thing is harder. The problem is that it's really convenient that == null checks for destroyed objects. It has a downside where it's surprising for experienced programmers that's new to Unity and it breaks around interfaces (see discussion here), so I'm a bit back-and-forth on what's the good option.

If the goal is that these statements should be equivalent:

Code (csharp):

1. var foo = bar != null ? bar : new Foo();
2. //and
3. var foo = bar ?? new Foo();

and that these statements should be equivalent:

Code (csharp):

1. if(foo != null) {
2.     foo.DoSomething();
3. }
4. //and
5. foo?.DoSomething();

There are some options:

1: Remove the == override, and have all of the statements above break on references to destroyed objects. This is consistent with how C# works, but requires retraining all Unity developers, retraining all of Unity's developers, and breaks every single free and paid plugin that includes scripts.
This also doesn't make either ? or ?? useful in Unity. It just makes things consistent over the board. It's not a very good solution.
2: Don't change anything. This doesn't break anything, but since the elvis operator is a lot more useful than ??, I'm assuming that a lot more people will do this and despair:

Code (csharp):

1. transformThatHasBeenDestroyed?.Translate(direction); //MissingReferenceException

Which means that no problem has been solved, and ? just can't be used in Unity (kinda like ?? really can't be used unless you're completely certain that the object will never be destroyed)

3: Somehow convince Microsoft to have ? and ?? use the class' == operator instead of the default one to check for null. This is potential breakage in ALL .NET CODE OUTSIDE UNITY, so the chances are slim.

4: Somehow convince Microsoft to make ? and ?? be overrideable on a per-class basis, and override it in UnityEngine.Object.
This has the least impact on things, and solves all problems. On the other hand, even if it happens, it probably won't happen before C# 8, which is probably years off. Unless they're still taking suggestions for C# 7?

5: Make Unity actually use UnityC#, which is a C# language that's equivalent to C# in every single way except that ? and ?? uses the class' == check. As an added bonus, the == check for object could be changed to do what UnityEngine.Object's == does.
This option sounds really good, but is also completely off the walls insane, so idk.

Is there any other option available?

 

This is the only option I like. At first I thought it's kind of crazy, but then realized that non-nullable reference types are expected to be one of the main themes for C# 8. Since null-whatever-operators will have absolutely no sense for non-nullable types, the override might become quite handy even outside of Unity.

Code (CSharp):

1. public class Foo
2. {
3.     public static bool operator ?(Foo foo) => foo.IsValid;
4. }

PS
Oops! It will never happen, because otherwise all the C# code all around the world will have to be recompiled. Not good.

 

There could be another solution, Unity is quite keen on patching assemblies in general so it could potentially be possible to remove the custom null check on the language side and patch it in directly at the IL level.

That should allow all the operators to function as expected, at the expense of having even more potentially unexpected behaviour.

Still wouldn't help when accessing Unity objects through interfaces and the like though.

 

Does anyone know how the compiler generates the code for the new ?. operator?

For example, why doesn't this code work out of the box:

Code (CSharp):

1. transformThatHasBeenDestroyed?.Translate(direction); //MissingReferenceException

 

Since transformThatHasBeenDestroyed isn't null, and ? probably uses the object's referenceEquals.

The latest version of the C# specs I can find (here, not official, but the latest official ones are from 2012 and doesn't contain the operator), says this about the operator:

Note that the mention is specifically "is non-null", not "== null resolves to false". So I'd guess the generated result is equivalent to:

Code (csharp):

1. if(!object.ReferenceEquals(transformThatHasBeendestroyed, null)) {
2.     transformThatHasBeenDestroyed.Translate(Direction);
3. }

If this "worked" in Unity, ie. the line would be passed by if the object was destroyed, then Unity would somehow be breaking the C# specs.

There is no solution to this that doesn't have major drawbacks. I think the UnityC# superset is the most appealing, but I've got no idea how much work that would be.

 

I just ran this test and peeked at the generated code (using ildasm):

Code (CSharp):

1. .method private hidebysig instance void  Start() cil managed
2. {
3.   // Code size       31 (0x1f)
4.   .maxstack  8
5.   IL_0000:  nop
6.   IL_0001:  ldarg.0
7.   IL_0002:  ldfld      class [UnityEngine]UnityEngine.Transform TestScript::transformThatHasBeenDestroyed
8.   [B]IL_0007:  brtrue.s   IL_000e[/B]
9.   IL_0009:  br         IL_001e
10.   IL_000e:  ldarg.0
11.   IL_000f:  ldfld      class [UnityEngine]UnityEngine.Transform TestScript::transformThatHasBeenDestroyed
12.   IL_0014:  call       valuetype [UnityEngine]UnityEngine.Vector3 [UnityEngine]UnityEngine.Vector3::get_zero()
13.   IL_0019:  callvirt   instance void [UnityEngine]UnityEngine.Transform::Translate(valuetype [UnityEngine]UnityEngine.Vector3)
14.   IL_001e:  ret
15. } // end of method TestScript::Start
16.  

What happens (if i understand correctly) is that an IL instruction (brtrue.s) is used to determine if an object is not-null, and if so, jumps to the correct location to continue evaluation.
This completely overrides Unity's (or in fact C# 's) operator evaluation and the overloading mechanism, so Unity's overloaded operator is not used in this case.

 

BTW - this new operator will only work incorrectly in the cases where it is triggered against a destroyed object. How often do you think this behaviour is encountered by developers? for the most cases it should be fine, though i still think there's something that should be done about it.

 

There was some chatter about this on Slack in the #code channel on Sep 27.

My question was why the heck the 'Missing' even exists, because it should just be null. It was pointed out that the C++ side of the data can be destroyed but the C# side cant because with C# there is no way to find all references to an object and null them, hence the C# wrapper has to stay (missing/kinda dead) until you drop all of the references to it and GC hits it.

Dunno if they're looking into a workaround for it or not.

 

The vast majority of null-checks I write are actually destroyed-checks. Most of them are of the kind "handle that the object you were following/checking/managing got destroyed".

I've got a hunch they are.

 

IMHO, all proposed solutions are not very good ones (e.g: change the universe to make this work) or other hacks.

it was said that it's not technically possible (or desired ? due to performance reasons), to scan for all references to a given object and invalidate them (make them null). If every native (C++) object holds a pointer to its managed (C#) counterpart, is it possible to nullify these (e.g: destroy the object) ? or is it the reference that should be nullified as well ?

Another (ugly) suggestion - a UnityEngine.Object will have another reference (e.g: this.self or something similar). This will be monitored by the native side and nullified in case the object is destroyed.

You would then check by something like:

Code (CSharp):

1. transformThatHasBeenDestroyed?.self?.Translate(Direction);

Ugly as f** i know

 

Yes, you can achieve that to work around the probem. It's just that this new operator was introduced for simplifying long (boring) code for null checking.. this kinda defeats the purposes (slightly), and also did i mention its pretty ugly?

 

It's also side-stepping the problem a bit.

Fundamentally what's going on here is the Unity uses some tricks to redefine the semantics of the language.

In C#, the semantics of "a == null" is "a has not been assigned".
In Unity, the semantics of "a == null" is "a has not been assigned OR the things a was assigned to has been destroyed"

I believe that it would be better if Unity changed the semantics of "null", so all the parts of the C# language that works with the concept of null resolves that as "not assigned or destroyed".


The only thing introducing an IsAlive() extension does is to push the responsibility of the semantic change from Unity to the user. It's not only ugly, it also essentially exactly the same as just removing the overload altogether and pushing the responsibility of handling destroyed objects to the user.

 

I have to disagree a null reference is just that - a reference to nothing. Trying to expand that to other things only leads to confusion and other (negative) things.

From my viewpoint - it won't be the end of the world if they dropped the overloaded equals (==) operator and asked me to check if an object is destroyed or null. Not that this will help this particular scenario anyway (as its not even using the overloaded operator).

Sometimes you have to correct things that were done wrong in the past

 

I also was envisioning exactly liortal's solution. Add a new property to UnityEngine.Object that is actual null if the base is fake null. Something like:

Code (csharp):

1. Object Checked => { if (this == null) return null; return this; }

Then you could do:

Code (csharp):

1. transformThatsBeenDestroyed?.Checked?.Translate()

Granted we could do this with an extension method, but maybe there's some trickery Unity can do to make it faster, and you could also add a new compiler warning for anyone using ?? or ?. on a Component without following it with Checked. It's not a great solution but it's better than doing nothing.

 

Well it's not EXACTLY the same as removing it; actually removing it would break every Unity project in existence. This is just adding a way for developers who know about "??" and "?." to have official support for those operators.

 

This would be the best as it wouldn't require people to code any differently - old stuff would continue working as normal, and ?. and ?? would suddenly work as expected. However, this doesn't work if you're using external dlls. In that case, a Checked() extension method that returns self/real null works better.

 

Why not? There's nothing stopping Unity from patching DLLs before loading their assemblies. I do wonder, does the automatic serialisation of UNET work if you compile to DLL and import? All that stuff is patched in.

 

If you're using non-Unity .dll's, that might break assumptions in those dlls. Like "If I do a null-check before putting an object into a list, I can safely assume that the list will never contain null"

 

their code weaving is operated at the IL level from what i know. so it never modifies any "source code", only compiled code.

In that case, they should have no issue patching DLLs that you throw at them, not sure if thats how it works though.

 

Yes, but that is already the case if you are using anything deriving from UnityEngine.Object

Yeah, as far as I'm aware it Cecils all that stuff in, on loading the assemblies in editor, and when building.

 

Yeah, but that is already the case now with any UnityEngine.Object. Any injected null check modification would only be applied to when the operands derive from that base class. I stepped through the ILSpy code to see how it decompiles such expressions and all that information is available to it from the IL at the time of matching the expressions, so it's possible.

 

Any news on this? Is Unity going to fix these operators to handle null checks similarly as "==" and "!="?

I just created an extension method workaround just to be a bit safer... https://github.com/Haapavuo/UnityExtensions/blob/master/UnityObjectExtensions.cs

 

I also feel that the time of the override == operator is over. It should have been over years ago when they were discussing it. The elvis/null coalescing op is too common outside of Unity. Overriding null is some cases and not others causes all sorts of unintended problems (performance and knowledge). I feel it is not worth the small convenience of having a IsAlive() or something similar that would be defined on the objects that need it. Obviously, this is a huge change and change is hard. Maybe have a Unity preference for enabling it in the player if people really need the legacy behavior. Going forward this is going to bring the scripting more inline with C# standards.

 

Oh dear, this is scary as hell, I just shot myself in the foot trying to do something like this in the new .NET 4.x api:

Code (CSharp):

1. shellPrefabRB = shellPrefabRB ?? (GameObject)Resources.Load(RES_FOLDER + shellPrefabRB_Res);

where shellPrefabRB is actually a serialised field in the Monobehavior, and I have already written code like this all over a project (luckily not so much of them), now I have to go back and re-wire those code with the old null check format.

 

I wrote a proposal to make them overridable on CSharp's GitHub:
https://github.com/dotnet/csharplang/issues/2020

 

@joncham There is quite the discussion going on there. What do you think about it? Is this something that will be addressed or discussed in the future?

 

Pardon me but any hackish improvisation deviating from standard is bound to backfire in ugly, devious ways at some point. And I'm talking about overriding "==" in the first place. Further prolonging the overload agony complicates things even more.
Why call
Let "null" mean "null", "==" mean "==" and "initialized with non-null yet non-usable value" be encompassed in something to mean just that, like "invalid" (as opposed to something working differently, hidden from the user).
So instead of

Code (CSharp):

1. if(reference != null)

let users do something like

Code (CSharp):

1. if(reference.isValid())

or maybe

Code (CSharp):

1. if(reference != invalid)

 

This behaviour, now with all the shiny runtime upgrades, still makes it hard to write code at times where you're for example have to work ontop of someone elses code and need to dive into whole inheritance hierarchies just to figure out if you're allowed to use standard language features. I hope Unity is still considering to rework this.

 

Not sure what you mean by that comment....

Why do u need to dig into someone else's code to know if u can safely use certain features?

 

I assume he means checking if what you’re dealing with is derived from UnityEngine.Object, although I always found that a case of pressing F12 once.

 

What workarounds to use ?. for serialized gameobject fields are there?

One I can think of: extension method for GameObject IsValid() which uses == null, and using it like this: obj.IsValid()?.Call(). However, it's not perfect.

 

You can actually do

Code (CSharp):

1. if(gameobject)

, which is overloaded and uses the overloaded == operator

 

-ish, both call CompareBaseObjects(). It actually puzzles me a bit that operator bool is not just (object)o != null && IsNativeObjectAlive(o). Surely there's no need to go through all the conditionals in case the compiler doesn't just optimise it all away.

 

Plus this doesn't help with the elvis operator.

It's still something easily missed when working with external code (that sometimes you don't even have the source or license to decompile to trace an objects root type)

 

While not exactly the same issue as the new operators, I think it's worth bringing up the fact that interfaces suffer from the same problem. A null check against a reference to a Unity Object through an interface also bypasses the checks.

 

I feel like there should probably be a documentation note about this at the top of the script ref or something. I just ran into this while implementing a property and it's pretty confusing. It's been a long time since I've been over the core docs but it would probably be very good to maintain some page of divergences from the current csharp standard within Unity.

It might be even more ideal if there's some way to do a compile time detection and throw a warning within Unity. Something like "Hey you're trying to coalesce a serialized field and that won't work!". Currently it just results in a somewhat confusing runtime break.

 

Visual Studio really can help here. I am running 16.6.2 and

Visual Studio 2019 Tools for Unity

(4.6.1.0). Building the project in VS shows Unity specific issues in the output window, such as:

info UNT0007: Unity objects should not use null coalescing

 

The easiest fix I can think of is

Code (CSharp):

1. [CanBeNull]
2. public static T N<T>([CanBeNull] this T o) where T  : UnityEngine.Object
3. {
4.     if(o == null)
5.         return null;
6.     return o;
7. }

and then use it whenver you want to use

??

or

.?

:

var name = gameObject.N()?.name ?? "Null";

But... as a programmer, I am lazy and I feel I am above this kind of repetative boilerplate stuff that just muddies the readability of the code.
If I had it my way, I'd put a precompilation step that, when

.?

or

??

is used on a reference to

UnityEngine.Object

, it just sneaks this extension in there. Problem solved. This is why C++ has macros: to get rid of boilerplate crap. But judging by the C# team, they seem more intent to police the way you do things.

 

Erm...

Code (CSharp):

1. m_RebindOperation = action.PerformInteractiveRebinding(bindingIndex)
2.     .OnCancel(
3.         operation =>
4.         {
5.             m_RebindStopEvent?.Invoke(this, operation);
6.             m_RebindOverlay?.SetActive(false); //rip
7.             UpdateBindingDisplay();
8.             CleanUp();
9.         })