Search Unity

[OpenSource] Fast.Reflection: delegates for lightning fast metadata reflection!

Discussion in 'Assets and Asset Store' started by vexe, Mar 1, 2015.

  1. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Went open source: Github repository link.


    Hi all,

    so if you ever wrote any editor extension before, then you've have to deal with reflection at some point. Reflection is all cool and powerful, it let's us do all sorts of funk and jazz. Unfortunately it's slow, really slow. Houston, we have a problem... So I decided to address the problem. I wrote an extension methods library for MemberInfos (fields, properties and methods) that emits IL to generate fast delegates to set/get fields/properties, call methods and invoke and constructors. I called the library, Fast.Reflection



    What?
    - an extension methods library that emits IL code to generate extremely fast
    field/property getters/setter, method calls and ctor invocations.
    Since it emits IL, it can't be used on AOT platforms

    Who's this for?
    - anyone who needs 'fast' reflection
    - editor developers (especially)
    - as a bonus, you could use it for your game if you're targetting standalone

    Features?
    - lightning fast (just a couple of milliseconds away from direct calls!)
    - very simple, easy to use and intuitive API
    - 0 GC column in many cases!
    - weakly typed API if you don't know the target and member types in advance
    - strongly typed API (generic)
    - works on both reference and value types! (no need to do anything special when dealing with structs)
    - handles ref/out method parameters
    - handles instance and static members (fields, properties and methods)
    - doesn't require Unity Pro
    - single source file with ~500 lines. No 3rd parties, no dlls.

    Price? - $5!

    API?
    - there are 4 delagate types to set/get fields/properties, call methods and invoke constructors
    Code (csharp):
    1.  
    2. public delegate void MemberSetter<TTarget, TValue>(ref TTarget target, TValue value);
    3. public delegate TReturn MemberGetter<TTarget, TReturn>(TTarget target);
    4. public delegate TReturn MethodCaller<TTarget, TReturn>(TTarget target, object[] args);
    5. public delegate T CtorInvoker<T>(object[] parameters);
    6.  
    - then we have the following extension methods:
    * strongly-typed API
    Code (csharp):
    1.  
    2. public static CtorInvoker<T> DelegateForCtor<T>(this Type type, params Type[] paramTypes)
    3. public static MemberGetter<TTarget, TReturn> DelegateForGet<TTarget, TReturn>(this FieldInfo field)
    4. public static MemberSetter<TTarget, TValue> DelegateForSet<TTarget, TValue>(this FieldInfo field)
    5. public static MemberGetter<TTarget, TReturn> DelegateForGet<TTarget, TReturn>(this PropertyInfo property)
    6. public static MemberSetter<TTarget, TValue> DelegateForSet<TTarget, TValue>(this PropertyInfo property)
    7. public static MethodCaller<TTarget, TReturn> DelegateForCall<TTarget, TReturn>(this MethodInfo method)
    8.  
    * weakly-typed API
    Code (csharp):
    1.  
    2. public static CtorInvoker<object> DelegateForCtor(this Type type, params Type[] ctorParamTypes)
    3. public static MemberGetter<object, object> DelegateForGet(this FieldInfo field)
    4. public static MemberSetter<object, object> DelegateForSet(this FieldInfo field)
    5. public static MemberGetter<object, object> DelegateForGet(this PropertyInfo property)
    6. public static MemberSetter<object, object> DelegateForSet(this PropertyInfo property)
    7. public static MethodCaller<object, object> DelegateForCall(this MethodInfo method)
    8.  
    Usage?
    - say we have the following struct or class:
    Code (csharp):
    1.  
    2. public struct|class Test
    3. {
    4.   public int Field;
    5.   public static string Property { get; set; }
    6.   public void ByRefMethod(ref int x, int y, out int z) { x = y = z = -1; }
    7. }
    8.  
    - we can then write:
    Code (csharp):
    1.  
    2.   // get metadata
    3.   var type = typeof(Test);
    4.   var field = type.GetField("Field");
    5.   var staticProperty = type.GetProperty("Property", BindingFlags.Static | BindingFlags.Public);
    6.   var byRefMethod = type.GetMethod("ByRefMethod");
    7.  
    1- weakly typed API:
    * useful if you don't know the target and member types at compile time
    * will generate around a small number of bytes of garbage when dealing with value-type members
    Code (csharp):
    1.  
    2.   // get a delgate for our type constructor, and construct an instance
    3.   var ctor = type.DelegateForCtor(Type.EmptyTypes); // get a ctor with no parameters
    4.   object inst = ctor(null); // null for no arguments.
    5.  
    6.   // get an open-instance weakly-typed delegate to set the field on a target instance
    7.   var fieldSetter = field.DelgeateForSet();
    8.   fieldSetter(ref inst, 10);
    9.  
    10.   // get a delegate to get the field value
    11.   var fieldGetter = field.DelgeateForGet();
    12.   Debug.Log(fieldGetter(inst)); // 10
    13.  
    14.   // do the same for the property
    15.   var propertySetter = property.DelegateForSet();
    16.   // when dealing with static members, there's no 'target'
    17.   // but we can't pass 'null' because our setter expect a 'ref' parameter
    18.   // we can have a null object around and pass that, or just pass our 'inst'
    19.   // we can pass whatever we want, the target is completely ignored...
    20.   propertySetter(ref inst, "test");
    21.  
    22.   var propertyGetter = property.DelegateForGet();
    23.   Debug.Log(propertyGetter(null)); // "test"
    24.  
    25.   // get a delegate to call our method
    26.   var byRefCaller = byRefMethod.DelegateForCall();
    27.   var args = new object[] { 1, 2, 3 };
    28.   byRefCaller(inst, args);
    29.   Debug.Log(args[0]); // -1
    30.   Debug.Log(args[1]); // 2
    31.   Debug.Log(args[2]); // -1
    32.  
    2- strongly-typed API
    * useful if you know the type of target in compile time
    * generates 0 bytes garbage even when dealing with value-type members
    Code (csharp):
    1.  
    2.   var ctor = type.DelegateForCtor<Test>(Type.EmptyTypes);
    3.   Test inst = ctor(null);
    4.  
    5.   // get an open-instance strongly-typed delegate to set the field on a target instance
    6.   var fieldSetter = field.DelgeateForSet<Test>();
    7.   fieldSetter(ref inst, 10); // generates 0 bytes garbage! no boxing cause we're strongly-typed
    8.  
    9.   var fieldGetter = field.DelgeateForGet<Test>();
    10.   Debug.Log(fieldGetter(inst)); // 0 bytes gc
    11.  
    12.   var propertySetter = property.DelegateForSet<Test>();
    13.   propertySetter(ref inst, "test");
    14.  
    15.   var propertyGetter = property.DelegateForGet<Test>();
    16.   Debug.Log(propertyGetter(null)); // "test"
    17.  
    18.   // get a delegate to call our method
    19.   // second generic arg is the return type.
    20.   // the generated code will return null in the case of method with void return
    21.   // so it doesn't matter much what the return type is as long as it's a reference type
    22.   // so we just pass 'object'
    23.   var byRefCaller = byRefMethod.DelegateForCall<Test, object>();
    24.   var args = new object[] { 1, 2, 3 };
    25.   byRefCaller(inst, args);
    26.   Debug.Log(args[0]); // -1
    27.   Debug.Log(args[1]); // 2
    28.   Debug.Log(args[2]); // -1
    29.  
    Benchmarks?

    (running 1,000,000 times)



    Please see source code for test here. (each measurement is running 1,000,000 times)

    GC column?

     
    Last edited: May 10, 2015
    forestrf likes this.
  2. kurylo3d

    kurylo3d

    Joined:
    Nov 7, 2009
    Posts:
    1,123
    Perhaps a youtube video would make this understandable to me.. and what it does other then code screenshots :p. Im an artist.. and figure reflections would be geared toward artists.
     
  3. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Hey there. I was expecting an artist to pop up :D - But by reflection here I don't mean reflective surfaces or shaders or any reflective material. I mean 'introspection', .NET metadata reflection Getting metadata information about a class fields, properties, methods, what attributes are applied on them etc. This is not geared towards artists, but to editor developers that make the tools for the artists.
     
    Tryz and TwiiK like this.
  4. hadicoco

    hadicoco

    Joined:
    May 10, 2014
    Posts:
    13
    Can you give an example of what code gets generated for those delegates?
     
  5. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    @hadicoco, sure! For example:

    Code (csharp):
    1.  
    2. public class|struct Test
    3. {
    4.      public int x;
    5.      public static string str { get; set; }
    6.      public void ByRef(ref int x, int y, out int z) { }
    7. }
    8.  
    9. var type = typeof(Test);
    10. var field = type.GetField("x");
    11. var property = type.GetProperty("str");
    12. var method = type.GetMethod("byRef");
    13.  
    14. 1- var weakFieldGetter = field.DelegateForGet();
    15. 2- var weakFieldSetter = field.DelegateForSet();
    16. 3- var strongFieldGetter = field.DelegateForGet<Test, int>();
    17. 4- var strongFieldSetter = field.DelegateForSet<Test, int>();
    18. 5- var weakPropertyGetter = property.DelegateForGet();
    19. 6- var weakPropertySetter = property.DelegateForSet();
    20. 7- var strongPropertyGetter = property.DelegateForGet<Test, string>();
    21. 8- var strongPropertySetter = property.DelegateForSet<Test, string>();
    22. 9- var weakMethodCaller = method.DelegateForCall();
    23. 10- var strongMethodCaller = method.DelegateForCall<Test, object>();
    24.  
    These will generate the following (in order)

    Code (csharp):
    1.  
    2. 1- public static object FieldGetter(object target)
    3. {
    4.     return ((Test)target).x; // this is where the boxing happens! - notice our return type is 'object' but 'x' is int, so we allocate an object on the heap and wrap x with it
    5. }
    6.  
    7. 2- public static void FieldSetter(ref object target, object value)
    8. {
    9.    // if Test was a class, we generate:
    10.    ((Test)target).x = (int)value;
    11.    // otherwise, we deal with boxing/unboxing the struct for the user ourself:
    12.    var tmp = (Test)target;
    13.    tmp.x = (int)value;
    14.    target = tmp;
    15. }
    16.  
    17. 3- public static int FieldGetter(Test target)
    18. {
    19.     return target.x;
    20. }
    21.  
    22. 4- public static void FieldSetter(ref Test target, int value)
    23. {
    24.     target.x = value;
    25. }
    26.  
    27. 5- public static object PropertyGetter(object target)
    28. {
    29.     return Test.str; // no boxing, str is a string - notice how we totally ignore the target. since str is static
    30. }
    31.  
    32. 6- public static void PropertySetter(ref object target, object value)
    33. {
    34.     Test.str = (string)value; // doesn't matter if Test is a class or struct, property is static we set it immediately
    35. }
    36.  
    37. 7- public static object PropertyGetter(Test target)
    38. {
    39.     return Test.str;
    40. }
    41.  
    42. 8- public static void PropertySetter(ref Test target, string value)
    43. {
    44.     Test.str = value;
    45. }
    46.  
    47. 9- public static object MethodCaller(object target, object[] args)
    48. {
    49.    Test tmp = (Test)target;
    50.    int arg0 = (int)args[0];
    51.    int arg1 = (int)args[1];
    52.    int arg2 = (int)args[2];
    53.    tmp.ByRef(ref arg0, arg2, out arg2);
    54.    args[0] = arg0;
    55.    args[2] = arg2;
    56.    return null;
    57. }
    58.  
    59. 10- public static object MethodCaller(Test target, object[] args)
    60. {
    61.    Test tmp = target;
    62.    int arg0 = (int)args[0];
    63.    int arg1 = (int)args[1];
    64.    int arg2 = (int)args[2];
    65.    tmp.ByRef(ref arg0, arg2, out arg2);
    66.    args[0] = arg0;
    67.    args[2] = arg2;
    68.    return null;
    69. }
    70.  
    Hope that helps! - Note in the examples, I show how you could generate an assembly that contains code for [s|g]etters for a field and/or property, a method call and a ctor invocation. You can load the assemblies in ILSpy and inspect the code yourself :)
     
  6. hadicoco

    hadicoco

    Joined:
    May 10, 2014
    Posts:
    13
    That helps thanks! but why not use Delegate.CreateDelegate instead of emitting IL?
     
  7. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    @hadicoco CreateDelegate works with MethodInfos only, which means you could use it to get delgates to call methods, and get/set properties. But it gets 'quite' tricky when working with structs. It also means can't get a delegate to set/get the value of a field like you do with properties. IL is pretty powerful, it gives you a whole lot of control, you can get 'exactly' what you have in mind and it just works!
     
  8. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Took long enough, but it finally made it to the asset store! Check it out!
     
  9. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Well that's interesting, running the same benchmarks above in Unity 5 yielded faster results for those 100+ ms weakly typed delegates. I updated the picture. The numbers now make sense. I wonder if they did any improvements on their GC or something.

    Another thing I noticed, is that the amount of garbage the weakly typed delegates generate depends on whether you have a 32-bit or 64-bit editor. My initial tests were in a 32-bit editor so I got 24 bytes, in a 64-bit it might slightly be more (not double the amount though). I guess it also depends on the size of the object being boxed. Just something to keep in mind.
     
    Last edited: Apr 19, 2015
  10. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    I was talking about Delegate.CreateDelegate in particular, not System.Linq.Expressions. I'm not an expert on them but I do remember giving them a try and having a hiccup somewhere along the road so I went with IL just cause it gives you more control.


    I would say that 99.99% is over exaggerating. Clearly it's inspired by it, that's true. You can see it in the public API (DelegateForSet/Get/Call etc) - It's the same idea - but the implementation is totally different. My asset actually came from the frustration I had with Fasterflect, as it requires you to wrap things when dealing with structs. I never got their delegate API to work with structs, I just couldn't. I gave up on it and decided to make my own solution. So I learned IL and went on with the job. FR uses the same API on classes|structs and it just works. Another point, Fasterflect doesn't offer strongly typed delegates, only weak ones in FR you could go strong or weak. Those two points make my version superior. The 'Fast' part comes from the fact that I have other 'Fast' products making use of IL emission in the making (Fast.Serializer and Fast.Save), it just so happens that Fast.Reflection sounded like Fasterflect. So I really don't see where the 99.99% comes from. Please look closely before implying that I'm copying other people's work and making a profit out of it, not very appreciated. If you still insist, then I would like you to show me the parts that matches "99.99%" (apart from the inspired public API method names)

    I'm not sure I fully understand your question, if you can 'normally' set a Vector3 components separately, what's denying you from doing the same via reflection?
     
    Last edited: Apr 29, 2015
  11. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Dude, why are you comparing Fasterflect with itself in that picture? Lol of course they're gonna be the same... I think you're confused somehow. My asset is a single file...

    You shouldn't modify a Quaternion's component's directly, it will give unexpected results. Here's an example on transform.position...

    Code (csharp):
    1.  
    2.   var pos = transform.position;
    3.   var setX = typeof(Vector3).GetField("x").DelegateForSet<Vector3, float>();
    4.   setX(ref pos, 10);
    5.   transform.position = pos;
    6.  
     
    Last edited: Apr 30, 2015
  12. Gekigengar

    Gekigengar

    Joined:
    Jan 20, 2013
    Posts:
    738
    Hello!, I need this system for a system I am making.
    (So far, It wouldn't work well without a "Lightning fast" reflection)

    Does that mean I can't sell my system on the asset store if am using this library?
     
  13. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    @Gekigengar, sure no problem. Just compile it in a Dll and use it in your project. Usually publishers will ask you to tell your users to buy their asset if you're using it, but I don't mind it here as long as it's not provided in plain text.
     
  14. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    @winxalex I can't believe you're serious... I told you I was using Fasterflect previously, which is what you're seeing in that older Vfw version. But after I wrote this asset I stopped using Fasterflect and used my library instead. Get the most recent version of Vfw and see for yourself. You're still comparing Fasterflect with Fasterflect...
     
    winxalex likes this.
  15. clunk47

    clunk47

    Joined:
    Apr 29, 2011
    Posts:
    22
    You're looking at the wrong S***...

     
    winxalex likes this.
  16. winxalex

    winxalex

    Joined:
    Jun 29, 2014
    Posts:
    166


    Code (CSharp):
    1. class Test{public int x;}
    2. var cls=new Test();
    3. var field = Test.GetType().GetField("x");
    4. var weakFieldSetter = field.DelegateForSet();
    if to pass "cls" to
    weakFieldSetter<object,object>
    weekFieldSetter(ref cls,3); <-- compiler would cannot convert `Test' expression to type `object'
    so you need to box before calling function
    Code (CSharp):
    1. object clsObj=cls;
    then you do unboxing
    Code (CSharp):
    1. ((Test)target).x = (int)value;
    then you say
    // otherwise, we deal with boxing/unboxing the struct for the user ourself ???

    Code (CSharp):
    1. struct Test{public int x;}
    2. var stct=new Test();
    3. var field = Test.GetType().GetField("x");
    4. var weakFieldSetter = field.DelegateForSet();
    again you need boxing
    Code (CSharp):
    1. object stctObj=stct;
    this do also copy.
    Then why you do copy of the copy inside weakFieldSetter again?
    Code (CSharp):
    1. var tmp = (Test)target;
    2.    tmp.x = (int)value;
    3.    target = tmp;
    Is there is some trail version or you share part of the code.
    What about creating types in runtime too like below;
    Code (CSharp):
    1. private static Delegate GetSetMemberInfoDelegateX(MemberInfo memberInfo){
    2.  
    3.                 Type typeDynamic = typeof(MemberInfoSetterDelegate<,>).MakeGenericType (new Type[] {
    4.                         memberInfo.DeclaringType,
    5.                         memberInfo.GetUnderlyingType ()
    6.                 });
    7.  
    8.  
    9.  
    10.                 Type instanceType = memberInfo.DeclaringType;
    11.                 Type valueType = memberInfo.GetUnderlyingType ();
    12.    
    13.                 var paramType = instanceType.MakeByRefType ();
    14.    
    15.    
    16.                 var setter = new DynamicMethod ("", typeof(void),
    17.                                         new[] { paramType, valueType },
    18.         memberInfo.DeclaringType.Module, true);
    19.    
    20.                 var generator = setter.GetILGenerator ();
    21.    
    22.    
    23.                 if (memberInfo.MemberType == MemberTypes.Field && ((FieldInfo)memberInfo).IsStatic) {
    24.                         generator.Emit (OpCodes.Ldarg_1);
    25.                         generator.Emit (OpCodes.Stsfld, (FieldInfo)memberInfo);
    26.                 } else {
    27.        
    28.                         generator.Emit (OpCodes.Ldarg_0);
    29.                         if (!instanceType.IsValueType)//this line is so we can use one SetterDelegate with ref in case of cls
    30.                                 generator.Emit (OpCodes.Ldind_Ref);
    31.                         generator.Emit (OpCodes.Ldarg_1);
    32.                         if (memberInfo.MemberType == MemberTypes.Field)
    33.                                 generator.Emit (OpCodes.Stfld, (FieldInfo)memberInfo);
    34.                         else
    35.                                 generator.Emit (OpCodes.Callvirt, ((PropertyInfo)memberInfo).GetSetMethod ());
    36.                 }
    37.    
    38.                 generator.Emit (OpCodes.Ret);
    39.  
    40.  
    41.  
    42.                 return setter.CreateDelegate (typeDynamic);
    43.  
    44.            
    45.         }

    If we were in .NET 4.0 we would have
    Code (CSharp):
    1. dynamic setter=GetSetMemberInfoDelegateX(field);
    2. setter(ref cls,3);
    in .NET 3.5 more cumbersom
    Code (CSharp):
    1. static void Executor<K,T>(Delegate del,ref K a,T b){
    2.  
    3.         (del as MemberInfoSetterDelegate<K,T>)(ref a,b);
    4.     }
    Code (CSharp):
    1. var setter=GetSetMemberInfoDelegateX(field);
    2. Executor(setter,ref cls,3);

    The test aren't quite accurate cos box/unbox time tend to grow with growth of struct/cls members.
     
    Last edited: May 3, 2015
  17. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    @winxalex If you have a strong reference why use the weakly typed API? if you have strong, you go strong. The weakly typed API is meant to use if all you had was pure base System.Object. If you use strong reference on weak API then yes of course you must unbox back to struct, but that's not how it's meant to be used.

    I went open source https://github.com/vexe/Fast.Reflection I'm done selling tools.
     
    Ghopper21 and winxalex like this.
  18. winxalex

    winxalex

    Joined:
    Jun 29, 2014
    Posts:
    166
    @vexe I gonna buy u barrel of bear cos u open source it. Thx alot.
    I could get types from MemberInfo thru ReflectedType and Property/Field Type but I couldn't build every case solutions without "dynamic" in my Monstrosity UnityVariable .
     
  19. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Very cool vexe, it's giving me up to 20x better performance in my Node Editor scaling system:) It hacks into the grouping system, which results one reflection call for every top group from where it's called, every GUI call. That adds up quickly, but this saved it;) Thank you so much!
    I hope you don't mind if I add this handy script to my framework (it's open source, too), I gave you credit in the readme;)
     
  20. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Sure thing buddy, always happy to have others use my stuff to make more awesome stuff :D

    But take special care when dealing with value-types/structs, even though I advertise that both value types and references types are dealt with in the same way that's true, but if you have some code that does reflection stuff on a value-type that's passed in as an argument to a function, you might want to make sure that the argument is passed by reference otherwise you'll be modifying a copy of the object.

    Another thing, I remember I faced a bug in this system when using it in VFW that I couldn't really replicate outside of that context, it was pretty weird. I had a delegate that points a function that returns bool, calling the delegate it will always return false even though if you step into the function and see that the value should be true. it's one of those bugs where it truly makes you doubt your sanity, you'd start to doubt that 1+1 might not be 2.

    But overall it's a good system with little issues. One improvement I had in mind was to make it so that you're able to generate and build strongly-typed delegates to functions so there's no boxing when you pass function arguments around.
     
  21. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Yeah I know that feeling ;) Fortunately I faced no such problem, I could implement it within five minutes, both on Node Editor and on UndoPro (you know from the other thread;) ).
    Use cases: function & property returning a rect; one void which modifies two lists boxed as objects. Works like a charm:)
     
  22. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Just a note, was trying to built for WebGL today, and did not work very well:( In backsight, this is pretty obvious, as it's explicitly stated that generating code at runtime is not supported. I am not sure about common Reflection though, I will have to try that...
     
  23. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    @Seneral I guess you could just generate pure C# code ahead of time instead. It's what I did in my native C reflection system. In C# it's easier cause you don't have to parse anything, you just annotate the types you're interested in with some attribute e.g. [Reflective] and then query and find all the types with that attribute, and generate the code for those at edit-time, not run-time.
     
  24. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    That sounds promising, but how would I do that? Particulary the storing in edit time, as it's a delegate, so I would need to store the actual code, which I do not know anything about:(
    In detail I just need this for 1) static method caller and 2) static property getter
     
  25. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    @Seneral You don't store the code in memory, it gets generated to a .cs file. You will still have delegates in a lookup table pointing to the functions in the generated .cs file. Nothing changed, except that instead of generating the code _at runtime_ and storing it _in memory_, we generate it _ahead of time_ to a _.cs file_ When you run the game all the code is generated there's no runtime IL emission.
     
  26. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Oh and, since you're generating to a file directly, you don't need to get involved with generating IL, it's a hassle. Instead you just spit out text, pure C# source code. Much easier to maintain and deal with. This is probably the approach I should have gone with in this system but I never considered it at that time for some reason. I'll see if I have time to add it in.
     
  27. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Yes I got the concept of how to do this, but how do I actually get the source code?
    I suppose I have to take a look at GenDelegateForMember and similar methods that actually generate the delegates and somehow retrive the code from 'DynamicMethod dynMethod'...
     
  28. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Sorry I'm not sure I understand. Get the source code of what?
     
  29. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    ^^ The delegate code ;)
     
  30. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    I'm still not 100% sure what you're asking about. The code that sets/gets the values? umm... you don't get it from anywhere, your metaprogram generates it. You just write the metaprogram (code that spits out code). You also generate an initialization function that populates your delegates lookup table. In your user code you have to call that function once. Let me know if that answers your question, if not please be more elaborate.
     
  31. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
  32. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Ok now I'm confused... :(
    As of my understanding, Fast.Reflection currently generates delegates for me that 'magically' speeds up the reflection access times. Don't know what's going on behind the scenes, unfortunately. Now you proposed me to store that C# source code (I assume that from the delegates you are generating). That seems to me pretty much like what you are describing...

    So the generated code (whatever it may be now) is in C or is this just and example from your C library you were mentioning? Because I'm pretty bad at understanding the concept and syntax of C. Should really learn it...
    In the generated code you linked, it seems to me meta_init stores those accessors, but those are simply referencing it, f.E:
    Code (csharp):
    1. inline void* get_Player_health(void* Target) { return &((Player*)Target)->health; }
    I won't be able to do that, the class I'm referencing is hidden...

    Thanks for your help anyway:)
     
  33. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Oh, wait. I think I see the source of your confusion. I think you thought there was a way to get the C# source code from the delegates I generate in FastReflection? If that's the case, I'm not aware of a method that lets you do that directly. You could however, once you have the delegates, spit them out to a DLL, then decompile it with ILSpy, copy-paste the code to a .cs file. But this is like someone's asking you to point to your left with your right hand, and instead of taking the short path and pointing to it across your face, you point to it all the way from behind the back of your head. it's unnecessarily complicated (but this technique is useful if you want to debug the IL code you're generating)

    The approach I'm saying here is _fundamentally_ the same. In both cases we generate code to set/get fields, allocate types etc. Except in the IL way you won't see the code, it's in memory. But when you generate it to text, you will see it.

    There are downsides though now that I think of it, (which I totally forgot about when I first mentioned this too you, I guess I'm writing too much C), since this is generated source code, that code will get compiled and dealt with the same way as ordinary code, which means you can't access private members of a class, which defeats the whole purpose of reflection. You might get by with public properties or just changing your stuff to be public and mark them with [NonSerializable] or something.

    The problem remain the same if you output IL to a DLL, I do remember researching that. There's a lot more freedom in those DynamicMethods and storing the IL in memory, lets you access everything.

    If all you need from this stuff is a static method call and a property getter, and the type is hidden/internal, why can't you just do Delegate.CreateDelegate and get a delegate to that method and getter?
     
  34. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Oh, ok that makes sense. But what about not outputting the source, but the compiled code instead, if possible? Would that bypass the limitations to acess private members?

    And isn't creating a delegate is what FastReflection is exactly doing? :confused:

    Btw, the members in question are the property "UnityEngine.GUIClip.topmostRect (returns Rect) (Static, Public)" and the Method "UnityEngine.GUIClip.GetTopRect (no Params) (returns Rect) (Static, Private)", where GUIClip is internal -> It cannot be inspected in the Assembly Browser, you have to trust me on the members;)
     
  35. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    You totally don't need FastReflection for that. You have to understand that it's not just giving you back a delegate that points _directly_ to the method you're interested in, instead, what it will do is generate a _function_ that will call your method, and gives you a delegate to that function. Why the indirection layer? Well, because this is supposed to be a framework that can deal with arbitrary methods of arbitrary parameters. The only way (at least I know of) to arbitrarily and generically pass parameters around is in an object[] (which is what that MethodCaller delegate takes). So you call that delegate it, give it the parameters in an object[], the function pointer behind the delegate is pointing to that function we generated, that function will take that object[], and extract the right number of parameters from it with the right type, and then it passes the values to the actual method.

    For example, if you have a method "void Fun(int x, float y)" inside a class "MyClass" - FastReflection will generate something that looks like (remember, this is just the C# equivalent of the IL that gets generated to memory):
    Code (csharp):
    1.  
    2. void __Fun(MyClass mc, object[] args)
    3. {
    4.      int x = (int)args[0]; // <-- Unboxing :(
    5.      float y = (float)args[1];
    6.      mc.Fun(x, y); // of course, in IL you don't call it like that, you do Fun(mc, x, y) :D (mc is the implicit 'this' pointer)
    7. }
    8.  
    FastReflection gives you back a delegate that points to __Fun, not to your class's Fun, if that makes sense. Of course I could put in more effort and implement completely strong typed delegates so you don't have to pass object[] and end up with boxing/unboxing, but meh.

    My suggestion to you was to use Delegate.CreateDelegate and get a delegate _directly_ to your method of interest, this doesn't generate any code. In our example, we would get a delegate directly to 'Fun'.

    So for your case, it would be something like (not tested):

    Code (csharp):
    1.  
    2. Type guiClipType = typeof(GameObject).Assembly.GetType("UnityEngine.GUIClip", maybe you need binding flags here?);
    3. MethodInfo topmostRectMethod = guiClipType.GetMethod("get_topmostRect", BindingFlags.NonPublic | BindingFlags.Static); // I think you can get the MethodInfo from the PropertyInfo value as well, guiClipType.GetProperty("topmostRect", BindingFlags.Public | BindingFlags.Static).Getter or something I can't remember :D
    4. Func<Rect> zeeFun = (Func<Rect>)Delegate.CreateDelegate(typeof(Func<Rect>), null, topmostRectMethod); // null for the target object, since it's static
    5.  
    Then 'zeeFun' is what you'd call to get the rect, and it's pretty fast. Note you might have to fiddle around with the code ^ to get it to work I remember Unity doesn't like it for some reason If I give it only the MethodInfo in CreateDelegate, if you get a null value back try passing the name of the function directly.

    Similar thing with GetTopRect, just make sure you pass the right BindingFlags. Hope that helps
     
    Last edited: Jan 2, 2016
  36. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Wow found out just that right now:D
    It works perfectly now, and it was actually pretty simple;)

    This is what I've come up with:
    Code (csharp):
    1.  
    2. private delegate Rect GetRectDelegate ();
    3. private static GetRectDelegate GetTopRectDelegate;
    4. private static GetRectDelegate topmostRectDelegate;
    5.  
    6. // Delegate accessors
    7. public static Rect getTopRect { get { return GetTopRectDelegate.Invoke (); } }
    8. public static Rect getTopRectScreenSpace { get { return topmostRectDelegate.Invoke (); } }
    9.  
    10. // Setup in a function
    11. Assembly UnityEngine = Assembly.GetAssembly (typeof (UnityEngine.GUI));
    12. Type GUIClipType = UnityEngine.GetType ("UnityEngine.GUIClip");
    13. PropertyInfo topmostRect = GUIClipType.GetProperty ("topmostRect", BindingFlags.Static | BindingFlags.Public);
    14. MethodInfo GetTopRect = GUIClipType.GetMethod ("GetTopRect", BindingFlags.Static | BindingFlags.NonPublic);
    15.  
    16. GetTopRectDelegate = (GetRectDelegate)Delegate.CreateDelegate (typeof(GetRectDelegate), GetTopRect);
    17. topmostRectDelegate = (GetRectDelegate)Delegate.CreateDelegate (typeof(GetRectDelegate), topmostRect.GetGetMethod ());
    18.  
    Thanks for clearing up the why behind it and helping me!
    I'll check if that finally works for WebGL:)

    Edit: Will replace custom delegate with Func<Rect>, totally forgot about that, thanks for the hint:)
    Edit2: Also noticed you CAN view GUIClip - stupid me - I was developing on it without knowing I could make it alot easier than with manual debugging :mad:
     
    Last edited: Jan 2, 2016
  37. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    For viewing stuff, I'm not sure what you're using. But I always use ILSpy, let's you see everything, good decompiler, very trustworthy.
     
  38. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    I'm using MonoDevelop's built-in assembly browser... But as I added, it was simply my mistakexD
    Seems I cannot use Reflection in WebGL, taking following code:
    Code (csharp):
    1. Assembly UnityEngine = Assembly.GetAssembly (typeof (UnityEngine.GUI));
    2. Type GUIClipType = UnityEngine.GetType ("UnityEngine.GUIClip", true);
    3. PropertyInfo topmostRect = GUIClipType.GetProperty ("topmostRect", BindingFlags.Static | BindingFlags.Public);
    4. MethodInfo GetTopRect = GUIClipType.GetMethod ("GetTopRect", BindingFlags.Static | BindingFlags.NonPublic);
    Both topmostRect and GetTopRect are null, not found:( they do work in the editor and standalone though, and the type is correctly fetched:confused:
    Do you have by chance any idea? I thought reflection was supported on WebGL, just Reflection.Emit not...
     
  39. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    No idea what's wrong, never tried WebGL. If you have a valid type handler, just iterate over all the members (with every possible binding flag value) and see if you get any fields. If you do but you don't see the one's you want, then maybe they're not included in WebGL builds? If you see them it means there's a problem with how you're specifying the binding flags.

    What are you doing with GUIClip anyways? Can you not implement whatever that topmostRect is doing yourself? or work-around it achieving the same thing with a different method/class?
     
  40. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    I'll try, needs some time to built. And no, I defenitely cannot do it otherwise; afaik it's the only way to hack and modify the Grouping system. I've built an universal scaling system for the GUI, and need to hack into the grouping system because GUI.matrix scaling messes up grouping/clipping by default...
     
  41. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Well I'm confused - In the Editor, everything get's logged apropriately, every (static) member I see in the Assembly Browser is logged. In WebGL though, it's only logging me a few, mostly missing the ones I need... Following are the logs I was creating:
    In the Editor:
    Code (csharp):
    1.  
    2. Members without Bindflags: Method-Unclip |-| Method-Unclip |-| Method-Clip |-| Method-Clip |-| Method-GetAbsoluteMousePosition |-| Method-get_enabled |-| Method-get_topmostRect |-| Method-get_visibleRect |-| Method-Equals |-| Method-GetHashCode |-| Method-GetType |-| Method-ToString |-| Constructor-.ctor |-| Property-enabled |-| Property-topmostRect |-| Property-visibleRect
    3. Both NonPublic and Public Instance Members: Method-Equals |-| Method-Finalize |-| Method-GetHashCode |-| Method-GetType |-| Method-MemberwiseClone |-| Method-ToString |-| Method-obj_address |-| Constructor-.ctor
    4. Nonpublic Static Members: Method-Push |-| Method-INTERNAL_CALL_Push |-| Method-Pop |-| Method-GetTopRect |-| Method-INTERNAL_CALL_GetTopRect |-| Method-Unclip_Vector2 |-| Method-INTERNAL_CALL_Unclip_Vector2 |-| Method-INTERNAL_get_topmostRect |-| Method-Unclip_Rect |-| Method-INTERNAL_CALL_Unclip_Rect |-| Method-Clip_Vector2 |-| Method-INTERNAL_CALL_Clip_Vector2 |-| Method-Internal_Clip_Rect |-| Method-INTERNAL_CALL_Internal_Clip_Rect |-| Method-Reapply |-| Method-GetMatrix |-| Method-INTERNAL_CALL_GetMatrix |-| Method-SetMatrix |-| Method-INTERNAL_CALL_SetMatrix |-| Method-INTERNAL_get_visibleRect |-| Method-Internal_GetAbsoluteMousePosition
    5. Public Static Members: Method-Unclip |-| Method-Unclip |-| Method-Clip |-| Method-Clip |-| Method-GetAbsoluteMousePosition |-| Method-get_enabled |-| Method-get_topmostRect |-| Method-get_visibleRect |-| Property-enabled |-| Property-topmostRect |-| Property-visibleRect
    6.  
    In WebGL
    Code (csharp):
    1.  
    2. Members without Bindflags: Method-Unclip |-| Method-Equals |-| Method-GetHashCode |-| Method-GetType |-| Method-ToString
    3. Both NonPublic and Public Instance Members: Method-Equals |-| Method-Finalize |-| Method-GetHashCode |-| Method-GetType |-| Method-MemberwiseClone |-| Method-ToString
    4. Nonpublic Static Members: Method-Push |-| Method-INTERNAL_CALL_Push |-| Method-Pop |-| Method-Unclip_Vector2 |-| Method-INTERNAL_CALL_Unclip_Vector2 |-| Method-GetMatrix |-| Method-INTERNAL_CALL_GetMatrix |-| Method-SetMatrix |-| Method-INTERNAL_CALL_SetMatrix
    5. Public Static Members: Method-Unclip
    6.  
    Sorry for the literal flood of information:/
    GUIClip has only static members (though it's not a static class) and inherits from nothing.

    In short: The missing types seem completely random at the first glance. F.E.
    Nonpublic Static Method 'INTERNAL_CALL_Unclip_Vector2' is shown but not it's other overload
    Nonpublic Static Method 'INTERNAL_CALL_Unclip_Rect'
     
  42. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    I can only guess these functions are not allowed in WebGL for some reason (like the File I/O stuff). Your best bet is to ask in a WebGL specific section of the forums, ask a Unity developer, or post a Q on UnityAnswers and try to get the attention of @Bunny83. Wish I could help more. It's why I started writing my own engine stuff, you get to have the stuff you _exactly_ want. Tired of Unity f***ing me each time I want to do something trivial and end up fighting its APIs for a long while... and then take a detour writing a tool to unf*** myself.
     
    winxalex likes this.
  43. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Well if they were not allowed the grouping system probably wouldn't work at all - if anything outside of the native code needs to access it. But thanks for the advice, I'll take a look around:)
    Thanks for all your advice so far, it has helped me alot. That being said, I understand your decision moving away from Unity, it can all be very frustrating. Just good I'm not AS experienced with the Unity API (just over a year now) to see the full madness/limitations of Unity;)
     
  44. vexe

    vexe

    Joined:
    May 18, 2013
    Posts:
    644
    Wish you the best of luck in your journey my friend!

    For the record, I did not completely 'move away' from Unity. Only for personal projects it's true. I still use it at work and have to provide support for my stuff.
     
  45. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    Thanks, wish you luck, too:)
     
  46. Trinary

    Trinary

    Joined:
    Jul 26, 2013
    Posts:
    395
    Hey vexe,

    Can you see any way that this method could be used to create extension methods on value types?
     
  47. Trinary

    Trinary

    Joined:
    Jul 26, 2013
    Posts:
    395
    @Seneral it says here that webGL doesn't support dynamic code generation. I suspect that's the reason for the error.
     
  48. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    System.Reflection IS supported, only System.Reflection.Emit not;) Also the type does get fetched correctly...
    But I have the suspect that code stripping may have cause this, actually it seems quite logic;)
    Will report back.

    Unfortunately don't know a solution to your question though...
    Can you access the valuetype without reflection?
    Maybe this is possible with code injection, but Idk.
     
  49. Seneral

    Seneral

    Joined:
    Jun 2, 2014
    Posts:
    1,206
    It works, after creating this link.xml file in the project to keep the members from being stripped:
    Code (csharp):
    1. <linker>
    2.     <assembly fullname="UnityEngine">
    3.         <type fullname="UnityEngine.GUIClip" preserve="all"/>
    4.     </assembly>
    5. </linker>
    Now, the WebGL type members match the Editor ones nearly exactly (one unimportant member is missing).
    After some tests, I even got it uploaded here;)
    The zooming is powered by my custom solution, it's hacking the grouping system, as mentioned before;)
     
    vexe and Trinary like this.
  50. winxalex

    winxalex

    Joined:
    Jun 29, 2014
    Posts:
    166
    They Introduce MonoDevelop 5.9.6, seems faster, some issues are solved. but they strip silently AssemblyBrowser....ILSpy works on windows (couldn't managed to maki it work on Mac with WINE)