Benchy Profiler - A simple benchmarking and profiling tool for Unity Free!

IMPORTANT!! Benchy Profiler 2.x end of life and Benchy Profiler 3.0 information here: http://forum.unity3d.com/threads/be...ool-for-unity-free.143895/page-4#post-1736819

-----

Looking for the FAQ? Please click here

The profiler for free and indie users is here! . That’s right – a simple benchmarking and profiling tool for Unity Free users.

Product status update: Version 3.x development has started
Current version on AS: The current version is available here

Features:

• Benchy Profiler can profile CSharp, UnityScript and Boo game and editor code without you writing anything!
• Profile your game without the editor on PC / Mac OSX and Android with the new OnDevice feature
• Save your current profiling session and reload it for comparison later
• Compare and analyse before and after results to establish performance improvements
• Powerful - useful - graphs showcasing expensive methods, call counts and more
• Interactive tutorial system allows you to get started with Benchy Profiler quicker
• Compatible with Unity 3.5.x+ under Windows and Mac OSX

Have you ever wanted to run Benchy Profiler without the overhead of the editor? Have you ever wanted to run Benchy Profiler directly on the device you’ve built your game for? Well now you can with Benchy Profiler OnDevice* available in the upcoming 2.0 release! More information available in the tutorials.

* Only available for PC / Mac OSX and Android game builds. iOS AOT prohibits the use of generics and dynamic code. Hands are tied there unfortunately.

Here are two screenshots running on a Nexus 7 - the first is a profiling session and the second a snapshot of two of the new widgets available for dragging directly into your scene.

Above screenshot shows Benchy Profiler recording a session whilst playing the game. The next step from here is to press the save/export button and view the results within the Benchy Profiler itself.


Above screenshot shows Benchy Profiler widgets in action. Their positions and GUI styles are configurable per widget and the two being showcase here do an fps benchmark (similar to 3D mark) and some overview stats respectively​

Editor profiling for asset store / in-house utility developers. If you're writing an editor tool and would like to hone in on performance bottlenecks then it's worth checking out. It works with C#, UnityScript / JavaScript and Boo editor code and a full tutorial (see next point) is available for it.

A new Interactive tutorial system is now available via the Benchy menu and there to assist new through to advanced users of Benchy Profiler. Topics covered at present are:

• Initial project setup
• Introduction to Benchy Profiler
• Saving and loading sessions
• Comparing recorded sessions
• Understanding and using Benchy Profiler OnDevice
• Benchy Profiler OnDevice for Android
• An introduction to Editor Profiling

Here's thanking you for making Benchy Profiler and its updates possible.

Univerb Studios

 

This looks nice. Is there any reason at all for Pro users to use benchy instead of Unity's Profiler?

 

Couldn't notice it from the video, but If it does deep-profiling any Unity Pro user could definitely take advantage of the speed, since Unity Pro's profiler is molasses-slow when you use deep-profiler.

 

Hi guys,

Benchy at this point is no where near the level of the Unity Profiler, one of the biggest differences is that the profiler that ships with Pro is fully integrated and can view into process immediately without having to define areas of code you'd like to benchmark.

This tool is very much geared towards those users that have no means of profiling to begin with and looking for a simple light tool that can quickly point them to problem areas.

Kindest Regards,

EDIT: Since posting this reply Benchy Profiler has far exceeded original ability. Please read this thread thoroughly to understand just how powerful this tool really is. If you're still unsure please read the reviews in the Asset Store for current application within industry.

 

It's live! http://u3d.as/content/univerb-gaming-studios/benchy/38K

 

Maddox, perhaps I misunderstood what you were looking for exactly. I've done some looking into the Profiler that ships with Pro (I'm a free user) and according to the documentation their deep profiling is each and every method a long with the graphical time line view stats. They also offer scripting options to lower overhead so you test only what you're wanting to.

In this sense Benchy can do a lot of this, that's how it's written (until I can figure out how to execute IL on the fly to inject benchy dynamically per method). I guess the question is what you use the deep profiling for exactly, I can give a better indication of if Benchy can be used instead for performance reasons.

I've got a lot planned for this utility, please stay tuned if you're not already a customer

 

What would you guys like to see in the next version of Benchy? Currently planning the next version.

Items definitely being considered:

- Platform and system information (useful for comparing results)
- Export results functionality
- Stop / Start recording ability (currently Benchy records immediately)
- General view improvements (text labels, output formatting etc.)

If this is the first time you're reading this thread, check the video here: YouTube // Vimeo

 

Maybe some graphing abilities like in this one:
http://u3d.as/content/indeego-games/graphical-object-profiler/2pC
?

Keep up the good work.

 

Thanks Maddox and agree with that feature request - something I've already begun work on a day or two ago

 

If I were to purchase the current version would I get access to the future versions or would I have to purchase it again?

 

Univerb didn't mention if he intends to eventually charge for it, but in the case it does, yes, according to the Asset Store rules you're eligible to any product updates for free.

 

Thanks Maddox! and thanks for the inquiry Tustvold - free updates for the life of the product in line with Asset store rules. To be clear though this means planned releases and features that we can accommodate and support.

 

Hello again!

Benchy 1.1! is currently under review and should be released on the store very soon! If you're an existing user, please get your upgrade free as soon as possible.

New features in this version include:

- Layout fixes for various window docking sizes
- Ability to toggle realtime view of benchmark results if required (use with caution, resource heavy)
- Non-GUI view available for those that prefer textual results (like me)
- Changed the method profiler to output in ms instead of seconds (also addressed an averaging issue on that tab)

Cheers for now! Still a lot more planned for future releases.

 

1.1 Has been released! Come get it here: http://u3d.as/content/univerb-gaming-studios/benchy/38K

 

Hello,

1.2 is currently under review - this release see's the introduction of micro profiling thanks to a suggestion from one of Benchy's users.

As always, existing users of Benchy get the upgrade free!

 

1.2 Has been released! Come get it here: http://u3d.as/content/univerb-gaming-studios/benchy/38K

 

I'm very interest.

Would this profiler allow me to see which parts of my enemy objects are causing some trouble? As, in functions I created myself or animation calls or even OnCollisionEnter();

I'm a little confused on how the code implementation works. Do I just add in Being(); and End(); wherever I want to profile?

Let me know here or PM as I'm interested in buying this sometime soon.

 

Hi there,

Great question and hope I can answer without asking too much around the code you want to get some profiling on. Refer to the pseudo code below, it should help answer the question:

Code (csharp):

1. void OnCollisionEnter()
2. {
3.      Benchy.Begin();
4.      Do some stuff;
5.      Call MethodA();
6.      Do some more stuff;
7.      // The next portion is the part you might be interested in checking for performance issues
8.      Benchy.Begin("Problem 1");
9.      Do some more stuff again;
10.      Benchy.End("Problem 1");
11.      Benchy.End();
12. }
13.  
14. void MethodA()
15. {
16.      Benchy.Begin();
17.      Do some code;
18.      Benchy.End();
19. }
20.  

The above will profile:

MethodA
OnCollisionEnter
+ Problem 1

You can mix up profiling sections as you wish to attain the desired test set. Hope this helps explain a bit more on how one would use the tool.

Kindest Regards,
Univerb Studios

 

Thanks this helps out a bunch! You will here from me sometime soon.

 

Hi!

I tried this tool for my project because I'm trying to find which function(s) cause bottlenecks. I'm experiencing FPS drops during specific sequences in my game.

I would assume that Benchy stores the timestamps of the Begin/End calls as well as counting the number of calls and their execution durations?

If so, it would be GREAT with a kind of time line feature that has the FPS over time displayed as a graph, together with ticks displaying where my functions are called.

Like this:

time: 012345678
FPS: ¨¨¨¨¨¨¨¨..¨¨¨¨¨
fcn1: __^_____^_
fcn2: ____^^____

Forgive my horrible illustration, but hopefully you get the point. If I see that fcn2 gets calls in during the poor fps period, then I can conclude that fcn2 is my culprit, and I can take steps. I could not conclude this from just measuring the number of calls and avg call duration. Am I right?

You should be able to do this using the data you've all ready collected in the begin/end calls (at least if you store the time stamps).

You could then also provide some additional sexy metrics like

* Per method: count the number (+ total duration of the call) of calls made during periods of low FPS, and let the user specify the low fps threshold.
(basically just show the same set of information as in the "method profiling" tab, but only for the subset of calls that occurred during low fps...)

* Average number of calls per second during a) the whole measurement period, b) the time where the class was alive (if possible)
* "Burst" number of calls / s, i.e. What is the highest calls/second? (could be inferred from the graph as well).

See what I mean? By combining the stuff you all ready have with a timeline, or some kind of "per second" concept, you could add a lot of value.

I'm new to programming in general and profilers in particular, so if I'm way off or if this is all ready available somehow, let me know =)

Hoping for an even more awesome 1.3!

 

Thanks for your reply, great feedback!

I see the problem clearly with regards to trying to isolate which part of code is causing you FPS drops. One could look at the method profiler to see if this code is hit often and work on improving performance within that section but that alone is driving optimizations for your requirement a bit blind.

Having the ability to view FPS over time per method will definitely help but at some points in time you're most likely going to hit more than just the one you're suspect (you'll perhaps hit others you're profiling). That aside combining this "new" view with the existing method profiler should aid in finding the root cause.

Timestamps of any kind were not required to date but definitely feasible with the way the component was designed. Just needs a bit of my time and some research, especially on the graphing side / idea you have.

I've had limited success with that but will see what can be done, alternatively the results will take the form of tabular information perhaps which will still add more value to the diagnosing of the problem code you're looking for.

Will also consider the above mentioned metrics enhancements / new features.

Expect a status update on the above next week Monday, 22nd October 2012.

 

Tabular would be perfectly fine! Maybe with user defined time buckets? Perhaps I start looking at 5 second buckets. Then i "drill down" to 0.5 second buckets or so, for the nitty gritty.

Lovely, I will expect it!

Thanks

 

Hi there,

Some extensive work has been done regarding the above and things are looking promising for both textual and graphical output. Will update this thread towards end of week but shooting to have submitted to Asset Store latest Friday for approval.

Kindest Regards,

 

Hi

Just purchased this, followed the instructions in the readme but I get a compiler error:

The best overload for the method 'Benchy.Begin(String)' is not compatible with the argument list '()'.

I've got Benchy loaded into the scene, and have Benchy.Begin(); and Benchy.End; added to a section of my code. The same error is showing for both calls.

U'm using Unity 3.5.6f4

Any ideas what I'm doing wrong?

Thanks,

Billy

-- UPDATE --

Sorry scratch that, I added a string "test" to both the begin and end calls and it works, didn't seem obvious to be from the docs (or maybe its just late in the day!!!)

 

Hi there!

I'll look into this, it's implied in the example scene but don't think referenced in the documentation. Thanks for letting me know you came right with this, also helps others.

EDIT: Just to clarify though it is totally possible to pass nothing to both begin and end (if you're profiling an entire method and not just sections for example). You could then use:

Code (csharp):

1.  
2.    Benchy.Begin():
3.    // do some work
4.    Benchy.End();
5.  

The named sections are useful for micro profiling, for example:

Code (csharp):

1.  
2.    Benchy.Begin();
3.    // do some work
4.    Benchy.Begin("Profiling doing some more work");
5.    // do some more work
6.    Benchy.End("Profiling doing some more work");
7.    // do some other work
8.    Benchy.End();
9.  

Regards.

 

Benchy 1.3 is almost finished guys and girls! Have one or two features I'd like to get in. Will try and get submitted to the asset store as soon as possible but the wait will be worth it.

 

This looks like a nice start to a helpful tool; very nice work. However, I'm using 4.0b and having the Benchy window open and idle oftentimes causes a full crash. I look forward to more updates to this! Cheers~

 

Hi Bfowle, glad you're enjoying the direction I'm going with Benchy and do hope that it fills the gap for many indie users. I'd like to investigate this issue you're having, can you give me steps to reproduce?

Will help a lot if I can address this before next asset store release, does sound like a memory leak of sorts. I've had no issues personally with 3.5.x but maybe something has cropped up with the newer versions.

 

Mainly have seen the crash when having the Benchy window docked inside the editor, and after running it at least once. Unity full-on crashes without any "send Bug report", etc. which tells me it's likely not something in Unity (also have rarely seen any Unity crashes since using this plugin.) I will see if I can get a more detailed bug report for you, whether it's when certain settings are checked/unchecked. Cheers~!

 

Thanks that was enough, it seems we've found a nasty Repaint() bug, profiling the profiler revealed a memory leak. Left as is, I can easily see unity crashing like this. Wonder if introduced when they patched the 3.5.x range.

Change will go in with the 1.3 submission. You guys are going to be very happy with the next version

 

Hi Guys,

Finishing up the quick tutorial video, updating the documentation and then a final round of tests, should be submitted to Asset Store tonight.

Cheers

 

Nice!

 

Submitted

Here's what you can look forward to in Benchy 1.3! http://www.youtube.com/watch?v=gLwdTie4a_0

 

Benchy 1.3 is here! Enjoy it guys / gals!

http://u3d.as/content/univerb-gaming-studios/benchy/38K

 

Great job, the trace adds a lot of value!

You should be able to scroll in the tracer though. And the method lines below the bars are kind of cryptic, they only show blue or red according to some hidden rule that the user has no control of (right?). Would be nice with a non graphical view with, per time bucket: time spent (ms) per method, and number of method calls per method. Maybe show that in the "view non-graphical" option? And allow the user to toggle between [number of calls] and [time spent (ms)]...

Keep it up!

EDIT: Also, sometimes my methods don't show up in the list even though i have the proper begin + end statements. Haven't been able to find a pattern as to when they show up and when they don't. Maybe i'm doing something wrong?

 

Great to hear back on the trace functionality. It will definitely be improved over time

In terms of your methods not showing up, if you mean actually not showing up at all for micro-profiling sections, you're free to send me a test project to univerbgamingstudios at gmail dot com

If you mean that some of the tracks aren't lighting up under the FPS trace but are shown to the left (ie: the lines aren't registering colour) this is because they haven't registered near enough activity / slow downs to warrant display.

Thanks again for this feedback!

 

About the scrolling... the bar graph should always show the far right of the bars... so as new bars are shown (as time passes), the last ones should always be shown. Kind of like cpu graphs, or network usage. You get the idea.

I mean methods not showing up at all. I'll see if i have the time to send you examples... This applies to both the trace view and the "summary" -one (the one without the bar graphs). I don't think this was an issue for me in 1.2.

As for "not having registered enough activity to warrant display"... Maybe that could be up to the user?

Another curious question... kind of a long shot... My game is for iPad, but I cant "benchy" on the iPad, right? In your experience, would you say that if something is an issue on the dev computer, it's also an issue on the iPad and vice versa? Any pitfalls to worry about there? I.e., are Benchy's results "guaranteed" to be relevant for mobile platforms. Kind of expecting a 'no' on that one, but it would be nice to hear your thoughts on it.

 

Correct, if you have a performance bottleneck before compiling for mobile, you should definitely experience the same sort of issue there. (Talking logic bugs, poor performing code).

Rendering performance however could be different between the two, just because PC / Mac is performing well does not mean the mobile build will (Texture sizes, particle effects etc.).

Are Benchy's results "Guaranteed" - As you pointed out - they are relative results for the platform run on. IE: snapshot in time, relative to the other methods profiled. As you point out though, these results are generated for a PC / Mac benchmarked session - which would not be subject to the same memory / texture etc limitations of a mobile device.

 

Got this tool a couple of days ago. Thanks a TON! It really helped me out. I ended up going from 22 fps to 120 fps. It was the least expected places that were causing the most damage! Please keep up the good work!

 

Hi mu-kow!

Thank you for your positive feedback and experience with Benchy! That's a remarkable improvement over the original code.

Please do leave us a honest review on the asset store

 

Done!

 

Thank you! Sir

 

1.4 is under review and should be released next day or two

What is new in 1.4?
==============
1.) The ability to unload Benchy from the scene added.
2.) Moved load and unload to Benchy->Tasks submenu.
3.) Shortcuts ctrl (cmd on mac) + alt works with (b) to display Benchy, (l) to load Benchy and (u) to unload Benchy.
4.) Included terms of use / disclaimer in readme
5.) Added shortcut to readme via Benchy's menu
6.) Added first time usage setup instructions

Enjoy and thank you for all your support

 

1.5 is superseding 1.4! Thanks to one of the community members for reporting an issue when building a final release and having the plug in installed. This is resolved and working as expected now so please get the update!

In even bigger news realtime / runtime benchmarking is introduced for the first time in this release. Although basic - you can view average, high / low fps, peak and current memory allocation, most executed method call and the most expensive one.

What makes this special is for the first time you can take your alpha build and test on the device you're targeting getting even more value out of this product. Enjoy!

What is new in 1.5?
==============

1.) Bug fix when building project for final export. Although it's recommended you remove the benchy folders when compiling your
final build some clients have requested we allow the product to build along side the main project.
2.) We have seperated Core and Editor code in this release.
3.) We have carried out extensive testing with 4.x Unity versions.
4.) We have fixed the FPS display issue as a result of the realtime warning flag.
5.) Last but not least Benchy introduces a first version of runtime result viewing in game. This is great for testing builds
on each platform

 

1.5 has been released! Enjoy folks

 

Great tool! Now I could use some help understanding the results.

Using the "red dot with clock" button, and "ignore Unity built-in functions" turned off, I get values of ~11000 for Unity.GameObject and Unity.Transform. Nothing else is higher than 1000. I'm also getting 18 fps in the editor, 40 fps in the built game when there's nothing going on besides a ship flying around the scene.

So it seems like I should be optimizing something, but I don't know what. What should be my next step?

Thanks!

 

Great to hear you're enjoying the tool. I think the very best way I can assist is if you mail the support address with a sample project for me to look at or mail more information on how you set benchy up (micro profiled sections etc.)

I'm sure we can isolate your slow down / problem code in terms of where you should be concentrating efforts.

univerbgamingstudios at gmail dot com

 

Just a quick post to let you guys know I am planning for some more development time on Benchy which will include some new features, I'm sure some GUI tweaks but most importantly - upgraded / updated documentation new to the concept of benchmarking / determining a place to start when profiling your projects.

This is all for Benchy 1.6 and should have this uploaded to Asset Store no later than 31st May 2013. Cheers for now and thank you for the support!

 

If you enjoyed previous versions, 1.6 is about to hit the asset store and this is what you can look forward to getting free of charge when you grab the update:

- Benchy can profile CSharp and UnityScript code without you writing any code! Thats right - automatic profiling without typing a thing
- Save your current benchmarking session and re-load it for viewing later
- Compare sets of benchmarks against one another to measure improvement
- Some Powerful new - useful - graphs showcasing expensive methods, call counts and more
- Data is exported in a format that you can use in downstream applications

Enjoy the update and expect documentation to follow next day or two!

NOTE! Grab Benchy now to save, as from 1.6 the price is raised by $5 on account of this utility being enhanced as far as it has

http://u3d.as/content/univerb-gaming-studios/benchy/38K

 

I just bought it, can't wait to try i didn't sleep nothing.