Utilities Script Inspector 3

Hey Unity Folks,

I'm pretty sure you are all familiar with Script Inspector 2, and many of you are already using it for viewing and editing your scripts, shaders, and text files right inside Unity Editor, so I'll skip the introduction and just present what is its current WIP status here.

For the last couple of months I've been working on automatic code completion feature, also known as "Intellisense" as Microsoft likes to call that. At first sight the feature may look not too complex, but as it turned out it is complex as hell if you try to do it all from scratch without any help of some of the few ready made libraries. One of those is NRefactory, an open-source library used in MonoDevelop/Xamarin for example, and another one is Roslyn by Microsoft which is still not finished and available as a preview for now. Both libraries can "understand" only C# code which was the main reason I've decided to go with my own solution since Script Inspector 2 besides C# scripts works also with UnityScript/JavaScript, Boo, and ShaderLab. The second reason was that I wanted to fully understand the mechanics behind the algorithms used in the library so I can easily fix bugs or extend the functionality when needed.

So, I did lots of research about the techniques and algorithms needed for implementing such feature before I started working on it, and at that time it looked like I might be able to invent quickly a minimalistic solution that will give good enough results... However, after trying a couple of methods and as I was getting deeper into the topic I realised I was wrong and that I'd still have to implement all the processing phases, almost same as it is done in NRefactory and Roslyn. That discovery was a little disappointing but it didn't discourage me to finish my job.

I begun again implementing the feature in the proper way, as it should be. First it needs a parser (a proper one, not just some hack with regular expressions) that will be able to recognize the syntax of the source file, generate a parse tree, and then incrementally update the parse tree while the user is editing previously parsed source code. Next step is the semantic analysis which traverses the parse tree and extracts symbol definitions, builds symbol tables, tries to resolve symbol references, etc... Once symbol tables are built and symbol references resolved, code-completion is simple: it takes the symbol to the left of the cursor, finds its definition in the symbol table, reads its type from the definition, and then list all the members from that type which are accessible in the current scope...

Sounds simple, eh? Well, there's still lots more than that... The parser needs to gracefully pass over syntactically incorrect input (there will be a lot of that as you type), and has to be able to deal with contextual keywords, for example. The semantic analysis has to deal with complex language features such as type inheritance, overloaded methods, type inference and generics, which is all needed for correct expression type evaluation... All that turned out to require tremendous amount of not only work, but also deep knowledge of all those topics! :-?

It took me more than a month to find all the related academic works and to study them, and then even more to implement some of them and to adapt the algorithms to work with the rest of Script Inspector 2 code. And still, it isn't finished... So far I've built a blasting fast parser with support for incremental parsing and pluggable LL(*) grammar definitions (so that I can reuse the same code for different scripting languages), C# 3.5 grammar with support for contextual keywords, syntax errors detection and recovery, syntax error highlighting in the editor, a little bit of the semantic analysis, and a little bit of the semantic highlighting shown in this screenshot:



The screenshot also shows proper handling of some of the contextual keywords in C#... Well, this may look like not too much, but in fact there's much more than a screenshot can show! I'll have to polish the error highlighting a little bit more and I'll post a video showing that in action! Stay tuned for updates

 

Sounds awesome! Keep up the good work, very much looking forward to it!

 

Thanks for the in depth summary of what's involved in intellisense. Looking forward to the videos!

 

Spectacular! This is clearly showing a difference between a quick solution and a well engineered and more comprehensive approach to the matter of in-editor scripting.

 

Thank you guys for taking time to read that lengthy post! Good to know there are at least three people interested in this, so I'll keep posting here whenever there's something new to show. And I'll try to keep the posts short and more visual

 

I am very interested. I am considering purchasing the version on the asset store, would have just prefered to see a demo version before purchasing.

 

Ahmmm... Livvy my friend, this is the same Script Inspector 2, the one you said you're loving it and it saved you a patch of hair on your head.

 

Thanks for the update. I look forward to the release and will continue to follow this thread. I

 

There's more than 3 people following this. And yes, I purchased already, just haven't used it yet. Too busy writing a book at the moment.

Terry

 

I am an "IDIOT", I was actually supposed to be replying to another thread. I actually had 2 instances of unity open. HAHA. This I already own and love!!!

 

Subscribed...

 

Oh, even more subscribers! That's so exciting now and great for motivation! Thank you all!!!

 

Subscribed, good work

 

Syntax error highlighting is working now!



Note that this is not highlighting any semantic errors, so undefined symbols are still not recognized as errors, for example. Only the syntax is checked for correctness against grammar rules of the language. Which is cool, but maybe not super-cool . Anyway, I had to do this so I can visualise output of the parser and correctness of the error recovery algorithm. Notice how error recovery kicks in after syntax errors restoring the parser state on a valid configuration, which then allows recognition of the rest of the code.

Next step is implementing the semantic analysis part, the last missing piece for the code auto-completion.

Stay tuned...

 

Ahhh this looks too cool! I'm looking forward to the final product! It will make everything flow so much nicer Keep up the great work! I really appreciate all the work and research you're putting in to this, so it's not just a half-ass product.

 

w007!

 

Hello Flipbookee, in next release we will have "auto indentation" feature too?

Thank you.

 

Yep, at least auto-indent and also auto-closing parentheses, quotes, brackets, and such. Depending on how will be beta testing going, it may also come with re-indent on paste, move and duplicate. All those require very little work, so they will be implemented at the end as part of the testing process. That's my favorite part when I'll be using Si2 to do the finishing touches ... Auto-formating is relatively easy too but it will come in some of the future versions, I guess.

 

i can't wait for this new update! I'm working with SI2 all the time and i'm loving it!! Keep up the great work!!

 

Will a future Update write all the code for me? ;p

JK

Looks great keep it up!!!!!!!!!!!!!!!!!

 

Thanks guys!

Error recovery got improved over the week for some special cases so it works even better now! This was one of the most difficult parts so far as resources describing the algorithm are rare and incomplete, so I had to come up with my own solution that's fast and gives nice results at the same time.

I hope to get the next update ready this weekend... stay tuned

 

So - you're saying I get to go away on holiday, and then come back to a rewarding present?

Yay!

 

WIP update with another screen capture, that is. Hope to get semantic analysis done for symbol references, so we can have reference highlighting, go to definition, type tooltips, and such... The last still missing feature before code-completion.

 

Anxiously awaiting update!

 

Hey jrDev, I've got the semantic analysis done for the most common symbol types, and reference highlighting like in Visual Assist X with two different colors for read and write. I was too excited about that and I jumped straight to auto-completion because I wanted to verify I'm on the right way to get that done too, and so far it looks cool! That proved the last thing I wasn't 100% sure about, so now it's only matter of time to get all this completed for the beta.

I'll capture the reference highlighting tonight, and all the other coolness as it comes...

 

You all love Visual Assist X for Visual Studio, don't you? If not that must be because you haven't tried it yet. I love it too, hehe

Reference highlighting for symbols a-la VA X:

 

And... there's a bug in that, as you can see Still WIP... Will be fixed

 

No one noticed that, right? Those two local scopes got blended into one, so locally declared variables using same names were getting highlighted as they'd be the same variables, which was wrong. This happened because my semantic analysis was normally distinguishing scopes by their names, and local scopes are unnamed so those were seen as one. I've fixed that and here's how it was supposed to look like:



Now I'm glad I've spent about one day more to implement the reference highlighting, otherwise I'd probably not notice this bug! :roll:

Go to definition should work properly now, but the most important thing for code completion is that I can extract the symbol type from its definition, and then list the members of that type as completion terms... Which is my next goal now... Stay tuned

 

Awesome man!

 

I purchased Script Inspector 2 last night from the Asset Store, but I get this strange issue when viewing scripts that have lots of public variables, and I am unable to adjust the size of the code window? Help!

 

Hi Greg!

It would be best if you open such scripts in Script Inspector's dedicated windows/views/tabs. Viewing such scripts in the Inspector view like that will be difficult, as you can see, because the inspector of the script importer takes all the space so there's very little left for the custom inspector of the imported object... To open the script in a new window you'd have to click the first toolbar button or press Ctrl+T while Si2 has the "keyboard focus", so you'll be able to resize that window, maximize it, dock it somewhere, or put it on the secondary monitor.

It might be just easier if you enable opening scripts on double-click from Project view in Si2 instead of opening them in your external editor. The option is turned off by default, so you'd have to enable that from the gear menu (last toolbar button) of any Si2 window (but not from the Inspector view): Gear -> Open on double-click -> Scripts. You can still open scripts in the external editor if you want, using the standard Open Asset Unity options, such as the "Open" option from the script asset's context menu, or Enter on Windows or Cmd-Down Arrow on Mac from the Project view, or from Script Inspector position the cursor on a desired line and press Ctrl+Enter (Cmd-Return on Mac).

There are also other ways to open scripts in Si2 dedicated windows, for example you can drag-and-drop scripts from the Project or game objects from the Hierarchy to the Si2 windows. Or you can open them from the component's context menu. Or from the SI Console window by double-clicking a log entry or from entry's context menu if you want to explore the call stack...

Let me know if this helps.

 

Hi, all sorted, enabled double-click to open scripts.

Looking forwards to editing scripts without leaving Unity Editor, tried other in-editor scripting assets before but were too buggy, hoping yours is solid, sounds it from the forums!

 

First impressions, just done some coding in Unity, and love it!

 

Thanks for your lovely comments, Greg! Make sure you subscribe to this thread if you'd like to follow the progress of the next huge update for Si2. First screen capture of code auto-completion is coming next! You shouldn't miss that one - it looks gorgeous

 

Does it have a good and useful autocomplete feature / will you add one

 

No, not yet, and yes, I'm adding it already. That's what this thread is about - to show my progress with the autocomplete feature mainly, besides the other smaller not yet released features.

 

Hey guys... I wanted to show you something, just quickly...



Semantic analysis works finally!

This is its third incarnation (the other two were just throw away code since they couldn't handle all the complexities of the C# language), and all of it is original code written from scratch. Some of the ideas used in its implementation are also original, and some are variations of other ideas presented in a couple of brilliant research papers.

Maybe I'm showing this too early, and maybe it doesn't look like too much... I could get the code completion working in a couple of days more, I guess, but yeah, I'll post again then

 

Looking nice!

We should all send a package of milk and cookies to keep you going.

 

This looks awesome! I'm waiting for "intellisense" to be added before I purchase this asset. When it's done I'm sure you'll have a number of new sales! Can't wait!

 

LOL, haha!

Well, just cheering from the sidelines also works for me

Thanks Callski

 

Waiting for the intellisense to make a purchase too! GO GO GO!

 

Keep up the good work, man. Unity should hire you !

 

Hmm, that's a very funny coincidence - just recently I've sent them my CV

 

One killer feature I wished for a long time is that going backward in stack traces. When double clicking on an error the editor got highlighted at the last function call, It would be absolutely great if we can go back and forth between the trace point. For example

We can go from StateView.Start <--> StateView.set_State <--> StateView.Show <--> .... all in correct line position.

I strongly belive that it's not too hard to add this feature, what do you think ?

 

Hmmm... What would you say if I tell you that feature is already available? It's been there for the last 10 months actually, since version 2.0! and yes, it's one of the killer features of Script Inspector 2

It makes finding the source of the issues so much easier, once you try that you'll never want to go back! Let me know if you dare to try this, I can send you a trial version by email...

 

Hey, thanks for your kindness, man. I've checked the screenshots again, It's wonderful. You should bloat more about it !

As the script is edited right inside the Editor, saving a script will trigger compile that can takes time. Do you have a solution to prevent unity from compiling until we actually saved all edited files ? How about adding a dummy script with invalid syntax to break the build and we can freely edit / save multiple files at the same time ?

It would be great if you send me the trial version, though, I will absolutely happy to be your beta tester

p/s : The package is really at good price, thanks for your generous man !

 

@thienhaflash: Yes, I have thought about that too all modified scripts are automatically saved before entering game mode in the Editor, so you don't have to save each file one by one, you just hit Play and all changes will be saved and compiled first and if there are no compile errors the game starts. Saves you at least one step and also makes sure you run the game with your changes. I hate it when I run the game and I realize I have forgotten to save a file, and then I have to run it again, so that will never happen with Si2

Thanks for your compliments! I'll send you a copy now...

 

The hit Play to save is a killer feature, man. Absolutely cool !

Some feedbacks (things I belive it's quick to apply) :
1. Automatically fix inconsistent EOL in windows / Mac project. It's really annoying now !
2. Ctrl+G : goto line
3. Ctrl+D : duplicate line ( I know there is ctrl + drag)
4. Selected text + F3 : find the selected text (instead of continue last find)
5. Allow copy text in SI Console
6. Allow bigger text (I'm on a really high resolution Laptop), maybe ctrl + mouseWheel ?
7. Close other script tabs
8. Open all scripts in a single tab (there only 1 script tab at all time)

Thanks man for let me be your beta tester. There should be a lot more to come !

 

I was struggling with a nasty bug for more than a week and finally I've found it deep in my parsing algorithm. It was only affecting the incremental parsing and messing up the parse tree piece by piece while typing, so I wasn't able to show the auto complete yet. I finally fixed that so hopefully the first presentation will happen in a day or two

@thienhaflash: Thanks a lot for your feedback and suggestions! See my comments below:

1. Si2 automatically fixes EOL's on save and unifies them! Only external tools can still mess that up. Once you get such warning you can double-click the warning in the SI Console to open the script up in Si2, then just save it with Ctrl+Alt+S without any modifications and that's it!

2. 3. coming soon

4. That's already there, Shift+Ctrl+Up/Down arrows finds the previous or next occurrence of the selected text or word at cursor if no selection

5. That should be possible, just press Ctrl+C. Same as in the original Unity Console.

6. That's already there Ctrl+Wheel, or Ctrl+Equals/Minus, or zoom in and zoom out gestures on touchpad. Note that this is only possible with dynamic fonts, so make sure you first select Source Code Pro font if you are on Unity 3.5 or change the font settings in Unity 4+ as it's explained in the readme file.

7. 8. Coming soon.

Thanks for all your suggestions! Let me know if there's anything else

 

First screen capture of the automatic code completion W.I.P. feature of the Script Inspector 2



Just note that this is still kind of raw and WIP. So far this feature only counts 8,000 lines of code, and it took about 8 months (of my own free time only) to get here!

I hope you like this gif even though it shows a couple of bugs... will be fixed...

Comments, suggestions, criticism, etc. are more than welcome, as always.