The plugin hangs the second time, in the initialization function of Berkelium. I sent a message to the Sirikata Dev List about it.
I am going to do a quick test and see if this also happens outside of Unity (so initializing Berkelium, creating a window, destroying the window, destroying Berkelium and then initialize it again).
I uploaded a build of that test project here: http://github.com/downloads/jdierckx...Test-win32.zip
Remember that you can fly around while holding the right mouse button.
So i can get the plug in to work but i can't type. but i can type in your test app what do you do to get it to work?
Did you download the latest version from http://github.com/jdierckx/UnityBerkeliumPlugin (using the Download Source button)?
If you want it to work in the editor, don't forget to copy all files from berkelium/bin to your editor directory (defaults to C:\Program Files\Unity\Editor in windows).
Since it ends up in a hard crash it seems to be a low level conversion issue between C#->C++->Berkelium. I remember that the guy from awesomium had a similar issue in the past due to inconsistent runtime settings. Can anybody confirm that?
Oh, I see. To be honest, the code to convert to a wstring was provided by someone else and I didn't really had a close look at it.
I will try this now, and commit the fix if it works (it probably will).
Edit: I committed the fix.
Thank you very much, I will test the latest changes ASAP.
One more thing I recognized:
I execute a js command eg. every second to update my HUD (800x600 texture which shows some simple icons like a map). Using the profiler I could confirm that the RenderWebsite.update() command in these frames jumps from <1ms to 73ms for this frame which leads to a significant lag during animations (even on my high end machine Iīm working here). I checked out a MT solution like from here but since the texture is written in C++ it seems to be necessary to deal with MT on this end instead (I got invalid Direct3D accesses here from C++ side...). Any idea how to approach this problem?
The problem is that the SetPixels and Apply calls are very expensive, but it works in both DirectX and OpenGL.
If we want rendering to be much more efficient, we could force OpenGL rendering and update the texture in C++. The code would be simpler too (including scrolling, which I can't get to work easily using the current method), but we would loose DirectX compatibility.
For now, try to keep dirty rectangles as small as possible. Are you using flash? Because one of the problems there is that it always marks the entire flash rectangle as dirty.
Ah, thank you for the information. I see now what to look for.
Was the Awesomium implementation using SetPixels, or did it update the OpenGL texture directly?
I would love to do some profiling of the different things involved in this implementation (using callbacks, using SetPixels, ...) once I can find some free time (or money) to do so.
Yes, MyGUI has integrated Berkelium, but they use the OGRE rendering engine, which has support for fast access to texture objects, using C++. They can also directly copy the buffer because OGRE supports the texture format directly.
The problem with SetPixels is that we have to provide it with a color array in stead of a buffer, so we have to convert the buffer pixel by pixel. When using OpenGL directly, we can also use the buffer directly and that would bring a huge speed increase.
Maybe Unity3 has better support for updating textures with different texture formats?
This has been discussed many times, and the conclusion is, that until we can make directX calls, we are stuck with "setPixels()". Unless we use OpenGL but who wants to limit themselves to that?
The reason we can't make directX calls is because "there is no way to get the video device pointer outside of Unity" as written by Aras. Requests for this pointer have gone unanswered. I think with UnityAR coming soon, SOMETHING must be done because AR will not run smoothly (or, as nearly smooth as it could) without these directX calls.
As for going through each pixel, I strongly recommend including the lookup table I suggested (and added to the berkelium source, at one point) so that you don't have to calculate the float value each channel of each pixel, rather you read the value from a precalculated lookup table.
As well, you can try to store the old buffer and memcmp it to the new "dirty" rectangle and determine if it is in fact dirty.
Yep, I will add the lookup table you implemented, I totally forgot about that Will that increase performance a lot though? I thought that would only still matter in mobile devices and such...
As for the dirty rectangles, I did some quick checks and it seems that it only reports rectangles that are indeed dirty, apart from flash which always reports its rectangle as dirty.
Thanks for the hints.
Interesting, This could revolutionize puzzle based gaming, or even gaming full stop, for example making a first person shooter and having a force field over a door with a key code entry, a website made specifically to the game with video logs could be implemented, say the player gets a log name to search for and from the log entry they get the code, this would allow more fluid gaming in both single and multiplayer gaming. Well done!
Playing god, so you don't have to.
Turns out you have to explicitly focus the web window (I thought that was done automatically when sending click events).
I added that functionality, and now text carets are correctly rendered when editing text fields.
Haven't had the time to incorporate the lookup table. Maybe later today...
How is the work in progress on the scrolling feature?
I'm currently working on a project where we really want to implement a browser feature for different infoboxes on various objects. Hopefully we can keep our data in plain html/flash content and use a browser for displaying inside unity.
Great work you guys have done so far!
Last edited by rasmusbs; 10-11-2010 at 08:27 AM.
This is very cool. Is this still in development, and are there any plans to wrap it for OSX?
I wanted to get this working with Unity GUI, and fortunately, basic compatibility wasn't too hard.
All I had to do was add a read-only accessor to the Texture2D in RenderWebsite. Since I wanted to attach this to an object with a renderer (my HUD is tied into my First Person Controller), I created an enum so designers could specifically set the desired render type (object renderer, gui texture, or generic render to texture). Displaying it with Unity GUI was pretty easy once I had access to the texture in the other scripts. The only caveat is that by default the texture is flipped vertically (it was doing the same thing for GUI Texture). In my GUI Code I used the GUIUtility.ScaleAroundPivot to fix this. (I put all this in the BerkeliumTestGui Script)
Attached is your original demo, but now with three scenes, demonstrating each rendering usage. Just as a warning, I moved all the files around to better match my project structure (I have multiple plugins, and wanted to separate the scripts that are only needed for demo purposes).
Now, getting input capture to work with the texture when in Unity GUI (or GUI Texture).... any ideas? It gets the keyboard input just fine, but not mouse clicks.