Unity Community


Page 1 of 8 123 ... LastLast
Results 1 to 20 of 150

  1. Posts
    52

    Berkelium plugin: rendering interactive websites

    Hi all,

    Berkelium is a wrapper around Google Chromium, and provides a way to render and interact with browser windows. I am creating a Unity plugin around this, so we can have also have interactive websites inside a Unity environment, in Windows (Mac users can use the htmlTexture plugin. I may sync my script API to match with that plugin once it is finished).

    Note that I had to use native code for this plugin, so this is Unity Pro only for now, and it can't be used inside a browser.


    You can find my first attempt at this plugin at http://research.edm.uhasselt.be/~nve...liumPlugin.zip

    The plugin still only has basic functionality, but I am actively working on it as we speak. This release is just proof that I am actually getting some results Wink

    The zip-file contains Berkelium, the source code and vs2008 solution of the plugin, a Unity test project and a test release so people without Unity Pro can see it working.


    Next things on my list is implementing support for scrolling, and caching the dirty rectangles to support partial updates using SetPixels (see reissgrant's comment above). After that, I will have a look at binding keyboard input.

    All contributions and comments are more than welcome of course!

    And a screenshot:



    Greetz,
    JeDi


  2. Posts
    310
    sweet,keep it up!


  3. Posts
    801
    So basically you can not only load the websites, but interact with them as well?
    e.g. I load website on a plane, run the app and it will allow me to navigate through the website which is on a plane, click links etc?

    if its that then sweet


  4. Posts
    52
    Yep, that's exactly what it does. You can render and interact with basically everything that Google Chrome can render. I have to investigate this a bit (I have no experience with neither Unity nor Berkelium), but I guess this can also be used as a GUI texture (using transparent websites). And it is just a texture, so you can use it on other objects than planes too (in-game terminals for example).

    Mind that the current implementation is not efficient yet, and scolling doesn't work properly. But those features are almost finished, and then it will be pretty efficient.


  5. Posts
    52
    I updated the plugin (same download link). These are the changes:
    - Fly around with the right mouse button pressed (very simple fly script)
    - Interaction can be enabled/disabled per object


  6. Posts
    983
    Awesome job. Think using a shader to do the BGRA to RGBA swap instead of doing it in the loop would save many cycles? Probably not many as you have to loop through the whole thing to convert it to a float anyways, but might help a little bit.


  7. Location
    Orlando, FL
    Posts
    724
    Use a float lookup table instead of calculating the conversion to float, it will save some time:

    Code:  
    1. static const float lookup[]=
    2. {
    3.          0.0f,  0.00392157f,  0.00784314f,   0.0117647f,   0.0156863f,   0.0196078f,   0.0235294f,    0.027451f,
    4.    0.0313726f,   0.0352941f,   0.0392157f,   0.0431373f,   0.0470588f,   0.0509804f,    0.054902f,   0.0588235f,
    5.    0.0627451f,   0.0666667f,   0.0705882f,   0.0745098f,   0.0784314f,   0.0823529f,   0.0862745f,   0.0901961f,
    6.    0.0941177f,   0.0980392f,    0.101961f,    0.105882f,    0.109804f,    0.113725f,    0.117647f,    0.121569f,
    7.      0.12549f,    0.129412f,    0.133333f,    0.137255f,    0.141176f,    0.145098f,     0.14902f,    0.152941f,
    8.     0.156863f,    0.160784f,    0.164706f,    0.168627f,    0.172549f,    0.176471f,    0.180392f,    0.184314f,
    9.     0.188235f,    0.192157f,    0.196078f,         0.2f,    0.203922f,    0.207843f,    0.211765f,    0.215686f,
    10.     0.219608f,    0.223529f,    0.227451f,    0.231373f,    0.235294f,    0.239216f,    0.243137f,    0.247059f,
    11.      0.25098f,    0.254902f,    0.258824f,    0.262745f,    0.266667f,    0.270588f,     0.27451f,    0.278431f,
    12.     0.282353f,    0.286275f,    0.290196f,    0.294118f,    0.298039f,    0.301961f,    0.305882f,    0.309804f,
    13.     0.313726f,    0.317647f,    0.321569f,     0.32549f,    0.329412f,    0.333333f,    0.337255f,    0.341176f,
    14.     0.345098f,     0.34902f,    0.352941f,    0.356863f,    0.360784f,    0.364706f,    0.368627f,    0.372549f,
    15.     0.376471f,    0.380392f,    0.384314f,    0.388235f,    0.392157f,    0.396078f,         0.4f,    0.403922f,
    16.     0.407843f,    0.411765f,    0.415686f,    0.419608f,    0.423529f,    0.427451f,    0.431373f,    0.435294f,
    17.     0.439216f,    0.443137f,    0.447059f,     0.45098f,    0.454902f,    0.458824f,    0.462745f,    0.466667f,
    18.     0.470588f,     0.47451f,    0.478431f,    0.482353f,    0.486275f,    0.490196f,    0.494118f,    0.498039f,
    19.     0.501961f,    0.505882f,    0.509804f,    0.513726f,    0.517647f,    0.521569f,     0.52549f,    0.529412f,
    20.     0.533333f,    0.537255f,    0.541176f,    0.545098f,     0.54902f,    0.552941f,    0.556863f,    0.560784f,
    21.     0.564706f,    0.568627f,    0.572549f,    0.576471f,    0.580392f,    0.584314f,    0.588235f,    0.592157f,
    22.     0.596078f,         0.6f,    0.603922f,    0.607843f,    0.611765f,    0.615686f,    0.619608f,    0.623529f,
    23.     0.627451f,    0.631373f,    0.635294f,    0.639216f,    0.643137f,    0.647059f,     0.65098f,    0.654902f,
    24.     0.658824f,    0.662745f,    0.666667f,    0.670588f,     0.67451f,    0.678431f,    0.682353f,    0.686275f,
    25.     0.690196f,    0.694118f,    0.698039f,    0.701961f,    0.705882f,    0.709804f,    0.713726f,    0.717647f,
    26.     0.721569f,     0.72549f,    0.729412f,    0.733333f,    0.737255f,    0.741176f,    0.745098f,     0.74902f,
    27.     0.752941f,    0.756863f,    0.760784f,    0.764706f,    0.768628f,    0.772549f,    0.776471f,    0.780392f,
    28.     0.784314f,    0.788235f,    0.792157f,    0.796079f,         0.8f,    0.803922f,    0.807843f,    0.811765f,
    29.     0.815686f,    0.819608f,    0.823529f,    0.827451f,    0.831373f,    0.835294f,    0.839216f,    0.843137f,
    30.     0.847059f,     0.85098f,    0.854902f,    0.858824f,    0.862745f,    0.866667f,    0.870588f,     0.87451f,
    31.     0.878431f,    0.882353f,    0.886275f,    0.890196f,    0.894118f,    0.898039f,    0.901961f,    0.905882f,
    32.     0.909804f,    0.913726f,    0.917647f,    0.921569f,     0.92549f,    0.929412f,    0.933333f,    0.937255f,
    33.     0.941177f,    0.945098f,     0.94902f,    0.952941f,    0.956863f,    0.960784f,    0.964706f,    0.968628f,
    34.     0.972549f,    0.976471f,    0.980392f,    0.984314f,    0.988235f,    0.992157f,    0.996078f,         1.0f,
    35. };

    To use it, call:
    Code:  
    1.                  for (int x = 0; x < *pRectW; x++)
    2.                 {
    3.                     floatpixel[0] = lookup[bytepixel[2]];
    4.                     floatpixel[1] = lookup[bytepixel[1]];
    5.                     floatpixel[2] = lookup[bytepixel[0]];
    6.                     floatpixel[3] = 1.0f; // A
    7.                     floatpixel+=4;
    8.                     bytepixel+=4;
    9.                 }


  8. Location
    Orlando, FL
    Posts
    724
    Ok, in case you need it, here is my entire update rect function for updating the dirty rect and converting to RGBA using the lookup table (for Awesomium, but think it is similar enough to help):
    Code:  
    1. void UpdateTextureInPluginRect(float* colors_rect)
    2. {
    3.     if (webCore!=NULL)
    4.     {
    5.         webCore->update();
    6.          if(webView->isDirty())
    7.          {
    8.             webView->render(buffer_rect, W4, 4, &rect);
    9.            
    10.             *pDirty = true;
    11.             *pRectX = rect.x;
    12.             *pRectY = rect.y;
    13.             *pRectW = rect.width;
    14.             *pRectH = rect.height;
    15.             *pRectBool = rect.isEmpty();
    16.  
    17.             floatpixel = colors_rect;
    18.                  
    19.             for (int y = 0; y < *pRectH; y++)
    20.             {
    21.                  if(*pRectY < 2 )
    22.                  {
    23.                      *pRectOffset = (((( *pRectY + y ) * WIDTH) + *pRectX)*4);
    24.                      
    25.                  }
    26.                  else
    27.                  {
    28.                      *pRectOffset = (((((*pRectY - 1) + y + 1) * WIDTH) + *pRectX)*4);
    29.                  }
    30.                 bytepixel = buffer_rect + *pRectOffset;
    31.  
    32.                
    33.                  for (int x = 0; x < *pRectW; x++)
    34.                 {
    35.                     floatpixel[0] = lookup[bytepixel[2]];
    36.                     floatpixel[1] = lookup[bytepixel[1]];
    37.                     floatpixel[2] = lookup[bytepixel[0]];
    38.                     floatpixel[3] = 1.0f; // A
    39.                     floatpixel+=4;
    40.                     bytepixel+=4;
    41.                 }
    42.             }
    43.         } else {*pDirty = false;}
    44.     }
    45. }


  9. Location
    Orlando, FL
    Posts
    724
    Here is my C# keyboard injection code (modified code from solmyr) This mimics a native windows key event:
    Code:  
    1. void keyHandler()
    2. {
    3.     if (flashHit ==true) // does Flash window have focus?
    4.     {
    5.         if(Input.inputString!="")
    6.         {
    7.             c = Input.inputString[0];
    8.             bs = Input.inputString;
    9.             injectKeyboard(0x0102, c);
    10.             if (bs == "\b")
    11.             {
    12.                 injectKeyboard(0x0100, (char)0x08);  // backspace
    13.             }
    14.         }
    15.     }
    16. }


  10. Posts
    213
    very cool stuff. On a similar note, I am still wanting to see the ability to do desktop sharing or application sharing. Like teleplace. Basically it would be nice to have an open office plugin for unity.

    I love the concept of using interactive textures. This goes so far beyond just websites. I wonder if I can make something in silverlight and render it via this plugin on a object.


  11. Posts
    983
    Probably not. While plugins should be able to be supported in the future, right now only Flash is supported by Berkelium.


  12. Location
    Hamburg
    Posts
    732
    Is there a plan to hand in-website events back to unity handling code?
    Making it possible to use html to build UI sounds interesting (at least for me and in theory).

    Cool work!


  13. Posts
    981
    I think I found a new ad system....


    MWU HA HA HA HAA.


  14. Location
    Orlando, FL
    Posts
    724
    Quote Originally Posted by TobiasS
    Is there a plan to hand in-website events back to unity handling code?
    Making it possible to use html to build UI sounds interesting (at least for me and in theory).

    Cool work!
    Yes, that is the intent, communication through javascript to Flash and back to Unity. I have a working prototype using Awesomium that does this, using Flash as my main menu and in game HUD system with transparency; but unfortunately Awesomium requires a license. This however, does not!


  15. Posts
    52
    @reissgrant Thanks for the info. I will have a look at using the tables, but first I have to implement using dirty rectangles in Unity, which will boost performance a whole lot more. Because I don't have any experience with .Net though, I am having trouble returning an array of structs from my C code back to .NET. I was hoping to do something like this in C#:
    Code:  
    1. struct Rect
    2. {
    3.   int top, left;
    4.   int width, height;
    5. }
    6.  
    7. [DllImport ("UnityBerkeliumPlugin")]
    8. private static extern Rect[] Berkelium_Window_getUpdates(int windowID);
    but I have no idea how to bind that to C code. I am reading up on C# and marshalling, but if anyone can give me some hints, that would be greatly apprectiated.
    Note that I have a struct in C++ which has the same members (4 ints), and I hold a std::vector of those in my class, so I just have to get that vector into C# somehow...

    @TobiasS As far as I know, callbacks aren't implemented in Berkelium yet, but they are definitely on their TODO list. Awesomium does have them, but as reissgrant said, Awesomium isn't free and/or open. For GUIs this is a necessary part of the plugin, but it could be useful for a whole bunch of other things too.
    You can however call javascript functions (also with result) from Berkelium, so you can poll values for now.


  16. Location
    Orlando, FL
    Posts
    724
    Unfortunately my knowledge in C# is limited too, so I resigned to returning a simple int for each one of my rect values (returnRectX(), returnRectY(), returnRectHeight() etc.) It's not pretty, but it works!


  17. Posts
    52
    Yes, I will do it like that for now too.

    On another note, I am trying to figure out keyboard injection. Could you elaborate on your code a couple of posts back please? Thanks!


  18. Location
    Orlando, FL
    Posts
    724
    Code:  
    1. injectKeyboard(0x0102, c);
    In the above code, 0x0102 is the WM_CHAR event (which is passed as an int) and c is the keycode (which is passed as a char).


    We'll need to pass the event and the vk_code to Berkelium for the keyEvent Function:
    Code:  
    1. virtual void Berkelium::Window::keyEvent(
    2.     bool    pressed,
    3.     int     mods, // not sure yet
    4.     int     vk_code, <--- put our "c" here (cast it to int)
    5.     int     scancode <--- put our "int" event here  
    6.    )
    7. -------------------------------------
    8. Parameters:
    9.        pressed  <---  if true indicates the key was pressed, if false indicates it was released
    10.        mods  <--- a modifier code created by a logical or of KeyModifiers
    11.        vk_code   <--- the virtual key code received from the OS
    12.        scancode   <--- the original scancode that generated the event

    Let me know if you need more


  19. Location
    Orlando, FL
    Posts
    724
    Don't think I've mentioned this yet, but great work getting this working!

    I downloaded your code to help and noticed a few things so far (All in the C# script):
    Code:  
    1.        m_Texture = new Texture2D (width, height, TextureFormat.ARGB32, false);
    Should be TextureFormat.RGB24 instead, slight performance boost

    Add this to counteract the reverse orientation of the texture (had the same issue in Awesomium):
    Code:  
    1. renderer.material.mainTextureScale = new Vector2 (1,-1);

    Also, you might want to add at the top, to be safe:
    Code:  
    1. using System.Collections;


    *EDIT TextureScale modified to display correctly(1,-1)


  20. Location
    Orlando, FL
    Posts
    724
    I was aiming to get the keyboard portion done tonight but am stuck now, because I'm getting a crash related to the mousemove() function. JeDi, are you getting the same crash or is it just me?

Page 1 of 8 123 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •