Search Unity

File I/O in Native Client

Discussion in 'Developer Preview Archive' started by Julien-Lynge, Feb 23, 2012.

  1. Julien-Lynge

    Julien-Lynge

    Joined:
    Nov 5, 2010
    Posts:
    142
    Poking around on the notes for Pepper, I came across a number of pages that mention (non-random) file I/O functionality in NaCl. The functionality is a bit different than System.File.IO in that it's non-blocking/asynchronous, so more analogous to WWW in Unity terminology. From https://developers.google.com/native-client/beta-docs/fileIO

    My question is: what level of support does Unity offer for taking advantage of this functionality? How would one go about writing code to do disk I/O from Chrome? Google does provide an example C++ read/write class at http://code.google.com/p/nativeclient-sdk/source/browse/trunk/src/examples/pong/pong.cc - however, we can't just compile this code to a plugin for the project, because as Unity says: "As with the standard webplayer plugin, native C/C++ plugins are not currently supported by NaCl."

    Our application is a data visualization tool and uses a large amount of caching, so it's important to know if there is or will soon be any way to cache to disk or have access to direct I/O from Chrome Native Client. NaCl seems like it would be fantastic (with the added bonus of potentially running on Linux), but if anyone has any other ideas for a way to cache a few hundred megs of images and binary data (not assetbundles, as we're working realtime) from a webplayer, perhaps taking advantage of a javascript bridge from Unity to HTML5 or similarly exotic things, let me know.
     
  2. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    There is currently no native way to do that in Unity with NaCl. I don't expect that we will directly expose NaCl FileIO APIs any time soon, as they don't map well to existing File.IO functionality (as you described), and exposing a specific NaCl File IO class seems to be too special cased. I think the preferred way for us to expose this would be to add the ability to load plugin DLLs in NaCl, but that is unlikely to happen in the short term either (I think that there are more interesting features to be added to NaCl at this point).

    So, right now your only option would indeed be using HTML5 file storage using js. From what I can tell with my limited knowledge of HTML5 file storage, this should generally be possible, though you would probably have to encode the data as text to submit it to Unity.
     
  3. Julien-Lynge

    Julien-Lynge

    Joined:
    Nov 5, 2010
    Posts:
    142
    Thank you Jonas, that's just what I needed to know.

    One issue I see with plugin dlls is that this would be Windows only (as I understand it). To be platform independent I suppose you'd have to be able to access other .nexe files from within the Unity compiled .nexe, and you'd then compile your File I/O interface script separately for NaCl and link it that way.

    On the other hand, even though File.IO functionality doesn't map well to the NaCl functionality, there are ways to do async read/write with streams in Mono, for instance http://msdn.microsoft.com/en-us/library/kztecsys(v=vs.71).aspx, and that should map pretty well from what I've seen of the NaCl examples.


    On a side note, my personal wish list would include the ability to cache any filetype (or even any non-executable type) with WWW.LoadFromCacheOrDownload, not just assetbundles (which are impossible to use for anything being generated in realtime). That would pretty much solve our problem, and would give us the incentive to purchase a caching license (as mentioned on http://unity3d.com/support/documentation/ScriptReference/Caching.Authorize.html) to get access to 4 GiB of disk from the webplayer.
     
  4. jeff-smith

    jeff-smith

    Joined:
    Feb 24, 2012
    Posts:
    19
    Yes, we appreciate the quick reply, jonas.
     
  5. TimCabbage

    TimCabbage

    Joined:
    Feb 13, 2011
    Posts:
    66
    NaCl is the only way to port things in Unity to Linux at the moment(and probably will be so for a while).
    It's ok with me if I want to make a game that just works on Linux.
    However I would need an option to mod things.

    I'm thinking like this: I don't *need* to access a user filesystem.
    What I need is a similar way of working as with a normal PC/MAC build ->
    A user has a directory in a userdir (~/games/MyGameName)
    A user can then either download and use mods and other files using the ingame "shop", or just put his previously downloaded files there. Or run an installer that does this for him.
    Now is that possible to achieve in NaCL or is the storage something like a DB file? If so - where is that folder? Can I at least get a location of it(to display it to a user).
    I've been looking for answers for two days now and can't seem to find them.

    Also, are there size limits to NaCl filestorage? It'd probably go into hundreds of megabytes easily.

    Thanks,
    Siv.

    Edit: Also, if I understand correctly portable NaCl will be able to run applications without the browser and without the limits? Kind of hard to find reliable info on these topics and you seem to know how it works;)
     
    Last edited: Feb 27, 2012
  6. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    Your question is essentially the same as the original thread opener, and the answer is the same as well. Currently the only possible way to do local file storage with Unity-NaCl is to use html5 file storage via JS. I don't think there is a generic limit for HTML5 file storage, but you generally request a storage quota from the browser, and then it is up to the browser to allow this or not (usually by asking the user).

    PNaCl is not about running without a browser or limits. It is about making NaCl use LLVM bitcode, instead of architecture specific machine code, in order to make NaCl executables truly cross-platform. The thing about limitations is that Google has decided not to enable NaCl (by default) for content outside of the Chrome Web Store until PNaCl is released, because they don't want the web to become polluted with platform specific content.
     
  7. TimCabbage

    TimCabbage

    Joined:
    Feb 13, 2011
    Posts:
    66
    Ok.
    Thank you very much for the reply.
    Thought that once the problems with the async file storage have been resolved, things would just work as usual. I don't want to develop it specifically for NaCl now, if modding would not be supported on Linux - well... happens. Still better than no Unity on Linux at all.

    Thanks for clarifying the pNaCl for me - had some misconceptions about it.
    And since we're here already - NaCl is an open standard, thus - would it be possible to make a linux distribution using NaCl that does not rely on Chrome but is just a truly native client(remove the filesystem restrictions(currently a chrome flag afaik) and make it an immersive fuillscreen?
    I'm talking license-wise and how it looks from the Unity's side.
     
  8. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    Since NaCl is Open Source and licensed under a BSD license, yes, this would be technically possible. You can run command line NaCl .nexe executables using the sel_ldr executable which comes with the NaCl SDK (with full file system access) just fine.

    But making it run Unity content would be a lot of work, since doing away with Chrome would mean that you would have to re-implement the whole rendering, audio and input interfaces from scratch, as those are provided by Chrome otherwise. So from Unity's side this would not make much sense, as at that point, we could just as well port over Unity to a native linux executable -- it would mean having to solve the same issues.
     
  9. TimCabbage

    TimCabbage

    Joined:
    Feb 13, 2011
    Posts:
    66
    Oh. ok. Thank you.
    I thought that the NaCl was handling all the audio, input and rendering as it was the sandbox and just "nested" inside Chrome.
    Oh well... learned a lot.

    Thank You Very Much:)