Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

What is "PackMan"?

Discussion in '2017.2 Beta' started by bdovaz, Jul 6, 2017.

  1. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,039
    From release notes:

    PackMan: In 2017.2, we are introducing Unity Package Manager. This will be the first stepping stone for the system. We are taking an incremental approach to integrate the system into the Unity ecosystem. It will grow with more features over time. For this first release, we avoid exposing user facing features. Instead, we are exposing an API for enabling internal components to be updated more frequently than the editor (the first pillar of the system).

    What is it's purpose?
     
  2. zyzyx

    zyzyx

    Joined:
    Jul 9, 2012
    Posts:
    226
    My guess: it will be a package manager a bit like nuget or npm.
     
  3. Alex-Lian

    Alex-Lian

    Guest

    Pretty much. We're working on making it easy to get these things like NavMesh, PostProcessing, and other future projects like Scriptable Render Pipelines easy to integrate into your project, even though they'll reside out in places like github or otherwise.

    This is the first steps of underlying functionality, and over time will build out that plus UI and other accoutrements to make it hopefully easy to use, upgrade and experiment.
     
  4. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    What happens when we move things around our end? I guess updating is the same procedure as asset store - you move it back to the default location first...?
     
  5. zyzyx

    zyzyx

    Joined:
    Jul 9, 2012
    Posts:
    226
    My guess again: packages will not be stored in the assets folder, just some kind of manifest file which describes download/source location and version of the package.
     
    superpig likes this.
  6. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,039
    So it should work like nuget (or similar) isn't it? And the generated code it's going to be on "Assets" folder or outside? It should be outside in a "packages" folder or similar like nuget that you can restore anytime.

    Also it's critical that those packages are tagged with versions so we can open an older project and restore old packages.
     
  7. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I'm struggling to understand the benefit of dev time allocated to packman vs just upping it on asset store.
     
    frosted likes this.
  8. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,039
    Seeing nuget, bower or composer and if Unity manages it like Facebook SDKs with Game Room (outside Assets folder). I can see multiple benefits:
    - Delegating to Unity package restoration like nuget restore. You don't need to watch Asset Store pages looking for new versions. You don't need to deal with not knowing what file belong to what package like what it happens with plugins sometimes.
    - Possibility to ignore on version control systems the folder that contains this packages because it should be possible to restore them. So we don't version multiple files that aren't "ours".
    - Been able to target a specific version of a package. This way you can open an old project a restore that package specific version that it's one year old for example.

    A nuget package example references on *.csproj:

    <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="10.0.3" />
    <PackageReference Include="System.ComponentModel" Version="4.3.0" />
    <PackageReference Include="System.ComponentModel.Annotations" Version="4.3.0" />
    <PackageReference Include="System.ComponentModel.Primitives" Version="4.3.0" />
    <PackageReference Include="System.Runtime.Serialization.Primitives" Version="4.3.0" />
    <PackageReference Include="System.Runtime.Serialization.Xml" Version="4.3.0" />
    </ItemGroup>

    Of course I want Unity to hear us to provide feedback and also hear them to see how are thinking on this.
     
    recursive likes this.
  9. frosted

    frosted

    Joined:
    Jan 17, 2014
    Posts:
    4,044
    maybe they feel that they can do earlier releases / beta releases using this system, whereas it 'feels weird' to release beta components from unity itself on asset store?

    Hopefully this is a very minor effort, since I agree, there is a ton of overlap with existing asset store.
     
  10. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Likely I'm going to want to put my assets from external sources (including Unity's and this) into Plugins folder. I like ALL the crap that's external in a subfolder for immediate clarity in the project. If this is able to update and remember the folder I've put it in, then to me it's worth it, otherwise... no - github would still be faster for me to work with.
     
  11. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    2,502
    I really like the idea a lot (Im used to package managers like npm and composer) and it was nice to see this stuff being talked about with enthusiasm and context in the video of the Unity Roadmap talk at Unite Europe this year.

    As for where the stuff will live, the only thing I can add to that debate at this stage is that projects I've opened using the bets of 2017.2 have a Packages folder that exists outside of the Assets folder. Just an empty manifest in there at this stage so I dont know what the exact plan is but I guess its a clue!
     
    superpig likes this.
  12. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I've got hopes that it'll work with rather than dictate! :)
     
  13. scvnathan

    scvnathan

    Joined:
    Apr 29, 2016
    Posts:
    75
    The other cool thing about the package manager is that it sounds like Unity will be using it themselves. So for example, down the line, Unity could ship modules of the core engine (they are in the process of laying the groundwork for that right now). That would let them ship small updates to core systems more easily. Theoretically, it should also let users revert easily. Find an annoying bug with no workaround? You could revert to the last version - say you're on 2019.3, you could revert the physics module to the one in 2019.2. :D Listen to some of their roadmap talks and they muse about this stuff a little.

    IMO, PackMan is one the best things Unity is doing right now, along with SRP, C# Jobs systems, and C# upgrades.
     
    charlesb_rm and zyzyx like this.
  14. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,646
    Lots of good guesses in this thread :)

    Indeed - package content will not be in your Assets folder at all. If you're using a package without wanting to modify it at all, then it's just an entry in your manifest file - the package content will be stored in a machine-wide cache. If you do want to modify it, then we "internalise" the package into the Packages folder of your project, where you can then edit the files to your heart's content.

    Yep, versioning is fully integrated.

    We will eventually be using Packman for Asset Store packages too.

    Yep, absolutely. There are still a lot of details around that - for example, we will not be QAing and supporting every possible combination of modules you might try to use, and some versions of modules might have dependencies on particular other versions of modules - but Packman gives us a framework to describe the different engine modules and their relationships, so that we can exploit that flexibility in situations where it makes sense.
     
    elbows likes this.
  15. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,039
    I like it that asset store packages will use it too so it don't get messed with our code particularly the root "Asset/Plugins" folder that it's used incorrectly by many many Asset Store Developers (since Unity 5 you don't need that magic folder despite of AndroidManifest.xml and such). I had always thought why Unity Asset Store reviewers don't disapprove packages that don't use that correctly because it's a source of problems in your project when you use multiple asset store packages specially mobiles device targeted ones.

    For the second paragraph I think that you can handle it having dependencies with versions attached so you can't have a module that has a dependency and that dependency being a very older one.

    Examples:

    <dependency package="test.package" version=">=1.2.3" />
    <dependency package="test.package" version="1.2.*" />
    <dependency package="test.package" version="*" />
     
  16. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,646
    Yes, I wasn't talking about the problem of "how do we specify a dependency" - as you say it is not difficult to just specify a version expression along with the dependency.

    The problem I was thinking of was the one of incompatible dependencies - if you want to use the animation system from 2019.2 with the physics system from 2019.1, then it may be that the former requires the engine core to be at version 2019.2 while the latter requires it to be at 2019.1, and it's not possible to satisfy both dependencies simultaneously.
     
  17. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    825
    One very large advantage is that the packages don't actually need to be on the Asset Store at all. They can be on a central disk location and referenced as needed by a project. This is excellent for example for studios that aren't actually connected to the internet due to security agreements.

    It also lets you essentially build internal packages that are not distributed on the asset store but are otherwise similar, but used internally.
     
    JesOb and hippocoder like this.
  18. Player7

    Player7

    Joined:
    Oct 21, 2015
    Posts:
    1,533
    Will the textures from these packages get put into a machine-wide library cache , so I don't have to see "Hold On" ..while unity does texture conversions for nth time just because I wanted to build to a different platform to test, where large/lots of textures that get used in multiple projects and tested on different platforms need to get converted every time.. Its so unnecessary for textures that have exact same settings, but are used in different projects (usually from the same core packages that get pulled into projects), thus currently go through the entire conversion process for per project, per platform. Its one seriously annoying part of Unity build process and its per project library junk folder that seems to think deleting per platform stuff is really helping me and saving space when usually its just wasting our time. And short of having completely separate projects setup per platform and using hardlinks or whatever to try maintain changes between versions etc is just a pain in the ass, especially users who aren't in even in local studio teams ... because currently its not really designed for quick testing on other platforms ..so a global library junk folder is needed! The cache server is broken node.js junk that doesn't work locally properly anyway..currently buggy to even use.. including fails to work with custom folder with spaces reported ages ago, confirmed, and still not fixed, and connection problems (locally :rolleyes: js )

    Sounds like a recipe for a bug soufflé.. I have zero interest in wanting mix and match low level parts of the engine. Kind of expecting the editor/engine releases to work cohesively without needing to further mix and match low level parts of them, else we all be talking about what the latest combination of unity low level bits actually work in combination. Maybe just leave pakman for the higher level things that are more plugin-like already, and don't even bother going down that route... like the post postprocessing stack etc because is it really needed for the low level things?

    Personally rather see attention to other areas like terrain, character/physic tooling, level editing and artist tools for 2019 than needing find if animation system 2019.2 will work with 2019.1's physics.
     
  19. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,646
    It's not part of the PackMan project itself, but yes, that's coming as part of the new Asset Import pipeline - a shared machine-wide 'Library' folder, along with background asset importing, so that you can open a project and begin working with it without having to wait for all of it to finish importing.

    It's not had much love for a while, but it has more recently been picked up again and we're working through the bugs (and, crucially, creating automated tests as we go, to ensure the issues do not regress). You should see improvements in 2017.1, and more in 2017.2/2017.3. Once the new asset import pipeline goes live, it will maybe become less valuable, at least for local usage, but I don't think it's going away for LAN usage.

    Sure, I'm not saying that we will encourage mixing and matching versions like that - certainly we will be putting out cohesive releases, with packages that were tested together, that you can just use. But it's good to have the flexibility - particularly if you are about to release, and you are blocked by a critical engine bug in one particular part of the engine, and you want to get an update to that individual part without having to worry about other fixes that may have been made in other areas. For example, using 2018.2.0 for most of the engine, but 2018.2.0p1 of the animation system.

    Well, a big part of doing this successfully is making more and more things plugin-like. You've already seen us do this with the Navmesh system, for example - moving from having the high-level parts built into the engine, to being an open-source package on Github. Something like the animation system might seem low-level today, but as we progress with making things more modular, it's going to become more plausible to look at it like a high-level system.

    Of course - and some of those are under active development alongside this. That's another thing that Packman will actually help with: making experimental versions of new features available for you to test out, without it requiring a whole new separate install of Unity, and without it being a pain to bring your project back to safer territory when you've finished trying it out.
     
    Deozaan, Player7 and scvnathan like this.
  20. jwvanderbeck

    jwvanderbeck

    Joined:
    Dec 4, 2014
    Posts:
    825
    One huge advantage of Unity Tech pulling things out of the core Editor/Engine and into Plugins is that it forces them to both expose more of the internals to us, as well as fix unknown issues with said APIs. This is great for everyone really. If Unity has one overarching problem, it is its tendency to be a black box to developers.
     
    dadude123 and scvnathan like this.
  21. scvnathan

    scvnathan

    Joined:
    Apr 29, 2016
    Posts:
    75
    +1

    I would also love it if Unity ends up collaborating with the community more by reviewing/accepting PRs from outside the company.
     
  22. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,646
    FWIW, we do already review/accept PRs on many of our GitHub and BitBucket repositories.
     
  23. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    1,039
    I don't know of other project but particularly UI has been a project that has many pull requests without anyone been accepted neither answered some of them by Unity staff:

    https://bitbucket.org/Unity-Technologies/ui/pull-requests/

    I hope to see some day Unity been actively working with community and accepting pull requests, I will help it too. A great example is Microsoft with .NET Standard that has multiple repositories with commits and merges every couple of hours.
     
    zyzyx likes this.
  24. Alex-Lian

    Alex-Lian

    Guest

    Ah, that was our first foray, and its setup is not awesome for us. Aside from the setup, our biggest hurdle is that our internal testing systems are unavailable to contributors, which means a lot of time contributions can have inadvertent performance or cross-platform impact.

    We _are_ working on trying to break down these barriers and make it easier to move things across, and the package manager here is part of that. The UI system will also be moved to be re-packed into the new model once it's moving.

    Indeed, we'd like to get there as well!
     
  25. nhold

    nhold

    Joined:
    May 2, 2017
    Posts:
    50
    If this package manager is done right I think it can be one of the best additions to Unity yet, at the very least the changes to have assembly definition files will allow better management of my own package manager.

    Some questions:
    • Will this allow us to have our own sources so we can provide our own (private or public) frameworks, utilities, code and art?
      • Can the sources just be a git url that requires ssh access?
    • It looks like it will be based off of NPM, is there a reason for this instead of just developing a custom one in C#?
    • Will we get a neat UI to install\upgrade\downgrade and view currently installed packages?
    To celebrate the start of Pacman in Unity here is a screenshot of my own Package Manager:

     
  26. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,646
    Eventually, yes and yes.
    Why reinvent the wheel?

    Eventually, yes.
     
    nhold likes this.
  27. nhold

    nhold

    Joined:
    May 2, 2017
    Posts:
    50
    Awesome! Thanks for the quick response.

    I was just hoping for a C# solution that I could contribute to :p. Not re-inventing the wheel is good as well I'll just have to learn that package definition format.