Search Unity

WSA batchbuild failing to restore nuget packages

Discussion in 'Windows' started by Kron0, Jan 25, 2017.

  1. Kron0

    Kron0

    Joined:
    Jun 26, 2013
    Posts:
    7
    I am trying to make a build process for a windows 10 app.
    The following is called from an editor script and functions in editor:

    string[] scenes = new string[] { "Assets/Scenes/Main.unity" };
    BuildPipeline.BuildPlayer(scenes,"../BatchBuild",BuildTarget.WSAPlayer,BuildOptions.None);

    this is the output log:
    http://pastebin.com/Zfvfq46L

    the relevant section is this:
    1. Restoring NuGet packages from 'C:\Users\Paperspace\Documents\Example Hololens Unity Project\UWP\project.json'.
    2. DisplayProgressbar: Building Player
    3. Project lock file validation failed.
    4. DisplayProgressNotification: Build Failed
    5. Error building Player: Failed to restore NuGet packages.

    Any help would be greatly appreciated
     
  2. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    Does it succeed if you build it manually on that machine? Does "<ProjectDir>\UWP\project.lock.json" exist after such a failed build? If it does, could you paste its contents?
     
  3. Kron0

    Kron0

    Joined:
    Jun 26, 2013
    Posts:
    7
    Yes, a manual build succeeds.
    No, the file "<ProjectDir>\UWP\project.lock.json" does not exist. (It should be a clean build so not require any pre-existing files).

    Building manually to that location (so as to generate the file) and then trying the batchbuild also fails in the same way.
     
  4. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    Weird. That file is supposed to be created by the build process, and it fails in the step that verifies that it managed to do so successfully. I assume the machine has network connection during the batch build, right? It seems NuGet.exe exits with 0 exit code, yet the file still does not exist.

    Could you try running NuGet.exe (located in <UnityEditorInstallation>\Editor\Data\PlaybackEngines\MetroSupport\Tools) with this command line and see if it succeeds?

    Code (csharp):
    1. nuget.exe restore "<UnityProjectDir>\UWP\project.json" -NonInteractive
     
  5. Kron0

    Kron0

    Joined:
    Jun 26, 2013
    Posts:
    7
    Machine is connected to internet.

    output with no changes:

    C:\Program Files\Unity\Editor\Data\PlaybackEngines\MetroSupport\Tools>nuget.exe restore "C:\Users\Paperspace\Documents\Example Hololens Unity Project\UWP\project.json" -NonInteractive
    MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
    Input file does not exist: C:\Users\Paperspace\Documents\Example Hololens Unity Project\UWP\project.json.

    output after building manually to \UWP:
    C:\Program Files\Unity\Editor\Data\PlaybackEngines\MetroSupport\Tools>nuget.exe restore "C:\Users\Paperspace\Documents\Example Hololens Unity Project\UWP\project.json" -NonInteractive
    MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
    Feeds used:
    C:\Users\Paperspace\AppData\Local\NuGet\Cache
    C:\Users\Paperspace\.nuget\packages\
    https://api.nuget.org/v3/index.json
    C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\

    Restoring packages for C:\Users\Paperspace\Documents\Example Hololens Unity Project\UWP\project.json...
    All packages are compatible with UAP,Version=v10.0.
    All packages are compatible with UAP,Version=v10.0 (win10-arm).
    All packages are compatible with UAP,Version=v10.0 (win10-arm-aot).
    All packages are compatible with UAP,Version=v10.0 (win10-x86).
    All packages are compatible with UAP,Version=v10.0 (win10-x86-aot).
    All packages are compatible with UAP,Version=v10.0 (win10-x64).
    All packages are compatible with UAP,Version=v10.0 (win10-x64-aot).

    So why does that not work when building from script? Verified problem still exists after manual build (exact same output as previous)
     
  6. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    Trying to figure it out :).

    It looks like the input file does not exist when you run nuget before building the project:

    Input file does not exist: C:\Users\Paperspace\Documents\Example Hololens Unity Project\UWP\project.json.

    This file is supposed to be created when you call BuildPlayer. Does it not get created when you run the build in batch mode?

    Either way, we'll probably need a bug report with a repro project in order to look into this. All our automated test suites are using batch mode to build tests, and I haven't ever seen an error like this.
     
  7. Kron0

    Kron0

    Joined:
    Jun 26, 2013
    Posts:
    7
    Would be happy to provide them. How do I go about doing that?

    Thanks for your help btw :)
     
  8. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    You can submit a bug report using the editor help -> report a bug menu.
     
  9. Kron0

    Kron0

    Joined:
    Jun 26, 2013
    Posts:
    7
  10. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    We'll take a look!
     
  11. StephenHodgson-Valorem

    StephenHodgson-Valorem

    Joined:
    Mar 8, 2017
    Posts:
    148
    Bump. Confirmed BuildPipeline.BuildPlayer() does not generate project.json
     
  12. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    The bug Kron0 reported was closed as "Not Qualified" because we could not reproduce the issue, and upon further communication with him he discovered that it wasn't working on his side because his CI setup launched Unity process without privileges to access nuget cache folder. Upon fixing that, things started working for him.

    Can you report another bug with the details of what you're seeing?
     
  13. StephenHodgson-Valorem

    StephenHodgson-Valorem

    Joined:
    Mar 8, 2017
    Posts:
    148
    Yes, I already submitted a bug report, but never got an email replay with the link. I was actually able to figure out exactly what the problem was. We are getting the project.json files generated but they are being generated in the wrong location. The default path is UWP but we have a build folder with a different name than UWP. I noticed that if I delete the UWP folder and run Pipeline.BuildPlayer, it will put the project.json and project.lock.json in a new UWP folder with the rest of the project in the specified location path.

    Here's a link to an example project on github

    1. Simply open the project.
    2. Switch to Windows Store platform
    3. Set target device to HoloLens
    4. Check Unity C# Projects
    5. No not press Build, instead use Test/Simple Build from menu.
    6. After editor builds, open project root folder, and see UWP and UWP_Test folders.
    7. project.json and project.lock.json files appear in UWP but not in UWP_Test like expected.
    My simple work around is just checking if the files exist, and if not then copying them from the editor build tools location.
    Editor\Data\PlaybackEngines\MetroSupport\Tools\project.json
     
    Last edited: Jun 11, 2017
    SillySnowman likes this.
  14. SillySnowman

    SillySnowman

    Joined:
    Jun 28, 2016
    Posts:
    1
    Also wanted to add that I encountered this same problem. I built an empty scene with just a cube and directional light and camera, then built using VS 2017 Community and Unity 5.6.1f3

    upload_2017-6-15_10-18-42.png

    Then, the generated projects were missing the project.lock.json files that are required in VS 2017, but NOT in VS 2015. Running a command line build, as follows, failed to generate the .appxbundle and .appx files required for a UWP deploy, like for what is required for a HoloLens build: "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\MSBuild.exe" "C:\<path_to_project>\WindowsStoreApp\<project>.sln" /t:Build /p:Configuration=Release /p:platform="x86"

    See the "C:\<path_to_project>\WindowsStoreApp\GeneratedProjects\UWP\Assembly-CSharp and Assembly-CSharp-firstpass as failing to generate the project.lock.json files required for VS 2017. To overcome this, simply run in cmd.exe:

    C:\<path_to_project>\WindowsStoreApp\> "C:\Program Files\Unity\Editor\Data\PlaybackEngines\MetroSupport\Tools\nuget.exe" restore

    which will auto-generate the project.lock.json file (which lists the dependencies your UWP app requires), and then your command line builds will run successfully! If you do not have nuget.exe, you can download it from here (I used version 4.1.0 latest release): https://dist.nuget.org/index.html

    Hope this helps somebody :)
     
  15. rihdus

    rihdus

    Joined:
    Jul 13, 2012
    Posts:
    10
    Can you explain at all what privileges needed to be changed, and where? I'm using Jenkins for our builds, and in trying to use BuildPipeline to make a Streamed Scene bundle, I'm hitting this problem. Specifically:

    • Calling the function in Unity from the command line (identical command line to what the CI uses) while logged in works.
    Code (csharp):
    1. Restoring NuGet packages from '<Path to Project>\UWP\project.json'.
    2. DisplayProgressbar: Building Player
    3. NuGet packages successfully restored.
    4. BuildPlayer: start building target 21
    • Calling nuget directly on the project.json file generated by Unity (from a UI build -- this file disappears when calling it from command line) while in the CI system works.
    Code (csharp):
    1.  
    2. "<UnityInstallation>\Editor\Data\PlaybackEngines\MetroSupport\Tools\nuget.exe" restore "<Path to Project>\UWP\project.json" -NonInteractive
    3. Restoring packages for <Path to Project>\UWP\project.json...
    4. Committing restore...
    5. Writing lock file to disk. Path:<Path to Project>\UWP\project.lock.json
    6. <Path to Project>\UWP\project.json
    7. Restore completed in 2249ms.
    8.  
    9. NuGet Config files used:
    10.    C:\Windows\system32\config\systemprofile\AppData\Roaming\NuGet\NuGet.Config
    11.  
    12. Feeds used:
    13.    https://api.nuget.org/v3/index.json
    • Calling the function in Unity while in the CI system fails to validate the lock file.
    Code (csharp):
    1. Restoring NuGet packages from '<Path to Project>\UWP\project.json'.
    2. DisplayProgressbar: Building Player
    3. Project lock file validation failed.
    4. DisplayProgressNotification: Build Failed
    5. Error building Player: Failed to restore NuGet packages.
    I've attempted to find every cache in a variety of "AppData" locations to remove any security permissions, but that hasn't worked as yet.
     
  16. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    What user are you running your CI process as?
     
  17. rihdus

    rihdus

    Joined:
    Jul 13, 2012
    Posts:
    10
    Local System. Running the CI Slave Service as the Administrator Account works as a temporary work around, but I'd prefer not to allow it to run as a full-privilege service permanently.
     
  18. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    Is that a different user than the one you built manually with? Perhaps you could try setting NuGetCachePath environment variable to a folder which the CI Process definitely has access to?
     
  19. rihdus

    rihdus

    Joined:
    Jul 13, 2012
    Posts:
    10
    I use an Administrator account when logging in most of the time, so any GUI or manual command-line builds use that. The Jenkins build system is running as a Service, so uses Local System by default (a special account). I have used it up to this point without any issues for making builds, including Win 8.1 builds. It's only with this attempt to move up to Universal 10 that I've had problems with permissions like this.

    I set the Environment Variable as you suggested, but it doesn't appear to be getting used by any builds even after a restart. Whether it succeeds (as Admin) or fails (as Local System), nothing is appearing in the folder I specified. I can see in the build output listing of Environment Variables that it has been picked up by the system -- just doesn't seem to be used by the call from Unity.
     
    Last edited: Jun 20, 2017
  20. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,675
    I suppose you don't want to create a "regular user" and run CI on that instead?

    Is this on a VM? If so, is there any chance you could share the image with us so we could reproduce it? There definitely is some kind of permission issue at play, but anything done by us right now would be a stab in the dark if we're not able to reproduce it.