[WIP] Release Manager. (Installer, Patcher, Launcher)

I also believe it's ssl downloads of large files breaking the download. But does the download has to be SSL?

64bit version of the release manager would be truly great. But is it possible to make it use the swap file? Also I have run into a few applications with problems addressing more than 4gb even to the swap files. This I had to use an external program to actually edit/hack the application to make it possible to address larger than 4gb in the swap file. Sony Vegas Studio was one of them. Anyway just trying to give you ideas on where the problem might be.

Is the 64 bit version far away? Just asking so I can plan for running closed beta with or without a real launcher.

Also for other people Reading this thread. This is a beta (test version) Product, and all these errors we find don't mean it's a bad Product. I have full faith in Release Manager and it's capability when the final version is out.

 

Not at all you can drop the https:// in front of your URL for download. I only use normal http. That having been said I am planning a hack proof private public key for the system down the line. (didn't want to make it more unstable with that off the bat) it will allow you to secure your downloads and make sure the stream been downloaded is not tampered with and also can only be downloaded if they have the original installation.

Its my next priority after the new Launcher and ReleaseManager are released. (This includes the retry on downloads and resuming partially completed files.) I have to complete the full 64 bit arc of LauncherUpdater->Launcher->Installer and ReleaseManager. So the test cycle is painful, but I really don't know why I didn't start with 64 and forget all about 32

Thanks for that, been surprised how many things can go wrong and really happy with the patience and perseverance you guys are showing. I don't just need this solid to sell it I wrote it because we really have to have a tool like this here. So every little thing fixed brings us one step closer to having a tool we can trust.

 

OK, just to give you some more info on the dropped download. I removed the "s" in the URL from the download directory and made a new package and installation. It still drops the download at around 100MB download. (It only fails every now and then now though.)

Log:

Unexpected end of ZLIB input stream
java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(Unknown Source)
at java.util.zip.InflaterInputStream.read(Unknown Source)
at java.util.zip.ZipInputStream.read(Unknown Source)
at java.io.FilterInputStream.<unknown>(Unknown Source)
at celestial.launcher.UpdateThread.unZipIt(Unknown Source)
at celestial.launcher.UpdateThread.run(Unknown Source)

So it seems like some other sensetive details in how the download is done towards some hosts.

 

I've been trying to compile a patch and i'm running into the fatal error: out of memory problem as well. Though i've noticed some odd behavior, right before the crash it seems to go down with the memory usage very fast (in my case the application never goes over 2gb so it still should be able to work properly i guess, i'm not sure how it works here). Got a screen http://take.ms/TBaTE

 

I've got a question: how does the html panel work?

...cause you know, i've tried some html5 parallax animated content on it (...) and it didn't work XD
It would be cool though, free animation support if you can expand the support.

 

First of all, great program, I've been looking for something like this that functions well for a long time.

I am having problems with the interface of the launcher client updating. When I create a new layout for the launcher in the Design Launcher tab and then hit "Upload this Layout," the transfer completes successfully, but the new design is never downloaded to the client launchers themselves. Am I missing a step in the process here?

 

Yes also looks like it tired to unzip an incomplete file. I will do a file length check if it fails we can assume the download failed and try again.

 

The HTML panel only has HTML 3.2 support. So no javascript or any advanced stuff saddly.
We are looking at making an image swopping panel that allows you to queue a bunch of images, but to build a fully complaint HTML pane would be a huge undertaking.

Whats the largest file in your project? I am pushing the 64 bit version forward as that the only way to get more memory in the application. I am quite sure the 64 bit version will solve your problem.

 

Thanks we are of course still making sure it functions well. A lot to polish still.

Currently the Launcher caches the design locally and assumes it wont change for a number of days. I am changing that to use the timestamp from the server to determine if the file has changed. For now you can delete the cache folder in your C:\ProgramData\{yourapp} the launcher will then reload the design.

 

Uhm image swapping is ok but not even close, you guys should really look into what's needed to put html5 into this because with it you can do the most advanced stuff you've ever seen, check out the Digital Extreme - Warframe launcher, i've looked into it and you can find the html5 files into the game directory. Apparently they made it with CEF3 if you can find it useful. This is really huge for this product, it probably has more potential than you ever thought and you just need an html5 panel to unlock it.

371mb, i'll be waiting for the 64 bit then if there's nothing else i can try

 

LOL I really like the interactions here. So I would never have considered HTML5 and now I can see how it will be done. I was planning to build a video playback panel, built instead HTML5 gives you youtube so you can put your youtube panel into the form. So good news its possible, bad news its going to cost me a bit $1,600. I will in the mean time evaluate the tool I am considering (HTML5, Flash, Silverlight supported). To make sure its possable. It will add a few megs to the Launcher download. Thats currently at 2.5 megs so I think we can afford to push that up in the name of video and html5.

What strikes me from here is that you could then build a shopping front into your launcher. So I see what you mean HitsuSan about the potential.

BTW Warframe's Launcher was one of the ones we did look at when we considered this project.

I currently have the 64bit version running here. Well ReleaseManager, but not the LauncherUpdater and the Launcher. Also WiX 64bit installer needs to be curated.

I will drop support for the 32 bit version of the ReleaseManager but the ReleaseManager will continue to support 32bit applications. We just going to hot the out of memory wall far to often in 32bit.

BTW I checked although I have configured my 32bit build to sue swap space when it runs out of memory. Its still limited to a 32bit address space. I would expect this to be 4gig but it appears to be lower then that.

 

Ah correction I may not end up spending cash on the HTML solution. I am looking at Chromium will be some work to get it in. My main concern remains added download Size. I might need to find a way to package it with HTML5 and Without for those that dont need the added weight.

 

You don't need youtube if you have HTML5, you can directly integrate videos with tags <video></video>

Size will go up for sure though, it might be around 50 to 70mb (plus your launcher files but we are talking about the current most high end solution there is).

Anyway yeah it seems it's using around 2gb not 4. I'm sure nobody will have a problem if there's no 32bit version, games are huge these days and i'd really like to meet a dev who's working on a 32bit environment ^^

 

I hope we can see fully working main features before any coolness stuff is added like html5.

 

Yes you will HTML5 has replaced my embedded Video but its not guaranteed yet. 64bit is priority and so are many of the other features.

 

Progress

We pretty close to the Windows 64bit version. Also the settings screen has a few options for LAF to make it more attractive.

http://info.celestial-games.com/doku.php?id=releasemanager:laf

The Launcher can not continue with partial downloads if the server supports them Failing that it will attempt to re-download the full file.

 

nice!

 

I didn't understand the last part but nice

Uhm now that i think about it, can't you just make the launcher check if there's an update for it's own layout? All you need it's a timestamp for the last update and a "redownload" call i assume.
And about the in engine options, how do you configure the launcher to make it work? I can't seem to make my game launch without it's own launcher and the options button doesn't show anything : \

 

Apologies I should save writing posts for when I wake up fresh not when I am too tiered. Thanks.

I was trying to explain that when the Launcher is downloading files. If the download breaks the Launcher will try to download the last part of the fill its still missing not start from scratch. This requires that the hosting provider supports Accept-Ranges="bytes". See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html I suspect most good ones will.

We are now doing that for ALL the files downloaded and cached.

I am not really sure I follow whats wrong. If you saying the Unity game still shows its launcher that's something you need to change in the outputting of your game. The menu is under Edit -> Project Settings -> Player Settings I think.

The Option button in should just work without any extra effort. You can specify the list of quality settings you have in your game as a comma separated list. Where you specify the default with an * after the name.



Does that answer your question at all?

 

It should be in theory but when i upload a new launcher it doesn't detect the new version right away and the only way i have to force it to update is to delete the Cached folder.

For the Option button i haven't changed anything, i just have Fantastic* as an option (only that one) under Action but no matter what i put in there that button is never active on the launcher.

 

Sorry I should have been more clear. this is in the new launcher. Its not released. It will be released with the new 64bit version. You game launcher should auto update.

Ok I tested only one quality setting, worked on here so that's not the issue. (I also added a ticket to not show the drop down if there is only one option, its just messy) could you paste your properties as an image perhaps I can see something there. Also in the ProgramData folder there is a log file log.txt. Please attach that here so I can see if there are any errors.

 

Oh ok cool

No errors in the log.txt (only the usual unzipping, nothing different).

This is how the button is setup. Can it be because i have the game compiled to have the options menu on startup? If that's the case i'll wait for the next release and i try disabling it with a patch.

 

You do need to disable unities option settings I cant overwrite those from startup. Does that config button not show the config though? Or does it simply not effect the game.

Also you do have the Prefab in your start scene from the Unity plugin?

 

It doesn't show the config at all, the button is there but pressing it doesn't show the config. I do have the prefab, yes.

 

Same here

 

When I try to start from scratch by picking a new working folder, creating a new (first) release etc. The new installed launcher still sees an old version from the old release. I can even start it even though it's not in this new install folder of the launcher.

Anyway I will wait for the 64 bit version Before doing any more testing and report any bugs.

 

Are you saying you installed a new version but it still picks up the old downloaded version despite uninstalling the previous app. I think this might be an issue with the installer not removing that programdata folder. I will check into that.

Thanks is close had to set up a 2nd build machine. Some of the 32 and 64 tooling didn't operate on the same machine so its taking a bit longer.

I have not been able to recreate this issue, but it might have something to do with having a non OS style button without a background image. I have made changes to the button class to cater for ignoring image masks since then so the logic might have been fixed in that process.

 

Alright, we'll try it with the next update then! (i'm using your standard button for the options though, it's the only one i didn't change at all ^^)

 

The new 64 bit version should be released tonight. The final hurdles have been overcome.

 

The 64 bit version is up, you should have received an email with directions on downloading your new build.

I have highlighted some of the changes in the list below that are key to many of the things mentioned in the forums before. Of course the main goal of this has to be solving the compile issues for files over 200mb.

Version 0.0.12 64Bit released

Launcher Updater
[x] Bring up the downloading dialog sooner
[x] Pass bit with and public key to launcher
[x] Make downloading retry and start from last knock position on failure

Launcher
[x] Allow buttons to ignore the mask and be considered clickable when the mouse is over the area not just the masked pixel
[x] Retry and restart on downloads
[x] Load the bit width from the properties and download the appropriate files
[x] Identify all OS&Bit width combinations and correct paths
[x] Make sure old LauncherUpdaters still work, so if no bits presented assume 32 and an empty key
[x] Change cache or images to work with the time stamps from the HTTP download not the 5 days as before
[x] Make sure Launcher does not try download files when offline if they are cached
[x] Launcher needs to save the cache info for layout.xml
[x] Change look and feel for display to choosen one from designer
[x] Hide quality settings drop down if there is only one option
[x] Bring up the downloading dialog sooner

General
[x] Create 2048 RSA key and distribute the public key with the config for the application
[x] Add the bit width of the release to the config application

Packager
[x] Make sure the uninstall removes the ProgramData folder completely
[x] Create package win 64

Designer
[x] Set look and feel type in designer

Thank you for your patience

 

Works like a charm Though it took almost 4 hours to compile a patch from a 400mb release (it became 1.4mb though that's cool!) and i've noticed it never used more than 20% of the cpu power, is there something that can be done?

Every other problem mentioned previously seems solved though, great work!

 

How much RAM do you have on your machine? It may be switching to swap space. You may also find that its using one CPU core 100%. Its not really possible to process one file with multiple threads. The work is very synchronized. I process multiple files but that would possible only lead to a serious RAM issue. I plan to make the compile screen more informative.

Does this include the config button not working? I could not replicate that issue myself.

 

I've got 32gb, it was using 4.5gb maximum during the whole operation. And no it wasn't using 1 CPU at 100% it didn't use more than half of a single thread (i've got an i7 3770k 4core/8thread cpu, 3.5ghz * core).

Yes! Fixed.

 

Nice machine. Thanks for the info, I will try setting the thread priority higher. There is no synchronization within the thread so why it would not consume the full available CPU is beyond me. There is one static used to pass the progress back and forth between the display and the compilation, but I doubt that's causing a synchronization issue.

Ah glad to hear, I changed buttons to use its area for mouse over if no images are added. There is an option to ignore Image Mask now btw. Not sure if you picked that up.

 

So really 4 hour per compile is not what's supposed to take for that small client isn't it? The bigger files took way too long compared to the rest of them. Anyway it will be better if we can have some of the compiling options available to choose so we can adjust the process to the power of our machines too.


Yes i've seen the ignore alpha mask option, very nice now we can do all sorts of cool buttons! I haven't tried yet but animated gifs are supported?
Anyway i was thinking that would be cool if we could have more configurable options for the ${status} formatting for the future, i'd like to be able to customize everything that's been displayed if possible. (i don't know what's in the todo list so i just throw it out there)

 

It does take some time to compile large files. There is a n-squared element to the manner changes are identified. I am testing a few changes today, hopefully we can cut that time down. Compiler options will be coming. One that might help a lot additional threads (assuming there is enough ram) each thread will process an individual file.

I thought about animating GIF's before, my main concern was defining how they animate. For example if you use the gif as a mouse over image for a button does it loop or play to the end and wait. Can we control the speed and so on. I will add it to the list for further consideration latter.

The main problem with the status line is that its conditional. So different things need to be shown at different times. There are a few conditions that come into play for that. I will give it further thought, we might be able to let you specify the way it looks for each state in the form. That way ${status} will show that configuration.

 

Oh nvm about the gifs, just add html5 that will sort gifs too! Now that i'm thinking about it it might be possible to add them on your current html 3.2. I'll test that later and i'll let you know.

 

Tried, it works we just need an option to remove the scrolling bar and you can already use gifs, if you don't want to click on the html pannel add a transparent image layer on top of that.

 

You should be able to put the html in a label

<html><a href="mygifurl.gif" /></html>

that should work

 

Apparently not, maybe i'm doing something wrong though.

 

Nope it doesn't work.

 

We have moved all the tasks to Trello. So if you keen to see the future of the Project please visist

https://trello.com/b/qxZ8ye1M/release-manager#

 

OK,

I didn't have much time to test the 64bit version, but here goes..

When I start a new project, and load a saved template I get this error when I select save:

Can not save file: java.util.logging.Logger

Are we forced to start from scratch?

 

Oh bugger, sorry man. I added a lot more logging on in the HTML definition I accidentally didn't mark it as static, so save is trying to save the logging. I have fixed it will try push a small patch out ASAP.

 

After restarting it actually stops me from opening the project.

 

Hi Zubba,

Each time Release Manager successfully opens a project it creates a backup.

The backup file is rm.project.backup you should be able to copy it to rm.project I would suggest backing it up yourself too before you try.

I am pushing out a patch now that should fix the issue you had.

 

Hey, i'm just reporting that i've made another patch today of the same size of the other one and it took minutes instead of hours

I couldn't understand if it was some kind of error or what... i think we need a report at the end of the compiling so we can check out if everything was fine.

 

Hi HitsuSan,

I am designing a complete overhaul of the compile screen to give that sort of feedback, size of patch number of changes files and so on. Also investigating threads compilation. If I can get it to respect memory limitations while it compiles it will work.

This is my next big update. (Also some Launcher changes that I consider critical)

By the way I have been thinking or reworking the resources so that they are contained in a resources folder (the view will remain) bu it means we would be able to create directories in that folder put html in there that we can feed back into the html panels. Its a bit messier but has many advantages would also allow you to update them on disk and have the system update when you finished saving. (Also a changed flag will be used so we only upload the changed files not all, this way Launchers will only need to download effected items)

Any thoughts about this?

 

Oh I have made an enhancement now there is a .log file with any issues. If you have a problem please email that to me. You should have my email from when I emailed the update to 64bit instructions.

 

Apparently it was all fine but with no report i couldn't tell (where's the file in any case?)
Anyway yeah it sounds nice especially if we start using html for enhancing the loader, it would be way better.

Looking forward for the next update!