Unity + SmartFox + Amazon EC2 = Win!!

Discussion in 'Unity Gossip' started by MetaZion, Mar 3, 2009.

  1. MetaZion

    MetaZion

    New Member

    Joined:
    Feb 21, 2009
    Messages:
    9
    I've been testing out Unity 3D ... with the goal of creating a light virtual world application. I've been using the SmartFox .NET/Unity API .. and it's been working great!! I have no prior networking type experience .. but I had a multiplayer game up and running in less than 24 hours.

    The only concern I had was the server side and being able to scale depending on traffic and space.

    Then I learned of Amazon EC2 (Elastic Compute Cloud) and it has been heaven sent ... I can now deploy Smart fox on EC2 and scale up or down dynamically at a lower cost than if I was using a dedicated server provider.

    If you haven't heard of Amazon EC2 .. check it out here http://aws.amazon.com/ec2/

    Also check out Smart Fox http://www.smartfoxserver.com/
  2. marco_gamelix

    marco_gamelix

    New Member

    Joined:
    Nov 13, 2008
    Messages:
    56
    Hey congratulations!! I'm very interested in the topic. Indeed I have heard about Amazon EC2 and SmartFox before. One question I have, which is the benefit of using SmartFox? I have read you can create a server just defining a Unity client as host. I want to learn :)
  3. Thomas Lund

    Thomas Lund

    New Member

    Joined:
    Jan 18, 2008
    Messages:
    465
  4. drunknbass

    drunknbass

    New Member

    Joined:
    Dec 4, 2008
    Messages:
    52
    sorry for being dumb but i cant figure out what this is..
    it sounds like it is a computer/server you can use and only get charged for how much/long you use it. but i have no way to calculate how much id pay since i dont really know what to input..
    ill assume it runs for 24 hrs a day and 30 days a month? that seems safe
    . also do you have an ip or domain, api etc you use to connect to this server? does it function like a dedicated server?
  5. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,586
    there is a cost calculator for the EC2 system on the pricing page or one of the related pages, that will give you an idea of the costs.

    but it won't help you much. without having your server and some realworld testing metrics you can neither estimate the cpu consumption of your server nor the database queries and especially you can not estimate the traffic
  6. MetaZion

    MetaZion

    New Member

    Joined:
    Feb 21, 2009
    Messages:
    9
    While testing EC2 for one month non-stop 24/7

    My bill came out to be $75 .... but I'm sure once I start using lots of bandwidth that will easily jump up past $150 ... max $200.

    But that fine because with Godaddy I'm paying $200 for a dedicated server monthly when I don't even use $200 worth of service every month.

    In my situation Amazon's service is great because I'm only paying for what I use.
  7. drunknbass

    drunknbass

    New Member

    Joined:
    Dec 4, 2008
    Messages:
    52
    thx! thats what i wanted to hear. its a little scary to put a cc down on something and not know what itll cost even in a ballpark range.

    and does ec2 require you to also you amazon s3 service? it seems like it does but isnt clear..

    right now i have a shared hosting plan on host monster and have recently looked into a dedicated server i can run root on and its expensive.. this seems like a nice option.

    do they charge you after 30 days? im wondering if i sign up now with my cc when i will receive my first bill. the first month or so would be internal testing so itd be very limited bandwidth etc.
  8. Charles Hinshaw

    Charles Hinshaw

    Unity Technologies

    Joined:
    Feb 6, 2008
    Messages:
    1,070
    It is part of the whole AWS thing. It may not be exactly what you are looking for -- so my advice is not to cancel anything you have going on just yet. Sign up for EC2 and play with some of their ready-to-go images. If you like it, and it clicks with you, you can look at how to make it meet more of your needs. If it doesn't work out, you'll be out a couple of dollars, max.

    Back to the original post -- yeah, this is a really good thing. Unity + SFS + EC2 has to be the lowest cost of entry for prototyping something and being able to scale it up through testing.

    Semi-related, I would also be really interested in seeing how viable it is to use CloudFront for serving static assets for the game.
  9. drunknbass

    drunknbass

    New Member

    Joined:
    Dec 4, 2008
    Messages:
    52
    yea i just dont really understand EXACTLY what this amazon stuff is. maybe i should call them.. like is ec2 an actual dedicated server or a disk image running on a server that i have root access to?

    the amazon s3 seems easier to understand.. it seems as though its just remote storage, and priced pretty well.. they seem to charge by posts / gets instead of by bandwidth? and the docs on amazon says you add a ec2 image to your s3 so it seems s3 is required? even if it isnt it might be a good option.

    if somebody could write a quick jist of the amazon crap id be very grateful.
  10. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,586
    ec2 is neither
    Its a cloud computing system.
  11. MetaZion

    MetaZion

    New Member

    Joined:
    Feb 21, 2009
    Messages:
    9
    drunknbass ... I'm not server savvy or anything but the simplest way for me to describe ec2 is....

    It's an expandable dedicated system in the clouds... :eek:

    yeah.. sounds crazy... but think about the service name ec2 = Elastic Compute Cloud. The way I see it (and I may be wrong) ... it's like having a dedicated server with no default limits... and its pay as you go.

    You can remote connect to your ec2 instance just like any other server.. you can install as many instances as you want. So no more buying another hosting account from your hosting provider just because you need a linux system to test on. Just run a linux instance and bam... terminate the instance when you want.. only paying for what you used..

    So lets say you just used 10 hours... for a base linux system that's just $1.00!!! Not bad if you ask me.
  12. Charles Hinshaw

    Charles Hinshaw

    Unity Technologies

    Joined:
    Feb 6, 2008
    Messages:
    1,070
    To tie this back into the original thread -- there is an AMI (which is an image used by EC2) with SmartFoxServer PRO running on Linux that is ready to go. So, that addresses an easy way to get it running -- you just load up that instance. I personally haven't used their image -- but I've used SmartFoxServer and EC2 independently. My guess is that addressing clustering still needs to be done by building your own image. That is, ideally, how you would set things up -- so that scale based on need more-or-less real time and not be manually launching new images and trying to get everything working together. SFS has a white paper on this.

    Why this is cool here is that SFS has been made to play extremely well with Unity thanks to Thomas's hard work.
  13. thecreatrix

    thecreatrix

    New Member

    Joined:
    Dec 19, 2008
    Messages:
    94
  14. dingosmoov

    dingosmoov

    Member

    Joined:
    Jul 12, 2005
    Messages:
    548
    Would Unity and Smartfox also be compatible with other cloud computing solutions? Like Mosso?
  15. Charles Hinshaw

    Charles Hinshaw

    Unity Technologies

    Joined:
    Feb 6, 2008
    Messages:
    1,070
    So, Unity is just the client in this case -- it isn't really part of the equation. SmartFoxServer is pretty flexible with regards to what it can run on. You couldn't use the AMI for a non-Amazon service (so it isn't just there waiting for you) but there is no reason why it shouldn't work with other similar hosting options as well.

    To really take advantage of the whole game back-end as a service thing, you will want something like Terracotta running with SFS -- they have a white paper on their web site with details. One of the really nice things is that a Terracotta cluster runs beautifully on EC2. In fact, if I recall, one of the Terracotta guys blogged about running on EC2 as soon as they let him in the original beta.
  16. Thomas Lund

    Thomas Lund

    New Member

    Joined:
    Jan 18, 2008
    Messages:
    465
    SmartFoxServer being a packaged java application server makes it super flexible in deployment platforms as Charles already mentioned. Anywhere you can get java to run, you can theoretically get a SFS server running. That should cover 99%+ of the server platforms out there in the wild.

    Now that doesnt necessarily mean that its cloud compatible with every cloud system out there. But Charles already summed it up - Terracotta being the key here.
  17. kroll

    kroll

    New Member

    Joined:
    Jun 11, 2007
    Messages:
    17
    I have to say I'm pretty excited about SFS as well, and Thomas really did a great job of making SFS "Just work" with Unity. Obviously, SFS is not for everyone: It doesn't "solve MMO" just like that, but it *does* do a super job of solving a particular subset of MMO style games. I have my own theories about where the limits are, but will need to measure some more before I know.

    Its a super way to get started on an MMO though.
  18. dingosmoov

    dingosmoov

    Member

    Joined:
    Jul 12, 2005
    Messages:
    548
    this may seem like a really basic question for all you gurus. But I was wondering...Does anyone have an architecture drawing, explaining how SmartFox Unity and the Amazon cloud would work together.

    I know it's a simplistic request, but it would really help.

    Thanks.
  19. Thomas Lund

    Thomas Lund

    New Member

    Joined:
    Jan 18, 2008
    Messages:
    465
    I dont know of any - but try to poke the SFS guys on the SFS forums about it! I'm pretty sure there are more guys over there that have had the same need.

    And link back if you get it from someone :)
  20. bliprob

    bliprob

    Member

    Joined:
    May 13, 2007
    Messages:
    901
    Mosso doesn't let you run server-side software (aside from php, asp.net and mysql). SMS is a java aplicaiton you need to install, configure and run.

    EC2 and Mosso are very different clouds. EC2 is basically a very large bank of Xen virtual machines; when you turn on an "instance" you are booting a disk image (called an AMI) into a Xen virtual machine. You can then ssh into your virtual machine and apt-get whatever, but there are lots of pre-built AMIs with various software already installed, so often you won't need to do anything but configure. You can build your own AMIs and store them on S3, and you can have persistent storage for your instances on S3 as well. You have to manage all of this through command line tools. (Huge hint -- there's a firefox plugin that lets you do everything through a GUI. It's an absolute must-have.) Command-line tools means there's an API, so you can build your own controls to start and stop instances (look at http://tarzan-aws.com/).

    Mosso's cloud is much simpler -- a large bank of web servers (some linux, some windows) in front of a very large SAN, and behind a smart load balancer that knows how to direct the .php stuff to linux servers and .aspx stuff to Windows servers. You store your files on the SAN (via FTP). You have to use Mosso's name servers, because that's where the real magic happens. There's no API because there's nothing to manage; web requests go in, data goes out.

    I use both, but for different things. Mosso is my fantastically-scaling never-need-to-think-about web server. EC2 is my personal playground for when I'd like to have 50 servers for some task, like a large video encoding batch, or when I've got thousands of live users to stream video to.

    You can handle thousands of live SMS users on a single EC2 instance. (I've done over a thousand concurrent live chat and video streams on a single EC2 "large" instance, running a Wowza Pro AMI.) I wouldn't worry about solving the > 1000 concurrent users problem until you've got a working game thats delivering >100 concurrent users.

    Scaling EC2 is no different than scaling a bunch of servers in your own rack. After all, they're just a bunch of (virtual) servers; you still need to do handle the hard stuff, like synchronization of data between servers. Terracotta is a solution for this (for Java-based server apps only). Terracotta handles synchronizing objects and data structures of java-based apps between servers. I've read about it working for SMS and Red5, and it seems to be the real deal, although I've never tried it myself.

    You could use Mosso as a Unity server if you polled a web-based API for updates, and stored the data in a MySQL backend. In fact, Mosso would be perfect for that. But most homemade MMOs I've heard of use socket servers (or even UDP, which you'll never get from Mosso) for the quickest update times.

    Mosso is $100 per month. I'm trying to find my Amazon bill for that large event... I can't find it, but as I recall it was super cheap. (A "small" server instance is $.10 per hour the server runs. Run it 24x7 for a month and your charge is $72, not including bandwidth charges. You can play with numbers here: http://calculator.s3.amazonaws.com/calc5.html)

    Also, Mosso now owns SliceHost, which is basically the same business as EC2 (virtual machines of various sizes) and the pricing is better. They don't have all the nifty pre-built AMIs, I don't know if there is an API, and I don't know how large their infrastructure is, but it's an option worth knowing about.

    At it simplest, you have [EC2 server running SMS] <--> [Unity client]. Just a server you turned on at EC2 running SMS, and the Unity client connects to it.

    Once you put Terracotta in the mix, I assume you'd have as many SMS servers as you'd like, plus another server instance that acts as a load-balancer. (If you so something simple like round-robin through a list of running instances via PHP, you could avoid this.)

    Also, all you potential MMO guys take note, there has been one time I know of when Amazon shut down all of the EC2 instances briefly. But that's better uptime than the datacenter at one wilshire, now that I think about it.
  21. dingosmoov

    dingosmoov

    Member

    Joined:
    Jul 12, 2005
    Messages:
    548
    @Thomas Lund - thanks for all your input and hardwork!

    @bliprob - you da man! you provided all the information I needed in that last post. Thank You!
  22. chonli

    chonli

    New Member

    Joined:
    Mar 13, 2009
    Messages:
    19
  23. KyleHatch85

    KyleHatch85

    Member

    Joined:
    Dec 13, 2011
    Messages:
    59
    Old topic but i was wondering what current thoughts were for people who host their game.

    We're looking to host a title around 20mb to play online, the game would only need to connect to the server whenever an asset is saved, or a challenge is completed, so always on isn't necessary.

    What are people currently using, i hear various things about EC2, Go Grid, 100tb, but no sort of real answer. Plus finding estimates for server costs seems difficult, i've tried using the EC2 calculator but having trouble getting accurate(ish) numbers to place in values.

    An idea of cost per user per day/week/month from anyone for any sized game would be useful.

    Thanks.
  24. npsf3000

    npsf3000

    Member

    Joined:
    Sep 19, 2010
    Messages:
    3,725
    It really depends on what your requirements are. Personally I find a lot of people, like the OP did [in 09], jump to the conclusion that the cloud is 'scalable' and therefor cheap. However when you do the maths you'll find that in a lot of cases it's dramatically overpriced and under-performs for what you want. Real world case see this post. Proton swapped and AFAIK hasn't looked back.

    That said I do use cloud for some multiplayer stuff - I particularly like using it for my DB stuff.

    Feel free to shoot me a PM to discuss your specific needs, and/or start up your own thread in the multi-player section.
    Last edited: Dec 13, 2011
  25. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,586
    Correct

    Additionally the Amazon Cloud is extremly overloaded and deadly for realtime applications. For games I would go with Rackspace if you want cloud or use real dedicated 4 / 8 core machines hosted within the same network and 'cluster' it yourself.
    Generally game server hosters (who offer servers for shooters etc) often have extremely competitive offers for full single machine dedicated hostings with full quad core i5 / i7 machiens on plenty ram and their low latency lines in the $100 range and alike per month, a thing that on a cloud service costs you easily multiple times that.

    the cloud benefit, that you can make use of the 'per demand' is a thing that only works out if you can really accept the impact of the multi server spread (for an mmo pottentially a major problem due to the data syncronization over the different VMs - SFS recommends TerraCotta which as you will work with it is strong but its not trivial to do it performantly so you don't lose more through the cloud than you gain)
  26. npsf3000

    npsf3000

    Member

    Joined:
    Sep 19, 2010
    Messages:
    3,725
    And dreamora brings in the notions of consumer vs server hardware, multiple cloud techs, data sync/latency/transport problems [the bane of computing - literally everywhere], hyrbid could/dedicated etc.

    I was trying to keep it simple.

    Who wants to talk about MANAGEMENT? Hehehe!
  27. Dreamora

    Dreamora

    Member

    Joined:
    Apr 5, 2008
    Messages:
    26,586
    I don't care about management.

    Thats technical basics, which you are either aware of or it does not matter if your server and technology can scale because you will not have a software architecture that will do it either and you won't have the player to need it pretty surely either (cause if you develop with that little forsight that you can ignore trivial technical limitations that will break your whole projects neck, the rest of the project will unlikely be driven more professional and with more foresight)
    Last edited: Dec 13, 2011
  28. Cascho01

    Cascho01

    Member

    Joined:
    Mar 19, 2010
    Messages:
    469
    A bit offtopic but maybe right here:
    In general, what about a powerful server running a hughe Unity-scene and streaming up the images directly to a mobile platform in realtime?
    Userinput would still be transmitted via TCP/UDP.

    This way the mobile does not require strong graphic hardware but only a little "reader".
    Think about increasing upload and downloadrates in the future ( LTE....)

    Are there any further thoughts about this?
  29. npsf3000

    npsf3000

    Member

    Joined:
    Sep 19, 2010
    Messages:
    3,725
    That would require:

    A good stable low latency link.
    Lots of bandwidth/quota.

    Neither of which mobile is well known for.
  30. niosop2

    niosop2

    Member

    Joined:
    Jul 23, 2009
    Messages:
    1,015
    Check out OnLive as this is exactly what they do. But they have designed a specialized video codec to send data, and dedicated hardware to do the encoding in realtime. They number they've come up with is 80ms RTT, any more than that and there is noticable lag between pressing a key and the game responding to that keypress.
  31. Cascho01

    Cascho01

    Member

    Joined:
    Mar 19, 2010
    Messages:
    469
    Very interesting, and what is RTT? (RenderTimeTransfer)

    So you mean its too slow?
  32. IsaactheDev

    IsaactheDev

    New Member

    Joined:
    Jul 22, 2013
    Messages:
    1
    Or you could use Google app Engine because all you need is a google account and it is free, you heard me right it is FREE!!!!!!!
  33. lmbarns

    lmbarns

    Member

    Joined:
    Jul 14, 2011
    Messages:
    1,246
    Have you checked out the photon networking package? It just works right out of the box now with their cloud as well.

    I downloaded the asset from the store, imported it, converted to android (using pro license), made a build, and loaded it on 2 tablets that were connected through the cloud and could interact with each other. Crazy how easy it's become...
  34. Gigiwoo

    Gigiwoo

    Member

    Joined:
    Mar 16, 2011
    Messages:
    1,635