Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Some suggestions on how to keep test data separate from production data.

Discussion in 'Unity Analytics' started by mpinol, Apr 29, 2016.

  1. mpinol

    mpinol

    Joined:
    Jul 29, 2015
    Posts:
    317
    I thought it would be helpful to give some quick tips on how to keep test and production data separated using Analytics. Please let me know if you found this helpful, whether you have any questions, or whether you would like to see some more examples like this!

    Suggestion 1
    I believe the best way to keep your test data separate from your production data and keep the data on the same Dashboard would be to send a Custom Event like Analytics.CustomEvent("tester", new Dictionary { {"testUser", userId } }); anytime a test player opens the game.

    This would then allow us the ability to create two Custom Segments, one that only fills with users who send the testUser event and another that only fill with users who do not send the testUser event.





    NOTE: it is very important that 'ALL' is selected for the 'Test users' segment and that 'NONE' is selected for the 'Live Users' segment.



    Then finally on the Data Explorer you would be able to separate any metric or Custom Event based on these segments.





    Also, remember that when you are creating builds for your live/production users to remove the 'testUser' Custom Event!


    Suggestion 2
    Another way would be to use two separate Unity Analytics projects for one game, one project id would be used for your test builds and another for your live builds.

    PLEASE NOTE: Going about solving this issue this way is only really possible if your game is not using any of the other Unity Services. Changing the project id will disassociate your game from all other services it was previously using!

    All of your testing builds would be created while linked to one project id and then once you are ready to create an official release you would unlink the testing project id and re-link to a new or previously unused project id before creating any of the new builds.


    You can unlink a project id by clicking on services in the Services window and selecting Settings.



    Then click Unlink Project.



    Then you would either create a new project id or link an already created production project id.


    This will give you two separate Dashboards for your testing and live data however, the biggest downside to this approach is that any funnels, segments, or saved reports that were created will have to be recreated under the new project id and if you are also using any other Unity Services everything related to those services would have to be changed/updated/migrated as well.

    I hope this helps!
     
    Last edited: Apr 30, 2016
  2. mobidus

    mobidus

    Joined:
    Mar 22, 2016
    Posts:
    22
    This is very useful. Thanks for the write up
     
    mpinol likes this.
  3. Antistone

    Antistone

    Joined:
    Feb 22, 2014
    Posts:
    2,836
  4. Antistone

    Antistone

    Joined:
    Feb 22, 2014
    Posts:
    2,836
    I don't have much experience with the segment system yet, but if you have to make a "NONE" rule for Live Users, is that going to make it difficult to define subsegments that are limited to live users? Maybe it would be more flexible to send an event for all users with a parameter indicating whether they're a test user or not?
     
  5. ap-unity

    ap-unity

    Unity Technologies

    Joined:
    Aug 3, 2016
    Posts:
    1,519
    Hi @Antistone,

    Yes, that would probably be a better way to go. Maybe even have some code to detect if you are in a dev build to send the event automatically.

    Code (CSharp):
    1. Analytics.CustomEvent("DevBuild", new Dictionary<string, object> {
    2.     {"is_dev_user", Debug.isDebugBuild}
    3. });
    And as you said, that would allow you to create segments with only non-test users.
     
    Last edited: Apr 19, 2017
  6. pthurston

    pthurston

    Joined:
    May 2, 2016
    Posts:
    3
    I'm a bit confused as to how this works. Does the Live Users segment filter out all users that have sent a true 'DevBuild' event at some point in the past? Do they use the device id to determine this?

    My main concern is what happens if the same device sends a "DevBuild true" and "DevBuild false" at different times (say I'm testing dev builds most of the time on a device, but put a production build to test it before release)
     
  7. ap-unity

    ap-unity

    Unity Technologies

    Joined:
    Aug 3, 2016
    Posts:
    1,519
    Not exactly. They will just look for any user that has sent DevBuild false in the past. And the analytics system keeps track of users via a unique user id that is generated on the device and sent with each event.

    In that case the user would just end up in both segments.
     
  8. creasaur

    creasaur

    Joined:
    Sep 5, 2017
    Posts:
    4
    Hi @mpinol,

    I applied your suggestion 2. Although I unlinked the project id I'm still receiving events for that particular project id along with the new project id. Any idea on that?
     
  9. ap-unity

    ap-unity

    Unity Technologies

    Joined:
    Aug 3, 2016
    Posts:
    1,519
    @creasaur,

    Are you re-linking your project to a different ID? Can you confirm that the new project ID is listed in the Settings menu.

    Also, can you check what is listed in ProjectSettings/UnityConnectSetting.asset. (You will need to have text serialization enabled to read this file, though that is now the default setting in Unity.)
     
  10. creasaur

    creasaur

    Joined:
    Sep 5, 2017
    Posts:
    4
    @ap-unity,

    I've linked the project to a different id & the new project id is listed in the Settings menu.

    I couldn't read the UnityConnectSettings.asset though. I'll ask one of the developers to try it as well.