Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[RELEASED] Dlib FaceLandmark Detector

Discussion in 'Assets and Asset Store' started by EnoxSoftware, Jun 4, 2016.

  1. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Dlib FaceLandmark Detector
    https://assetstore.unity.com/packages/tools/integration/dlib-facelandmark-detector-64314



    dlib1.3.6_features.png
    dlib1.2.6_features.png


    Requires Unity2020.3.48f1 or higher.

    Works with Unity Cloud Build

    ChromeOS
    support
    iOS & Android support
    Windows10 UWP support
    WebGL support
    Win & Mac & Linux Standalone support
    Preview support in the Editor


    DlibFaceLandmarkDetector
    can ObjectDetection and ShapePrediction using Dlib19.7 C++ Library.


    Official Site | ExmpleCode | Android Demo WebGL Demo | Tutorial & Demo Video | Forum | API Reference


    Features:

    • You can detect frontal human faces and face landmark (68 points, 17points, 6points) in Texture2D, WebCamTexture and Image byte array. In addition, You can detect a different objects by changing trained data file.
    • ObjectDetector is made using the now classic Histogram of Oriented Gradients (HOG) feature combined with a linear classifier, an image pyramid, and sliding window detection scheme. You can train your own detector in addition to human faces detector. If you want to train your own detector, please refer to this page.
    • ShapePredictor is created by using dlib's implementation of the paper (One Millisecond Face Alignment with an Ensemble of Regression Trees by Vahid Kazemi and Josephine Sullivan, CVPR 2014). You can train your own models in addition to human face landmark model using dlib's machine learning tools. If you want to train your own models, please refer to this page.
    • Advanced samples using “OpenCV for Unity” are Included. (The execution of this samples are required “OpenCV for Unity”.)
    • By utilizing the VisualScripting With DlibFaceLandmarkDetector Example, you can leverage all the methods available in DlibFaceLandmarkDetector within the Unity’s Visual Scripting development environment..VisualScripting With DlibFaceLandmarkDetector Example (GitHub)

    Examples:

    • Texture2DExample
    • WebCamTextureExample
    • CatDetectionExample
    Advanced Examples(require OpenCV for Unity):
    • Texture2DToMatExample
    • WebCamTextureToMatHelperExample
    • VideoCaptureExample
    • ARHeadExample
    • VideoCaptureARHeadExample
    • FrameOptimizationExample

    ExampleCode using Dlib FaceLandmark Detector is available.


    DlibFaceLandmarkDetector uses Dlib under Boost Software License ; see Third-Party Notices.txt file in package for details.
    The Shape Predictor model files included with this asset are available for commercial use.


    System Requirements:

    Build Win Standalone & Preview Editor : Windows8 or later
    Build Mac Standalone & Preview Editor : OSX 10.13 or later
    Build Linux Standalone & Preview Editor : Ubuntu18.04 or later
    Build Android : API level 21 or later
    Build iOS : iOS Version 11.0 or later

    More Info >>


    Release Notes:
    1.3.7

    [Common]Changed the minimum supported version to Unity2020.3.48f1.
    [WebGL]Added support for "WebAssembly 2023".
    [iOS]Changed “Target minimum iOS Version” to 11.0.
    1.3.6
    [WebGL]Added a plugin file with threads and simd enabled for the WebGL platform. This update removes support for the WebGL platform in Unity 2021.1 and below. (Select MenuItem[Tools/Dlib FaceLandmark Detector/Open Setup Tools/WebGL Settings])
    1.3.5
    [Windows]Added Support for ARM64.
    [WebGL]Added Unity2023.2 or later support.
    [Lumin]Removed Lumin platform support (for MagicLeapOne).
    [Common]Add a button to SetupTools to automatically add scenes under the "Examples" folder to "Scenes In Build".
    1.3.4
    [Common]Changed the setup procedure to use the SetupToolsWindow.
    [Common]Change the namespase under "DlibFaceLandmarkDetector/Editor" folder from "DlibFaceLandmarkDetector" to "DlibFaceLandmarkDetector.Editor".
    [Common]Added "DlibFaceLandmarkDetector" folder under "StreamingAssets" folder.
    [Common]Added function to automatically move the StreamingAssets folder.
    [WebGL]Added Unity2022.2 or later support.
    1.3.3
    [Android]Added Support for ChromeOS (x86 and x86_64 architectures).
    1.3.2
    [Common]Added Assembly Definitions.
    1.3.1
    [Common]Fixed a small issue.
    1.3.0
    [UWP]Added ARM64 Architecture.
    1.2.9
    [Common]Added optimization code using NativeArray class. ( require PlayerSettings.allowUnsafeCode flag, "DLIB_USE_UNSAFE_CODE" ScriptingDefineSymbol and Unity2018.2 or later. )
    [Common]Added support for Unicode file path ( objectDetectorFilePath and shapePredictorFilePath ).
    [Common]Added ImageOptimizationHelper to ARHeadWebCamTextureExample.
    [Common]Added some converter methods to OpenCVForUnityUtils.cs.
    1.2.8
    [Lumin]Added the code for MagicLeap.
    1.2.7
    [WebGL]Added Unity2019.1 or later support.
    1.2.6
    [Common]Added “sp_human_face_17.dat”, “sp_human_face_17_mobile.dat” and “sp_human_face_6.dat”.
    [Common]Changed the training dataset of Shape Predictor model. Since the training dataset consists of Flickr CC0 licensed images, the Shape Predictor model files are available for commercial use.
    [Common]Added BenchmarkExample.
    1.2.5
    [Common]Re-assined namespace.
    [Common]Support for OpenCVforUnity2.3.3 or later.
    1.2.4
    [macOS]Removed 32bit architecture (i386) from dlibfacelandmarkdetector.bundle.
    1.2.3
    [Android,UWP]Fixed Utils.setDebugMode() method on the IL2CPP backend.
    1.2.2
    [iOS]Added a function to automatically remove the simulator architecture(i386,x86_64) at build time.
    [Common] Improved DlibFaceLandmarkDetectorMenuItem.setPluginImportSettings() method.
    [Common]Updated to WebCamTextureToMatHelper.cs v1.0.9.
    [Common]Added support for Utils.setDebugMode() method on all platforms.
    1.2.1
    [Common]Updated to WebCamTextureToMatHelper.cs v1.0.8.
    [Common]Updated to LowPassPointsFilter v1.0.1. Updated to KFPointsFilter v1.0.2. Updated to OFPointsFilter v1.0.2.
    [Common] Added updateMipmaps and makeNoLongerReadable flag to DrawDetectResult () and DrawDetectLandmarkResult() method.
    [Common]Fixed Utils.getFilePathAsync() method.(Changed #if UNITY_2017 && UNITY_2017_1_OR_NEWER to #if UNITY_2017_1_OR_NEWER.)
    1.2.0
    [Common]Updated to WebCamTextureToMatHelper.cs v1.0.7.
    [Common]Fixed WebCamTextureExample and OpenCVForUnityUtils.cs.
    [Common]Added NoiseFilterVideoCaptureExample and NoiseFilterWebCamTextureExample.
    [Commo]Added useLowPassFilter option to ARHeadVideoCaptureExample and ARHeadWebCamTextureExample.
    [Common]Added throwException flag to Utils.setDebugMode() method.
    [Common]Added drawIndexNumbers flag to DrawFaceLandmark() method.
    1.1.9
    [Android]Added arm64-v8a Architecture.
    1.1.8
    [Common]Updated WebCamTextureExample.(support Portrait ScreenOrientation)
    [Common]Updated to WebCamTextureToMatHelper.cs v1.0.4.
    1.1.7
    [Common]Updated “human_face_68_sp.dat” and “human_face_68_sp_for_mobile.dat”.
    1.1.6
    [Common]Updated to dlib19.7.
    [Common]Updated to WebCamTextureToMatHelper.cs v1.0.3.
    [Common]Updated “human_face_68_sp.dat” and “human_face_68_sp_for_mobile.dat”.
    1.1.5
    [Common]Switched to the shape predictor file trained using new datasets.
    1.1.4
    [Common]Updated WebCamTextureToMatHelper.cs v1.0.2
    [Common]Improved Utils.getFilePathAsync().
    1.1.3
    [Common]Fixed to improve the pose estimation performance.
    [Common] Changed DetectLandmarkArray (int left, int top, int width, int height) to DetectLandmarkArray (double left, double top, double width, double height).
    [WebGL]Fixed Utils.getFilePathAsync() method.
    1.1.2
    [Common]Updated WebCamTextureToMatHelper.cs and OptimizationWebCamTextureToMatHelper.cs(Changed several method names.).
    [Common]Changed the Example name.
    1.1.1
    [Common]Improved Utils.getFilePath() and Utils.getFilePathAsync().
    1.1.0
    [Win][Mac][Linux][UWP]Added the native plugin file enabled SSE4 or AVX compiler option.
    1.0.9
    [WebGL]Added WebGL Plugin for Unity5.6.
    1.0.8
    [Common]Changed the name of asset project.("Sample" to "Example")
    [Common]Fixed VideoCaptureARExample and WebCamTextureARExample.
    1.0.7
    [Common]Fixed WebCamTextureToMatHelper.cs.(flipVertical and flipHorizontal flag)
    1.0.6
    [Common]Fixed OpenCVForUnityMenuItem.cs.(No valid name for platform: 11 Error)
    [Common]Added OptimizationWebCamTextureToMatHelper.cs.
    1.0.5
    [Common]Fixed WebCamTextureToMatHelper class.
    [Common]Added Utils.getVersion().
    [Common]Fixed Utils.getFilePathAsync().
    1.0.4
    [Common]Updated shape_predictor_68_face_landmarks_for_mobile.dat.
    1.0.3
    [WebGL]Added WebGL(beta) support.(Unity5.3 or later)
    [Common]Fixed missing script error.(WebCamTextureToMatHelper.cs)
    [Common]Added shape_predictor_68_face_landmarks_for_mobile.dat.
    1.0.2
    [Common]Improved WebCamTextureHelper class.
    1.0.1
    [Common]Added OptimizationSample.
    [Common]Added DetectRectDetection() method.
    1.0.0
    [Common]Initial Commit
     
    Last edited: Jan 18, 2024
  2. lucasmd

    lucasmd

    Joined:
    Apr 3, 2016
    Posts:
    6
    Hi,

    Hope you are doing well !

    Is it possible to get it before the review ? I already purchased OpenCV Modules from Enox

    Cheers,
    L.
     
    theolagendijk likes this.
  3. theolagendijk

    theolagendijk

    Joined:
    Nov 12, 2014
    Posts:
    117
    Hi EnoxSoftware,

    I bought your "OpenCV for Unity" and I'm very impressed with it. I would love to get "Dlib FaceLandmark Detector" too. Any chance that I can get a sneak preview ahead of the pending review?

    Cheers!
     
  4. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Unfortunately, 'Dlib FaceLandmark Detector' has been declined from AssetStore.
    I have submitted a fixed version to the asset store again. Please wait a while.
     
  5. theolagendijk

    theolagendijk

    Joined:
    Nov 12, 2014
    Posts:
    117
    Too bad. Good luck with the update!
     
  6. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Dlib FaceLandmark Detector v1.0.0 is now available.

    Version changes
    1.0.0
    Initial version
     
    theolagendijk likes this.
  7. Reaper93

    Reaper93

    Joined:
    Mar 1, 2016
    Posts:
    2
    Hello guys.I've got the newest version of both Open CV and D-lib .Great work. How can i improve head tracking so 3d model would move more smoothly and increase the framerate?
    I do not have experience working with head tracking and augmentation.
    I would appreciate if you will help me with that)
     
  8. qooqinc

    qooqinc

    Joined:
    Jul 19, 2016
    Posts:
    1
    Hello, I was experimenting with the AR Sample that uses OpenCV + D-lib but i noticed that there seems to be a bug with the input mat.

    When passing the image to the faceLandmarkDetector:

    Mat rgbaMat = extractSubMat (webCamTextureToMatHelper.GetMat ());

    OpenCVForUnityUtils.SetImage (faceLandmarkDetector, rgbaMat);

    List<UnityEngine.Rect> detectResult = faceLandmarkDetector.Detect ();
    The detection doesn't work if i reduce the number of cols, but it does if i reduce the number of rows. This is quite strange since all I'm doing on the "extractSubMat" method is:

    return originalMat.submat(newRange(0,originalMat.rows()),newRange(0,originalMat.cols()-100));
    The previous AR sample version that doesn't use d-lib used to work with this code. I need to be able to resize the input mat before the detection arbitrarily so that I can optimize the time it takes to search for a face while still showing to the user a high definition video input.
     
  9. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    The steps on the next page might be effective.
    Resize Frame
    Skip frame
    http://www.learnopencv.com/speeding-up-dlib-facial-landmark-detector/
     
  10. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Pixel data in "FaceLandmarkDetector.SetImage()" method must be continuous.In other words, ”Mat.isContinuous ()” must be true.I plan to add a document in the next version up.

    Code (CSharp):
    1.            Mat imgMat = new Mat (imgTexture.height, imgTexture.width, CvType.CV_8UC4);
    2.  
    3.             OpenCVForUnity.Utils.texture2DToMat (imgTexture, imgMat);
    4.             Debug.Log ("imgMat.ToString " + imgMat.ToString ());
    5.  
    6.             Mat subMat = imgMat.submat (new Range (0, imgMat.rows ()), new Range (0, imgMat.cols () - 100));
    7.  
    8.             Debug.Log ("subMat.isContinuous() " + subMat.isContinuous ());
    9.  
    10.             Mat copyMat = new Mat (subMat.rows (), subMat.cols (), CvType.CV_8UC4);
    11.  
    12.             Debug.Log ("copyMat.isContinuous() " + copyMat.isContinuous ());
    13.  
    14.             subMat.copyTo (copyMat);
    15.  
    16.  
    17.             FaceLandmarkDetector faceLandmarkDetector = new FaceLandmarkDetector (DlibFaceLandmarkDetector.Utils.getFilePath ("shape_predictor_68_face_landmarks.dat"));
    18.  
    19.             OpenCVForUnityUtils.SetImage (faceLandmarkDetector, copyMat);
    20.      
    21.  
    22.             List<UnityEngine.Rect> detectResult = faceLandmarkDetector.Detect ();
    23.          
    24.             foreach (var rect in detectResult) {
    25.                 Debug.Log ("face : " + rect);
    26.              
    27.                 OpenCVForUnityUtils.DrawFaceRect (copyMat, rect, new Scalar (255, 0, 0, 255), 2);
    28.              
    29.              
    30.                 List<Vector2> points = faceLandmarkDetector.DetectLandmark (rect);
    31.              
    32.                 Debug.Log ("face points count : " + points.Count);
    33.                 if (points.Count > 0) {
    34.                     OpenCVForUnityUtils.DrawFaceLandmark (copyMat, points, new Scalar (0, 255, 0, 255), 2);
    35.                  
    36.                 }
    37.             }
    38.  
    39.          
    40.             faceLandmarkDetector.Dispose ();
    41.  
    42.  
    43.             Texture2D texture = new Texture2D (copyMat.cols (), copyMat.rows (), TextureFormat.RGBA32, false);
    44.  
    45.             OpenCVForUnity.Utils.matToTexture2D (copyMat, texture);
    46.  
    47.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
     
  11. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    OptimizationSample (Resize Frame+Skip frame)
    http://www.learnopencv.com/speeding-up-dlib-facial-landmark-detector/

    Code (CSharp):
    1. using UnityEngine;
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using System;
    5. using System.Runtime.InteropServices;
    6.  
    7. #if UNITY_5_3 || UNITY_5_3_OR_NEWER
    8. using UnityEngine.SceneManagement;
    9. #endif
    10. using OpenCVForUnity;
    11. using DlibFaceLandmarkDetector;
    12.  
    13. namespace DlibFaceLandmarkDetectorSample
    14. {
    15.     /// <summary>
    16.     /// Face Landmark Detection from WebCamTextureToMat Sample.
    17.     /// </summary>
    18.     [RequireComponent(typeof(WebCamTextureToMatHelper))]
    19.     public class OptimizationSample : MonoBehaviour
    20.     {
    21.    
    22.         /// <summary>
    23.         /// The colors.
    24.         /// </summary>
    25.         Color32[] colors;
    26.  
    27.         /// <summary>
    28.         /// The texture.
    29.         /// </summary>
    30.         Texture2D texture;
    31.  
    32.         /// <summary>
    33.         /// The web cam texture to mat helper.
    34.         /// </summary>
    35.         WebCamTextureToMatHelper webCamTextureToMatHelper;
    36.  
    37.         /// <summary>
    38.         /// The face landmark detector.
    39.         /// </summary>
    40.         FaceLandmarkDetector faceLandmarkDetector;
    41.  
    42.         /// <summary>
    43.         /// The FAC e_ DOWNSAMPL e_ RATI.
    44.         /// </summary>
    45.         public int FACE_DOWNSAMPLE_RATIO = 2;
    46.  
    47.         /// <summary>
    48.         /// The SKI p_ FRAME.
    49.         /// </summary>
    50.         public int SKIP_FRAMES = 2;
    51.  
    52.         /// <summary>
    53.         /// The count.
    54.         /// </summary>
    55.         int count;
    56.  
    57.         /// <summary>
    58.         /// The rgba mat_small.
    59.         /// </summary>
    60.         Mat rgbaMat_small;
    61.  
    62.         /// <summary>
    63.         /// The detect result.
    64.         /// </summary>
    65.         List<UnityEngine.Rect> detectResult;
    66.  
    67.         // Use this for initialization
    68.         void Start ()
    69.         {
    70.             faceLandmarkDetector = new FaceLandmarkDetector (DlibFaceLandmarkDetector.Utils.getFilePath ("shape_predictor_68_face_landmarks.dat"));
    71.  
    72.             webCamTextureToMatHelper = gameObject.GetComponent<WebCamTextureToMatHelper> ();
    73.             webCamTextureToMatHelper.Init ();
    74.         }
    75.  
    76.         /// <summary>
    77.         /// Raises the web cam texture to mat helper inited event.
    78.         /// </summary>
    79.         public void OnWebCamTextureToMatHelperInited ()
    80.         {
    81.             Debug.Log ("OnWebCamTextureToMatHelperInited");
    82.  
    83.             Mat webCamTextureMat = webCamTextureToMatHelper.GetMat ();
    84.  
    85.             colors = new Color32[webCamTextureMat.cols () * webCamTextureMat.rows ()];
    86.             texture = new Texture2D (webCamTextureMat.cols (), webCamTextureMat.rows (), TextureFormat.RGBA32, false);
    87.  
    88.             rgbaMat_small = new Mat ();
    89.             detectResult = new List<UnityEngine.Rect> ();
    90.             count = 0;
    91.  
    92.             gameObject.transform.localScale = new Vector3 (webCamTextureMat.cols (), webCamTextureMat.rows (), 1);
    93.             Debug.Log ("Screen.width " + Screen.width + " Screen.height " + Screen.height + " Screen.orientation " + Screen.orientation);
    94.                                    
    95.             float width = gameObject.transform.localScale.x;
    96.             float height = gameObject.transform.localScale.y;
    97.                                    
    98.             float widthScale = (float)Screen.width / width;
    99.             float heightScale = (float)Screen.height / height;
    100.             if (widthScale < heightScale) {
    101.                 Camera.main.orthographicSize = (width * (float)Screen.height / (float)Screen.width) / 2;
    102.             } else {
    103.                 Camera.main.orthographicSize = height / 2;
    104.             }
    105.  
    106.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
    107.  
    108.         }
    109.  
    110.         /// <summary>
    111.         /// Raises the web cam texture to mat helper disposed event.
    112.         /// </summary>
    113.         public void OnWebCamTextureToMatHelperDisposed ()
    114.         {
    115.             Debug.Log ("OnWebCamTextureToMatHelperDisposed");
    116.  
    117.         }
    118.  
    119.         // Update is called once per frame
    120.         void Update ()
    121.         {
    122.  
    123.             if (webCamTextureToMatHelper.isPlaying () && webCamTextureToMatHelper.didUpdateThisFrame ()) {
    124.  
    125.                 Mat rgbaMat = webCamTextureToMatHelper.GetMat ();
    126.  
    127.                 // Resize image for face detection
    128.                 Imgproc.resize (rgbaMat, rgbaMat_small, new Size (), 1.0 / FACE_DOWNSAMPLE_RATIO, 1.0 / FACE_DOWNSAMPLE_RATIO, Imgproc.INTER_LINEAR);
    129.  
    130.  
    131.                 OpenCVForUnityUtils.SetImage (faceLandmarkDetector, rgbaMat_small);
    132.  
    133.  
    134.                 // Detect faces on resize image
    135.                 if (count % SKIP_FRAMES == 0) {
    136.                     detectResult = faceLandmarkDetector.Detect ();
    137.                 }
    138.                
    139.                 foreach (var rect in detectResult) {
    140.  
    141.                     List<Vector2> points = faceLandmarkDetector.DetectLandmark (rect);
    142.  
    143.                     if (points.Count > 0) {
    144.                         List<Vector2> originalPoints = new List<Vector2> (points.Count);
    145.                         foreach (var point in points) {
    146.                             originalPoints.Add(new Vector2(point.x * FACE_DOWNSAMPLE_RATIO, point.y * FACE_DOWNSAMPLE_RATIO));
    147.                         }
    148.  
    149.                         OpenCVForUnityUtils.DrawFaceLandmark (rgbaMat, originalPoints, new Scalar (0, 255, 0, 255), 2);
    150.                     }
    151.  
    152.                     UnityEngine.Rect originalRect = new UnityEngine.Rect(rect.x * FACE_DOWNSAMPLE_RATIO, rect.y * FACE_DOWNSAMPLE_RATIO, rect.width * FACE_DOWNSAMPLE_RATIO, rect.height * FACE_DOWNSAMPLE_RATIO);
    153.                     OpenCVForUnityUtils.DrawFaceRect (rgbaMat, originalRect, new Scalar (255, 0, 0, 255), 2);
    154.                 }
    155.  
    156.                 Imgproc.putText (rgbaMat, "Original: (" + rgbaMat.width () + "," + rgbaMat.height () + ") DownScale; (" + rgbaMat_small.width () + "," + rgbaMat_small.height () + ") SkipFrames: " + SKIP_FRAMES, new Point (5, rgbaMat.rows () - 10), Core.FONT_HERSHEY_SIMPLEX, 1.0, new Scalar (255, 255, 255, 255), 2, Imgproc.LINE_AA, false);
    157.  
    158.                 OpenCVForUnity.Utils.matToTexture2D (rgbaMat, texture, colors);
    159.  
    160.                 count++;
    161.             }
    162.  
    163.         }
    164.    
    165.         /// <summary>
    166.         /// Raises the disable event.
    167.         /// </summary>
    168.         void OnDisable ()
    169.         {
    170.             webCamTextureToMatHelper.Dispose ();
    171.  
    172.             faceLandmarkDetector.Dispose ();
    173.         }
    174.  
    175.         /// <summary>
    176.         /// Raises the back button event.
    177.         /// </summary>
    178.         public void OnBackButton ()
    179.         {
    180.             #if UNITY_5_3 || UNITY_5_3_OR_NEWER
    181.             SceneManager.LoadScene ("DlibFaceLandmarkDetectorSample");
    182.             #else
    183.             Application.LoadLevel ("DlibFaceLandmarkDetectorSample");
    184.             #endif
    185.         }
    186.  
    187.         /// <summary>
    188.         /// Raises the play button event.
    189.         /// </summary>
    190.         public void OnPlayButton ()
    191.         {
    192.             webCamTextureToMatHelper.Play ();
    193.         }
    194.  
    195.         /// <summary>
    196.         /// Raises the pause button event.
    197.         /// </summary>
    198.         public void OnPauseButton ()
    199.         {
    200.             webCamTextureToMatHelper.Pause ();
    201.         }
    202.  
    203.         /// <summary>
    204.         /// Raises the stop button event.
    205.         /// </summary>
    206.         public void OnStopButton ()
    207.         {
    208.             webCamTextureToMatHelper.Stop ();
    209.         }
    210.  
    211.         /// <summary>
    212.         /// Raises the change camera button event.
    213.         /// </summary>
    214.         public void OnChangeCameraButton ()
    215.         {
    216.             webCamTextureToMatHelper.Init (null, webCamTextureToMatHelper.requestWidth, webCamTextureToMatHelper.requestHeight, !webCamTextureToMatHelper.requestIsFrontFacing);
    217.         }
    218.        
    219.     }
    220. }
     
  12. davidosullivan

    davidosullivan

    Joined:
    Jun 9, 2015
    Posts:
    387
    Hopefully someone could clarify a couple of things for me.
    First of all its not very clear if this is a standalone asset or if OpenCV for Unity is also requiired, if so its gonna be too expensive for me!
    Second I see the demo App is 152M, this would make it about 120M too big for me! What would you say is the minimum number of megabytes this asset would add to a build if all the examples were stripped out and it just ran face detection a bit like the one in the post above?
     
  13. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    First
    You can use all the features of ”Dlib FaceLandmark Detector” without "OpenCV for Unity".

    Second
    Because Dlib's default trained file(shape_predictor_68_face_landmarks.dat) is 100M, Demo App size is over 150M.
    You can reduce the app size if you use smaller files.
     
  14. davidosullivan

    davidosullivan

    Joined:
    Jun 9, 2015
    Posts:
    387
    Thanks for the info there... Do you know if the .dat file is basically in XML format? When I did my thing I was able to make the files massively smaller by stripping out all the XML stuff and just storing arrays of numbers which could be reconstructed later when they were read... Maybe yours does something similar- i.e. reads the file into an array, and that array itself could just be saved and be much smaller?

    Also do you know if there are any trained files available that have a lot less landmarks, and if not how difficult is it to 'train' one?
     
  15. Urajin

    Urajin

    Joined:
    Jul 21, 2016
    Posts:
    3
    Hello, I just wanted to thank you for creating such a useful tool, it has saved me a lot of time on my current project. I would like to know what changes were made in the newest update: Version: 1.0.1 (Aug 02, 2016), is there a link you can direct me to for details about plugin updates that you've made? thanks again!
     
  16. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Unfortunately, I couldn't find another trained file.

    Also, ”frontal_cat_face.svm" and "shape_predictor_68_cat_face_landmarks.dat" are the files that I trained.
    Please refer to Read.pdf(https://github.com/EnoxSoftware/DlibFaceLandmarkDetector/blob/master/ReadMe.pdf) for detail.
     
  17. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Dlib FaceLandmark Detector v1.0.1 is now available.

    Version changes
    1.0.1
    [Common]Added OptimizationSample.
    [Common]Added DetectRectDetection() method. You are able to obtain detailed data of the detected face.(detection_confidence, weight_index)
     
  18. davidosullivan

    davidosullivan

    Joined:
    Jun 9, 2015
    Posts:
    387
    Dont suppose 'OptimizationSample' is a sample that optimizes file size?! :)
    I cant afford to buy it to look and I there is no point in me buying it unless the file size is MUCH smaller LOL...

    Let me know if you have done something on that front :)
     
  19. Rowlan

    Rowlan

    Joined:
    Aug 4, 2016
    Posts:
    4,178
    I just tried the Android demo. What is the cat face detection example in the demo supposed to do? All you show is some cat image with lines on it. That could also be a static image. Is there a live demo available? From what I've learned cat detection isn't easy and the OpenCV demos don't work. That's why I'd rather try on a real example before I buy your asset.
     
  20. Urajin

    Urajin

    Joined:
    Jul 21, 2016
    Posts:
    3
    Hi, I was wondering can you post code on how to change the web cam size? I'm working with the dlib+opencv webcam AR example. I'd like the web cam view to be a smaller, and it would really help if there was an easy way to resize it, is this possible? thanks
     
  21. shawhu

    shawhu

    Joined:
    Mar 11, 2016
    Posts:
    17
    Just recently bought Dlib face landmark detector, great work!
    And currently I only have one problem.

    In iOS, using webcamtexturesample, it works great in landscape mode but not in portrait mode.
    It can detect faces in any of 2 landscape modes but when the phone is in portrait mode, then no face can be detected.

    Also, I've tried to make the front facing camera default by change the line
    shouldUseFrontFacing = false to shouldUseFrontFacing = true
    But it doesn't seems to work either.

    I hope you can help me on this.

    Thanks!!

    Harry
     
  22. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    ”frontal_cat_face.svm" and "shape_predictor_68_cat_face_landmarks.dat" are the files that I trained. Because this file is not good detection accuracy, It is not work well on WebCamTexture.
     
  23. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    You can change the size of WebCamTexture in the Inspector.
    facetracker_webcamhelper.PNG
     
  24. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    WebCamTextureSample can detect landscape direction only.
    写真 2016-08-10 17 20 50.png
    But, WebCamTextureToMatSample can detect landscape and portrait direction using OpenCV's method.
     
  25. eddyvor

    eddyvor

    Joined:
    Jul 12, 2016
    Posts:
    2
    just bought the package and run a sample scene file. Everything works great except it's
    failed to load shape_predictor_68_face_landmarks.dat
    Why is that? I checked and the file is there. and point the the correct file path. Anyone have t his problem?
     
  26. HereVR

    HereVR

    Joined:
    Jul 4, 2014
    Posts:
    218
    Hello, I sent you a mail and a PM, too.
     
  27. Cushmily

    Cushmily

    Joined:
    Mar 25, 2015
    Posts:
    10
    It this the only way solve that issue?
     
  28. shawhu

    shawhu

    Joined:
    Mar 11, 2016
    Posts:
    17
    Well, I managed to figure it out how to detect faces in portrait mode. You just need to rotate the Color32[] 90 degrees, detect the faces, draw boxes, rotate it back and that's it. works for me.

    Here are 2 rotating little functions that I wrote.
    public Color32[] RotateCW( Color32[] input, int width, int height)
    {
    Color32[] result = new Color32[input.Length];
    int i = 0;
    for ( int x = 0; x < width; x++)
    {
    for ( int y = height - 1; y >= 0; y--)
    {
    result = input[x + y * width];
    i++;
    }
    }
    return result;
    }
    public Color32[] RotateCCW( Color32[] input, int height, int width)
    {
    Color32[] result = new Color32[input.Length];
    int i = 0;
    for ( int x = height - 1; x >= 0; x--)
    {
    for ( int y = 0; y < width; y++)
    {
    result = input[x + y * height];
    i++;
    }
    }
    return result;
    }

    also, to make it work, we need to set the current width and height according to the rotated color32[]
     
    Last edited: Aug 11, 2016
  29. louis2009

    louis2009

    Joined:
    Jun 4, 2013
    Posts:
    9
    does it support more than one player in AR sample?
     
    Joy0023 likes this.
  30. Robin_Ripper

    Robin_Ripper

    Joined:
    May 14, 2016
    Posts:
    1
    What exactly is the difference between this asset and the free face tracker asset?
     
  31. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    ARSample support one player.
     
  32. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    DlibFaceLandmarkDetector's face landmark detection accuracy and stability is higher than FaceTrackerSample's.
     
  33. nadinen

    nadinen

    Joined:
    Jul 12, 2016
    Posts:
    2
    Is there a way to use PORTRAIT detection with WebCamTextureSample ?
    if not what could we modify to use it ?

    Thx
     
  34. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
  35. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    How does one rescale the AR objects in the included example for portrait mode?
     
  36. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    it looks like you are missing a index for result

    are you feeding in rgbaMat.width() or another width, height?

     
  37. NeedNap

    NeedNap

    Joined:
    Nov 1, 2012
    Posts:
    22
    Hi,
    I am using the library to detect 2d faces and it works great.
    I don't know how I can map a 3d object that overlaps to the detected face, taken by the webcam.
    There is a sample in OpenCV FaceTrackerSample library but it uses a different approach from "Dlib face landmark detector" library, so I can't migrate the code in my project.

    Can someone help me, please?
     
  38. pzstar2014

    pzstar2014

    Joined:
    Aug 22, 2014
    Posts:
    1
    Hi
    I am using dlib to detect faces, then draw a cartoon mouth(texture) on their faces.
    If there is just one person everything is OK. but if there are 2 people, the mouth will flickering between 2 people. So I am wandering is something like "ID" so I can distinguish which one is which one?
     
  39. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    I think that RectangleTracker class is useful. This class has been used to track the detected face in the FaceSwapperSample.
    https://github.com/EnoxSoftware/FaceSwapperSample/tree/master/RectangleTracker/Scripts
     
  40. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    DlibFaceLandmarkDetectorWithOpenCVSample/WebCamTextureARSample is the same approach as FaceTrackerARSample.
    https://github.com/EnoxSoftware/Dli...ebCamTextureARSample/WebCamTextureARSample.cs
     
  41. MoonCrowned

    MoonCrowned

    Joined:
    Oct 25, 2015
    Posts:
    3

    Hello! Just bought your plugins and playing with them, and I like them, thank you! :)

    I have a question about cats, thats very funny! How did you train the cats? Did you use your own cats photos, and place points on them manually? Or maybe you used the database with 10k cat's faces? I cannot find it in the internet, but in 2014 year it has been placed on the url 137.189.35.203/WebUI/CatDatabase/catData.html

    Maybe you have one? If yes - could you share it, please?
     
  42. NeedNap

    NeedNap

    Joined:
    Nov 1, 2012
    Posts:
    22
    Thank you Enox for your reply.
    The example you're referring works very well: I adjusted the code to manage multiple faces detection.

    To simulate mirror effect, I need to flip horizontally the webcam image, so I changed the line:

    bool flipHorizontal = true;
    gameObject.transform.localScale = new Vector3(webCamTextureMat.cols() * (flipHorizontal ? -1 : 1), webCamTextureMat.rows(), 1);
    ...
    float width = Mathf.Abs(gameObject.transform.localScale.x);
    float height = Mathf.Abs(gameObject.transform.localScale.y);
    ...


    adding a minus to the y value.
    This works on webcam image, but the 3D face object overlapped needs to be flipped accordingly. What can I do?

    gameObject.transform.localScale = new Vector3(webCamTextureMat.cols(), -webCamTextureMat.rows(), 1);

    adding a minus to the y value.
    This works on webcam image, but the 3D face object overlapped needs to flip concordly. What can I do?


    UPDATE

    I found the solution; I flip the camera using the following instruction:
    ARCamera.projectionMatrix = ARCamera.projectionMatrix * Matrix4x4.Scale(new Vector3(-1, 1, 1));

    and it works.

    UPDATE 2

    It doesn't work very well because the 3D face object is shown from the back (even if the orientation is ok)! What can I do to flip the 3D object in the right position?
     
    Last edited: Sep 17, 2016
  43. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    the included cat traindata is using 18 photos of cats from Internet image search. and I place points on them manually.

    train_objectdetector.PNG
    train_shapepredictor.PNG
     
  44. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Please try Flip Horizontal flag on Inspector.
    facetracker_webcamhelper.PNG
     
  45. NeedNap

    NeedNap

    Joined:
    Nov 1, 2012
    Posts:
    22
    It works, THANK YOU!!!!
    You are my hero
     
  46. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    For portrait mode, it seems the video image is always flipped. When flip horizontal is used, it's inverted vertically - and flip vertical it's inverted horizontally but in the wrong direction.
     
  47. MoonCrowned

    MoonCrowned

    Joined:
    Oct 25, 2015
    Posts:
    3
    Thank you for your answer! But i could not place any point in imglab, just rectangles (Left shift + mouse drag). I cant find any comments or tutorials, where you can place 68 points like on your second cat image. How did you place them?
     
  48. detournment_19

    detournment_19

    Joined:
    Jan 20, 2015
    Posts:
    22
    How would I go about distinguishing the individual facial features given the data returned from dLib? I'm getting a list of points, but I'd like to get a list of point for just eyes, or just mouth, etc. Is that possible with dLib? How about OpenCV?
     
  49. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Could you tell me about your test environment?
    build platform
    Unity version
    OpenCV for Unity version
     
  50. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,555
    Please look menubar "help".
    and
    http://stackoverflow.com/questions/37189615/create-an-xml-file-for-dlib-training

    C:\dlib-19\tools\imglab\build\Release>imglab ./cat/c
    at_training_landmarks.xml --parts "00 01 02 03 04 05 06 07 08 09 10 11 12 13 14
    15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
    42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 69 61 62 63 64 65 66 67"