[RELEASED] OpenCV for Unity

I have a question, do you think it is possible making Vuforia marker detection and openCV hand detection together? I really would like make a project where I could play Tictactoe where I can control the game with my hand/fingers. I know it is possible with vuforia virtual buttons, but they dont work very well if you want a fast and a precise system.

 

I think that it is probably possible. However, I do not know whether the performance is good more than vuforia virtual buttons.

 

OpenCV for Unity
Released Version 2.0.5

Version changes
2.0.5
[Common]Added HOGDescriptorSample.

 

Hi,
the face detection script work well, i have to instantiate a gameobject in the center of the rectangle , i can get the center of the rectangle but i can't convert the center position (center.x, center.y ) they are exprimed in pixel how can i convert them to float so i can use them inside vector3 like this

Code (CSharp):

1. GameObject instance = Instantiate(cube, new Vector3(center.x,center.y,0), transform.rotation) as GameObject;

Thank you.

 

Hi,
I've been trying to get the videoWriter class to work on my Android app, when I looked around I found lots of mixed info whether it works on Android or not.

Is videoWriter implemented in the current version when building for Android or not? If not are there any plans to implement it?

 

Code (CSharp):

1.                public GameObject point3D;
2.  
3.                 // Use this for initialization
4.                 void Start ()
5.                 {
6.  
7.                         Texture2D imgTexture = Resources.Load ("lena") as Texture2D;
8.          
9.                         Mat imgMat = new Mat (imgTexture.height, imgTexture.width, CvType.CV_8UC4);
10.          
11.                         Utils.texture2DToMat (imgTexture, imgMat);
12.                         Debug.Log ("imgMat dst ToString " + imgMat.ToString ());
13.  
14.  
15.                         //CascadeClassifier cascade = new CascadeClassifier (Utils.getFilePath ("lbpcascade_frontalface.xml"));
16.                         CascadeClassifier cascade = new CascadeClassifier (Utils.getFilePath ("haarcascade_frontalface_alt.xml"));
17.                         if (cascade.empty ()) {
18.                                 Debug.LogError ("cascade file is not loaded.Please copy from “OpenCVForUnity/StreamingAssets/” to “Assets/StreamingAssets/” folder. ");
19.                         }
20.  
21.                         Mat grayMat = new Mat ();
22.                         Imgproc.cvtColor (imgMat, grayMat, Imgproc.COLOR_RGBA2GRAY);
23.                         Imgproc.equalizeHist (grayMat, grayMat);
24.  
25.  
26.                         MatOfRect faces = new MatOfRect ();
27.      
28.                         if (cascade != null)
29.                                 cascade.detectMultiScale (grayMat, faces, 1.1, 2, 2,
30.                                            new Size (20, 20), new Size ());
31.  
32.                         OpenCVForUnity.Rect[] rects = faces.toArray ();
33.                         for (int i = 0; i < rects.Length; i++) {
34.                                 Debug.Log ("detect faces " + rects [i]);
35.  
36.                                 Imgproc.rectangle (imgMat, new Point (rects [i].x, rects [i].y), new Point (rects [i].x + rects [i].width, rects [i].y + rects [i].height), new Scalar (255, 0, 0, 255), 2);
37.  
38.  
39.  
40.                         }
41.  
42.  
43.  
44.                         Texture2D texture = new Texture2D (imgMat.cols (), imgMat.rows (), TextureFormat.RGBA32, false);
45.  
46.                         Utils.matToTexture2D (imgMat, texture);
47.  
48.                         gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
49.  
50.  
51.  
52.                         GameObject quad = gameObject;
53.  
54.                         //center point of detected face
55.                         Point point2D = new Point (rects [0].x + rects [0].width / 2, rects [0].y + rects [0].height / 2);
56.  
57.                         float textureWidth = GetComponent<Renderer> ().material.mainTexture.width;
58.                         float textureHeight = GetComponent<Renderer> ().material.mainTexture.height;
59.  
60.                         Matrix4x4 transCenterM =
61.                 Matrix4x4.TRS (new Vector3 (((float)point2D.x) - textureWidth / 2, (textureHeight - (float)point2D.y) - textureHeight / 2, 0), Quaternion.identity, new Vector3 (1, 1, 1));
62.          
63.          
64.                         Vector3 translation = new Vector3 (quad.transform.localPosition.x, quad.transform.localPosition.y, quad.transform.localPosition.z);
65.          
66.                         Quaternion rotation =
67.                 Quaternion.Euler (quad.transform.localEulerAngles.x, quad.transform.localEulerAngles.y, quad.transform.localEulerAngles.z);
68.          
69.                         Vector3 scale = new Vector3 (quad.transform.localScale.x / textureWidth, quad.transform.localScale.y / textureHeight, 1);
70.          
71.                         Matrix4x4 trans2Dto3DM =
72.                 Matrix4x4.TRS (translation, rotation, scale);
73.          
74.                         Matrix4x4 resultM = trans2Dto3DM * transCenterM;
75.  
76.                         Vector3 point3DVec = new Vector3 (0, 0, 0);
77.                         point3DVec = resultM.MultiplyPoint3x4 (point3DVec);
78.  
79.                         point3D.transform.position = point3DVec;
80.                         point3D.transform.eulerAngles = gameObject.transform.eulerAngles;
81.  
82.                 }

 

VideoWriter class is not implemented on Android platform. Perhaps, I think that it is difficult to implement.
https://enoxsoftware.com/opencvforunity/documentation/support-modules/

 

Thank you, please can i detect user foot like they are juggling balls, so i want to develop an effect like this one in the video

 

EnoxSoftware , finally found out why CardboardMarkerBasedARSample wasnt working in comparison to the MarkerBasedARSample with the same marker. So it seems that on CardboardMarkerBasedARSample the screens are inverted so I had to change the Marker Settings. Difficult to explain in english, but I inverted the marker detection grid, and started to show those 3d objects.
Just to warn people about this

 

Please try this code.

Code (CSharp):

1. #!/bin/sh
2.  
3. sudo apt-get -y install build-essential cmake git libgtk2.0-dev pkg-config libdc1394-22 libdc1394-22-dev libjpeg-dev libpng12-dev libtiff5-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libxine2-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev libtbb-dev libqt4-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils unzip
4. mkdir opencv
5. cd opencv
6. wget https://github.com/opencv/opencv/archive/3.1.0.zip -O opencv-3.1.0.zip
7. unzip opencv-3.1.0.zip
8. wget https://github.com/opencv/opencv_contrib/archive/3.1.0.zip -O opencv_contrib-3.1.0.zip
9. unzip opencv_contrib-3.1.0.zip
10. cd opencv-3.1.0
11. mkdir build
12. cd build
13. cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_opencv_python2=OFF -D BUILD_opencv_python3=OFF -D WITH_V4L=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-3.1.0/modules -D BUILD_SHARED_LIBS=OFF ..
14. make -j $(nproc)
15. sudo make install
16. sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
17. sudo ldconfig

 

Please change "FlipHorizontal" flag.

Code (CSharp):

1. //if WebCamera is frontFaceing,flip Mat.
2.                         if (webCamTextureToMatHelper.GetWebCamDevice ().isFrontFacing) {
3.                                 webCamTextureToMatHelper.flipHorizontal = true;
4.                         }else{
5.                                webCamTextureToMatHelper.flipHorizontal = false;
6.                         }

 

Is it possible to read in a short video, take out the background, and then save the video?

 

VideoWriter class has been implemented from "OpenCV for Unity".
However, this class is not supported in Android.
https://enoxsoftware.com/opencvforunity/documentation/support-modules/

VideoWriter Sample

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. using OpenCVForUnity;
5.  
6. namespace OpenCVForUnitySample
7. {
8.     /// <summary>
9.     /// VideoCapture sample.
10.     /// </summary>
11.     public class VideoWriterSample : MonoBehaviour
12.     {
13.         private int fps = 10;
14.         private double frameWidth = 768;
15.         private double frameHeight = 576;
16.  
17.         VideoCapture capture;
18.         VideoWriter writer;
19.         Mat rgbaMat;
20.         Texture2D texture;
21.        
22.         // Use this for initialization
23.         void Start ()
24.         {
25.            
26.            
27.             rgbaMat = new Mat ();
28.            
29.             #if UNITY_PRO_LICENSE || ((UNITY_ANDROID || UNITY_IPHONE) && !UNITY_EDITOR) || UNITY_5
30.            
31.            
32.             capture = new VideoCapture ();
33.             capture.open(Utils.getFilePath("768x576_mjpeg.mjpeg"));
34.            
35.             if(capture.isOpened()){
36.                 Debug.Log ("capture.isOpened() true");
37.             }else{
38.                 Debug.Log ("capture.isOpened() false");
39.             }
40.             #endif
41.  
42.  
43.             Debug.Log("format: " + capture.get(8));
44.             Debug.Log("preview format: " + capture.get(Videoio.CV_CAP_PROP_PREVIEW_FORMAT));
45.             Debug.Log("CAP_PROP_POS_MSEC: " + capture.get(Videoio.CAP_PROP_POS_MSEC));
46.             Debug.Log("CAP_PROP_POS_FRAMES: " + capture.get(Videoio.CAP_PROP_POS_FRAMES));
47.             Debug.Log("CAP_PROP_POS_AVI_RATIO: " + capture.get(Videoio.CAP_PROP_POS_AVI_RATIO));
48.             Debug.Log("CAP_PROP_FRAME_COUNT: " + capture.get(Videoio.CAP_PROP_FRAME_COUNT));
49.             Debug.Log("CAP_PROP_FPS: " + capture.get(Videoio.CAP_PROP_FPS));
50.             Debug.Log("CAP_PROP_FRAME_WIDTH: " + capture.get(Videoio.CAP_PROP_FRAME_WIDTH));
51.             Debug.Log("CAP_PROP_FRAME_HEIGHT: " + capture.get(Videoio.CAP_PROP_FRAME_HEIGHT));
52.  
53.  
54.             texture = new Texture2D((int)(frameWidth), (int)(frameHeight), TextureFormat.RGBA32, false);
55.             gameObject.GetComponent<Renderer>().material.mainTexture = texture;
56.            
57.             #if (UNITY_ANDROID || UNITY_IPHONE || UNITY_WP_8_1) && !UNITY_EDITOR
58.             gameObject.transform.eulerAngles = new Vector3 (0, 0, -90);
59.             #endif
60.            
61.             gameObject.transform.localScale = new Vector3((float)frameWidth, (float)frameHeight, 1);
62.            
63.            
64.             #if (UNITY_ANDROID || UNITY_IPHONE || UNITY_WP_8_1) && !UNITY_EDITOR
65.             Camera.main.orthographicSize = (float)frameWidth / 2;
66.             #else
67.             Camera.main.orthographicSize = (float)frameHeight / 2;
68.             #endif
69.            
70.            
71.             double ex = capture.get(Videoio.CAP_PROP_FOURCC);
72.             //char EXT[] = {ex & 0XFF , (ex & 0XFF00) >> 8,(ex & 0XFF0000) >> 16,(ex & 0XFF000000) >> 24, 0};
73.             Debug.Log("CAP_PROP_FOURCC: " + (char)((int)ex & 0XFF) + (char)(((int)ex & 0XFF00) >> 8) + (char)(((int)ex & 0XFF0000) >> 16) + (char)(((int)ex & 0XFF000000) >> 24));
74.  
75.  
76.  
77.             writer = new VideoWriter();
78.  
79.             Debug.Log ("persistentDataPath "+ Application.persistentDataPath);
80.  
81. #if UNITY_STANDALONE_OSX || UNITY_EDITOR_OSX
82.             writer.open(Application.persistentDataPath + "/output.mjpeg", VideoWriter.fourcc('a','v','c','1'), fps, new OpenCVForUnity.Size(frameWidth,frameHeight));
83. #else
84.             writer.open(Application.persistentDataPath + "/output.mjpeg", VideoWriter.fourcc('M','J','P','G'), fps, new OpenCVForUnity.Size(frameWidth,frameHeight));
85. #endif
86.  
87.             if(writer.isOpened()){
88.                 Debug.Log ("writer.isOpened() true");
89.             }else{
90.                 Debug.LogError ("writer.isOpened() false");
91.             }
92.  
93.            
94.         }
95.        
96.         // Update is called once per frame
97.         void Update ()
98.         {
99.             //error PlayerLoop called recursively! on iOS.reccomend WebCamTexture.
100.             if (capture.grab ()) {
101.                
102.                 #if UNITY_PRO_LICENSE || ((UNITY_ANDROID || UNITY_IPHONE) && !UNITY_EDITOR) || UNITY_5
103.                 capture.retrieve (rgbaMat, 0);
104.  
105.                 writer.write(rgbaMat);
106.  
107.                 Imgproc.cvtColor (rgbaMat, rgbaMat, Imgproc.COLOR_BGR2RGB);
108.  
109.                 #endif
110.                
111.                 //                                Debug.Log ("Mat toString " + rgbaMat.ToString ());
112.                
113.                 Utils.matToTexture2D (rgbaMat, texture);
114.                
115.                 gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
116.                
117.             }
118.         }
119.        
120.         void OnDestroy ()
121.         {
122.             capture.release ();
123.             writer.release();
124.         }
125.        
126.         public void OnBackButton ()
127.         {
128.             Application.LoadLevel ("OpenCVForUnitySample");
129.         }
130.     }
131.    
132. }

 

Dose it support facial expression detection?

 

Hi, Crash again.

ERROR: SymGetSymFromAddr64, GetLastError: '试图访问无效的地址。

' (Address: 700BA2C4)
0x700BA2C4 (opencvforunity)
0x71003E74 (opencvforunity) xphoto_Xphoto_inpaint_10
0x71003564 (opencvforunity) xphoto_Xphoto_inpaint_10
0x7100422E (opencvforunity) xphoto_Xphoto_inpaint_10
0x71139668 (opencvforunity) xphoto_Xphoto_inpaint_10
0x71127C8B (opencvforunity) xphoto_Xphoto_inpaint_10
0x71126D3A (opencvforunity) xphoto_Xphoto_inpaint_10
0x711271C6 (opencvforunity) xphoto_Xphoto_inpaint_10
0x71126ADA (opencvforunity) xphoto_Xphoto_inpaint_10
0x70D4D881 (opencvforunity) objdetect_CascadeClassifier_detectMultiScale_10
0x056FC078 (Mono JIT Code) (wrapper managed-to-native) OpenCVForUnity.CascadeClassifierbjdetect_CascadeClassifier_detectMultiScale_10 (intptr,intptr,intptr,double,int,int,double,double,double,double)
0x056FC001 (Mono JIT Code) OpenCVForUnity.CascadeClassifier:detectMultiScale (OpenCVForUnity.Mat,OpenCVForUnity.MatOfRect,double,int,int,OpenCVForUnity.Size,OpenCVForUnity.Size)
0x056BCE3B (Mono JIT Code) AutoPhotograph:Update ()
0x0569F541 (Mono JIT Code) (wrapper runtime-invoke) object:runtime_invoke_void__this__ (object,intptr,intptr,intptr)

////////////////////////////////////////////

Code (CSharp):

1.  cascadeFace.detectMultiScale(equalizeHistMat, faces, 1.1f, 2, 0
2.                             | Objdetect.CASCADE_FIND_BIGGEST_OBJECT,
3.                             new OpenCVForUnity.Size(equalizeHistMat.rows() * 0.20f, equalizeHistMat.rows() * 0.20f), new Size());
4.  
5.                     Imgproc.rectangle(rgbaMat, new Point(maxRect.x, maxRect.y), new Point(maxRect.x + maxRect.width, maxRect.y + maxRect.height), new Scalar(255, 255, 0, 255), 4);
6.  
7.                     if (faces.rows() > 0)
8.                     {
9.                         //draw face rect
10.                         OpenCVForUnity.Rect[] rects = faces.toArray();
11.                         {
12.                             faceRect = rects[0];
13.  
14.                             using (MatOfRect eyes = new MatOfRect())
15.                             using (Mat equalizeHistEyeMat = new Mat())
16.                             using(Mat eyeMat = new Mat(grayMat, faceRect))
17.                             {
18.                                 // check eyes
19.                                 Imgproc.equalizeHist(eyeMat, equalizeHistEyeMat);
20.                                 cascadeEye.detectMultiScale(equalizeHistEyeMat, eyes, 1.1f, 2, 0 | Objdetect.CASCADE_FIND_BIGGEST_OBJECT,
21.                                             new OpenCVForUnity.Size(equalizeHistEyeMat.rows() * 0.15f, equalizeHistEyeMat.rows() * 0.15f), new Size());

 

There is no facial expression detecter class in OpenCV.
It is necessary to implement it by yourself.
http://answers.opencv.org/question/10583/moodemotion-detection-happy-to-sad-using-opencv/

 

Hello,

I have a problem with the validation of my app on XCode with a project which use this OpenCV plugin.

The code signing process always produce an error, and I can't finalize my application because of that !

Part of the log with the problem it seems (from IDEDistribution.standard.log) :

Project with OpenCV plugin 2.0.2, from Unity 5.3.4p6 and XCode 7.3.1.
Tested with and without IL2CPP.

Also I always have to manually fix the Framework and Library Search Paths on XCode (from "$(PROJECT_DIR)/Framework\Plugins/iOS" to "$(PROJECT_DIR)/Framework/Plugins/iOS" for example)

EDIT : I will try to test on a empty project with only the OpenCV plugin

 

Xphoto.inpaint() seems to be the cause of the error.
Is Xphoto.inpaint() used in this code?

 

It seems to "import settings" are not correct.
Please make sure it is set as follows.

 

Thank you for the answer !
Here is the current (default) config for this folder in my project :



Do I need to edit it with the same parameters as yours ?
Maybe I need to add that this is for an iOS application, not OS X !

 

ImportSettings of "opencvforunity.bundle" seems to affect the build of iOS application.

you can easily set the ImportSettings as follows.

• Move "Plugins" folder to "Assets/Plugins/OpenCVForUnity/".

• Select MenuItem[Tools/OpenCV for Unity/Set Plugin Import Settings].

 

Face swapper demo very slow with 1920x1080 resolution. How could I fix this problem? Is this possible?

 

The steps on the next page might be effective.

• Resize Frame
• Skip frame

http://www.learnopencv.com/speeding-up-dlib-facial-landmark-detector/

 

I've tried using the VideoWriter class both in Unity Editor and standalone on windows and It never succeds in opening the videowriter. Is there someway to get a more detailed error message than a simple "videowriter didnt open" ?

My code related to the videowriter is:

Code (CSharp):

1.  
2.  
3. private VideoWriter videoWriter;
4.  
5.  
6. videoWriter=new VideoWriter();
7.  
8.  
9. videoWriter.open(Application.dataPath + "/Pictures/" + fileName + ".mjpeg" ,VideoWriter.fourcc('M','J','P','G'), 25, new Size(res.x, res.y));
10.  
11. if(videoWriter.isOpened()){
12. Debug.Log("videoWriter open");
13. } else{
14.   Debug.LogError("videoWriter didnt open");
15. }
16.  
17.  

 

If you use the “VideoCapture(string filename) or VideoWriter class”, require setup.
1)Download "OpenCV for Windows Version 3.1.0"(http://opencv.org/downloads.html).
2)Set PATH variable to "opencv_ffmpeg310.dll" or "opencv_ffmpeg310_64.dll".
if 32bit, "¥path¥to¥opencv¥build¥x86¥vc12¥bin¥".
if 64bit, "¥path¥to¥opencv¥build¥x64¥vc12¥bin¥".
Or
2)Copy to Project Folder.

 

I downloaded and installed 3.1.0 from the link and tried placing the 32bit dll in

• the project top directory
• the compiled .exe directory
• the compiled .exe_data directory
• and the project/Asset directory

When I place it in the Asset directory Unity picks up on it and adds it to the build and brings it along to the exe_Data/plugins folder together with the original opencvforunity.dll.

But no matter where I place the 32 or 64 bit dll I still get a error when trying to do videoWriter.open(...). Do you have any other suggestion to what I might be doing wrong?

Big thanks for all the help so far!

 

"filename" seems to be the cause of the problem.

videoWriter.open(Application.dataPath + "/Pictures/" + fileName + ".mjpeg" ,VideoWriter.fourcc('M','J','P','G'), 25, new Size(100, 100));
> "videoWriter didnt open"

videoWriter.open(Application.persistentDataPath + "/output.mjpeg",VideoWriter.fourcc('M','J','P','G'), 25, new Size(100, 100));
> "videoWriter open"

 

Using

Code (CSharp):

1.  
2. videoWriter.open(Application.persistentDataPath+"/output.mjpeg",VideoWriter.fourcc('M','J','P','G'),25,newSize(100,100));
3.  

I'm still getting "videoWriter didnt open". Any other idea what can be wrong?

 

There is no implementation of a ImgProc.floodfill function that does not utilize a mask. Is this an oversight or something that is planned for in the future? Thanks!

 

No, the code isn't used which "Xphoto.inpaint()"

this is internal code in opencv

 

I was tested in the following environments.
Windows10 64bit
Unity5.3.3f1 64bit
OpenCV for Unity 2.0.5

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. #if UNITY_5_3 || UNITY_5_3_OR_NEWER
5. using UnityEngine.SceneManagement;
6. #endif
7. using OpenCVForUnity;
8.  
9. namespace OpenCVForUnitySample
10. {
11.     /// <summary>
12.     /// Video writer sample.
13.     /// </summary>
14.     public class VideoWriterSample : MonoBehaviour
15.     {
16.  
17.         // Use this for initialization
18.         void Start ()
19.         {
20.  
21.             VideoWriter videoWriter;
22.  
23.  
24.             videoWriter = new VideoWriter ();
25.  
26.  
27. //            videoWriter.open (Application.dataPath + "/Pictures/" + fileName + ".mjpeg", VideoWriter.fourcc ('M', 'J', 'P', 'G'), 25, new Size (res.x, res.y));
28.             videoWriter.open(Application.persistentDataPath+"/output.mjpeg",VideoWriter.fourcc('M','J','P','G'),25,new Size(100,100));
29.  
30.             if (videoWriter.isOpened ()) {
31.                 Debug.Log ("videoWriter open");
32.             } else {
33.                 Debug.LogError ("videoWriter didnt open");
34.             }
35.         }
36.    
37.         // Update is called once per frame
38.         void Update ()
39.         {
40.    
41.         }
42.  
43.         public void OnBackButton ()
44.         {
45.             #if UNITY_5_3 || UNITY_5_3_OR_NEWER
46.             SceneManager.LoadScene ("OpenCVForUnitySample");
47.             #else
48.                         Application.LoadLevel ("OpenCVForUnitySample");
49.             #endif
50.         }
51.     }
52. }
53.  

Copy opencv_ffmpeg310_64.dll to the project top directory.

 

Since this package is a clone of OpenCV Java, you are able to use the same API as OpenCV Java 3.1.0.
ImgProc.floodfill function that does not utilize a mask is not implemented.
Please use the Imgproc.floodFill( Mat image, Mat mask, Point seedPoint, Scalar newVal ) in the following manner.

Code (CSharp):

1. Imgproc.floodFill(img, new Mat(), new Point(matSize / 2, matSize / 2), new Scalar(1));

 

Could you send me the code?　enox.software@gmail.com

 

Hi
i have a problem after converting texture2d to mat and trying to use copyTo function to attach the mat to other mat i get this error :

Code (CSharp):

1. CvException: Native object address is NULL
2. OpenCVForUnity.Mat..ctor (IntPtr addr) (at Assets/OpenCVForUnity/org/opencv/core/Mat.cs:459)
3. OpenCVForUnity.Mat.submat (OpenCVForUnity.Rect roi) (at Assets/OpenCVForUnity/org/opencv/core/Mat.cs:2763)
4. OpenCVForUnitySample.mask.statt () (at Assets/FaceSwapeLive/mask.cs:326)
5. OpenCVForUnitySample.mask.Start () (at Assets/FaceSwapeLive/mask.cs:86)
6.  

this is my code :

Code (CSharp):

1.         imgTexture = Resources.Load("cgcg") as Texture2D;
2.              Downloaded_mask = new Mat(imgTexture.height, imgTexture.width, CvType.CV_8UC4);
3.              Utils.texture2DToMat(imgTexture, Downloaded_mask);  
4.              Downloaded_mask.copyTo(rgbaMat.submat(new OpenCVForUnity.Rect(10, 10, Downloaded_mask.cols(), Downloaded_mask.rows())));

i spend more than 6 hrs and i can't get it work i hope you can help me , thanks

 

Hi,
I have a problem with the overloading of operators. I'm converting a code written in C++ with opencv to C# using your lib, however expressions such as
A * B; (where A and B are of Mat type: I want a matrix multiplication, not element-wise, I don't want A.mul(B))
or
A /= 10.0f;
...
seem to be forbidden in C#,
can you help me with that?

Thanks a lot

 

The Mat.clone() function seems to be sporadically (though not frequently) returning a mirrored and flipped vertically Mat. Any idea what might be causing this in the code below? mat_lastFrame is fine, however every occasionally mat_dubug is flipped vertically and mirrored horizontally.

Code on helper thread interacting with a camera:

Code (CSharp):

1. this.lastFrame = buffer[buffer.Length-1];
2. this.mat_lastFrame = DuoOpenCVHelper.ByteArrayToMat(this.lastFrame, this.frameRecorder.WIDTH, this.frameRecorder.HEIGHT);
3. this.mat_debug = this.mat_lastFrame.clone();
4.  

Code on Main Thread displaying Mat's being read:

Code (CSharp):

1. OpenCVForUnity.Utils.matToTexture2D(this.mat_lastFrame, this.texture1);
2. OpenCVForUnity.Utils.matToTexture2D(this.mat_debug, this.texture4);

Helper function for converting byte[] off camera to an OpenCV Mat:

Code (CSharp):

1. public static Mat ByteArrayToMat(byte[] b, int width, int height) {
2.         Mat mat = Mat.ones(new Size(width, height), OpenCVForUnity.CvType.CV_8U);
3.         OpenCVForUnity.Utils.copyToMat<byte>(b, mat);
4.         //mat.put(0, 0, b);
5.         return mat;
6.     }

 

To follow up on previous comment about Mat.clone() returning an occasional flipped and inverted image, the same appears to happen internally in BackgroundSubtractorMOG2. When I pass it "this.mat_lastFrame" I occasionally get an inverted background subtraction result despite this.mat_lastFrame remaining in its original format.

 

rgbaMat has been initialized?

 

A * B
In the same way as a case using ”OpenCV Java”, please use Core.gemm().
http://enoxsoftware.github.io/OpenC...1_core.html#a7ec35e6f5d2b7cb428cb5bae2c1385ed
http://answers.opencv.org/question/10770/product-of-two-matrices/
A /= 10.0f
http://stackoverflow.com/questions/20848263/mathematical-operations-on-mat-objects-opencv

 

Perhaps, I think it's because the same Mat on a multi-thread is being operated.
Please refer to the following sample.
https://github.com/EnoxSoftware/Ope...eSample/WebCamTextureAsyncDetectFaceSample.cs

 

Thanks a lot.

 

Hi, i m working with open cv for unity and Dlib face landmarks detection, and i'm wondering how to offset the web cam texture and AR objects (y += 150) in a way to see through the UI interface ... i m trying to move the quad and AR objects up but it seems like the AR proportions are falses like the field of view of the AR camera and the scale of AR ... can you find a solution to this issue in a way to make the web cam texture's position and scale ables to been updated and modified from a sample variables, thanks.

 

Thank you! You are correct it must have been a thread access issue because that fixes the problem. Just being careful to make sure the main thread is only seeing clones of Mat's from my helper thread (and I marked the clones as volatile) and the mirroring issue has been resolved.

 

Hi, i'm wondering how to offset the AR web cam texture (Quad + AR object) without disabling the correct proportions of the AR camera and AR objects scaling ... any help with this please, Thanks.

 

EnoxSoftware
yes i work with your example rgbaMat is initialized

 

I tried copying your example in to a new Unity project and got it working. I also got it working in my own project aswell.

But now after calling open it creates the file as expected, but after calling a few writes and then releasing the videowriter the file is still empty, any guess to what could be going wrong there?

I have checked that the size is correct, does the mat have to use a specific color format to work aswell?

 

Hi! I have a piece of paper with printed square rectangle frame + a random picture printed inside of it. How can I grab a rectangle + fix the perspective so I get 2048x2048 square texture as a result? With OpenCVForUnity (or maybe with OpenCVForUnity+Vuforia)? Thanks!

 

I think it is feasible using Imgproc.findContours() and Imgproc.warpPerspective().
A similar process has been implemented in MarkerBasedARSample.
https://github.com/EnoxSoftware/MarkerBasedARSample/blob/master/MarkerBasedAR/MarkerDetector.cs

 

Hello,
the cam shift sample works fine and i need to get the center of the yellow rect that track object after touching 4 points and make a cube following the center of rect position
Thank you
Lafi

 

Hey! I just sent you an e-mail, but thought I would also post here.

We're building an app that is primarily going to be downloaded via cell, in areas where there is no wifi. So it's pertinent that we get our binaries to be as small as possible.

We're really just trying to use the OpenCV+dlib WebCamTextureARSample, basically. While we can build an OpenCV binary that only has these features and cuts our filesize to less than half, we're stuck in that libOpenCVForUnity.a is trying to make calls to functions that are not in this library.

What would be super, super awesome would be a build of libOpenCVForUnity.a and opencv2.framework that was slimmed down for this specific use. Heck, I just paid $150 for these assets, I'd be willing to pay more if it solved our problem