Unity3d + Arduino (Read serial in Unity3d)

So, I've been bored, and trying to learn the limits of Unity3d free edition. I also work with arduino here and there, and came up with the idea that many people have.

Why can't use use an accelerometer and use it to view the 3d world that way?

Well, I did some research on the matter, and it seemed that everyone decided that you had to have a plugin in order to send and receive information. I've found that is not true... to a point.

Importing the right stuff gets you to be able to read serial, and after a bit of annoyance on getting it to read the information right, and delaying the arduino by 10ms so that I could actually read the information, it works fine. At least in the standalone player in Windows. I'm not sure on mac, but I know that web player doesn't support it.

So, here I sit with some spiffy code, and decided that even though I haven't been here long, I would simply post a bit of code that lets you get information from a serial device. (Note: You can use ANY serial device as long as you know how to read the information.)

C# Code:

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4. using System.IO.Ports;
5.  
6. public class serialRotation : MonoBehaviour {
7.  
8.     SerialPort stream = new SerialPort("COM4", 9600); //Set the port (com4) and the baud rate (9600, is standard on most devices)
9.     float[] lastRot = {0,0,0}; //Need the last rotation to tell how far to spin the camera
10.    
11.    
12.     void Start () {
13.         stream.Open(); //Open the Serial Stream.
14.     }
15.    
16.     // Update is called once per frame
17.     void Update () {
18.         string value = stream.ReadLine(); //Read the information
19.         string[] vec3 = value.Split(','); //My arduino script returns a 3 part value (IE: 12,30,18)
20.         if(vec3[0] != ""  vec3[1] != ""  vec3[2] != "") //Check if all values are recieved
21.         {
22.             transform.Rotate(           //Rotate the camera based on the new values
23.                                 float.Parse(vec3[0])-lastRot[0],
24.                                 float.Parse(vec3[1])-lastRot[1],
25.                                 float.Parse(vec3[2])-lastRot[2],
26.                                 Space.Self
27.                             );
28.             lastRot[0] = float.Parse(vec3[0]);  //Set new values to last time values for the next loop
29.             lastRot[1] = float.Parse(vec3[1]);
30.             lastRot[2] = float.Parse(vec3[2]);
31.             stream.BaseStream.Flush(); //Clear the serial information so we assure we get new information.
32.         }
33.     }
34.    
35.     void OnGUI()
36.     {
37.         string newString = "Connected: " + transform.rotation.x + ", " + transform.rotation.y + ", " + transform.rotation.z;
38.         GUI.Label(new Rect(10,10,300,100), newString); //Display new values
39.         // Though, it seems that it outputs the value in percentage O-o I don't know why.
40.     }
41. }
42.  

Arduino test code:

Code (csharp):

1.  
2. float value = 0;
3. boolean flip = false;
4.  
5. void setup()
6. {
7.  
8.   Serial.begin(9600);
9. }
10.  
11. void loop()
12. {
13.   if(value >= 1023)
14.   {
15.       flip = true;
16.   }else if(value <= 0)
17.   {
18.       flip = false;
19.   }
20.  
21.   if(flip == false)
22.   {
23.      value += 1;
24.   }else{
25.      value -= 1;
26.   }
27.  
28.   float values[] = {0,0,value};
29.  
30.   Serial.flush(); //clears any possible left over information
31.   Serial.print(map(values[0],0,1023,0,360)); //Resizes values from the range on arduino input to 360 of rotation.
32.   Serial.print(",");
33.   Serial.print(map(values[1],0,1023,0,360));
34.   Serial.print(",");
35.   Serial.print(map(values[2],0,1023,0,360));
36.   Serial.println();
37.  
38.   delay(10);
39. }
40.  

What's the practical use? Well, I want to stick a screen to my face and the arduino to the top of my head and get some VR kind of effects from it. Wouldn't be too bad to get true VR effects for less then $200. Add to that a decent controller and a great game and you have the VR of the future!

I'd also like to be able to use my arms and such in this kind of system, but that's going a little overboard at this point. (though, with some bend sensors and some potentiometers {volume knobs} I could probably create a decent control system)

So, let me know what you guys think! O-o ... Also, did I put this in the right forum?

 

Nice, thanks for sharing. One of the things on my to-do list is to pick up an Arduino board and start playing with it. I don't know if you found them already, but there have been a couple other threads on the topic:
http://forum.unity3d.com/viewtopic.php?t=34257
http://forum.unity3d.com/viewtopic.php?p=206356

 

Awesome. I have been meaning to start learning arduino, but I don't know where to start. Any suggestions on what bored I should get?

 

The Duemilanove is probably the best for starting with. (It's what I have)

It's made for people that don't have much experience in it, so you can plug solid core wires into it like a bread board. No soldering needed.

 

Sounds perfect. Thanks a lot.

 

Heyyy, this looks very interesting. Im already reading serial data in from my Duemilanove board, but that is via Processing, which sends data over the OSC protocol. However, if your code can skip this whole step, then thats a whole lot easier.

Definately go for your VR project, it sounds super exciting. For my bachelors, we're reading gyroscope data (Wii motion plus) via the Arduino. You will probably need some sort of gyroscope to provide yaw data for your VR stuff. The wiimote "mimics" this by using the IR camera, but thats a limit of approx 45 degrees horizontal FoV. Good luck, and thanks for this, ill check it out when I get a few minutes

 

Well, there is either the gyro way like you were saying, or there is the more accurate way of a 3-axis accelerometer and a digital compass... at least, that's my theory on if it doesn't work like I want.

 

As Caliber Mengsk mentioned, the Duemilanove is the best bet for beginners. I highly recommend a starter kit like this one, which comes with necessities like wires, a breadboard, and various sensors. RadioShack (at least the ones around my area) still carries things like LEDs and resistors, though I find their selection is pretty hit and miss.

 

I'm getting my stuff from pololu.com, but another suggested site is sparkfun.com (sparkfun has better selection, but most things are a buck or two more)

 

Hi. I have a question. How to send an instruction from the serial port ?

Sorry for my english...

By

 

hey caliber....
i'm totally noob with unity but know a bit on arduino...
where do you refer to the camera in your script...
just made a c# behaviour script and put in your code, hope that's right so far...
any hint would be awesome..i'm just using the standard main camera!!!
thx

 

@MasterLoLo
Sorry, I didn't notice the topic was brought back up. I haven't been super active on the forums again until recently, and didn't see your post.

First you create a SerialPort variable (generally serial port communications are known as streams), then open the stream (in the start function is best) then it's as simple as streamVariable.Write("My string") or streamVariable.WriteLine("My string")

@palimpalim
transform.Rotate( //Rotate the camera based on the new values
float.Parse(vec3[0])-lastRot[0], //x
float.Parse(vec3[1])-lastRot[1], //y
float.Parse(vec3[2])-lastRot[2], //z
Space.Self
);

That's where I set the rotation of the camera. The script is attached to the main camera in this case. If you say wanted to have a cube move in the rotation, then you would attach the script to the cube. XD I really need to go through and redo this script some time. I did this way back when I was first learning Unity, so I'm sure there are much more efficient ways to do the same thing.

 

Haven't read the thred yet, so attracted by unity-arduino integration, and jusy downloaded integration pack from google code. thanx for the source...

 

Caliber, are you sure this code still works?

If I try it in unity 3.3 ,then I get the message

error CS0246: The type or namespace name 'SerialPort' could not be found. Are you missing a using directive or an assembly reference?

I use

using UnityEngine;
using System.Collections;
using System.IO.Ports;

 

Serial port access is basically broken on Unity for Mac.
http://forum.unity3d.com/threads/54112-U3.b3-System.IO.Ports/page4

 

why my unity3D can't run as soon sa I put "new SerialPort" code in script.

 

I played with the arduino a while back. Very fun piece of hardware I like the Unity3D integration. Keep it up.

 

To those that have had the problem of it not working, that's because unity is now set to use the .net framework subset instead of the full framework (helps lower file size I think). You simple have to turn it on.

Click Edit>project settings>player

At the bottom of the settings window there is a pulldown box that says .Net 2.0 Subset. Change that to just be .Net 2.0.

At least, that's what I'm assuming is the problem. I've had issues with importing libraries without changing that.

 

hey everyone...i'm new to unity and i've been trying to follow up on the little documentation on interfacing unity with arduino...I'm currently doing a couple joysticks for a project and i got them to interface through the google code http://code.google.com/p/unity-arduino-serial-connection/ but since i need to read 12 inputs the code is crazy hard to understand and debug....580 lines on the unity side alone....this sounds way simpler but i can't quite figure out how to implement it...

my code (3 buttons example) ALL CREDITS GO TO Caliber Mengsk

Arduino

Code (csharp):

1.  
2. int buttonPin[3] = {2,3,4};
3. int buttonState[3] = {0,0,0};
4.  
5. void setup() {
6.  
7.   Serial.begin(9600);
8.   for(int i=0; i<3; i++){
9.     pinMode(buttonPin[i], INPUT);    
10.   }
11.  
12.   Serial.begin(9600);    
13. }
14.  
15. void loop () {
16.  
17. for(int i=0; i<3; i++){
18.     buttonState[i] = digitalRead(buttonPin[i]);
19.  
20.     if (buttonState[0] == HIGH) {    
21.       Serial.println("button1 pressed") ;
22.     }
23.          
24.     if (buttonState[1] == HIGH) {    
25.        Serial.println("button2 pressedn") ;
26.     }
27.      
28.     if (buttonState[2] == HIGH) {    
29.       Serial.println("button3 pressed") ;
30.     }
31.  
32. }
33. }
34.  

unity

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3. using System.IO.Ports;
4.  
5. public class arduino : MonoBehaviour {
6.  
7.  
8.     SerialPort stream = new SerialPort("COM3", 9600); //Set the port (com4) and the baud rate (9600, is standard on most devices)
9.  
10.     float[] initState = {0,0,0}; //Need the last rotation to tell how far to spin the camera
11.  
12.  
13.    
14.  
15.     void Start () {
16.  
17.         stream.Open(); //Open the Serial Stream.
18.  
19.     }
20.  
21.      // Update is called once per frame
22.  
23.     void Update () {
24.  
25.         string value = stream.ReadLine(); //Read the information
26.  
27.         string[] vec3;
28.  
29.         if(vec3[0] != ""  vec3[1] != ""  vec3[2] != "") //Check if all values are recieved
30.  
31.         {
32.  
33.             transform.Move(           //Rotate the camera based on the new values
34.  
35.                                 float.Parse(vec3[0])-initState [0],
36.  
37.                                 float.Parse(vec3[1])-initState [1],
38.  
39.                                 float.Parse(vec3[2])-initState [2],
40.  
41.                                 Space.Self
42.  
43.                             );
44.  
45.             initState [0] = float.Parse(vec3[0]);  //Set new values to last time values for the next loop
46.  
47.             initState [1] = float.Parse(vec3[1]);
48.  
49.             initState [2] = float.Parse(vec3[2]);
50.  
51.             stream.BaseStream.Flush();
52.  
53.         }
54.  
55.     }
56.  
57.    
58.  
59.     void OnGUI()
60.  
61.     {
62.  
63.         string newString = "Connected: " + transform.move.x + ", " + transform.move.y + ", " + transform.move.z;
64.  
65.         GUI.Label(new Rect(10,10,300,100), newString); //Display new values
66.  
67.  
68.     }
69.  
70. }

Is this "right"?

After i attach it to an object....should I somehow define the keyinputs?

Thanks in advance for any help you can give me!

Antonio

 

OH MY GOD! ive been looking for this, i had a go but never got it to work im going to make a custom game controler one day

 

You genius! Now I have to alter my summer project -.-

 

It's fun doing Theremin stuff with the Arduino.

 

A game using a theremin?

 

More as a input device for altering sfx and gfx but it's fun nonetheless, at least to me.

But nothing is stopping you using it for a game as well.

 

Please stop with the good ideas :/

 

Why not just stick an android to your face?

http://roadtovr.wordpress.com/2011/10/17/make-your-own-head-mounted-display-with-an-android-phone/

 

I want to send data from a Pic. how do I store the data in a variable and compare that data with a letter or character.

 

hola a todos!!

me gustaría leer los datos de puerto serie. pero en una función diferente al UPDATE. estoy enviando una trama de datos y desde el UPDATE no he podido, podrían ayudarme!!

 

Has anyone tried running a servo motor from Unity through Arduino ?

Create a motion controller for games etc... I see the asset "Uniduino" can do this... but I can not find anything regarding servos or stepper motors.

 

Ya encontraste algo al respecto o abandonaste el proyecto!? "aparte de uniduino"

 

This is nice stuff Man. I am able to have a serial connection setup in Unity. And its working fine. Now second step I want is connect Arduino with Bluetooth to Unity. Here I am getting issues as there is no direct way to connect via Bluetooth. Please help with examples