WII remote and FLASH example

After Johnny Lee, we all do the the trick with the Wii remote and Flash.
It is just more fun steering the computer with the Wii than with the keyboard or mouse!

Just using Glove Pie we can easily catch the sensor values of the Wii remote and use these values in FLASH.

For the class Computer Science and Media, I use a wii steering the presentation of the Kick-off of a half year course. I even bought a steering wheel for the wii at dealextreme.com!

This is the file of the flash movie, with scripts and the glovepie script.
Included is also microphone activity and webcam activity, although I turned this off during the presentation :-)
The presentation is done in my old website which was presented inside this beautifull violet nds...

ARDUINO and FLASH (problems)

The ARDUINO is great for catching input, and FLASH is very convenient for designers to work with. Combining the two is not difficult at all, although there are some details which could be complicated. Or so we thought....indeed sometimes it works, but switching from XP to MAC OS for instance requires the right port setting, and still the serial data, from the analog ARDUINO pins comes in erratically, also changing from the ATmega168 to the ATmega2328 board does seem to change the way the serial data come in.
Using a Socket in FLASH or using an XML Socket differs too...

So there are a few problem areas:

• the config file for Serproxy, which is the port number in the MAC OS case?
• the FLASH socket, XML or not?
• the serial data coming in, sometimes something seems to be skipped, do we have to use a checksum?

There are several links in the Arduino docs explaining what to use and how:
http://www.arduino.cc/playground/Interfacing/Flash
Personally I use simply the Serial lib from the Arduino, Serproxy and a socket in FLASH.
You have to configure the serproxy.cfg, that it uses the right com port. Then you have the two way communication. (Tinkerproxy is only one way.)

Setting the serproxy config file is explained here: http://protolab.pbworks.com/TutorialFlashSetup
Especially for the MACOS fans...

The digital pins work fine, being only true or false.
So you start the arduino script, just talking into the serial port:
like Serial.println (digitalRead(pinNumber));

Serproxy is started, it says waiting for clients...

Then in FLASH you use:
import flash.net.Socket;
you add an eventlistener:
arduinoSocket = new Socket("localhost",5331);
arduinoSocket.addEventListener(Event.CLOSE, closeHandler); arduinoSocket.addEventListener(ProgressEvent.SOCKET_DATA, socketDataHandler);

and you read the data:
var nb : uint = arduinoSocket.bytesAvailable;
var bytes:ByteArray = new ByteArray();
var input:String = String(arduinoSocket.readShort( ));
//or like this
//var input:String = String(arduinoSocket.readBytes( bytes, 0, 5));

the reading of the HIGH or LOW is simple and reliable, but the potmeter on an analog port is more complex, due to the bits being sent through the Serial port which have to be caught in the right way.

basic flash script (will be improved shortly)

Using the analog pins ( a potentiometer or a flex sensor) gives more problems.
You have to experiment with the bytes coming in.

We had the arduino flash connections running on XP using serproxy in no time.

On the other hand we had trouble getting it running on MAC OS. The problem was the comport naming. Apparently normally ARDUINO software choses /dev/tty.usbserial-*, but for Serproxy you need the /dev/cu.usbserial-*
so using this serproxy.cfg:
# Config file for serproxy

# Do not Transform newlines coming from the serial port into nils
newlines_to_nils=true

# Comm ports used
comm_ports=3

# Default settings
comm_baud=9600
comm_databits=8
comm_stopbits=1
comm_parity=none

# Idle time out in seconds
timeout=300

# Settings for COM3
serial_device3=/dev/cu.usbserial-A7006vzH
net_port3=5333

we got it right. Also why is this called port 3? We found that out using Tinkerproxy, which showed this port number, but the list of ports in the terminal gave no clue about this number.

Anyway, now it works on both operating systems....
Well this was what we thought, but experimenting with our own and scripts of other found on the internet, with different ARDUINO boards, with XMLSocket and the plain "Socket" class, we still have erratic data coming in the FLASH part, skipping things etc.

(If you open the serial window in the ARDUINO IDE then the data come in fine.)

Evidently more experience and or knowledge of what is going on is needed.

OO FLASH starter

Here the source you can find an example of a simple OO structure in FLASH CS3 / 4

It needs an empty FLASH movie with in the properties window setting the class to OOstarter

This OOstarter imports all the other classes it needs.

It just loads an image and a textfield, waits 2 seconds and loads another image and starts moving a small circle around.

the movement is quite bad!

It illustrates the use of classes, and could be the beginning of a wonderful mini golf game.

the source

FLAR: avoid lib problems!

Sometimes the movie does not start showing an avalanche of errors which are not very understandable, like this:



Then it could be that the movie does not find the right as files.

You can solve this by going to Publish settings->flash->actionscript settings and trying to add the files the movie needs. But then these path's are hard coded.

The problem lies in the header of the as files: (PV3DARApp.as)

(image pv3d is situated in support folder)

Very important is the PV3DARApp.as file, situated in the same folder as your SImpleCube.as or SimplePlane.as.

It can be openend in FLASH.

In the header of the PV3DARApp.as file you can find these lines:

import org.libspark.flartoolkit.support.pv3d.FLARBaseNode;
import org.libspark.flartoolkit.support.pv3d.FLARCamera3D;

sometimes the lines are like this: (apperently there are two versions of this starter toolkit.)

import org.libspark.flartoolkit.pv3d.FLARBaseNode;
import org.libspark.flartoolkit.pv3d.FLARCamera3D;

so looking in the folder called org -> libspark -> then either you have directly pv3d or ->support-> pv3d (see image of folders)

Also apparently when the support folder is not used: you have to use _baseNode.addChild and when the support is installed _markerNode.addChild etc.

When you have sorted this out, you do not need the external path in action script settings, which avoids hard coding this path, which avoids problems when copying the movies to other locations, or to other laptops.

AVOID HARD CODED FILE PATH's!!!!

FLASH AR Open Door Movie

At the moment this is a test movie:

http://www.contrechoc.com/flash/arExample/SimplePlane.html

On this page you see a movie starting. If nothing happens:

Is there a webcam installed?

Right click and ask for settings, allow and select the webcam.

The marker is the usual marker from the flyer of the Open Door event.

Flash AR example, streaming video and changing images

We did a few experiments using the FLAR toolkit, see also other posts with FLAR

For the open door event we made a movie with masks, this was the moving testing the principles:

http://www.contrechoc.com/flash/arExample/SimplePlane.html

On this page you see first three images of an unhappy hamburger (which gets gradually eaten) and then a movie projected on the plane is starting. The images are changed using a timer.

Troubleshooting:
Is there a webcam installed?
Right click and ask for settings, allow and select the webcam.

The marker is the usual marker from the flyer of the wdka Open Door event:
or print this file:
flarlogo-marker.pdf (there has to be a rather large area of white surrounding this file)

find all the details of the FLAR toolkit here:

http://saqoosha.net/en/flartoolkit/start-up-guide/
users.fbihome.de/~roeder/MT4/FLARToolkit_HowTo.pdf

FLAR, import troubleshooting

Sometimes the movie does not start showing an avalanche of errors which are not very understandable, like this:

It is clear that the movie does not find the right as files.

You can solve this by going to Publish settings->flash->actionscript settings and trying to add the files the movie needs. But then these path’s are hard coded.

The problem lies in the header of the as files: (PV3DARApp.as)

(image pv3d is situated in support folder)

Very important is the PV3DARApp.as file, situated in the same folder as your SimpleCube.as or SimplePlane.as.

This file is included in the FLAR files but can also be openend in FLASH.

In the header of the PV3DARApp.as file you can find these lines:

import org.libspark.flartoolkit.support.pv3d.FLARBaseNode;
import org.libspark.flartoolkit.support.pv3d.FLARCamera3D;

sometimes the lines are like this: (apperently there are two versions of this starter toolkit.)

import org.libspark.flartoolkit.pv3d.FLARBaseNode; //so without support
import org.libspark.flartoolkit.pv3d.FLARCamera3D;

so looking in the folder called org -> libspark -> then either you have directly pv3d or ->support-> pv3d (see image of folders)

Also apparently when the support folder is not used: you have to use _baseNode.addChild and when the support is installed _markerNode.addChild etc.

So the problem can be solved choosing the right import lines, without or with the support folder and using the right Node, _markerNode or _baseNode.

When you have sorted this out, you do not need the external path in action script settings, which avoids hard coding this path, which avoids problems when copying the movies to other locations, or to other laptops.

AVOID HARD CODED FILE PATH’s!!!!

FLAR code to get a plane with a png image for PaperVision

We want to trigger masks with the AR marker. So we have to be able to put a png image on a plane, this is the code:

private function addPlaneForMask()
{
var mat:BitmapFileMaterial= new BitmapFileMaterial(“textureMap/cross.png”);
_plane1 = new Plane(mat , 150, 160);
this._plane1.z = 20;
this._plane1.rotationX = 180;
this._baseNode.addChild(this._plane1);
}

then we want to change the image shown by the marker:

private function removeAndAddPlaneForMask()
{
this._baseNode.removeChild(this._plane1);
var mat:BitmapFileMaterial= new BitmapFileMaterial(“textureMap/otherImage.png”);
_plane1 = new Plane(mat , 150, 160);
this._plane1.z = 20;
this._plane1.rotationX = 180;
this._baseNode.addChild(this._plane1);
}

and if you call this last function by a timer then the marker get refreshed every time the timer fires:

var refreshMaskTimer: Timer = new Timer(2000);
refreshMaskTimer.addEventListener(TimerEvent.TIMER, removeAndAddPlaneForMask );

refreshMaskTimer.start();

You can get the next image by using a counter in a the name (String) of an image:

adding: private var imageNumberOfMask : Number =1;//with the other private vars

in the function removeAndAddPlaneForMask:

var imageName : String = “textureMap/” + String(imageNumberOfMask) + “.png”;

var mat:BitmapFileMaterial= new BitmapFileMaterial(imageName);

imageNumberOfMask = imageNumberOfMask + 1; //increasing the number of the image

of course then you have to take care the number of images becomes not to high, so:

if ( imageNumberOfMask > 37 ) imageNumberOfMask = 1; //restart at 1 if your number of masks is 37

FLAR: Having fun with Augmented Reality

This week before the winter break we will do something funny and interesting: we will make a simple model in BLENDER, export it with Collada and import the model in FLASH PaperVision the 3D engine of FLASH. Then with a marker we will see the image appear in the webcamview.

We use the FLAR TOOLKIT and an extended SimpleCube FLASH Movie.

Of course we focus on results and concepts and don’t bother for the moment about the technical details of Papervision or FLAR.

This is our starting download:

http://www.contrechoc.com/flash/AR_Crosslab.zip

www.blender.org for BLENDER