Merging Retrieval of Thresholded Points from feed and feeding it into Planar Estimation.

I had encountered a bug with the "ideal case" for pose-estimation, which I apparently missed while working on random sample cases, and have since fixed it.

Since Integrating the feed to the program I had encountered odd behavior occurring with the normal calculation, and decided to go back to testing smaller cases.  Doing so I realized the following problem:





THIS is what all the people have been talking about when they said there are two corresponding planes that could result from coplanar points. I had initially thought they mean their were simply two normals, one going one direction, the other going the other direction.

Also, there are a number of things I've forgotten to do for the transition, particular going from Cartesian Coordinates to Barycentric Coordinate Systems, which would be useful for visual debugging, however I neglected to do because I thought RELATIVELY it'd still calculate the normals correctly.

Fixed some visual issues, and am able to display some MARGINAL success:

blue detection point still looks off.  Probably an error in my conversion of coordinate systems.

CoPlanar Pose Estimation - Iterated Error Minimization approach

So this week I finished testing out the approach using 3 coplanar points with the following givens:
- Two of the points are equidistant from a third point.  (generating two vectors rg and rb, where:
 |rg| = |rb|)
- Those two vectors are also perpendicular to each other. (allowing you to assume dot(rg, rb) = 0 ).

Breaking down the two equations, you reach:
rgZ = - (rgX*rbX + rgY*rbY)/rbZ
and
rbZ = sqrt( rgX^2 + rgY^2 + rgZ^2 - rbX^2 - rbY );

Using these two formulas, it is possible to iteratively guess at one (or both) of the possible normals, in my demo I chose only to guess at one of the normals, thinking it should be possible to determine which one you want by always being the one that points towards you.  This needs to be tweaked a bit, as this result isn't being reached based on my current approach.

Some screen shots (thick transparent lines are expected normals, thin opaque lines are estimated normals ):

As you can see the guesses are by no means perfect, as there is both a large error visible here, and the cyan is facing the wrong direction.


What seems like a nice estimation is rendered unlikely to occur for the magenta normal as those src points would be very unlikely to attain on a planar target.

Another example of the normal facing the other direction.

Log of random points ((rgX,rgY, rgZ) and (rbX,rbY, rbZ)) being fed to algorithm:
VecG is simply rgX,rgY with the computed rgZ. Same is true for VegB.
Normal is the cross-product of both.
"Error" is computed by using the second (or the unused) equation used to compute the values.

VecG: x: 31.5836 y: -78.3898 z: -84.347
VecB: x: -29.5511 y: -90.051 z: 72.6256
NormalVector is: x: 31.5836 y: -78.3898 z: -84.347
"Error", should be very close to Zero: 1.42109e-014

VecG: x: -68.688 y: -80.2789 z: -43.0637
VecB: x: -37.2845 y: -24.4362 z: 105.024
NormalVector is: x: -68.688 y: -80.2789 z: -43.0637
"Error", should be very close to Zero:  1.42109e-014

VecG: x: 47.2518 y: -11.8229 z: -49.4074
VecB: x: 48.9731 y: -7.34275 z: 48.5935
NormalVector is: x: 47.2518 y: -11.8229 z: -49.4074
"Error", should be very close to Zero: 1.42109e-014

VecG: x: -20.7617 y: -77.7093 z: -95.7285
VecB: x: 24.3629 y: -97.7477 z: 74.0646
NormalVector is: x: -20.7617 y: -77.7093 z: -95.7285
"Error", should be very close to Zero: 1.42109e-014

VecG: x: -71.3187 y: -10.7273 z: 103.581
VecB: x: 99.1394 y: -45.4115 z: 63.5577
NormalVector is: x: -71.3187 y: -10.7273 z: 103.581
"Error", should be very close to Zero: 1.42109e-014

VecG: x: 16.3549 y: -28.9468 z: -30.6332
VecB: x: 28.4036 y: -16.7058 z: 30.9507
NormalVector is: x: 16.3549 y: -28.9468 z: -30.6332
"Error", should be very close to Zero: 1.42109e-014

VecG: x: -8.72524 y: -24.6681 z: -76.4579
VecB: x: -70.22 y: -35.023 z: 19.3131
NormalVector is: x: -8.72524 y: -24.6681 z: -76.4579
"Error", should be very close to Zero: 1.42109e-014

VecG: x: -50.6455 y: -88.3694 z: 19.7959
VecB: x: 57.7013 y: -14.008 z: 85.0901
NormalVector is: x: -50.6455 y: -88.3694 z: 19.7959
"Error", should be very close to Zero: 1.42109e-014

VecG: x: -19.48 y: -20.7312 z: -95.9996
VecB: x: -96.9359 y: -11.6703 z: 22.1902
NormalVector is: x: -19.48 y: -20.7312 z: -95.9996
"Error", should be very close to Zero: 4.26326e-014

VecG: x: -36.8877 y: -35.8715 z: 109.191
VecB: x: 87.1273 y: -83.517 z: 1.99698
NormalVector is: x: -36.8877 y: -35.8715 z: 109.191
"Error", should be very close to Zero: 1.98952e-013



I also looked into other approaches of Coplanar Pose Estimation and found a REALLY NICE demo using AForge.. a C# framework...  here is a screenshot:



The theory is described in a paper titled "Iterative Pose Estimation using Coplanar Feature Points" written by Oberkampf, Daniel DeMenthon and Larry Davis.  It uses four source points and generates the matrix formed by the translation and rotation transformations, along with the alternate matrix (as it is possible to have two normals.)


Goals for Next time:
- Integrate normal estimation with captured source points.
- Improve upon estimation approach through other algorithms (reverse engineer Coplanar Class from AForge?)

Posit Algortithm, OpenGL + OpenCV libraries

EDIT:
I am back tracking and scaling down the problem that I'll be working on this week to detect the orientation of a plane, given known points, as I mistook the scope of this problem's difficulty level.

I've incorporated openGL libraries in order to allow me to visually track the progress of my work with pose estimation, and have currently integrated a posit demo found on willow garage.


I've also created an improvised "AR target" with a valid detected face image, along with 3 points to use as control points for orientation, using the concept of thresholding to derive the source points to be used by the Posit algorithm for pose estimation.


I am currently reading up on the Posit Algorithm (http://www.cfar.umd.edu/~daniel/daniel_papersfordownload/Pose25Lines.pdf), described by Daniel F DeMenthon and Larry S Davis.

A concern arises at the start of the first paragraph:
"We assume that we can detect and match in the image four or more noncoplanar feature points of the object and that we know their relative geometry on the object "

An alternative that I plan to look into after I finish reading the paper is located at: http://nghiaho.com/?page_id=576 , which describes a library for Robust Pose Estimation from a Planar Target

Week 2 Face Detection with Haar Classifier, Next Step Pose Estimation

At the end of week one, following a tutorial on how to write a Haar Classifier for Face detection, I am able to now track a face, and using multi-threading am able to present a program (for now, the display of the webcam feed) on a separate layer from the detection calculation.

Added in a frame capture to document what I have so far.
Some images of the face detection capture:

(Good)

Unexpectedly, the image also has the previous rectangle in the scene. Not sure why exactly, however, perhaps this could lead to work in motion estimation.
Some Examples:
(First Image of Sequence)


(Second Image of Sequence, lingering rectangle in image)


(Third Image of Sequence, only one lingering rectangle in image)



My next steps will be to do pose estimation, perhaps beginning on static images.
Currently I am studying the following documentation of previous work on pose estimation to get a better idea of what I need to implement:
Real-Time Face Pose Estimation in Video Sequence, Xiaoping Chen, Qianqian Yang, Honghong Liao, Weiping Sun, Shengsheng Yu, College of Computer Science and Technology, Huazhong University of Science and Technology

and

Head Tracking and Gesture Recognition Library, Louis-Philippe Morency

- Alfred