Baby Sitting

While using Ipi Soft I've been trying to monitor how I use the program and what I'm having to teach myself while I work.
I've found that the new updated version does quite a bit of the calibration work for you now - which is great. It was the most time consuming part.
However, it still makes errors on matching the skeleton to the kinect information. The errors can be as small as losing a hand for a frame or two, to whisking away the whole model, and these errors can really build on themselves. This is why I've got into the habit of baby sitting the process - pausing it, moving parts of the body into what i think should be the correct position and moving ahead again.

I've even started to try and trick the software. I think it must use the previous few frames (at least the one previous frame) as a starting point, then shifts into the correct position. To correct an error that continuously errors, and affects the next frames, I've started moving a body part for the last few frames - then tracking forward from the latest point.
In effect this is telling the software that the previous 2-3 frames are correct, use these and then shift slightly. By doing this I have a little more control over where the software 'thinks' that certain wayward body parts should go.

The T-Pose

First Hurdle

So my first major issue was getting the system up and running and able to work.
This was installing drivers, getting the kinects online and making sure I could record.
After scouring the internet for answers as to why I couldn't use two kinects on the current software I found that we needed an update - so grabbed a demo. This is what I'll be using, the demo of Ipi Soft 2.0 with two Kinects.

For quite a while I was having issues with frame rates and the amount of data I was getting from the infra red. It was quite minimal, and had quite an effect on the output.

FirstBVHFast from Joshua Griffin on Vimeo.

This is the first BVH animation. The reason the skeleton is so fast is due to the drops in the framerate of the recording. It went down to about 12fps - meaning it looks like an old black and white rollup film speed. This is pretty dire for what I need to be doing.

[edit] !! Figured out why it was at such a low framerate and can now use both kinects at 30fps perfectly. There were two main issues - and it's silly really. I was recording on Ipi Recorder both the infra red depth AND RGB data. This was too much for the kinects' bandwidth, which is why it was bottlenecked and started killing frames. Another issue (a slightly more minor one) was that I was running Ipi Studio (opposed to Ipi Recorder) at the same time. This obviously taxed the laptop too much and affected the output. Scary really.

Where I was at

Before embarking on this project, I had trialled version 1.3 of Ipi Soft using 6 Playstation Eye cameras. The 6 cameras were formed in a complete circle and captured RGB data at 30fps. This was then calibrated in a dark space using a light (the luma from the RGB would be used in the calibration). The performance was then captured and later processed through Ipi Studio. Both calibration and processing would take 2-3 hours, and possibly longer if there was an issue with a recording.
The data from this, and subsequently the animation, wasn't of a high enough quality. The process also couldn't capture some faster movements.

I also was a complete novice when it came to cleaning motion capture. I had never used motionbuilder and had no idea how to clean keys (I still don't!!)

What's going on

So to build on my previous experience with the Ipi Soft kit, I've decided to research it's updated features and look into a pipeline to fit this into my main game creation project.

This will involve researching and trialling tests with the current software, creating a best practice and working method, creating a short wiki and several motion captures.
I also plan to look into motion capture clean-up. I'll be doing this in an academic capacity and also cleaning a choice few of my animations.

As I'll be looking into the pipeline, I'll also have quite an in-depth look at rigging and how I can best build a rig to edit the motion capture data.