We propose to decompose a human behavior sequence into relevant motion segments.

Each motion segment is described by both human motion and depth appearance around hands.

We model the sequence dynamics using a Dynamic Naive Bayesian classifier.

We evaluate the method for various types of behavior: gesture, action and activity.

The challenge of online detection of successive behaviors is also investigated.