Near-optimal Character Animation with Continuous Control
- 1. Near-optimal Character Animation with Continuous ControlAdrienTreuille, Yongjoon Lee, ZoranPopović2008.10.14 HA SE HOON
- 3. Motion GraphA directed graph to synthesize a new motionNode = PoseEdge = Motion ClipWe already discussed in 4th presentation“Group Motion Graph”, Yu-Chi Lai, Stephen Chenney, ShaoHua FanPresented by Heo Jae PilWe will use a similar but different method!Prerequisite #1: Motion GraphShort MotionRunningJumpingWalkingFBASShort MotionShort MotionNShort MotionDrawn by Heo Jae Pil
- 4. A sub-area of machine learningHow agent should take action?Goal: Maximize rewardModel of reinforcement learningA set of environment states SA set of action AA set of scalar reward in RWe will use reinforcement learning to find a near-optimal actionPrerequisite #2: Reinforcement Learning
- 5. Finding a “near optimal” character animationWith real-time continuous controllerTasksNavigation (Walking)Spinning NavigationFixed Object Avoidance (FOA)Moving Object Avoidance (MOA)Introduction: The goal of the paperReal Time Continuous Controller(Arrows under the char.)
- 6. You should know…How to represent motions, states, policiesHow to blend motion clipsHow to define the cost functionHow to find the near optimal policyIntroduction
- 7. Motion Graph KOVAR, L., GLEICHER, M., AND PIGHIN, F. 2002Motion synthesis from annotationsARIKAN, O., FORSYTH, D. A., AND O’BRIEN, J. F. 2003Precomputing avatar behavior from human motion dataLEE, J., AND LEE, K. H. 2004Related Works
- 8. A set of motion clips СOne of each clip C = (p1, p2, …, pm)Each pose p = ℝnA Vector specifying all joint positionsMotion Model: Definition of terms
- 9. Motion Model:Motion ClipsAssumption: A clip C represents one walk cycleConstraint frames: (b) for Cin, (d) for CoutCinCoutDivided into two subsequences Cin and CoutEach subsequence covers one foot plant frameThis frame became a ‘constraint’ frame
- 10. Allow any transition between two clipsUnlike “motion graph”!AlgorithmStep 1: C is mirrored if necessaryStep 2: Overlap constraint frames of Cout and C’inStep 3: Rotate C’ to match its foot (C’ is reoriented so that its ground-contact foot coincides with that of C)Step 4: Blend Cout to C’in(With ground-contact foot as the root of kinematic skeleton)Motion Model: Blending motion clips
- 11. Motion Model: Example of motion blendingConstraint frames (foot-planted frames)Prevent foot-skating!!! Why?
- 12. Control: Goal tasksNavigationSpinning NavigationUser controls gait, path, torso orientation(example of gait: walking, running,…)User controls the motion direction as character spinsFixed Obstacle AvoidanceMoving Obstacle AvoidanceThe character follows a line, avoiding fixed planar objectThe character follows a line, avoiding fixed planar object with linear motion
- 13. State XC: current clipx, z, Ɵ : position and orientationu, v, u’, v’: relative position and speed of obstacleG, T, W: desired Gait, Torso Orientation, SpinControl: Definition of the stateIntention of user
- 14. Transition function f: S x C SX = (C, …) X’ = (C’, …)Control: Transition function
- 15. State Cost Cs: S ℝIf current clip is not desired gait then Cs(X) = ∞(If X = (C, …) but C ∉ G, then Cs(X) = ∞)Transition Cost Ct : S x S ℝControl: CostsWeights of each term.
- 16. Policy ∏ : S CDecide next clip C from given state SThen we can move to next state S’ with transition function f(X, C) X’ (X/X’ = current/next state)Policies: DefinitionDefinition of state and transition function
- 17. Greedy Policy ∏greedyFind the clip that minimize the cost!Just look one stateBut we should consider the following case:Policies: Greedy Policy
- 18. We will minimize the entire costTaking into account the futureRedefine the cost functionFor given some policy ∏ that produces (X1, X2, … Xn)α ∈ [0, 1) : future uncertainty factorPolicies: Optimal Policy
- 19. Value function V: S ℝLong term cost under optimal policy ∏*V(X) = C∏*(X)Def: Optimal policy isThus, if we can calculate value function V(X), We can find long-term optimal next state!Policies: Value function
- 20. Now issue is approximation of V(X)Linear approximation of basis functionsBasis functions Ф = (Ф1, Ф2, … ,Фn)Each basis functions Ф : S RPolynomials or GaussiansThen, Approximation of V isBasis functions are pre-selected by human!So, how can we calculate weights r1, r2, … rn?Policies: Near optimal policy
- 21. Draw set of sample clipsInit state transition pairs and rDef: r = a vector of weights (r1, r2, … rn)Step1X’ is the optimal next state from X under current VStep 2Recalculate r by solving the linear programPolicies: Near optimal policy (Algorithm)
- 22. Policies: Near optimal policy
- 23. SwitchabilitySwitching between tabulated value functionWhen transition from a walking to a running, the algorithm still picks near-optimalSeperabilityLearn separate value function for each taskEx) MOA and FOAPolicies: Dimensionality
- 24. Let’s see the videoResults
- 25. Presents a new control modelHigh dimensionalContinuousReal timeNear optimalBut…Needs a large amount of databaseConclusion
- 26. Any question?