Immersive 3D Environment Using Kinect and Voice Commands
Download as PPTX, PDF1 like545 views
This document discusses the use of Kinect technology to create a more immersive experience in 3D gaming environments, emphasizing user interaction through hand gestures and voice commands. It critiques current gaming limitations, proposes solutions for content creation and interaction, and outlines future perspectives for enhancing multiplayer experiences and integrating with other technologies. The application aims to engage multiple senses, allowing users to sketch and edit game levels dynamically.
Immersive 3D Environment Using Kinect and Voice Commands
- 1. Utilizing Kinect control for a more immersive interaction with 3D environments Supervised by Dr. Ammar Joukhadar Dr. Noor Shaker Eng. Mohammad Shaker Designed, Implemented and Tested by Mohammed Saed Haj Ali Marah Halawa Kinda Tarboush F.I.T.E of Damascus, Syria – AI Department 2014
- 2. Content • Motivation • The Problem with Today’s Games • Modeling • Sketching a Level • Generating the Basic Level Structure • Allow Edit and Creation on Content through in-game Interactions • Hand Gesture • Voice Command • Our Application • Future Perspectives • Demo
- 3. Content • Motivation • The Problem with Today’s Games • Modeling • Sketching a Level • Generating the Basic Level Structure • Allow Edit and Creation on Content through in-game Interactions • Hand Gesture • Voice Command • Our Application • Future Perspectives • Demo
- 4. “Why using a stylus when God gave us ten fingers!” Steve Jobs when introducing the first iPhone, 2010
- 6. Good design looks great, yes,
- 7. Good design looks great, yes, but
- 8. smell great and sound great? Good design looks great, yes, but why shouldn't it also feel great,
- 9. smell great and sound great? Good design looks great, yes, but why shouldn't it also feel great,
- 10. Jinsop Lee “Design for all 5 senses” The Concept of
- 11. Content • Motivation • The Problem with Today’s Games • Modeling • Sketching a Level • Generating the Basic Level Structure • Allow Edit and Creation on Content through in-game Interactions • Hand Gesture • Voice Command • Our Application • Future Perspectives • Demo
- 12. The player mostly has only one method to interact with the game (most games has the same playing peripherals) like joystick or mouse
- 13. Most games don’t let the player to create their own content worlds and share it with others.
- 14. For most games there’s no direct interaction between the game and the player (only one way from the player to the game)
- 15. For most games there’s no direct interaction between the game and the player (only one way from the player to the game)
- 16. A Game For You to Use Your Senses
- 17. Content • Motivation • The Problem with Today’s Games • Modeling • Sketching a Level • Generating the Basic Level Structure • Allow Edit and Creation on Content through in-game Interactions • Hand Gesture • Voice Command • Our Application • Future Perspectives • Demo
- 18. Sketching a Level Sketch Your Level
- 19. Using Kinect
- 20. Interaction with the environment Hand Gesture
- 21. Interaction with the environment Voice Command
- 22. Our Game Put here a picture or a short video of the game. Just so that people know what are you talking about
- 23. The Immersion Process Player Sketch Generating basic Level Canvas Allow Edit and Generation of Content
- 24. The Immersion Process Player Sketch Generating basic Level Canvas Allow Edit and Generation of Content
- 25. The Immersion Process Player Sketch Generating basic Level Canvas Allow Edit and Generation of Content
- 26. Sketching a Level Put here a picture of a sketch and a picture of the height map of the same sketch
- 27. The Immersion Process Player Sketch Generating basic Level Canvas Allow Edit and Generation of Content
- 28. Sketch to Level Conversion Talk about the conversion from sketch to level and the cuboid algorithm you did (with picture)
- 29. The Immersion Process Player Sketch Generating basic Level Canvas Allow Edit and Generation of Content
- 30. Hand Gesture Interaction with Dynamic Hand Gesture Interaction with Static Hand Gesture
- 31. Dynamic Hand Gesture • Animated gestures
- 32. Static Hand Gesture • Comparison with reference images • Using machine learning • Rule based system
- 33. Select Rotate Delete Move The gestures that we will use
- 34. Hand gesture in our application Dynamic Hand Gesture Static Hand Gesture
- 35. Hand gesture in our application Dynamic Hand Gesture Discrete Markov Processes Static Hand Gesture Rule based system
- 36. Hand gesture in our application
- 37. Interaction via Voice Commands
- 38. Voice Command • Audio to text conversion
- 39. Voice Command • Speech Recognition Hidden Markov Model
- 40. Voice Command • Words Meaning Conceptual Grammar
- 41. Content • Motivation • The Problem with Today’s Games • Modeling • Sketching a Level • Generating the Basic Level Structure • Allow Edit and Creation on Content through in-game Interactions • Hand Gesture • Voice Command • Our Application • Future Perspectives • Demo
- 42. Our Application Generate 3D environment and edit it using most of your senses!
- 43. Content • Motivation • The Problem with Today’s Games • Modeling • Sketching a Level • Generating the Basic Level Structure • Allow Edit and Creation on Content through in-game Interactions • Hand Gesture • Voice Command • Our Application • Future Perspectives • Demo
- 44. Future Perspective • Enhance the social aspect of the game • Multiple player playing at the same time at the same level • Assess the game immersions metrics • Allow for more immersive and enhanced gestures • Analyze the applicability of the approach on other hardware (like Oculus Rift) • Analyze the applicability of the approach on other game genres
- 45. Content • Motivation • The Problem with Today’s Games • Modeling • Sketching a Level • Generating the Basic Level Structure • Allow Edit and Creation on Content through in-game Interactions • Hand Gesture • Voice Command • Our Application • Future Perspectives • Demo