03 synchronized distance learning
Download as PPTX, PDF1 like2,191 views
The document discusses synchronized distance learning and summarizes key topics covered, including: 1. Whiteboard and chat room tools that allow for online annotation and floor control. 2. Multimedia synchronization methods like Microsoft ASF and SMIL that integrate different media formats and allow for event-based synchronization. 3. Synchronization models like Petri nets and interval temporal logic that can be used to model and analyze multimedia presentations.
![Definition 5
• A marking μ of a Petri net C = ( P,T,I,O ) is a
function from the set of places P to the
nonnegative integers N; μ : P → N
• The marking can also be defined as an n-vector,
μ = [ μ1 μ2 …μn ]
• The number of tokens in place pi is μi, i =
1, …, n
– μ(pi ) = μi](https://image.slidesharecdn.com/03-synchronizeddistancelearning-130929222404-phpapp02/85/03-synchronized-distance-learning-89-320.jpg)
![A Marked Petri Net of Example 1
• The marking μ = [ 1 2 0 0 1 ]
t1 t2
t3
t4
P1 P5
P2
P4
P3](https://image.slidesharecdn.com/03-synchronizeddistancelearning-130929222404-phpapp02/85/03-synchronized-distance-learning-91-320.jpg)
![Another Marked Petri Net of Example 1
• The marking μ = [ 0 2 0 5 1 ]
t1 t2
t3
t4
P5
P2
P4P1
P3](https://image.slidesharecdn.com/03-synchronizeddistancelearning-130929222404-phpapp02/85/03-synchronized-distance-learning-92-320.jpg)
![User Edges and Derived
Edges
User edges:
( A, B ) = { < } = [1]
( B, C ) = { m } = [7]
( C, D ) = { d } = [3]
( C, E ) = { s } = [9]
( F, D ) = { < } = [1]
2. Path Length = 3
(A, E) = (A, B ) o (B, C) o (C, E) = (A, C ) o (C, E)
= [1] o [9] = [1] = { < }
(A, D) = (A, B ) o (B, C) o (C, D) = (A, C ) o (C, D)
= [1] o [3] = [22] = { <, o, m, d, s }
(B, F) = (B, C ) o (C, D) o (D, F) = (B, D ) o (D, F)
= [16] o [1]-1 = [23] = { >, oi, mi, di, si }
(E, F) = (E, C ) o (C, D) o (D, F) = (E, D ) o (D, F)
=[14] -1 o [2] = [15] o [2] = [2] = { > }
3. Path Length = 4
(A, F) = (A, B ) o (B, C) o (C, D ) o (D, F)
= ( (A, B ) o (B, C) ) o ( (C, D ) o (D, F) )
= (A, C ) o (C, F) = [1] o [2] = [29]
= { <, >, d, di, o, oi, m, mi, f, fi, s, si, e }
Derived edges:
1. Path Length = 2
(A, C) = (A, B ) o (B, C) = [1] o [7] = [1] = { < }
(B, D) = (B, C ) o (C, D) = [7] o [3] = [16] = {o, d, s }
(C, F) = (C, D ) o (D, F) = [3] o [1]-1 = [3] o [2] = { > }
(D, E) = (D, C ) o (C, E) = [4] o [9] = [14] = {o, di, fi }
(B, E) = (B, C ) o (C, E) = [7] o [9] = [7] = { m }](https://image.slidesharecdn.com/03-synchronizeddistancelearning-130929222404-phpapp02/85/03-synchronized-distance-learning-148-320.jpg)
![Adding Edges
Adding an user edge to Example 4.1:
Add ( G, E ) = { f } = [11]
Derived edges :
Derive ( E, G ) = ( G, E )-1 = [11]-1 = [12] = { fi } ( inverse )
Derive ( A, G ) = ( A, E ) o ( E, G ) = [1] o [12] = [1] = { < } ( association )
Derive ( B, G ) = ( B, E ) o ( E, G ) = [7] o [12] = [1] = { < } ( association )
Derive ( C, G ) = ( C, E ) o ( E, G ) = [9] o [12] = [18] = {<, o, m }
Derive ( D, G ) = ( D, E ) o ( E, G ) = [14] o [12] = [24] = { <, o, m, di, fi } ( association )
Derive ( F, G ) = ( F, E ) o ( E, G ) = ( E, F )-1 o ( E, G ) = [1] o [12] = [1] = { < } ( inverse, association )](https://image.slidesharecdn.com/03-synchronizeddistancelearning-130929222404-phpapp02/85/03-synchronized-distance-learning-150-320.jpg)
03 synchronized distance learning
- 2. Outline • White Board and Chat Room – Designing Shared Whiteboard • Multimedia Synchronization – Microsoft ASF – SMIL • Synchronization Models – Petri Net – Interval Temporal Logic (optional)
- 3. White Board and Chat Room • Chat Room – Almost Everywhere • White Board – On-Line Annotation with Floor Control
- 4. Text Acknowledgement Timothy K. Shih and Yi-Chun Liao, MINE Lab, Tamkang University, Taiwan • Text and simple graphic drawing • Annotation message broadcasting via multiple uni-casting • NetMeeting 3.0, COM (Component Object Model), ActiveX control • Acknowledgement from Remote Stations is included • Floor Control: Control Modes, Channel, and Group On-Line Annotation
- 5. Request Button Request to SpeakErrorAcknowledgement Question Button Speaking Return Control Interface and Demo
- 7. • 30 Client connections • Heterogeneous Network Demo Video
- 8. • The control of ―Who to Speak‖ in a communication environment • Concepts: – Channel: communication modal (text, audio, video, etc.) – Mode: communication style of a channel (Free Access vs. Equal Right) – Group: the IP domain of a channel • Example: 4 Channels, 3 Groups, 10 Persons, 2 Modes Video Conferencing (ER) and Chat Room Tool (FA) Chat Room Tool (FA) Annotation Tool (ER) Floor Control
- 9. Chat RoomAnnotation Virtual PanoramaShared Notebook Video Audio Video-on-Demand Multi-modal and Multi- channel Communication
- 10. • Enhance the clearness of video presentation • Add bitmapped animation to the video Augmented Video Conferencing Tool
- 11. Augmented Video Presentation Screen Capture Video Control Video + Audio MPEG-2
- 13. Implementation • Configuration • Five Control Modes – Normal – Resized Screen – Resized Video – Full Video – Full Screen
- 14. • Audio and Video Devices and Codec • Screen and CCD Camera Resolutions Device and Layout Configuration
- 16. Normal Mode
- 19. Full Video Mode
- 20. Full Screen Mode
- 21. • Problems to Solve – Distance Precision – Space Discontinuity – Video Precision – Eye Contact – Object Tracking • The instructor • The whiteboard – Interaction Protocol • Floor Control – Performance Challenges
- 22. Demo Demo
- 23. Multimedia Synchronization • Why Multimedia Synchronization • Synchronization Methods – Microsoft ASF – SMIL • Synchronization Models (Theories) – Petri Net – Interval Temporal Logic
- 24. Video Music Audio Slide Message Annotation ASF Records Timothy K. Shih, Y. R. Liu and Sheng-En Yeh, MINE Lab, Tamkang University, Taiwan • Automatic Recording • Use Commercial Codec and the Advanced Streaming Format (ASF) from Microsoft • Video and Slide Synchronization • Text and Drawing Annotation Synchronization • Slide Selection • Presentation Editing • Automatic Loading Multimedia Synchronization Using ASF
- 25. • Event markers are embedded in a ASF file • Arbitrary activation of markers • Do not record the entire stream as one video clip • Video streaming depends on Codec Video Stream Text User Interrupt User Interrupt User Interrupt Slide Slide Slide MS Office XP – Presentation Broadcast provides video and slide synchronization (PowerPoint and IE) Event-based Synchronization
- 26. Choose Device Choose Codec Presentation Recording Tool
- 27. Select Output Slide Selection Presentation Playback Control Functions Presentation Tool
- 28. • Hard to record the entire lecture without making a mistake • Different presentation needs different level of details • Lecture summarization and editing • Atomic presentation object – a PowerPoint Slide (with its video and annotation events) Text Atomic Object Adaptive Presentation
- 29. Three Levels of Details Automatic Loading Presentation Editing Multi-Level Presentation
- 31. Synchronized Multimedia Integration Language (SMIL) • Recommended by W3C • XML-based language • Designed for Internet use • Easy to write similar to HTML • Integrates several media formats • Plain text documents with a ―.smi‖ or ―.smil‖ suffix Part of slides about SMIL are summarized from: http://service.real.com/help/library/guides/production/realpgd.htm
- 32. History • Dec 1995 - SMIL designing started • Nov 1997 - SMIL 1.0 Recommendation • Mar 1998 - First SMIL implementation • Aug 1999 - First public draft of SMIL 2.0 • Aug 2001 - SMIL 2.0 Recommendation
- 33. Support for SMIL in Players / Browsers 2.0 • X-Smiles 0.40: SMIL 2.0 Basic Profile 2.0 • Oratrix GRiNS player: SMIL 2.0 Language 2.0 • IE 5: many SMIL 2.0 modules 1.0 • QuickTime Player 4.1 1.0 • RealPlayer 8 1.0 • Compaq HPAS 1.0 • Productivity Works Lp Player 1.0 • SOJA from Helio 1.0 • S2M2 by NIST 1.0 • Schmunzel by SunTREC Salzburg
- 34. Use Cases for SMIL • Slideshows • Advertisements • Internet TV • Education • Corporate communications • Product information • User’s Guides • Net meetings • etc.
- 35. General Rules • Documents start with <smil> and end with </smil> • <head> section, which is optional; and a <body> section which is required • Tags and Attributes in lowercase only • All elements must have closing tags or / • Attribute values must be enclosed in double- quotes • HTML-style comments are allowed. – <!-- comments go here -->
- 36. Typical SMIL Elements <head> - Head element <meta> - Meta data, such as author, copyright... <layout> - Layout of the presentation <region> - Region describing positioning <body> - Body element <seq> - Sequential time container <par> - Parallel time container <img>, <audio>, <video> - Media to be played <switch> - Selects between elements <a>, <area> - Linking
- 37. A SMIL Example <smil> <head> <layout> <region id=”reg1” top=”0” left=”0” width=”300” height=”200” /> <region id=”reg2” left=”300” width=”300” height=”200” /> </layout> </head> <body> <seq> <img id=”id1” src=”intro.jpg” region=”reg1” dur=”4s” /> <audio id=”id2” src=”music.wav” dur=”2s” /> <par> <video id=”id3” src=”movie5s.mpg” region=”reg1” /> <img id=”id4” src=”intro.jpg” begin=”3s” region=”reg2” /> </par> </seq> </body> </smil>
- 38. <head> element • Information not related to the temporal presentation of the material • May contain the following children – <meta> – <switch> – <layout>
- 39. <meta> element • Just like the <meta> element in HTML • An element used to specify information about the document • Common uses include: – <meta name="author" content="Jim Dabrowski"/> – <meta name="title" content="Jim’s SMIL Presentation"/> • Other uses: copyright, abstract, etc.
- 40. <layout> element • Probably the most important element • Specify the layout of the document, as well as the layout of the multimedia elements in the document • If not specified, how document looks will depend on the player • Two children allowed here – <root-layout> – <region>
- 41. <root-layout> element • An empty element • Defines the size of the main window • You cannot specify media elements in here • There can only be one <root-layout>
- 42. <root-layout> attributes • background-color – Can use named or hexadecimal values. • height, width – All specified in pixels • title
- 43. <region> element • Areas within <root-layout> in which media elements will appear • May define as many as you want • Very similar to frames in HTML • Are allowed to overlap — controlled by z- order • Any regions that lay outside the <root- layout> will be cut off
- 44. Required attributes • id – A unique text name for the region • top, left, width, height – Pixels or percentages or mixed
- 45. Example • <head> <root-layout width="480" height="300"/> <region id=“someid" top="60" left="120" width="240" height="180"/> </head> Screen (60, 120) 240 180
- 46. <body> element • Information related to the temporal and linking behavior of the document • Children include – Synchronization elements • <par>, <seq> – Media elements • <animation>, <audio>, <img>, <text>, <textstream>, <video>, <ref>
- 47. Synchronization elements • <par> element — children of a <par> element occur in parallel • <seq>element — children of a <seq> element appear in sequential order • Valid children of <seq> and <par> elements include – <seq> & <par> – Any of the media elements
- 48. Combining <seq> and <par> tags <seq> clip 1 <par> clip 2 clip 3 </par> clip 4 </seq> <par> clip 1 <seq> clip 2 clip 3 </seq> clip 4 </par> 1 2 3 4 3 1 2 4
- 49. Media elements • The various pieces of multimedia that go into the SMIL document • Continuous media – <animation>, <audio>, <textstream>, <video> – Have an intrinsic duration • Discrete media – <text>, <img> – Have no intrinsic duration • The actual tag name doesn’t matter • Type attribute is used to determine media type • Could use <ref> tag
- 50. Media element attributes • All media elements have certain attributes in common • You use these attributes to specify how the children of <seq> and <par> elements will appear in the presentation
- 51. Specifying media and location • src – Specifies the source of the media element – Identical to the src attribute of the <img> tag in html • region – The area of the root-layout in which the media element is to be displayed – e.g., <video src="hal.mov" region="video_region" /> – If you don’t specify a region, it’s implementation dependent
- 52. Setting begin and end time • begin & end attributes • Specifies the explicit beginning and ending time of the element – e.g., <video src="hal.mov" begin="10.5s" end="45.5s" /> • Relative to parent element regardless of own length • May use with <par> and <seq> elements to affect entire group
- 53. Setting internal begin & end times • clip-begin & clip-end attributes • Specifies the internal beginning and ending time of a clip – E.g., <video src="hal.mov" clip-begin="2.5s" clip- end="15.8s" /> • May combine with begin and end attributes
- 54. Setting a duration • dur attribute • Explicit duration of the media element • Time value, or ―indefinite‖ • Should not use dur with end attributes
- 55. Filling up time • fill attribute – Specifies what happens to the clip when it is done – Freeze or remove • repeat – Specifies how many times the clip should repeat itself – Any integer is valid – Can add repeat attribute to the <par> and <seq> elements
- 56. Synchronizing elements • By default, <par> elements end when all elements finish • Can be modified with the end attribute to force an end time for the <par> element • Or you can use the endsync attribute of the <par> element to synchronize the ending of its clips • Three values for the endsync attribute – first, last, id(element id)
- 57. Linking • SMIL provides two ways to link to other documents • <a> tag – Identical to the <a> tag in html – Specify an href attribute with a URL – Can surround and turn an entire media element into a link • <anchor> tag – Defines a ―hot spot‖ inside a media element – Similar to HTML image maps
- 58. <anchor> tag • Specified as a child of a media element – <video src="video.rm" region="video_region"> <anchor href="url here" coords="20, 40, 80, 120"> </video> • Where the coords attribute specifies the location within the media element that is the hot spot • May use pixels or percentages
- 59. Control Structure • SMIL provides a <switch> tag • Can use it to allow player to choose one of several test-attributes • Format <switch> <choice1 test-attribute="value1"/> <choice2 test-attribute="value2"/> … </switch> • Various test-attributes have been defined
- 60. Using <switch> for language • You can use the <switch> tag to allow player to choose the appropriate language • <par> <video src="video/seattle.rm" /> <switch> <audio src="french/seattle.rm" system-language="fr"/> <audio src="german/seattle.rm" system-language="de"/> <audio src="spanish/seattle.rm" system-language="es"/> <audio src="english/seattle.rm"/> </switch> </par>
- 61. Using <switch> for system bitrates • <switch> <par system-bitrate="75000"> <!-- dual ISDN or faster --> <audio src="audio/newsong1.rm"/> <video src="video/newsong1.rm"/> </par> <par system-bitrate="4700"> <!-- single ISDN --> <audio src="audio/newsong2.rm"/> <video src="video/newsong2.rm"/> </par> <par system-bitrate="20000"> <!-- 28.8 modem --> <audio src="audio/newsong3.rm"/> <video src="video/newsong3.rm"/> </par> </switch>
- 62. For information • W3C’s Synchronized Multimedia Group – http://www.w3.org/AudioVideo • Just SMIL – http://smw.internet.com/smil
- 63. Petri Net • What are Petri Nets – A tool for system study • System Modeling – Structure – Dynamic Behaviors • System Evaluation Source: Feng-Cheng Yang These slides are modified based on Feng-Cheng Yang’s presentation
- 64. Using Petri Net • Modeling – Representation for a System – Often in Mathematical Terms – Manipulated to reveal information • Example of Discrete System Analysis – Bank System • Teller Components – Idle state – Busy state • Customer Components – Idle state (waiting in the line) – Busy state (being served)
- 65. Multimedia Presentation SMIL <par> clip 1 <seq> clip 2 clip 3 </seq> clip 4 </par> 3 1 2 4 Petri Net (simplified) 3 1 2 4
- 66. Features of a System • Components – Subsystems – Separated and Interacting • States – Depend on the past history – Change over time
- 67. System Behaviors • Concurrency (Parallelism) – Activities of different components occur simultaneously – Difficult to model • Sequencing – Serial activities occur one by one between components – Easy to model
- 68. History • Basic concept was created in 1962 by Carl Adam Petri in his doctoral thesis (W. Germany) – Communication between asynchronous components of a computer system • Early theory, notation, and representation are developed by the Information System Theory Project (Applied Data Research, US) – Conduct by A.W. Holt (1968)
- 69. History (cont.) • Considerable research and publication in 1970 – Directed by Prof. Jack B. Dennis at M.I.T. • Conference on Petri Nets and Related Methods in 1975 at M.I.T. • Workshop on Petri Nets held in Paris in 1977 • In 1979 an advanced course on General Net Theory is held in Hamburg • A Special Interest Group on Petri nets was formed in Germany
- 70. Petri Net Structure (Mathematical Model) • Four parts – P: a set of places – T: a set of transitions – I: an input function • A mapping from a transition tj to a collection of places I(tj ) • The places are input places of tj – O: an output function • A mapping from a transition tj to a collection of places O(tj ) • The places are output places of tj Pi tj Pi tj
- 71. Definition 1 • A Petri net structure is a four-tuple: C = (P,T,I,O) • P = { p1, p2,…, pn }, n ≥ 0 • T= { t1, t2,…, tm }, m≥0 • P∩T = ∅ • I:T→P∞, a mapping from transitions to bags of places • O:T → P∞, a mapping from transitions to bags of places
- 72. Petri Net Representation • A place pi is an input place of a transition tj if pi ∈ I(tj) • A place pi is an output place of a transition tj if pi ∈ O(tj) • A bag is a generalization of sets which allows multiple occurrences of an elements in a bag. – E.g.: O(tj) = { p1, p1, p1, p2, p3, p4, p4, p5 } Pi tj Pi tj
- 73. Example 1 P2 P4P1 P5 P3 t1 t2 t3 t4
- 74. Example 1 • C = ( P,T,I,O ) • P = { p1 , p2 , p3 , p4 , p5 }, n = 5 • T = { t1 , t2 , t3, t4 }, m = 4 • I(t1 ) = { p1 } O(t1 ) = { p2 , p3 , p5 } • I(t2 ) = { p2 , p3 , p5 } O(t2 ) = { p5 } • I(t3 ) = { p3 } O(t3 ) = { p4 } • I(t4 ) = { p4 } O(t4 ) = { p2 , p3 } P2 P4P1 P5 P3 t1 t2 t3 t4
- 75. Multiplicity • #(a, A) : The number of occurrences of element a in bag A. • The multiplicity of an input place pi for a transition tj is #( pi, I(tj) ) • A transition tj is an input of a place pi, if pi is an output of tj • A transition tj is an output of a place pi, if pi is an input of tj Pi tj Pi tj # = 3
- 76. Definition 2 • Input and output functions are extended as follows: – I:T→P∞, O:T → P∞ , such that • #( tj, I(pi) ) = #( pi, O(tj) ) and • #( pi, I(tj) ) = #( tj, O(pi) ) Pi tj Pi tj
- 78. Example 2 • C = ( P,T,I,O ) • P = { p1 , p2 , p3 , p4 , p5 , p6 }, n = 6 • T = { t1 , t2 , t3, t4, t5 }, m = 5 • I(t1 ) = { p1 } O(t1 ) = { p2 , p3 } • I(t2 ) = { p3 } O(t2 ) = { p3 , p5 , p5 } • I(t3 ) = { p2, p3 } O(t3 ) = { p2, p4 } • I(t4 ) = { p4 , p5 , p5 , p5 } O(t4 ) = { p4 } • I(t5 ) = { p2 } O(t5 ) = { p6 } P2 P4P1 P5 P3 t1 t2 t3 t4 P6 t5
- 80. Example 3 • C = ( P,T,I,O ) • P = { p1 , p2 , p3 , p4 , p5 , p6 , p7 , p8 , p9 }, n = 9 • T = { t1 , t2 , t3, t4, t5 , t6 }, m = 6 • I(t1 ) = { p1 } O(t1 ) = { p2 , p3 } • I(t2 ) = { p8 } O(t2 ) = { p1 , p7 } • I(t3 ) = { p2 , p5 } O(t3 ) = { p6 } • I(t4 ) = { p3 } O(t4 ) = { p4 } • I(t5 ) = { p6 , p7 } O(t5 ) = { p9 } • I(t6 ) = { p4 , p9 } O(t6 ) = { p5 , p8 } P2 P4 P1 P5 P3 t1 t2 t3 t4 P6 t5 P7 P8 P9 t6
- 81. Petri Net Graphs • Bipartite directed multigraph • Vertices: – Circles (places) and Bars (transitions) • Links: – Directed arcs (represent the relations between transitions and places) • Represented by functions I and O • Multiple arcs directed from a place to a transition is allowed
- 82. Definition 3 • A Petri net graph G is a bipartite directed multigraph, G = ( V, A ), where V = { v1, v2, …, vs } is a set of vertices and A = {a1, a2, …, ar} is a bag of directed arcs, ai = (vj , vk ), with vj, vk ∈ V • The set V can be partitioned into two disjoint sets P and T such that V = P ∪ T, P ∩ T = ∅ , and for each directed arc, ai ∈ A, if ai = (vj , vk ), then either vj ∈ P and vk ∈ T or vj ∈ T and vk ∈ P
- 83. Definition 4 • Define V = P ∪ T. • Define A as a bag of directed arcs such that for all pi ∈ P and tj ∈ T , – #( (pi , tj ), A ) = #( pi , I(tj) ) and – #( (tj , pi ), A ) = #( pi , O(tj) ) • G = ( V, A ) is a Petri net graph which is equivalent to the Petri net structure C = ( P, T, I, O )
- 84. The Dual of a Petri Net • The dual of a Petri net C = ( P, T, I, O ) is the Petri net C’ = (T, P, I, O ) which results from interchanging places and transitions
- 85. The Dual of Example 1 P2 P4P1 P3 t1 t5 t3 t4 t2 P2 P4P1 P5 P3 t1 t2 t3 t4
- 86. The Inverse Petri Net • The inverse Petri net for a Petri net C = ( P, T, I, O ) is defined by interchanging the input and output functions, -C = ( P, T, O , I ).
- 87. The Inverse of the Dual of Example 1 P2 P4P1 P3 t1 t5 t3 t4 t2 P2 P4P1 P3 t1 t5 t3 t4 t2
- 88. Petri Net Markings • A marking μ is an assignment of tokens to the places of a Petri net • The number and position of tokens may change during the execution of a Petri net • The tokens are used to define the execution of a Petri net
- 89. Definition 5 • A marking μ of a Petri net C = ( P,T,I,O ) is a function from the set of places P to the nonnegative integers N; μ : P → N • The marking can also be defined as an n-vector, μ = [ μ1 μ2 …μn ] • The number of tokens in place pi is μi, i = 1, …, n – μ(pi ) = μi
- 90. Marked Petri Nets • M = ( C, μ ) is a Petri net structure and a marking μ – M = (P,T,I,O, μ ) • Tokens are represented by small dots in the circles which represent the places of a Petri net • The number of tokens assigned to a place is unbounded • The number of possible markings of a Petri net is infinitive
- 91. A Marked Petri Net of Example 1 • The marking μ = [ 1 2 0 0 1 ] t1 t2 t3 t4 P1 P5 P2 P4 P3
- 92. Another Marked Petri Net of Example 1 • The marking μ = [ 0 2 0 5 1 ] t1 t2 t3 t4 P5 P2 P4P1 P3
- 93. Execution Rules for Petri Nets • Tokens reside in places and control the execution of the transitions of the net • A Petri net executes by firing transition • A transition fires by removing tokens from its input places and creating new tokens which are distributed to its output places – The tokens are not moving from one place to the next place – They are consumed by the transition and regenerated from the transition to the next place • A transition may fire if it is enabled
- 94. Execution Rules for Petri Nets (Conti.) • A transition is enabled if each of its input places has at least as many tokens in it as arcs from the place to the transition • The tokens in the input places which enable a transition are its enabling tokens • A transition fires by removing all of its enabling tokens from its input places and then depositing into each of its output places one token for each arc from the transition to the place
- 95. Definition 6 • A transition tj ∈ T in a marked Petri net C = ( P,T,I,O ) with marking μ is enabled if for all places pi , μ(pi) ≥ #( pi, I(tj) ) tj 1 token ≥ one arc 3 tokens ≥ three arcs 2 tokens ≥ two arcs
- 96. Definition 7 • A transition tj in a marked Petri net with marking μ may fire whenever it is enabled • Firing an enabled transition tj results in a new marking μ’ defined by • μ’(pi ) = μ (pi ) - #(pi , I(tj) ) + #(pi , O(tj) ) tj Pi tj Pi Remove 2 tokens (for 2 arcs) Remove 3 tokens (for 3 arcs) Add 1 token (for 1 arc) Add 1 token (for 1 arc)
- 97. Firing Examples • Example a: – I(t3 ) = { p2} O(t3 ) = {p7, p13 } – t3 is enabled whenever there is at least one token in place p2 • Transition t3 fires by removing one token from p2 and depositing one token in place p7 and one token in place p13 t3 t3 P2 P7 P13 P7 P13 P7P2
- 98. Firing Examples (Cont.) • Example b: – I(t2 ) = {p21, p23} O( t2 ) = {p23, p25, p25 } – t2 is enabled whenever there is at least one token in place p21 and at least one token in place p23 – Transition t2 fires by removing one token from p21 and one token from p23 and then deposits one token in place p23 and two tokens in place p25 t2 t2 P21 P23 P25 P23 P25 P21
- 99. Firing Examples (Cont.) • t1 , t3 , and t4 are enabled t1 t2 t3 t4 P4 P1 P3 P5P2
- 100. • t4 is fired t1 t2 t3 t4 P4 P1 P3 P5P2
- 101. • t1 is fired t1 t2 t3 t4 P4 P1 P3 P5P2
- 102. • t3 is fired t1 t2 t3 t4 P4 P1 P3 P5P2
- 103. Summary • Petri Net Definitions • Petri Net Examples • For multimedia presentations – Transitions are used for synchronization – Places can represent multimedia resources • Related to Interval Temporal Logic 3 1 2 4
- 104. Multimedia Presentation Based on Petri Net • Defining a operation model of multimedia presentation by features of Petri Net • Constructing an algorithm based on Timed Petri Net for multimedia presentations
- 105. Background • Petri net is a graph and a mathematical tool • Multimedia presentations includes timing control and user interaction • Describe multimedia objects by Timed Petri Net • Create new definitions based on Petri Net for interactions • Implement a tool for multimedia presentation designs
- 106. An operation example of Petri net Place A rc Tokens Transition A rc A rc Place Place Tokens Place Place Place A rc A rc A rc Transition Token Token Token (a) A initial marking before transition firing (b) A new marking after transition firing
- 107. Interpretations of places and transitions Tansitions A ttributesInput Places O utput Places Input Places Transition A ttributes O utput Places D ata Processing Inform ation R equest A vailable Service R esource Perm ission A pproach Precondition Event Postcondition M ultim dia Tim ing Playing
- 108. Two special cases in Petri net (a) A sink transition of Petri Net (End of Presentation) (b) A source transition of Petri Net (Begin of Presentation)
- 109. Self-loop and Weights 1 1 1 2 1 (a) A self_loop of Petri Net (b) Weights of synchronous arcs of Petri Net
- 110. New Components for Multimedia Objects : Place and token is for playing a m ultim edia resource. : U ser transition is for accepting a m essage and causing the activation of connected transitions. : U se r arc is fo r connec ting fr om a user transition to a transition. : Sy nc hro nou s arc is for con ne cting fr om a p la ce to a transition or from a transition to a place. : Transition is for synchronous control. New New
- 111. Example of Multimedia Petri net im age text video sound m usic Syn. arc Syn. arc Syn. arc Syn. arc Syn. arc Syn. arc Syn. arc Syn. arc Syn. arc Syn. arc U ser arc U ser arc navigation m essage navigation m essage user transition transition transition transition transition
- 112. Definitions of place, transition and user transition in the Multimedia Petri net O bject attributes File form at D uration File nam e Execution tool File attributes Place node Transition U ser Transition A ttributes M essages Tim ing O bject relations Interrupt Pause Stop C hange
- 113. Algorithm: Compute_Node_Time • Construct a Transition_Graph reachable from one of the starting • Navigation_Messages; • Initialize the node pointed by the Navigation_Message to • Node_Time = 0; • Sort the Transition_Graph to a Transition_List by Topological sort; • For each node N in the Transition_List, except the first, • Begin • For each incoming edge, compute the Estimated_Time = • Node_Time + Edge_Duration; • Set Node_Time of node N to the maximum of all Estimated_Time • of incoming edges; • End
- 114. Algorithm: Run_Petri_Net – Begin – For each node N in the Transition_List Do Begin – Collect places pointed by Sync_Arcs from node N in Sync_Set(N); – Sort the Transition_List according to Node_Time in a non- – decreasing order; – End – CoBegin – Process1: – For each node N in the Transition_List Do Begin – Sequentially play resources in Sync_Set(N) at Node_Time(N) – concurrently; – End – Process2: – LoopBegin – If Process1 ends, Then Process2 ends; – If there is a Navigation_Message of a User_Transition Then – Exits with the next starting Navigation_Message; – If there is a Selection, Assignment, or Condition Then – Perform the action accordingly; – LoopEnd – CoEnd – End – Algorithm: Run_Presentation – Begin – LoopBegin – If there is a starting Navigation_Message Then – Compute_Node_Time; – Run_Petri_Net; – Else – End Presentation – LoopEnd – End
- 115. A initial graph of Multimedia Petri Net Text V ideo W ave Sound Bm p M idi
- 116. Transition graph for simplified Multimedia Petri Net Sound Bm p Text V ideo U ser m essage W ave U ser m essage M idi
- 117. An example for the Compute-Node-Time algorithm 70 35 20 15 15 20 40 30 20 a b c d e
- 118. Topological sort of the transition nodes a c d e b 20 70 152030 35 15 40 20
- 120. Presentation Schedule of the Example A nim ation Sound Text M usic Im age V ideo Tim e Transition "a" Transition "b" Transition "c" Transition "d" U ser transition
- 121. Synchronous Sets for the Order of Transitions Firing a b c d b c d C ase1 : W ithout firing of user transition : {anim ation,sound,text} : {m usic,im age,text} : {m usic,video} : End C ase2 : W ith firing of user transition : {m usic,im age,video} : End
- 122. Layered Multimedia Presentation Subnet Subnet Subnet Transition Transition U ser Transition
- 126. Summary • Provide a model for the multimedia presentation design by extending concepts of Petri Net • Could be extended for layered multimedia presentations • Petri Net is used as an example of synchronization control
- 127. Collaborative Learning with Petri Net • Different students learn different contents and in different speed • Team members have to learn skill and gathered together to solve a mission start C B A Dispatch Discuss Grade
- 128. The Same Structure but Different Skill Learner A Cooperative Learning Learner B Learner C L2L1 L3 L4 L1,L3,L4 L1,L2,L3,L4 L1,L2,L3,L4
- 129. Capability & Assessment Learner A Learner B Learner C L1,L3,L4 L1,L2,L3,L4 Exam Give different color tokens condition Learner D L1,L2,L3,L4 L1,L2,L3,L4 Learner E L1,L3,L4 C.L.
- 131. Interval Temporal Logic • Temporal Intervals • Fast Interval Relation Composition • Solving the Composition Conflicts • Interval Relation Distances • Qualitative Spatial Knowledge • The Applications
- 132. • James F. Allen, "Maintaining Knowledge about Temporal Intervals," Communications of the ACM, Vol. 26, No. 11, 1983. • Relations between two time intervals • Relation composition • Can be used in many research areas Temporal Intervals
- 133. Allen’s 13 Temporal Interval Relations
- 134. Composition of the 13 Interval Relations
- 135. From Point Relations to Interval Relations
- 136. Revised 18 Temporal Interval Relations
- 137. Fast Interval Relation Composition • Relation composition involves set operations • The amount of relation sets is finite • Table13
- 138. The Relation Composition Algorithm Algorithm : RelComp Input : rs1 29RelSet, rs2 29RelSet Output : rs 29RelSet Preconditions : true Postconditions : true Steps : 1. rs = r1 rs1 , r2 rs2 (r1 , r2) rs1 rs2 Table13 (r1 , r2)
- 139. Computing All Possible Relations Algorithm : ComputeTable29 Input : Table13 Output : Table29 Preconditions : true Postconditions : relation composition is closed under I Steps: 1. Construct a set of 13 atomic sets from the 13 relations, assuming that this set is called I, which is an index set for table look up. 2. Let Table29( i, j ) = Table13( i, j ), i I, j I 3. Table29( i, j ), i I , j I , do 3.1: if k = Table29( i, j ) I then 3.1.1 : I = I Table29( i, j ) 3.1.2 : m I ,do 3.1.2.1 Table29( k, m ) = Relcomp( k, m ) 3.1.2.2 Table29( m, k ) = Relcomp(m, k )
- 140. The 29 Possible Relation Sets IDs Relation Sets IDs Relation Sets ------------------------------------------------------------------------------------------ 1 { < } 16 { o, d ,s } 2 { > } 17 { oi, di, si } 3 { d } 18 { <, o, m } 4 { di } 19 { >, oi, mi } 5 { o } 20 { f, fi, e }* 6 { oi } 21 { s, si, e }* 7 { m } 22 { <, o, m, d, s } 8 { mi } 23 { >, oi, mi, di, si } 9 { s } 24 { <, o, m, di, fi } 10 { si } 25 { > , oi, mi, d, f } 11 { f } 26 { o, oi, d, di, s, si, f, fi, e }* 12 { fi } 27 { <, m, d, di, o, oi, f, fi, s, si, e } 13 { e }* 28 { > , mi, di , d, oi, o, fi, f , si, s, e } 14 { o, di, fi } 29 { < , > ,m, mi, di , d, oi, o, fi, f , si, s, e }* 15 { oi, d, f } rs rs-1 rs rs-1 ------------------------------------------- 1 2 18 19 3 4 20 20 5 6 21 21 7 8 22 23 9 10 24 25 11 12 26 26 13 13 27 28 14 15 29 29 16 17 The Inverse Relations
- 141. Table29 o | 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 ----------------------------------------------------------------------------------------------------------------------------- -------------- 01 |01 29 22 01 01 02 01 22 01 01 22 01 01 01 22 22 22 01 29 22 01 22 29 01 29 22 22 29 29 02 |29 02 25 02 25 02 25 02 25 02 02 02 02 25 25 25 02 29 02 02 25 29 02 29 25 25 29 25 29 03 |01 02 03 29 22 25 01 02 03 25 03 22 03 29 25 22 29 22 25 22 25 22 29 29 25 29 29 29 29 04 |24 23 26 04 14 17 14 17 14 04 17 04 04 14 26 26 17 24 23 17 14 27 23 24 28 26 27 28 29 05 |01 23 16 24 18 26 01 17 05 14 16 18 05 24 26 22 27 18 28 22 14 22 29 24 28 27 27 29 29 06 |24 02 15 23 26 19 14 02 15 19 06 17 06 28 25 26 23 27 19 17 25 27 23 29 25 28 29 28 29 07 |01 23 16 01 01 16 01 20 07 07 16 01 07 01 16 22 22 01 28 22 07 22 29 01 28 22 22 29 29 08 |24 02 15 02 15 02 21 02 15 02 08 08 08 25 25 15 02 27 02 08 25 27 02 29 25 25 29 25 29 09 |01 02 03 24 18 15 01 08 09 21 03 18 09 24 15 22 27 18 25 22 21 22 29 24 25 27 27 29 29 10 |24 02 15 04 14 06 14 08 21 10 06 04 10 14 15 26 17 24 19 17 21 27 23 24 25 26 27 28 29 11 |01 02 03 23 16 19 07 02 03 19 11 20 11 28 25 16 23 22 19 20 25 22 23 29 25 28 29 28 29 12 |01 23 16 04 05 17 07 17 05 04 20 12 12 14 26 16 17 18 23 20 14 22 23 24 28 26 27 28 29 13 |01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 14 |24 23 26 24 24 26 24 17 14 14 26 24 14 24 26 27 27 24 28 27 14 27 29 24 28 27 27 29 29 15 |24 02 15 29 27 25 24 02 15 25 15 27 15 29 25 27 29 27 25 27 25 27 29 29 25 29 29 29 29 16 |01 23 16 29 22 28 01 23 16 28 16 22 16 29 28 22 29 22 28 22 28 22 29 29 28 29 29 29 29 17 |24 23 26 23 26 23 14 23 26 23 17 17 17 28 28 26 23 27 23 17 28 27 23 29 28 28 29 28 29 18 |01 29 22 24 18 27 01 27 18 24 22 18 18 24 27 22 27 18 29 22 24 22 29 24 29 27 27 29 29 19 |29 02 25 23 28 19 28 02 25 19 19 23 19 28 25 28 23 29 19 23 25 29 23 29 25 28 29 28 29 20 |01 23 16 23 16 23 07 23 16 23 20 20 20 28 28 16 23 22 23 20 28 22 23 29 28 28 29 28 29 21 |24 02 15 24 24 15 24 08 21 21 15 24 21 24 15 27 27 24 25 27 21 27 29 24 25 27 27 29 29 22 |01 29 22 29 22 29 01 29 22 29 22 22 22 29 29 22 29 22 29 22 29 22 29 29 29 29 29 29 29 23 |29 23 28 23 28 23 28 23 28 23 23 23 23 28 28 28 23 29 23 23 28 29 23 29 28 28 29 28 29 24 |24 29 27 24 24 27 24 27 24 24 27 24 24 24 27 27 27 24 29 27 24 27 29 24 29 27 27 29 29 25 |29 02 25 29 29 25 29 02 25 25 25 29 25 29 25 29 29 29 25 29 25 29 29 29 25 29 29 29 29 26 |24 23 26 29 27 28 24 23 26 28 26 27 26 29 28 27 29 27 28 27 28 27 29 29 28 29 29 29 29 27 |24 29 27 29 27 29 24 29 27 29 27 27 27 29 29 27 29 27 29 27 29 27 29 29 29 29 29 29 29 28 |29 23 28 29 29 28 29 23 28 28 28 29 28 29 28 29 29 29 28 29 28 29 29 29 28 29 29 29 29 29 |29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29
- 142. Solving the Composition Conflicts • Unknown derivation If X < Y and Y > Z, then X ? Z • Multiple derivation If X < Y and Y d Z, then X {<, o, m, d, s} Z • Conflict derivation If X < Y, Y < Z, and X > Z are declared by the user, then there exists a conflict
- 143. The Domain of Relations • An user edge denotes a relation between a pair of objects defined by the user. The relation may be reasonable or non-reasonable. • A derived edge holds a non-empty set of reasonable relations derived by our algorithm. The relation of the two objects connected by the derived edge can be any reasonable relation in the set. • complete relation domain (a complete graph): contains user edges and derived edges, with possible cycles and possible conflicts. • reasonable relation domain (a graph): contains user edges and derived edges, with possible cycles but no conflict. • reduced relation domain (a graph): contains only user edges, with possible cycles and possible conflicts. • restricted relation domain (a tree): contains only user edges, without cycle.
- 145. Computes the Reasonable RD from the Reduced RD Algorithm : ComputeRD1 Input : G = ( GV,GE ) Output: Kn = ( KnV, KnE ) Preconditions : true Postconditions : GV = KnV GE KnE Steps : 1 : G = EliminateConflicts (G) 2 : Kn = G pl = 2 3 : repeat until | KnE | = | KnV | * ( | KnV | -1 ) / 2 3.1 : for each e = ( a, b ) e KnE a KnV b KnV such that there is a path of user edges from a to b , with path length = pl 3.2 : suppose (( n1, n2 ), ( n2, n3 ),…, ( nk-1, nk ) ) is a path with a = n1 b = nk k = pl + 1 3.3 : set e.rs = Table29 (( a, nk-1 ).rs, (nk-1, b ).rs ) 3.4 : KnE = KnE { e } 3.5 : pl = pl + 1
- 146. Eliminate Conflicts Algorithm : EliminateConflicts Input : G = ( GV, GE ) Output : G = ( G V, G E) Preconditions : G contains only user edges G = G Postconditions : G = G , but the reasonable sets of edges in G may be changed. Steps : 1. for each P = ((n1, n2), (n2, n3), , (nk-1, nk) ) in G with n1 = nk k >3 1.1 : for each i, 1 i k-2 1.1.1 : set (ni, ni+2).rs = Table29 ((ni, ni+1).rs, (ni+1, ni+2).rs ) 1.2 : rs = Table29 ((nk, nk-2).rs, (nk-2, nk-1).rs ) 1.3 : if (nk, nk-1).r rs then 1.3.1 : ask user to choose a r rs 1.3.2 : set (nk, nk-1).r = r
- 147. An Example
- 148. User Edges and Derived Edges User edges: ( A, B ) = { < } = [1] ( B, C ) = { m } = [7] ( C, D ) = { d } = [3] ( C, E ) = { s } = [9] ( F, D ) = { < } = [1] 2. Path Length = 3 (A, E) = (A, B ) o (B, C) o (C, E) = (A, C ) o (C, E) = [1] o [9] = [1] = { < } (A, D) = (A, B ) o (B, C) o (C, D) = (A, C ) o (C, D) = [1] o [3] = [22] = { <, o, m, d, s } (B, F) = (B, C ) o (C, D) o (D, F) = (B, D ) o (D, F) = [16] o [1]-1 = [23] = { >, oi, mi, di, si } (E, F) = (E, C ) o (C, D) o (D, F) = (E, D ) o (D, F) =[14] -1 o [2] = [15] o [2] = [2] = { > } 3. Path Length = 4 (A, F) = (A, B ) o (B, C) o (C, D ) o (D, F) = ( (A, B ) o (B, C) ) o ( (C, D ) o (D, F) ) = (A, C ) o (C, F) = [1] o [2] = [29] = { <, >, d, di, o, oi, m, mi, f, fi, s, si, e } Derived edges: 1. Path Length = 2 (A, C) = (A, B ) o (B, C) = [1] o [7] = [1] = { < } (B, D) = (B, C ) o (C, D) = [7] o [3] = [16] = {o, d, s } (C, F) = (C, D ) o (D, F) = [3] o [1]-1 = [3] o [2] = { > } (D, E) = (D, C ) o (C, E) = [4] o [9] = [14] = {o, di, fi } (B, E) = (B, C ) o (C, E) = [7] o [9] = [7] = { m }
- 149. Adds User Edges to the Reasonable RD Algorithm : AddUERD Input : l = (a, b ), Kn = ( KnV, KnE ) Output : Kn+1 = ( Kn+1V, Kn+1E ) Preconditions : l KnE a KnV b KnV Postconditions : | Kn+1V | = | KnV | + 1 | Kn+1E | = | KnE | + n Steps : 1: Kn+1E = KnE { l } 2: for each e = ( c, b ) c a c KnV 2.1: e.rs = d KnV, ( c, d ) KnE, (d, b ) KnE ( Table29(( c, d ).rs, ( d, b ).rs )) 2.2: Kn+1E = Kn+1E {e} 3: Kn+1V = KnV { b }
- 150. Adding Edges Adding an user edge to Example 4.1: Add ( G, E ) = { f } = [11] Derived edges : Derive ( E, G ) = ( G, E )-1 = [11]-1 = [12] = { fi } ( inverse ) Derive ( A, G ) = ( A, E ) o ( E, G ) = [1] o [12] = [1] = { < } ( association ) Derive ( B, G ) = ( B, E ) o ( E, G ) = [7] o [12] = [1] = { < } ( association ) Derive ( C, G ) = ( C, E ) o ( E, G ) = [9] o [12] = [18] = {<, o, m } Derive ( D, G ) = ( D, E ) o ( E, G ) = [14] o [12] = [24] = { <, o, m, di, fi } ( association ) Derive ( F, G ) = ( F, E ) o ( E, G ) = ( E, F )-1 o ( E, G ) = [1] o [12] = [1] = { < } ( inverse, association )
- 151. Interval Relation Distances • Use relation distances to estimate the similarity of object positions • There are three types of relation distances – The Point Relation Distance – The Interval Relation Distance – The Extended Point-Interval Relation Distance
- 152. The Point Relation Distance
- 153. The Interval Relation Distance
- 154. The Extended Point-Interval Relation Distance
- 155. The Distance Graph < ols m loe o s d fi e f di si oi m i ole los > oo
- 156. Qualitative Spatial Knowledge • Relation Embedding – two points on a line – two points on a plan – two line segments on a line – two line segments on a plan – two 2-D objects on a plan
- 157. Projection of Two Line Segments on a 2-D Plan
- 158. Projection of Two 2-D Objects on a 2-D Plan
- 159. Generalized Spatial Relation f 1 = 29RelSet o 29RelSet 29RelSet f 2 = 29RelSet o 29RelSet 29RelSet o 29RelSet 29RelSet 29RelSet f 3 = 29RelSet o 29RelSet 29RelSet o 29RelSet 29RelSet o 29RelSet 29RelSet 29RelSet 29RelSet where 29RelSet 29RelSet { {<} {<}, {<} {>}, … , { = } { = } } 29RelSet 29RelSet 29RelSet { {<} {<} {<}, {<} {<} {>},…, {=} {=} {=}} i1 j1, i2 j2 P (29RelSet 29RelSet ) f 2 (i1 j1, i2 j2 ) = f 1 (i1 , i2 ) f 1 ( j1 , j2 ) i1 j1 k1 , i2 j2 k2 P (29RelSet 29RelSet 29RelSet ) f 3 (i1 j1 k1, i2 j2 k2) = f 1 (i1 , i2 ) f 1 ( j1 , j2 ) f 1 (k1 , k2 ) where A B = { a b | a A, b B } A B C = { a b c | a A, b B, c C }
- 160. The Applications • Multimedia Presentation Generation • Shape-Based Image Retrieval (not discussed)
- 168. Notes • Synchronization of Multimedia Objects • Computation Efficiency • Extension toward N-Dimension • Multimedia Presentation Scheduling
- 169. Summary • White Board and Chat Room • Multimedia Synchronization • SMIL • Petri Net • Interval Temporal Logic