US 20 1 0022573 6A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US 2010/0225736 A1 King et a]. (43) Pub. Date: Sep. 9, 2010 (54) VIRTUAL DISTRIBUTED MULTIPOINT Publication Classi?cation CONTROL UNIT (51) Int CL _... _ H04N 7/15 (2006.01) (76) lnvemors~ Ken! C- Kmg Ausnn, TX ms) (52) us. Cl...... 348/14.09; 348/E07.084 AshIsh Goyal, Bangalore (IN); Hrishikesh Gopal Kulkarni, (57) ABSTRACT Bangalore (IN) Correspondence Address: MEYERTONS, HOOD, KIVLIN, KOWERT & GOETZEL, P.C. PO. BOX 398 AUSTIN, TX 78767-0398 (US) (21) App1.No.: 12/712,947 (22) Filed: Feb. 25, 2010 Related US. Application Data (60) Provisional application No. 61/157,511,?led on Mar. 4, 2009. A virtual distributed multipoint control unit in a videoconfer ence may be implemented using a master endpoint, one or more facilitator endpoints, and one or more leaf endpoints. The facilitator endpoint(s) may receive video and/or audio from the leaf endpoints (Which may provide/receive video conference audio and video to one or more local participants). The master endpoint may receive video and/or audio from one or more facilitator endpoints (Which may include cumu lative video stream from multiple leaf endpoints) and/or directly from one or more leaf endpoints. The master end point may arrange received video and/or audio and provide cumulative video and/or audio streams to the leaf endpoints (e.g., directly and/or through the facilitator endpoints). The virtual distributed multipoint control unit may support a con ference With more leaf endpoints than the master endpoint has input ports and/ or decoders. Videoconferencing Endpoint 103a Videoconferencing Endpoint 103b / Videoconferencing Endpoint M 100 Network M Videoconferencing Endpoint M
Patent Application Publication Sep. 9, 2010 Sheet 1 of 25 US 2010/0225736 A1 Videoconferencing Endpoint M Videoconferencing En1%%(gnt f Videoconferencing Endpoint M 100 Network M Videoconferencing Endpoint M FIG. 1a
Patent Application Publication Sep. 9, 2010 Sheet 3 0f 25 US 2010/0225736 A1 288m 32m lam: 5: 282m 3: 88m as E55 5: 28% a2 28% 5: 25%5 E55 2 % E35 E55 82 8 EsQEm $2 m2 wesqnzm E2563
Patent Application Publication Sep. 9, 2010 Sheet 4 0f 25 US 2010/0225736 A1 IN Q A FIG. 2 209
Patent Application Publication Sep. 9, 2010 Sheet 5 0f 25 US 2010/0225736 A1 A leaf endpoint may communicate with a facilitator endpoint. @ V The facilitator endpoint may receive video images from a plurality of leaf endpoints. M V The facilitator endpoint may generate a composite image comprising two or more video images. 5 V The facilitator endpoint may transmit the composite video image and coordinate information to the master endpoint. M i The master endpoint may generate a new composite image based on user preference. E v The master endpoint may transmit the new composite video image to a leaf endpoint. 0 V The leaf endpoint may separate out the video images of the composite image and form a different composite image to display. 31 7 v The leaf endpoint may display the composite image. V The master endpoint may receive the composite video image and the coordinate information. @ V The master endpoint may separate out the video images using the coordinate information. M FIG. 3
Patent Application Publication Sep. 9, 2010 Sheet 6 0f 25 US 2010/0225736 A1 Leaf Endpoints 103e \ [- 103f [- 103g 1 2 3 Decoders (x3) @ Scalers (x4) IP Pon M C _t Facilitator omposl ors - m Engggmt 4 Local Camera [- 103b 455a \ S2 S3 \ 4 55d _\ 4 55b Displayable 4 La out1 81 S \ 4550 y
Patent Application Publication Sep. 9, 2010 Sheet 9 0f 25 US 2010/0225736 A1.QI 2» 588m lemvaléglm. _ \_ m? Ba 8» Q. E? S? r mm E 3 E 3 a E+BT mcmaxmz @ 23% Em some momq 90m
Patent Application Publication Sep. 9, 2010 Sheet 13 0f 25 US 2010/0225736 A1 719 <- 721 700 \
Patent Application Publication Sep. 9, 2010 Sheet 14 0f 25 US 2010/0225736 A1 FIG. 8a 719
Patent Application Publication Sep. 9, 2010 Sheet 15 0f 25 US 2010/0225736 A1 707 m C) Q 701 FIG. 8b
Patent Application Publication Sep. 9, 2010 Sheet 16 0f 25 US 2010/0225736 A1 0,0 / Leaf endpoint 103e Video image 43 507 r 0,360 Leaf endpoint 103i Video image @ f- 0, 720 Leaf endpoint 1039 Video image @ 1279, 719 Cal! ID, System Name, Number, IP Address, Left, Top, Right, Bottom, Mute 0, User 1, 99.99.99.999, 99. 99. 99. 999, 0, 0, 1279, 359, 0 1, User 2, 98.99.99.999, 99.99.98.999, 0, 360, 1279, 719, 1 3, User 3, 99.98.99.999, 99.98.99.999, 0, 720, 1279, 1079, 1 W 519 FIG. 9
Patent Application Publication Sep. 9, 2010 Sheet 17 0f 25 US 2010/0225736 A1 1001 [- 1003 1 0 2 1 3 [- 1005 [- 1007 0 2 1 1 2 3 0 3 [- 1009 [- 1011 1 2 3 4 4 5 Local C 0 5 6 $222; 0 6 1 2 3 FIG. 10
Patent Application Publication Sep. 9, 2010 Sheet 18 0f 25 US 2010/0225736 A1 The leaf endpoints may send their audio to respective facilitator endpoints. 1 101 l The facilitator endpoints may send received audio streams to the master endpoint. 1 103 l Cumulative audio streams may be created for each endpoint with participants participating in the videoconference. 1 105 l The cumulative audio streams may be sent to the respective endpoints. 1 107 l The endpoint may provide the cumulative audio stream to one or more speakers. 1 109 FIG. 11
Patent Application Publication Sep. 9, 2010 Sheet 19 0f 25 US 2010/0225736 A1 A leaf endpoint may call a master endpoint to start/join a videoconference. 1201 l The master endpoint may assign the leaf endpoint to a facilitator endpoint. 1203 l The leaf endpoint may call the assigned facilitator endpoint. 1205 i The master endpoint may receive an indication that a facilitator endpoint in the videoconference will or has become unavailable. 1207 v The master endpoint may locate a different facilitator endpoint to use in the videoconference. 1209 l The master endpoint may send instructions to the respective leaf endpoints and/or substitute facilitator endpoint to establish communications. 121 1 FIG. 12