Pinos. Wim Taymans. Principal Software Engineer October 8, Wim Taymans. Gstreamer Conference Dublin

Size: px

Start display at page:

Download "Pinos. Wim Taymans. Principal Software Engineer October 8, 2015. Wim Taymans. Gstreamer Conference Dublin"

Jack Hamilton
10 years ago
Views:

1 Pinos Principal Software Engineer October 8,

2 Pinos what Daemon that manages access to multimedia streams Capture streams (from v4l2, pulseaudio, ) Upload streams 2

3 Pinos history GStreamer conference 2014 William Manley ideas for stb-tester Multiplexing of v4l2 between processes Socketsrc, multifdsink, metadata, fdpay, fddepay 'pulsevideo' name was coined We had a long talk 3 RED HAT

4 Pinos history Gstreamer Hackfest March 2015 Merged pending patches into main GStreamer branch Part of upcomming 1.6 release 4

5 Pinos history Meanwhile Thinking about giving access to v4l2 devices in sandboxed applications Patches to pulseaudio to implement security framework 5

6 Pinos history First attempts in April 2015 DBus API design prototypes 6

7 Pinos history Renamed to Pinos in June 2015 First announcement 7

8 Pinos goals Design goals High performance Extensible, use properties in the API Reuse existing components Capture of video for now 8

9 Pinos components DBus for client daemon interaction org.pinos Connection of client Selection of source Negotiation of formats Start/stop streaming of media Introspection with ObjectManager 9

10 Pinos components Socket for media transfer Metadata is written to/read from socket Media is passed as file descriptor in ancillary data Protocol also supports passing metadata via shared memory 10

11 Pinos components GStreamer As module providing the v4l2 devices Pinospay to convert stream to pinos protocol and fd in metadata Multisocketsink to send media to clients Media formats described using GStreamer caps For server/client clock slaving 11

12 Pinos api Powerful client API Inspired by pulseaudio Asynchronous Gio/Glib/Gobject based GDBus internally Threadsafe/threadaware 12

13 Pinos DBus protocol ConnectClient CreateSourceOutput fd Client Start Stop Remove org.pinos.sourceoutput1 org.pinos.source1 org.pinos.client1 org.pinos.daemon1 Disconnect 13

14 Pinos fd protocol Version Length Type Len Data Type Len Data

15 Pinos message types (2) Header Flags, PTS, DTS, seqnum (3) Fd-payload Id, offset, size, fd-index (4) Release fd-payload Id 15

16 Pinos client part Start org.pinos.sourceoutput1 Stop Client fd recvmsg() sendmsg() metadata fd fd metadata fd fd metadata fd fd

17 Pinos other message types (1) Continuation Type-len-value in shared memory at offset/size (5) Format change In-band format change (6) Property change Key/value 17

18 Pinos server part org.pinos.sourceoutput1 Start Stop Client fd v4l2src pinospay multifdsink pipeline org.pinos.sourceoutput1 Start Stop Client fd 18

19 Pinos stream upload Client can also provide buffers to pinos Other clients can see and capture from this stream Use loopback to do interesting things 19

20 Pinos upload server part... recvmsg() sendmsg() metadata fd fd metadata fd fd Client Start Stop org.pinos.sourceinput1 fd socketsrc pipeline multifdsink org.pinos.sourceoutput1 org.pinos.sourceoutput1 Start Stop Start Stop Client Client fd fd... 20

21 Pinos GStreamer GStreamer integration Pinossrc, pinossink pinospay, pinosdepay when dealing with raw sockets Device monitor 21

22 Pinos crazy ideas metadata fd fd recvmsg() metadata fd fd sendmsg() Client Start Stop org.pinos.sourceinput1 fd socketsrc Mixer Sink metadata fd fd recvmsg() metadata fd fd sendmsg() Client Start Stop org.pinos.sourceinput1 fd socketsrc pipeline 22

23 Pinos status API unstable GStreamer device manager for Pinos Lists devices exported by pinos Needed some GStreamer infrastructure to hide duplicates (in 1.6) Ported cheese to device manager Cheese can capture from Pinos Fedora package pending 23

24 Pinos next Get it used in distros Config files.. Work on zero-copy... 24

25 25 RED HAT

Integrating Video Conferencing into Everyday Applications. Olivier Crête

Integrating Video Conferencing into Everyday Applications Olivier Crête Calls integrated Calls in their own app Easy! Skype, Ekiga, WLM, etc Call directly in other apps NEW! EXCITING! INNOVATIVE! Ingredients