Video Streaming: from the native Android player to unconventional devices
1 like949 views
The document discusses video streaming using the Android platform, focusing on various approaches and libraries such as MediaPlayer, ExoPlayer, and the Android Multimedia Framework. It outlines the architecture for creating Android TV applications, including the use of Leanback support libraries, Chromecast device integration, and the development flow for sender applications. Additionally, it provides code examples and resources for building multimedia applications on Android devices.
![Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Leanback UI component
BroswerFragment
allows the developer “to create a primary layout for browsing categories and rows of
media items [with a minimum of code]”](https://image.slidesharecdn.com/presentazione-slideshareversion-141114055223-conversion-gate01/85/Video-Streaming-from-the-native-Android-player-to-unconventional-devices-21-320.jpg)
![Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
Contacts
a.martellucci[at]reply.it
@martellux
+AlessandroMartellucci
m.bonifazi[at]reply.it
@mbonifazi
+MatteoBonifazi
http://www.reply.eu http://www.gaiareply.com](https://image.slidesharecdn.com/presentazione-slideshareversion-141114055223-conversion-gate01/85/Video-Streaming-from-the-native-Android-player-to-unconventional-devices-42-320.jpg)
Video Streaming: from the native Android player to unconventional devices
- 1. Video Streaming from the native Android player to uncoventional devices Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci
- 2. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci • Reply is today a leading IT Services Company • Operates in Italy, Germany, UK, Benelux, USA and Brasil. • Open Reply is the company of Reply Group focused on open source software, multichannel web solutions and mobile applications. • Based in Rome, Open Reply’s Android Lab is a young team of over 20 engineers 100% focused on Android development. • We are specialised in broadcasting, banking and Android OS Customisation. Open Reply & Android Lab
- 3. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci • Playing a content on your handheld device. • Entertaiment tailored for the user with • Easiest way to enjoy online video and music on user TV on Presentation Millestones
- 4. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Android multimedia framework • Android support for playing several media types from media file stored inside the application (raw resources, standalone files) or for OTT streaming. • Documentation of the media framework is enough just for simple test case. • Underneath MediaPlayer Framework documentation is kind of nightmare (lot of events and error are not documented at all).
- 5. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Supported type HttpLiveStreaming Protocol 3 is supported by Android 4.0. Best implementation is in Kitkat. Google seems putting all his efforts to support new media type ( SmoothStreaming, Dynamic adaptive streaming)
- 6. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Playing a video VideoView Three different approaches Mediaplayer & SurfaceView ExoPlayer
- 7. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci //1. Find the view from the layout VideoView myVideoView = (VideoView)findViewById(R.id.myvideoview); //2. Setup video url myVideoView.setVideoURI(Uri.parse(SrcPath)); //3. Setup Video controller myVideoView.setMediaController(new MediaController(this)); //4. Start playing myVideoView.requestFocus(); myVideoView.start(); android.widget.VideoView MediaPlayer code example Programmer doesn’t directly handle the MediaPlayer
- 8. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci //0. Get SurfaceView and its holder mPreview = (SurfaceView)findViewById(R.id.surfaceView); holder = mPreview.getHolder(); //1. Create MediaPlayer object: mp = new MediaPlayer(); //2. Add SurfaceHolder callback - Aware when SurfaceView is created holder.addCallback(new SurfaceHolder.Callback(){ .... @Override public void surfaceCreated(SurfaceHolder holder) { //3. Attach the surface to the player mp.setDisplay(holder); try { mp.setDataSource(filepath); //4. Prepare the Mediaplayer in sync or async mode ( prepareAsync() ) mp.prepare(); } catch (Exception e) {// Catch the exception} //5. Start the player mp.start(); }... }); android.media.Mediaplayer & android.view.SurfaceView MediaPlayer code example Application creates SurfaceView and MediaPlayer. More control on MediaPlayer state
- 9. Droidcon London – 2014 – Matteo Bonifazi e Alessandro Martellucci // 1. Instantiate the player. player = ExoPlayer.Factory.newInstance(RENDERER_COUNT); // 2. Construct renderers. MediaCodecVideoTrackRenderer videoRenderer = … MediaCodecAudioTrackRenderer audioRenderer = ... // 3. Inject the renderers through prepare. player.prepare(videoRenderer, audioRenderer); // 4. Pass the surface to the video renderer. player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface); // 5. Start playback. player.setPlayWhenReady(true); ... player.release(); // Release when everything is done! ExoPlayer MediaPlayer code example Pre-built player that can be extend. Implement features not currently supported in the normal mediaplayer.
- 10. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci What kind of Mediaplayer exist? • Built-in players ü AwesomePlayer (default player selected) ü NuPlayer (Apple HLS) • Extra player factories can be registered • DIY mediaplayer ü Demuxing: android.media.mediaExtractor ü Decoding: android.media.MediaCodec ü Video rendering: android.media.MediaCodec ü Audio rendering: android.media.AudioTrack ü Implement the interface frameworks/av/include/media/MediaPlayerInterface.h
- 11. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Android TV Entertainment tailored for you Source:h)p://www.televisedrevolu4on.com/
- 12. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Nexus Player Source:h)p://www.televisedrevolu4on.com/ 1.8GHz Quad Core, Intel® Atom™ Imagination PowerVR Series 6 Graphics 2D/3D Engine 1GB RAM 8GB storage Asus device – 235 g
- 13. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Android TV app features • Android TV devices have Android Lollipop 5.0 on board. • Android TV has inside the same Android mul4media framework of normal devices. • Android TV app can be built on API 17 towards. • Based on Leanback Android Support library
- 14. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci 10 feet experience Smartphone 5” 320 dp TV Full HD 30” 320 dpi 10 feet 16 inches
- 15. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Manifest features Support landscape Portrait activity are forbidden
- 16. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Manifest features No touch screen <uses-feature android:name="android.hardware.touchscreen" android:required="false"/>
- 17. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Manifest features Limit sensor <uses-feature android:name="android.hardware.sensor.accelerometer” android:required="false" />
- 18. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Keep calm and lean back!! Provides built-in tailored for 10 feet experience
- 19. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Leanback support library <uses-feature android:name="android.software.leanback" android:required="true" /> <application android:allowBackup="false” android:label="@string/app_name” android:theme="@style/Theme.Leanback" > <activity android:name="MainActivity” android:screenOrientation="landscape" > <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LEANBACK_LAUNCHER" /> </intent-filter> </activity>
- 20. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Leanback support library Model View Presenter Presenter Model View
- 21. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Leanback UI component BroswerFragment allows the developer “to create a primary layout for browsing categories and rows of media items [with a minimum of code]”
- 22. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Leanback UI component DetailFragment Display information about the content that the user has selected
- 23. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Chromecast a cast-ready device for multi-screen experience source: www.google.it
- 24. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Chromecast hardware specifications source: www.pcworld.com
- 25. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Chromecast components • Google Cast technology • Multi-Screen experiece • Google Cast SDK • Sender Application • Android app • iOS app • Chrome app • Receiver Application • Default Media Receiver • Styled Media Receiver • Custom Media Receiver
- 26. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Android Client Application library dependencies • Minimum SDK version supported by Google Cast is 9 (Gingerbread) • MediaRouter API of android-support-v7 • Google Play Services • AppCompat API of android-support-v7
- 27. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Android Client Application typical sender application flow • Sender app starts MediaRouter device discovery: MediaRouter.addCallback • MediaRouter informs sender app of the route the user selected: MediaRouter.Callback.onRouteSelected • Sender app retrieves CastDevice instance: CastDevice.getFromBundle • Sender app creates and uses GoogleApiClient: GoogleApiClient.Builder • Sender app launches the receiver app: Cast.CastApi.launchApplication • Sender app creates a communication channel: Cast.CastApi.setMessageReceivedCallbacks • Sender sends a message to the receiver over the communication channel: Cast.CastApi.sendMessage source: developers.google.com
- 28. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Cast-Ready Device Discovery capabilities Remote Playback Live Audio Live Video MediaRouteSelector.Builder mediaRouteSelectorBuilder = new MediaRouteSelector.Builder(); mediaRouteSelectorBuilder.addControlCategory(MediaControlIntent.CATEGORY_REMOTE_PLAYBACK); mediaRouteSelectorBuilder.addControlCategory(MediaControlIntent.CATEGORY_LIVE_AUDIO); mediaRouteSelectorBuilder.addControlCategory(MediaControlIntent.CATEGORY_LIVE_VIDEO); MediaRouterSelector mediaRouterSelector = mediaRouterSelectorBuilder.build();
- 29. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Media Cast Button easy approach for discovering source: developers.google.com source: developers.google.com public boolean onCreateOptionsMenu(Menu menu) { MenuItem mediaRouteMenuItem = menu.findItem(R.id.media_route_menu_item); MediaRouteActionProvider mediaRouteActionProvider = (MediaRouteActionProvider) MenuItemCompat.getActionProvider(mediaRouteMenuItem); mediaRouteActionProvider.setRouteSelector(mMediaRouteSelector); …. }
- 30. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Receiver Application what a mistery? What is? HTML5 and Javascript application What does it do? Display the media content on TV Message handling Which type? Default Media Receiver Styled Media Receiver Custom Media Receiver
- 31. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Default Media Receiver simplest • Off-the-shelf • No UI customization • No registration Source: developers.google.com
- 32. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Styled Media Receiver simple and customizable • Similar to Default Media Player • CSS UI customization • Registration Source: developers.google.com
- 33. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Custom Media Receiver(1/3) whatever you want • Fully Web Applicaiton • Debug(able) at 9222 • Registration Source: developers.google.com
- 34. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Custom Media Receiver(2/3) example <html> <head> <title>Example minimum receiver</title> <script src="//www.gstatic.com/cast/sdk/libs/receiver/2.0.0/cast_receiver.js"></script> </head> <body> <video id='media'/> <script> window.mediaElement = document.getElementById('media'); window.mediaManager = new cast.receiver.MediaManager(window.mediaElement); window.castReceiverManager = cast.receiver.CastReceiverManager.getInstance(); window.castReceiverManager.start(); </script> </body> </html>
- 35. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Custom Media Receiver(3/3) advanced features • Video Codification/Decodification • H.264 High Profile Level 4.1, 4.2 and 5 • VP8 • Adaptive Bitrate Streaming • HTTP Live Streaming (HLS) • Dynamic Adaptive Streaming over HTTP (MPEG-DASH) • Smooth Streaming • Digital Rights Management • Play Ready DRM • Widevine DRM • Media Player Library
- 36. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Channel, Namespace and Protocol (1/2) communication • Protocol: a set of well-known messages • Namespace: a labeled protocol • Channel: the communication layer class CustomChannel implements Cast.MessageReceivedCallback { public String getNamespace() { return “urn:x-cast:com.channel.custom”; } @Override public void onMessageReceiver(CastDevice castDevice, String namespace, String message) { … } } … Cast.CastApi.setMessageReceivedCallbacks(mApiClient, mCustomChannel.getNamespace(), mCustomChannel); …
- 37. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Channel, Namespace and Protocol (2/2) communication • Media Namespace: urn:x-cast:com.google.media.cast • RemoteMediaPlayer • MediaManager RECEIVER MEDIA CHANNEL window.mediaManager = new cast.receiver.MediaManager(window.mediaElement); … CLIENT MEDIA CHANNEL Cast.CastApi.setMessageReceivedCallbacks(mApiClient, mRemoteMediaPlayer.getNamespace(), mCustomChannel); …
- 38. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Google Api Client and Media Route Provider manufacturing Source: developers.google.com CastDevice selectedCastDevice = CastDevice.getFromBundle(selectedRouteInfo.getExtras); Cast.CastOptions.Builder apiOptionsBuilder = new Cast.CastOptions.Builder(selectedCastDevice, …); googleApiClient = new GoogleApiClient.Builder().addApi(Cast.API, apiOptionsBuilder.build()).build(); googleApiClient.connect(); Cast.CastApi.launchApplication(googleApiClient, applicationId, launchOptions); Cast.CastApi.joinApplication(googleApiClient); Cast.CastApi.stopApplication(googleApiClient); Cast.CastApi.leaveApplication(googleApiClient);
- 39. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci RemoteMediaPlayer …finally MediaInfo mediaInfo = new MediaInfo.Builder("http://your.server.com/video.mp4") .setContentType("video/mp4") .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) .setMetadata(mediaMetadata) .build(); … mRemoteMediaPlayer = new RemoteMediaPlayer(); mRemoteMediaPlayer.load(mApiClient, mediaInfo, true); …
- 40. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Enjoy the video Source: www.huffingtonpost.ca Thank you to all of you
- 41. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Resources • Google Cast -> https://developers.google.com/cast • Chromecast App -> http://www.google.it/chrome/devices/chromecast/apps.html • Google Cast Downloads -> https://developers.google.com/cast/docs/downloads • Github -> https://github.com/googlecast • Android TV -> https://developer.android.com/tv • Nvidia Guideline -> https://developer.nvidia.com/android-tv-developer-guide • Github -> https://github.com/googlesamples/androidtv-Leanback.git • Android Mediaplayer-> http://developer.android.com/guide/topics/media/mediaplayer.html • ExoPlayer -> https://github.com/google/ExoPlayer
- 42. Droidcon London – 2014 – Matteo Bonifazi & Alessandro Martellucci Contacts a.martellucci[at]reply.it @martellux +AlessandroMartellucci m.bonifazi[at]reply.it @mbonifazi +MatteoBonifazi http://www.reply.eu http://www.gaiareply.com