Video playback using HTTP Live Streaming and HTTP Adaptive Streaming

Introduction to multimedia streaming in TPC/IP networks.

Tomasz Ścisło (scislot@gmail.com)

tscislo.rootnode.net

Warsaw, 23.05.2011

Agenda

• concept overview
• detailed information about each part of multimedia streaming protocols over HTTP (MSPH) system
• different types of MSPH
• adaptive streaming concept
• media presentation description for different MSPH
• pros and cons of using MSPH in general

MSPH overview

Media encoder

• takes real-time signal from audio-video device, encodes the media and encapsulates it for delivery
• it prepares in real time MPEG-2 transport stream that can be passed to media segmenter
• it can encapsulate many elementary streams in one transport stream - means that many services can be transmitted that way
• hardware used as Video Server should be efficient one, because of real time encoding

Stream segmenter

• this is typically a software process that reads trasport stream from local network and devides it into series of small media files of equal duration
• it also creates an index file (format depends on MSPH standard) containing references to the individual media files
• in case of live streaming each time the segmenter completes a new media file, the index file is updated
• each segment is saved as .ts file (more than one service could be in such transport stream)

Distribution components

• the distribution system can be any web server (Apache, JBoss, Tomcat)
• no custom server modules are required to deliver content
• typically only content-type and MIME-type HTTP headers should be set for XML, m3u8 and ts files

Client side

• in MSPH client is typically a software process which is able to read index files and decode services taken from transport stream
• the complexity of client software may be different - it could be able to decode each service taken from transport stream or just treat transport stream atomically
• using Apple iPhone and Apple streaming standard it is possible to put index file in HTML5 video element and browser (only Safari) will be the client

Different MSPH solutions

Standard name	Vendor	Distribution method	Live support
HTTP Live Streaming	Apple	m3u8
HTTP Dynamic Streaming	Adobe	XML	?
Smooth Streaming	Microsoft	XML
HTTP Adaptive Streaming	OIPF	XML
Dynamic Adaptive HTTP Streaming	MPEG	?	?

What are the advanteges of adaptive streaming?

• usually client software starts playback using the lowest possible bitrate (slow start)
• than, for example if in time t=1,3s the client was able to download d=325kB it can switch to higher bitrate
• new optimal bitrate: B = d/t = 2Mbit/s
• to enable seamless switching segmented transport streams have to be encoded in the same media format, but may be different in the profile of that format

Real bitrates and encoding type using HLS

• Video encoding: H264 MPEG-4 AVC in 1280x720px HD resolution (2,44Mbit/s)
• Audio encoding: MPEG AAC Audio (132kbit/s)
• Above encoding methods use around 2,5Mbit/s bandwidth for audio and video
• It would be theoretically possible to encapsulate 2 more services in segmented transport strem with bandwidth equal to 7,5Mbit/s
• Broadband 10Mbit/s Internet connections are becoming more common these days
• Using this encoding and HLS standard, while having 10Mbit/s internet connection it is possible to watch sport event from 3 different perspectives

Media presentation description (MPD) for HLS (1)

• Media presentention description for HLS standard is in ExtM3U text file encoded in UTF-8 => so the extention is .m3u8
• #EXT - is the tag prefix for ExtM3U
• There are only two tags for ExtM3U format: #EXTM3U, #EXTINF
• #EXTM3U tag should be located in the first line of ExtM3U file
• #EXTINF: [duration in seconds], [title] => information about next transport stream on list

Media presentation description (MPD) for HLS (2)

• Tags for HLS MDP starts with a prefix: #EXT-X, such as: #EXT-X-TARGETDURATION, #EXT-X-PROGRAM-DATE-TIME, #EXT-X-ENDLIST, #EXT-X-STREAM-INF
• only one tag is MANDATORY => #EXT-X-TARGETDURATION

• the client software is able to descent into EXTM3U file that contains URL's to other EXTM3U files
• then the client software returns to the initial file and continuous playback until the #EXT-X-ENDLIST tag is reached

Media presentation description (MPD) for HLS (3)

• #EXT-X-TARGETDURATION: [time in seconds] - defines max duration of playlist file
• #EXT-X-PROGRAM-DATE-TIME: [YYYY-MM-DDThh:mm:ssZ] - associates the beginning of the next media file with an absolute date and/or time
• #EXT-X-ENDLIST - tag indicates that no more media files will be added to playlist file (this is the last ts to play)
• #EXT-X-STREAM-INF: BANDWIDTH=[Mbit/s], PROGRAM-ID: [number], CODECS="[names]", RESOLUTION=[number]x[number] - this tag is usually in main index file, it indicates that next URI in the playlist is another playlist file
• #EXT-X-KEY: METHOD=[AES_128][URI][IV] - media files can be encrypted using AES with 128bit key, this tag indicates how to decrypt media files

Media presentation description (MPD) for HLS (4)

• in this example we see main index file for HLS - new tags are intended
• PROGRAM-ID=1 parameter for #EXT-X-STREAM tag means that all given playlists contains the same video program, which can be accessed with different parameters
• in this case different parameter is BANDWIDTH (adaptive streaming!)
• it could be other parameter, such as different codec or resolution

Media presentation description (MPD) for HLS (5)

• this is the example of one alternative playlist file - new tags are intended
• using #EXT-X-PROGRAM-DATE-TIME client software is able to determine when current transport stream was recorded

Different playback modes in HTTP Live Streaming

• VoD (Video on Demand) In this mode client software reads constatn playlist, which is not updated by media segmenter, in fact for this kind of playback there is no need for media encoder. Only distribiution server is needed. In this mode #EXT-X-ENDLIST tag must be set at the end of playlist.
• Live with sliding window algorithm In this mode Video Server is needed. Playlist is updated in real time and one transport stream is removed from the top and new one is added at the bottom.
• Live with new transport streams added at the bottom In this mode Video Server is needed. Playlist is updated in real time and transport streams are added at the bottom of the file.

Media presentation description for HAS (1)

• Media presentation description for HAS is in XML with oipf namespace
• mimeType defines what kind of transport streams (MPEG-2, pure MP3) are used
• HAS uses UrlTemplate XML tag => due to that all transport stream segments doesn't have to be mentioned in MDP XML. Only URL template, startIndex and endIndex should be given.
• width and height attribiutes are used to determinate the dimensions of video playback

Media presentation description for HAS (2)

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.

<?xml version="1.0" encoding="UTF-8"?>
<MPD type="OnDemand" minBufferTime="PT2S" mediaPresentationDuration="PT596S" xsi:schemaLocation="urn:3GPP:ns:PSS:AdaptiveHTTPStreamingMPD:2009 3GPP-MPD-r1.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:3GPP:ns:PSS:AdaptiveHTTPStreamingMPD:2009">
    <ProgramInformation>
        <ChapterDataURL>
            chapter.xml
        </ChapterDataURL>
    </ProgramInformation>
    <Period start="PT0S" segmentAlignmentFlag="True">
        <Representation id="0" mimeType="video/mpeg" bandwidth="500000" startWithRAP="True" width="960" height="540" group="0">
            <SegmentInfo duration="PT6S" baseURL="500k/" startIndex="0">
                <UrlTemplate sourceURL="T0_Big_Buck_Bunny_HAS_s$Index$.ts" endIndex="99"/>
            </SegmentInfo>
        </Representation>
        <Representation id="0" mimeType="video/mpeg" bandwidth="1200000" startWithRAP="True" width="960" height="540" group="0">
            <SegmentInfo duration="PT6S" baseURL="1200k/" startIndex="0">
                <UrlTemplate sourceURL="T1_Big_Buck_Bunny_HAS_s$Index$.ts" endIndex="99"/>
            </SegmentInfo>
        </Representation>
        
        <Representation id="0" mimeType="video/mpeg" bandwidth="2200000" startWithRAP="True" width="1440" height="810" group="0">
            <SegmentInfo duration="PT6S" baseURL="2200k/" startIndex="0">
                <UrlTemplate sourceURL="T0_Big_Buck_Bunny_HAS$Index$.ts" endIndex="99"/>
            </SegmentInfo>
        </Representation>
        <Representation id="1" mimeType="video/mpeg" bandwidth="3200000" startWithRAP="True" width="1440" height="810" group="0">
            <SegmentInfo duration="PT6S" baseURL="3200k/" startIndex="0">
                <UrlTemplate sourceURL="T1_Big_Buck_Bunny_HAS$Index$.ts" endIndex="99"/>
            </SegmentInfo>
        </Representation>
    </Period>
</MPD>

Disadvantages of using MSPH

• all MSPH are on the higher level of abstraction that they really should be (not HTTP/TCP/IP, but some protocol/UDP/IP)
• many vendors of MSPH solutions, hard to guess which will be a world wide standard
• one MSPH standard is IETF Internet Draft (not yet a recommendation)
• more storage space on server side is needed

Advantages of using MSPH

• no specialized servers to distribute content (for VoD no need for Video Server)
• uses HTTP and/or HTTPS traffic which is by default unblocked on every firewall and router
• possibility to prepare live streaming, which is impossible for single mp4 or mpeg files
• this playback is adaptive!
• client is able to choose only bitrates suitable for it
• it is possible to use AES-128bit algorithm for secure data transfer

Credits

• Adaptive Streaming - a brief tutorial by Niels Laukens, VRT Medialab
• OIPF Release 2 Specification HTTP Adaptive Streaming [V2.0] - [2010-09-07]
• HTTP Live Streaming Overview - Apple Inc., 2010-02-05

• Thank you. Questions?