Creating HLS content

MPEG-DASH is one way to create adaptive bitstream playback, but it’s not the only one. In this post, we’ll explore Apple’s HLS (HTML Live Streaming) a direct competitor for MPEG-DASH and see how to build a package ready for Video On Demand.

I will use FFMPEG to encode the video as required in the HLS specification. This is the same tool that I’ve used to create videos with other codecs.

HLS Creation: Single Rendition

At WWDC 2017, Apple announced support for HEVC (H265), letting you work with both types of streams in the same playlist.
 
The technologies to implement HEVC in HLS is significantly different than what we discuss here so I’ll probably do a separate post on it rather than confuse matters. I’m also banking on the fact that H264 has better support in both Apple and non-Apple devices.

HLS works differently than DASH with different formats and parameters. We will use FFMPEG command line tool to create the HLS video. It provides a lot of tools and supports the creation of the necessary files out of the box.

HLS works with Transport Stream segments and has much tighter control over the characteristics of the streams (or renditions in HLS parlance).

Encoding a single rendition of your video looks like this for an H264 product.

# Create the directory to store the files
mkdir serenity
# run the command
ffmpeg -i serenity.mp4 \
-vf scale=w=640:h=360:force_original_aspect_ratio=decrease \
-c:a aac -ar 48000 -b:a 96k \
-c:v h264 \
-profile:v main \
-crf 20 \
-g 48 -keyint_min 48 \
-sc_threshold 0 \
-b:v 800k -maxrate 856k -bufsize 1200k \
-hls_time 4 \
-hls_playlist_type vod \
-hls_segment_filename serenity/360p_%03d.ts \
serenity/360p.m3u8

The explanation for each of the parameters in the example are described in the following list:

  • -i serenity.mp4 – set serenity.mkv as input file
  • -vf "scale=w=1280:h=720:force_original_aspect_ratio=decrease" – scale video to maximum possible within 1280×720 while preserving aspect ratio
  • -c:a aac -ar 48000 -b:a 96k – set audio codec to AAC with sampling of 48kHz and bitrate of 128k
  • -c:v h264 – set video codec to be H264 which is the standard codec of HLS segments
  • -profile:v main – set H264 profile to main – this means support in modern devices
  • -crf 20 – Constant Rate Factor, high level factor for overall quality
  • -g 48 -keyint_min 48 – create key frame (I-frame) every 48 frames (~2 seconds) – will later affect correct slicing of segments and alignment of renditions
  • -sc_threshold 0 – don’t create key frames on scene change – only according to -g
  • -b:v 800k -maxrate 856k -bufsize 1200k – limit video bitrate, these are rendition specific and depends on your content type
  • -hls_time 4 – segment target duration in seconds – the actual length is constrained by key frames
  • -hls_playlist_type vod – adds the #EXT-X-PLAYLIST-TYPE:VOD tag and keeps all segments in the playlist
  • -hls_segment_filename beach/360p_%03d.ts – explicitly define segments files’ names
  • serenity/360p.m3u8 – path of the playlist file – also tells ffmpeg to output HLS (.m3u8)

Running the code above will produce a set of files:

  • One or more transport stream segments with a .ts extension. In this case, it produced 34 segments for the video I chose
  • The playlist with a .m3u8 format

The playlist for the video looks like this:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:4.004267,
360p_000.ts
#EXTINF:4.004267,
360p_001.ts
#EXTINF:4.004267,
360p_002.ts
#EXTINF:4.004267,
360p_003.ts
#EXTINF:4.004267,
360p_004.ts
#EXTINF:4.004267,
360p_005.ts
#EXTINF:4.004267,
360p_006.ts
#EXTINF:4.004267,
360p_007.ts
#EXTINF:4.004267,
360p_008.ts
#EXTINF:4.004267,
360p_009.ts
#EXTINF:4.004267,
360p_010.ts
#EXTINF:4.004267,
360p_011.ts
#EXTINF:4.004267,
360p_012.ts
#EXTINF:4.004267,
360p_013.ts
#EXTINF:4.004267,
360p_014.ts
#EXTINF:4.004267,
360p_015.ts
#EXTINF:4.004267,
360p_016.ts
#EXTINF:4.004267,
360p_017.ts
#EXTINF:4.004267,
360p_018.ts
#EXTINF:4.004267,
360p_019.ts
#EXTINF:4.004267,
360p_020.ts
#EXTINF:4.004267,
360p_021.ts
#EXTINF:4.004267,
360p_022.ts
#EXTINF:4.004267,
360p_023.ts
#EXTINF:4.004267,
360p_024.ts
#EXTINF:4.004267,
360p_025.ts
#EXTINF:4.004267,
360p_026.ts
#EXTINF:4.004267,
360p_027.ts
#EXTINF:4.004267,
360p_028.ts
#EXTINF:4.004267,
360p_029.ts
#EXTINF:4.004267,
360p_030.ts
#EXTINF:4.004267,
360p_031.ts
#EXTINF:4.004267,
360p_032.ts
#EXTINF:4.004267,
360p_033.ts
#EXTINF:3.503733,
360p_034.ts
#EXT-X-ENDLIST

HLS Creation: Multiple Renditions And Master Playlist

All the work we did in the previous section was for one rendition. You will definitely want more than one to acommoadte both your low-end, low-bandwidth users as well as those who are watching in the latest 4k iMac on fiber connections. We can create multiple renditions and let the player decide what’s the best one to play at any given time of the playback.

This command will create 4 versions of the video along with a playlist for each:

  • 360p
  • 480p
  • 720p
  • 1080p
ffmpeg -hide_banner -y -i serenity.mp4 \
  -vf scale=w=640:h=360:force_original_aspect_ratio=decrease \
  -c:a aac -ar 48000 -c:v h264 -profile:v main \
  -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod  -b:v 800k -maxrate 856k \
  -bufsize 1200k -b:a 96k -hls_segment_filename serenity/360p_%03d.ts serenity/360p.m3u8 \
  -vf scale=w=842:h=480:force_original_aspect_ratio=decrease \
  -c:a aac -ar 48000 -c:v h264 -profile:v main \
  -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 1400k -maxrate 1498k \
  -bufsize 2100k -b:a 128k -hls_segment_filename serenity/480p_%03d.ts serenity/480p.m3u8 \
  -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease \
  -c:a aac -ar 48000 -c:v h264 -profile:v main \
  -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 2800k -maxrate 2996k \
  -bufsize 4200k -b:a 128k -hls_segment_filename serenity/720p_%03d.ts serenity/720p.m3u8 \
  -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease \
  -c:a aac -ar 48000 -c:v h264 -profile:v main \
  -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 \
   -hls_playlist_type vod -b:v 5000k -maxrate 5350k \
  -bufsize 7500k -b:a 192k -hls_segment_filename serenity/1080p_%03d.ts serenity/1080p.m3u8

This created the media but there’s no way for the player to know what’s available. To fix this we create a master playlist that tells the player what renditions are available for the video.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
360p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1400000,RESOLUTION=842x480
480p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2800000,RESOLUTION=1280x720
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080
1080p.m3u8

Note that this playlist doesn’t include the stream transport segments. The master list only tells the player where to look for the corresponding renditions.

Yes, this is cumbersome to do by hand and, when also working with DASH packages, it can be hard to manage.

There are different tools to automate the generation of the HSL Renditions and playlists.

Newer versions of Google’s Shaka Packager support creating HLS segments and playlists. They also support creating HLS and DASH content with a single command (although this appears to only with single MP4 files).

Assuming that you’ve already downloaded/compiled the packager, the command to generate DASH and HSL look like this:

packager \
  in=h264_baseline_360p_600.mp4,stream=audio,output=audio.mp4,playlist_name=audio.m3u8,hls_group_id=audio,hls_name=ENGLISH \
  in=h264_baseline_360p_600.mp4,stream=video,output=h264_360p.mp4,playlist_name=h264_360p.m3u8,iframe_playlist_name=h264_360p_iframe.m3u8 \
  in=h264_main_480p_1000.mp4,stream=video,output=h264_480p.mp4,playlist_name=h264_480p.m3u8,iframe_playlist_name=h264_480p_iframe.m3u8 \
  in=h264_main_720p_3000.mp4,stream=video,output=h264_720p.mp4,playlist_name=h264_720p.m3u8,iframe_playlist_name=h264_720p_iframe.m3u8 \
  in=h264_main_1080p_6000.mp4,stream=video,output=h264_1080p.mp4,playlist_name=h264_1080p.m3u8,iframe_playlist_name=h264_1080p_iframe.m3u8 \
  --hls_master_playlist_output h264_master.m3u8 \
  --mpd_output h264.mpd

The result produces the following files:

.
├── audio.m3u8
├── audio.mp4
├── h264.mpd
├── h264_1080p.m3u8
├── h264_1080p.mp4
├── h264_1080p_iframe.m3u8
├── h264_360p.m3u8
├── h264_360p.mp4
├── h264_360p_iframe.m3u8
├── h264_480p.m3u8
├── h264_480p.mp4
├── h264_480p_iframe.m3u8
├── h264_720p.m3u8
├── h264_720p.mp4
├── h264_720p_iframe.m3u8
├── h264_baseline_360p_600.mp4
├── h264_main_1080p_6000.mp4
├── h264_main_480p_1000.mp4
├── h264_main_720p_3000.mp4
└── h264_master.m3u8

And these can be used to play either DASH or HLS video. We’ll tackle playback next.

Playback

I know how to use Shaka Player to play DASH content. It should be possible to use the same player to play both DASH and HLS content although it shouldn’t be necessary to play both, the player scripts should take care of cross-platform issues.

Another option is to use Video.js with the HSL plugin and the DASH if you need to serve both formats on the same page.

The basic functionality looks like this:

<video  id=example-video
        width=600 height=300
        class="video-js vjs-default-skin"
        controls>
  <source
        src="https://example.com/index.m3u8"
        type="application/x-mpegURL">
</video>

<!--
  Put the lines below at the bottom of the page,
  right before the closing body tag
-->
<script src="video.js"></script>
<script src="videojs-contrib-hls.min.js"></script>
<script>
var player = videojs('example-video');
player.play();
</script>

I’m tracking issue 1365 in the Video.js HSL Plugin Repository to track the issue of content not playing in Chrome.

I’m also researching if I can combine DASH and HLS in a single player for either Shaka Player or Video.js.

Encoding Suggestions

Like DASH, HLS allows for multiple renditions of content at different resolutions. What these resolutions are will depend on your content and your target audience.

Each row presents to bitrate values, one for low motion content like conference presentations and other activities where there is little motion. The other is for the opposite type of videos like sports and other high motion activities. The audio bitrate remains constant for both low and high motion videos.

The table is taken from Peer 5’s Creating A Production Ready Multi Bitrate HLS VOD stream

Quality Resolution bitrate – low motion bitrate – high motion audio bitrate
240p 426×240 400k 600k 64k
360p 640×360 700k 900k 96k
480p 854×480 1250k 1600k 128k
HD 720p 1280×720 2500k 3200k 128k
HD 720p 60fps 1280×720 3500k 4400k 128k
Full HD 1080p 1920×1080 4500k 5300k 192k
Full HD 1080p 60fps 1920×1080 5800k 7400k 192k
4k 3840×2160 14000k 18200 192k
4k 60fps 3840×2160 23000k 29500k 192k

References