Ubuntu vaapi-ffmpeg开发环境搭建

本篇文章有些内容比较过时，最新请参考：https://blog.jianchihu.net/intel-gpu-hw-video-codec-develop.html。包含CentOS以及Ubuntu的开发环境搭建

最近要在Linux上做编解码开发，为了成本考虑，没用NVIDIA的方案，用了Intel编解码方案。大家都知道Intel在Windows上有个Intel Media SDK的方案，比较常用，支持的CPU型号也多，在Linux上也有类似方案，叫做Intel Media Server Studio。但是Intel Media Server Studio支持的型号比较少，如下是官方文档说明的支持型号:

System Requirements
Hardware
Intel® Media Server Studio supports the following platforms with the integrated graphics:
 Intel® Xeon® E3-1200 v4 Family with C226 chipset
 Intel® Xeon® E3-1200 and E3-1500 v5 Family with C236 chipset
 5th Generation Intel® Core™
 6th Generation Intel® Core™
Additionally, for Intel® Xeon® E5 v4 and v5 processors, support of software-only (CPU)
HEVC decode and encode, select video pre-processing (Color Space Conversion, Scaling),
and virtualization (KVM*, Xen*) is available.
Note: Individual components could have specific requirements, please read the
corresponding release notes.

System Requirements

Hardware

Intel® Media Server Studio supports the following platforms with the integrated graphics:

 Intel® Xeon® E3-1200 v4 Family with C226 chipset

 Intel® Xeon® E3-1200 and E3-1500 v5 Family with C236 chipset

 5th Generation Intel® Core™

 6th Generation Intel® Core™

Additionally, for Intel® Xeon® E5 v4 and v5 processors, support of software-only (CPU)

HEVC decode and encode, select video pre-processing (Color Space Conversion, Scaling),

and virtualization (KVM*, Xen*) is available.

Note: Individual components could have specific requirements, please read the

corresponding release notes.

不过还有一个叫做VA API(Video Acceleration API) 的方案，VA API是一个统一的编解码规范，类似Windows上的Dxva方案，主要由各大显卡厂商在驱动中实现。目前主要Intel与AMD实现这个VA API方案，不过AMD上支持的编解码特性比较少，也只是部分支持。对于Intel来说，基本上带集显的都支持VA API。所以为了成本，通用性考虑，我选用了这个VA API做Linux上的编解码开发。

如果基于原生VA API开发的话，比较复杂点，好在ffmpeg支持VA API，所以我们只需要编译支持VA API的ffmpeg即可。本篇文章主要讲下VA API开发环境的搭建，主要针对H264编解码。开发系统为ubuntu18.04.2 LTS server版本。需要注意的是，目前没看到好的显卡直通（passthrough）以及虚拟化方案，我们还是老实在实体机Linux开发，不能在虚拟机里，要不就用不了显卡硬件加速了。

基本库安装

sudo apt-get -y install autoconf automake build-essential libass-dev libtool pkg-config \
texinfo zlib1g-dev libva-dev cmake mercurial libdrm-dev libvorbis-dev libogg-dev git \
libx11-dev libperl-dev libpciaccess-dev libpciaccess0 xorg-dev git-core libfreetype6-dev \
libsdl2-dev libtool libvdpau-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev wget zlib1g-dev

sudo apt-get -y install autoconf automake build-essential libass-dev libtool pkg-config \

texinfo zlib1g-dev libva-dev cmake mercurial libdrm-dev libvorbis-dev libogg-dev git \

libx11-dev libperl-dev libpciaccess-dev libpciaccess0 xorg-dev git-core libfreetype6-dev \

libsdl2-dev libtool libvdpau-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev wget zlib1g-dev

VA API相关库驱动安装

Libva

Libva是VA API的实现。

git clone https://github.com/01org/libva
cd libva
./autogen.sh
./configure
make
make install

git clone https://github.com/01org/libva

cd libva

./autogen.sh

./configure

make

make install

intel-vaapi-driver

主要在我们的程序与Intel 集显之间起桥梁作用。传输打包的缓存以及命令到i965驱动（开源的intel集显驱动，已集成在Linux内核中），用于硬件加速的视频编解码，着色器处理等。

git clone https://github.com/01org/intel-vaapi-driver
cd intel-vaapi-driver
./autogen.sh
./configure
make
make install

git clone https://github.com/01org/intel-vaapi-driver

cd intel-vaapi-driver

./autogen.sh

./configure

make

make install

libva-utils

提供一系列 VA API相关的测试。比如vainfo命令，可以用来检测我们的硬件支持哪些VA API编解码特性。

git clone https://github.com/intel/libva-utils
cd libva-utils
./autogen.sh
./configure
make
make install

git clone https://github.com/intel/libva-utils

cd libva-utils

./autogen.sh

./configure

make

make install

这一步安装完后，我们开始检测安装的成果，首先查看我们的显卡设备：

root@ubuntu:~/libva-utils# ls /dev/dri/
by-path  card0  card1  renderD128  renderD129

1 2	root@ubuntu:~/libva-utils# ls /dev/dri/ by-path card0 card1 renderD128 renderD129

可以看到我的电脑有两张显卡，一张独显，一张集显。下面开始通过vainfo命令验证显卡支持情况：

root@ubuntu:~/libva-utils# vainfo --display drm --device /dev/dri/renderD128
libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/r600_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.4.0.pre1)
vainfo: Driver version: Mesa Gallium driver 19.1.0-devel for AMD OLAND (DRM 2.50.0, 4.15.0-45-generic, LLVM 7.0.1)
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple            :	VAEntrypointVLD
VAProfileMPEG2Main              :	VAEntrypointVLD
VAProfileVC1Simple              :	VAEntrypointVLD
VAProfileVC1Main                :	VAEntrypointVLD
VAProfileVC1Advanced            :	VAEntrypointVLD
VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
VAProfileH264Main               :	VAEntrypointVLD
VAProfileH264High               :	VAEntrypointVLD
VAProfileNone                   :	VAEntrypointVideoProc
root@ubuntu:~/libva-utils# vainfo --display drm --device /dev/dri/renderD129
libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.4.0.pre1)
vainfo: Driver version: Intel i965 driver for Intel(R) Kaby Lake - 2.1.0
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple            :	VAEntrypointVLD
VAProfileMPEG2Simple            :	VAEntrypointEncSlice
VAProfileMPEG2Main              :	VAEntrypointVLD
VAProfileMPEG2Main              :	VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
VAProfileH264Main               :	VAEntrypointVLD
VAProfileH264Main               :	VAEntrypointEncSlice
VAProfileH264Main               :	VAEntrypointEncSliceLP
VAProfileH264High               :	VAEntrypointVLD
VAProfileH264High               :	VAEntrypointEncSlice
VAProfileH264High               :	VAEntrypointEncSliceLP
VAProfileH264MultiviewHigh      :	VAEntrypointVLD
VAProfileH264MultiviewHigh      :	VAEntrypointEncSlice
VAProfileH264StereoHigh         :	VAEntrypointVLD
VAProfileH264StereoHigh         :	VAEntrypointEncSlice
VAProfileVC1Simple              :	VAEntrypointVLD
VAProfileVC1Main                :	VAEntrypointVLD
VAProfileVC1Advanced            :	VAEntrypointVLD
VAProfileNone                   :	VAEntrypointVideoProc
VAProfileJPEGBaseline           :	VAEntrypointVLD
VAProfileJPEGBaseline           :	VAEntrypointEncPicture
VAProfileVP8Version0_3          :	VAEntrypointVLD
VAProfileVP8Version0_3          :	VAEntrypointEncSlice
VAProfileHEVCMain               :	VAEntrypointVLD
VAProfileHEVCMain               :	VAEntrypointEncSlice
VAProfileHEVCMain10             :	VAEntrypointVLD
VAProfileHEVCMain10             :	VAEntrypointEncSlice
VAProfileVP9Profile0            :	VAEntrypointVLD
VAProfileVP9Profile0            :	VAEntrypointEncSlice
VAProfileVP9Profile2            :	VAEntrypointVLD

root@ubuntu:~/libva-utils# vainfo --display drm --device /dev/dri/renderD128

libva info: VA-API version 1.1.0

libva info: va_getDriverName() returns 0

libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/r600_drv_video.so

libva info: Found init function __vaDriverInit_1_1

libva info: va_openDriver() returns 0

vainfo: VA-API version: 1.1 (libva 2.4.0.pre1)

vainfo: Driver version: Mesa Gallium driver 19.1.0-devel for AMD OLAND (DRM 2.50.0, 4.15.0-45-generic, LLVM 7.0.1)

vainfo: Supported profile and entrypoints

VAProfileMPEG2Simple : VAEntrypointVLD

VAProfileMPEG2Main : VAEntrypointVLD

VAProfileVC1Simple : VAEntrypointVLD

VAProfileVC1Main : VAEntrypointVLD

VAProfileVC1Advanced : VAEntrypointVLD

VAProfileH264ConstrainedBaseline: VAEntrypointVLD

VAProfileH264Main : VAEntrypointVLD

VAProfileH264High : VAEntrypointVLD

VAProfileNone : VAEntrypointVideoProc

root@ubuntu:~/libva-utils# vainfo --display drm --device /dev/dri/renderD129

libva info: VA-API version 1.1.0

libva info: va_getDriverName() returns 0

libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so

libva info: Found init function __vaDriverInit_1_1

libva info: va_openDriver() returns 0

vainfo: VA-API version: 1.1 (libva 2.4.0.pre1)

vainfo: Driver version: Intel i965 driver for Intel(R) Kaby Lake - 2.1.0

vainfo: Supported profile and entrypoints

VAProfileMPEG2Simple : VAEntrypointVLD

VAProfileMPEG2Simple : VAEntrypointEncSlice

VAProfileMPEG2Main : VAEntrypointVLD

VAProfileMPEG2Main : VAEntrypointEncSlice

VAProfileH264ConstrainedBaseline: VAEntrypointVLD

VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice

VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP

VAProfileH264Main : VAEntrypointVLD

VAProfileH264Main : VAEntrypointEncSlice

VAProfileH264Main : VAEntrypointEncSliceLP

VAProfileH264High : VAEntrypointVLD

VAProfileH264High : VAEntrypointEncSlice

VAProfileH264High : VAEntrypointEncSliceLP

VAProfileH264MultiviewHigh : VAEntrypointVLD

VAProfileH264MultiviewHigh : VAEntrypointEncSlice

VAProfileH264StereoHigh : VAEntrypointVLD

VAProfileH264StereoHigh : VAEntrypointEncSlice

VAProfileVC1Simple : VAEntrypointVLD

VAProfileVC1Main : VAEntrypointVLD

VAProfileVC1Advanced : VAEntrypointVLD

VAProfileNone : VAEntrypointVideoProc

VAProfileJPEGBaseline : VAEntrypointVLD

VAProfileJPEGBaseline : VAEntrypointEncPicture

VAProfileVP8Version0_3 : VAEntrypointVLD

VAProfileVP8Version0_3 : VAEntrypointEncSlice

VAProfileHEVCMain : VAEntrypointVLD

VAProfileHEVCMain : VAEntrypointEncSlice

VAProfileHEVCMain10 : VAEntrypointVLD

VAProfileHEVCMain10 : VAEntrypointEncSlice

VAProfileVP9Profile0 : VAEntrypointVLD

VAProfileVP9Profile0 : VAEntrypointEncSlice

VAProfileVP9Profile2 : VAEntrypointVLD

可以看到我的AMD独显VA API支持很少，Intel集显基本都支持了。如果通过vainfo命令我们如上所示得到显卡支持情况，说明我们VA API相关驱动以及库安装成功了。下面介绍下支持VA API的ffmpeg的编译。

ffmpeg编译

参照官方编译，创建相关目录用于存放源码以及编译后的程序。

mkdir -p ~/ffmpeg_sources ~/bin

1	mkdir -p ~/ffmpeg_sources ~/bin

安装nasm与yasm。ffmpeg一些汇编程序用到。

sudo apt-get install nasm
sudo apt-get install yasm

1 2	sudo apt-get install nasm sudo apt-get install yasm

编译安装libx264,用于软编H264。

cd ~/ffmpeg_sources && \
git -C x264 pull 2> /dev/null || git clone --depth 1 https://git.videolan.org/git/x264 && \
cd x264 && \
PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure --prefix="$HOME/ffmpeg_build" --bindir="$HOME/bin" --enable-static --enable-pic && \
PATH="$HOME/bin:$PATH" make && \
make install

cd ~/ffmpeg_sources && \

git -C x264 pull 2> /dev/null || git clone --depth 1 https://git.videolan.org/git/x264 && \

cd x264 && \

PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure --prefix="$HOME/ffmpeg_build" --bindir="$HOME/bin" --enable-static --enable-pic && \

PATH="$HOME/bin:$PATH" make && \

make install

下载最新的ffmepg源码，编译。

cd ~/ffmpeg_sources && \
wget -O ffmpeg-snapshot.tar.bz2 https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2 && \
tar xjvf ffmpeg-snapshot.tar.bz2 && \
cd ffmpeg && \
PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \
--prefix="$HOME/ffmpeg_build" \
--pkg-config-flags="--static" \
--extra-cflags="-I$HOME/ffmpeg_build/include" \
--extra-ldflags="-L$HOME/ffmpeg_build/lib" \
--extra-libs="-lpthread -lm" \
--bindir="$HOME/bin" \
--enable-gpl \
--enable-vaapi \
--enable-libx264 \
--enable-nonfree && \
PATH="$HOME/bin:$PATH" make && \
make install && \
hash -r
source ~/.profile

cd ~/ffmpeg_sources && \

wget -O ffmpeg-snapshot.tar.bz2 https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2 && \

tar xjvf ffmpeg-snapshot.tar.bz2 && \

cd ffmpeg && \

PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \

--prefix="$HOME/ffmpeg_build" \

--pkg-config-flags="--static" \

--extra-cflags="-I$HOME/ffmpeg_build/include" \

--extra-ldflags="-L$HOME/ffmpeg_build/lib" \

--extra-libs="-lpthread -lm" \

--bindir="$HOME/bin" \

--enable-gpl \

--enable-vaapi \

--enable-libx264 \

--enable-nonfree && \

PATH="$HOME/bin:$PATH" make && \

make install && \

hash -r

source ~/.profile

最后编译生成的静态库文件都在主目录ffmpeg_build里，用于我们的开发。ffmepg可执行程序在主目录bin下。运行ffmpeg程序，执行如下命令：

root@ubuntu:~/bin# ./ffmpeg -codecs | grep vaapi
ffmpeg version N-93240-g976dae8 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
configuration: --prefix=/root/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/root/bin --enable-gpl --enable-vaapi --enable-libx264 --enable-nonfree
libavutil      56. 26.100 / 56. 26.100
libavcodec     58. 47.102 / 58. 47.102
libavformat    58. 26.101 / 58. 26.101
libavdevice    58.  6.101 / 58.  6.101
libavfilter     7. 48.100 /  7. 48.100
libswscale      5.  4.100 /  5.  4.100
libswresample   3.  4.100 /  3.  4.100
libpostproc    55.  4.100 / 55.  4.100
DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_v4l2m2m h264_vaapi )
DEV.L. hevc                 H.265 / HEVC (High Efficiency Video Coding) (encoders: hevc_vaapi )
DEVIL. mjpeg                Motion JPEG (encoders: mjpeg mjpeg_vaapi )
DEV.L. mpeg2video           MPEG-2 video (decoders: mpeg2video mpegvideo mpeg2_v4l2m2m ) (encoders: mpeg2video mpeg2_vaapi )
DEV.L. vp8                  On2 VP8 (decoders: vp8 vp8_v4l2m2m ) (encoders: vp8_v4l2m2m vp8_vaapi )
DEV.L. vp9                  Google VP9 (decoders: vp9 vp9_v4l2m2m ) (encoders: vp9_vaapi )
root@ubuntu:~/bin#

root@ubuntu:~/bin# ./ffmpeg -codecs | grep vaapi

built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)

configuration: --prefix=/root/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/root/bin --enable-gpl --enable-vaapi --enable-libx264 --enable-nonfree

libavutil 56. 26.100 / 56. 26.100

libavcodec 58. 47.102 / 58. 47.102

libavformat 58. 26.101 / 58. 26.101

libavdevice 58. 6.101 / 58. 6.101

libavfilter 7. 48.100 / 7. 48.100

libswscale 5. 4.100 / 5. 4.100

libswresample 3. 4.100 / 3. 4.100

libpostproc 55. 4.100 / 55. 4.100

DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_v4l2m2m h264_vaapi )

DEV.L. hevc H.265 / HEVC (High Efficiency Video Coding) (encoders: hevc_vaapi )

DEVIL. mjpeg Motion JPEG (encoders: mjpeg mjpeg_vaapi )

DEV.L. mpeg2video MPEG-2 video (decoders: mpeg2video mpegvideo mpeg2_v4l2m2m ) (encoders: mpeg2video mpeg2_vaapi )

DEV.L. vp8 On2 VP8 (decoders: vp8 vp8_v4l2m2m ) (encoders: vp8_v4l2m2m vp8_vaapi )

DEV.L. vp9 Google VP9 (decoders: vp9 vp9_v4l2m2m ) (encoders: vp9_vaapi )

root@ubuntu:~/bin#

可以看到我们编译的ffmppeg已经支持VA API了。

VA API编解码开发

ffmepg官方example有VA API使用教程,具体可以看官方示例代码。不过需要注意的是VA API编码时，输入的YUV格式必须是NV12，其他格式YUV得转为NV12格式。官方example：vaapi_encode.c有个AVFrame(sw_frame) ，用于存放我们输入的YUV数据，该AVFrame的data[0]用于存放Y数据，data[1]存放UV数据，由于输入格式是NV12，所以data[1]中UV数据的内存布局为：UVUVUVUV···UVUV。