Method for optimizing two-pass coding

This application is based upon and claims priority to Chinese Patent Application No. 201911292705.4, filed on Dec. 16, 2019, the entire contents of which are incorporated herein by reference.

The present invention relates to a field of coding technologies, and in particular to a method for optimizing two-pass coding.

With the rapid development of electronic information technologies and the use of various video data collection methods, digital video has become a main carrier of multimedia information. However, the amount of uncompressed digital video data is tremendous, for example, for an 8-bit RGB color video with a resolution of 1920×1080 and a frame rate of 30 Hz, the amount of data per hour is as high as 4.89 TB. Such a large amount of data brings great challenges to the transmission and storage of videos. Therefore, since the 1980s, video compression technologies have continued to become a hot field of research and application at home and abroad. With the development of digital video codec technologies, the application of digital videos covers various fields such as TV broadcasting, digital movies, remote education, remote medicine, video surveillance, video conversation and streaming media transmission, and many well-known video application companies have emerged. In order to ensure the interoperability among codec products from different suppliers, corresponding video coding standards were born.

Due to the existence of various redundant data in digital videos, it is impossible for any single coding tool to achieve efficient video compression alone. The first-generation video coding standard H.261 released in the 1980s uses a hybrid video coding framework that contains a plurality of compression tools used for such as prediction, transformation, quantization, and entropy coding, which can effectively remove temporal, spatial, visual, information entropy and other redundancy to achieve efficient video data compression. Therefore, the hybrid video coding structure has been used to this day by every generation of subsequent video coding standard. The Joint Collaborative Team on Video Coding (JCT-VC) jointly established by the Video Coding Experts Group (VCEG) of the International Telecommunication Union-Telecommunication standardization sector (ITU-T) and the Moving Picture Experts Group (MPEG) of the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) has released the first version of High Efficiency Video Coding (HEVC) standard in 2013, the compression performance of which has doubled compared with the previous generation standard H.264/AVC. At present, the market share of HEVC is increasing year by year, and some coding optimization methods for HEVC coders have appeared in the past few years.

In video coding, a quantization parameter (QP) and a Lagrange multiplier λ determine the distortion of the coded video and the number of bits required for coding. There is a close relationship between them. The HEVC coder uses a hierarchical coding structure, and the quantization parameter of the coded frame is determined by its position in the Group of Pictures (GOP) and an input quantization parameter QP 0 of the coder: QP IIM =QP 0 +ΔQP. In an HEVC low-delay coding configuration, the GOP size is 4, and values of corresponding ΔQP are respectively 3, 2, 3 and 1. Then, the Lagrange multiplier used in the coding is calculated based on QP HM :

λ P = W L · 2 ( ( QP HM - 12 ) / 3.0 ) ,

wherein W L is a weighting factor related to the level to which the coded frame belongs, and its value takes into account the importance of the rate distortion of coded frames of different levels to an extent. Furthermore, in HEVC's R-λ code rate control algorithm, the Lagrange multiplier of a frame or a Coding Tree Unit (CTU) to be coded is first determined; and then the quantization parameter is calculated based on λ: QP=4.2005×ln(λ)+13.7122.

Under the hybrid video coding framework, predictive coding technology has made a great contribution to efficient video compression of a coder. Intra-frame prediction and inter-frame prediction effectively remove spatial and temporal redundancy in the video data. However, predictive coding also causes generation of great rate distortion dependence between coded frames and between basic coding units, that is, a current coding decision will affect the maximum rate distortion performance that can be achieved in subsequent coding process. Self-adaptive bit resource allocation based on effective utilization of the above rate distortion dependence can further improve compression performance of the coder, and the bit resource allocation can be achieved by adjusting the quantization parameter and the Lagrange multiplier in the coding process.

The present invention uses rate distortion dependence relationship to optimize allocation of coded bit resources and provides a method for optimizing two-pass coding.

The technical solution of the present invention is as follows:

A method for optimizing two-pass coding comprising the following steps:

S 1 , setting a start frame of a video sequence as a frame I with a level set to Level 0, which is used as a separate GOP; optimizing the start frame, namely the frame I, specifically setting a quantization parameter QP I of the frame I according to a range of an input quantization parameter QP 0 of a coder:

S 2 , reading in a frame to be coded of the GOP by the coder;

S 3 , first coding: performing first coding on a current frame by using a default quantization parameter QP HM,i set by an HEVC, and obtaining a frame-level temporal impact factor k i of the current frame and a block-level temporal impact factor k B,j of all the 16×16 pixel blocks in the frame:

wherein D i and D i MCP are respectively a coding distortion and a motion compensation prediction error of the current frame; and D B,j and D B,j MCP are respectively a coding distortion and a motion compensation prediction error of a jth 16×16 pixel block in the current frame;

S 4 , restoring reference list information of the coder after the first coding is completed, that is, the first coding does not output a code stream of the current frame or store a reconstructed image; after the coding, resetting a linked image list in the coder to a state before the current frame is coded, including restoring a reference frame identifier in the linked image list;

S 5 , determining whether a scene is switched, if yes, then going to step S 6 , if no, then going to step S 7 ; a determination method is: if p i >4 p and p i >10, it is determined that scene switching occurs in the current frame, wherein p i is an average motion compensation prediction absolute error of the current frame, and p is an a mean value of the average motion compensation prediction absolute error of previous 6 frames;

S 6 , setting a quantization parameter of the current frame as QP i =QP 0 , then performing coding, and outputting the code stream and storing the reconstructed image in a default way of the coder, going to step S 8 ;

S 7 , setting the quantization parameter of the current frame as

QP i = { Q ⁢ P HM , i + Round ⁢ ⁢ ( 10 × k i - 5 ) ⁢ , k i ⁢ > 0.75 QP HM , i , ⁢ else ,

wherein QP HM,i is the quantization parameter of the current frame set in an original coder, and Round(⋅) is a rounding operator; if the current frame is a key frame, coding quality loss caused by increasing QP i will be multiplied due to direct distortion propagation of the key frame to a plurality of subsequent frames, so that the number of bits saved is not sufficient to offset the total loss of coding quality, which may eventually result in a decrease in coding performance, and therefore the current step does not adjust the quantization parameter of the key frame;

after a frame-level Lagrangian multiplier λ p is calculated through QP i , obtaining a Lagrangian multiplier λ n and a quantization parameter QP n for coding each CTU in the current frame by the following formulas:

wherein M is the number of the 16×16 pixel blocks included in an nth CTU, N is the number of the CTUs in a coded frame, w n , w and W n are intermediate variables;

coding each CTU in the current frame by using the calculated Lagrangian multiplier λ n and the quantization parameter QP n , and outputting the code stream and storing the reconstructed image in a default way of the coder;

S 8 , determining whether the coded frame is the last frame of the video sequence, if yes, then ending the coding; otherwise, continuing to determine whether the coded frame is the last frame in the current GOP, if yes, then going back to step S 2 , otherwise, going back to step S 3 .

The differences between the solution of the present invention and the traditional method include: a frame-level temporal influence factor and a block-level temporal influence factor k B,j are proposed to measure rate distortion dependence. A start frame I of a video sequence is optimized. The coding quality of the frame I determines the maximum rate distortion performance that can be reached by subsequent P/B frame coding, and proper improvement of the quality of the frame I can bring a certain improvement of the rate distortion performance for the entire coded video. Since the rate distortion dependence is weak at high code rates and is strong at low code rates, a quantization parameter QP I of the frame I is provided according to the range of an input quantization parameter QP 0 of a coder in the present invention. The frame-level temporal influence factor k i and the block-level temporal influence factor k B,j are obtained in a simplified coding process. In order to reduce computational complexity, an RDO mode selection process of the first coding skips most of the mode determination steps, and only uses 64×64, 32×32 and 16×16 inter-frame prediction modes. Different optimization strategies are used for coding frames depending on whether a scene is switched.

The beneficial effects of the present invention are:

Rate distortion dependence intensity between the current frame and a reference frame, and importance of each coding block in the current frame for the rate distortion in a subsequent coding process are obtained by means of the simplified first coding, and are then used to guide optimization of a frame-level and a CTU-level coding resources in the two-pass coding. In HEVC coder HM-16.7, in a case where coding complexity is increased by an average of 23%, average code rate savings of 5.1% and 5.3% are respectively achieved in two coding configurations of a low-delay B frame (LDB) and a low-delay P frame (LDP) according to the present invention.

FIG. 1 is a main flowchart according to the present invention;

FIG. 2 is a schematic diagram of an example of reference relationship in HEVC low-delay coding;

FIG. 3 is a statistical diagram of a frame-level temporal impact factor k i according to the present invention;

FIG. 4 is a schematic diagram of an example of a block-level temporal impact factor k B,j that can be visualized in video;

FIG. 5 is a schematic diagram of comparison of rate distortion curves; and

FIG. 6 is a schematic diagram showing coding time increase and code rate savings of the present invention relative to an HEVC coder.

The present invention will be further described below with reference to the drawings and simulation embodiments, and the effectiveness of the present invention will be pointed out.

Embodiment

This embodiment uses a development environment of Visual Studio 2013 and is implemented based on HEVC reference software HM-16.7.

FIG. 1 is a flowchart of main steps of method for optimizing two-pass coding, specifically including:

S 1 : optimizing a start frame of a video sequence. Setting a quantization parameter QP I of a frame I according to a range of an input quantization parameter QP 0 of a coder:

S 2 : reading in a frame to be coded of a GOP in a default way of HM-16.7.

S 3 : performing first coding in a simplified coding process. Performing first coding on a current frame by using a default quantization parameter set by the HEVC and a Lagrange multiplier calculation method, so as to obtain information such as frame-level and 16×16 pixel block-level motion compensation prediction error and coding distortion, and then calculating a frame-level temporal impact factor k i of the current frame and a block-level temporal impact factor k B,j of all the 16×16 pixel blocks in the frame according to the formula.

S 4 : restoring information such as a reference list. The first coding does not output a code stream of the current frame or store a reconstructed image; after the coding, resetting a linked image list in the coder to a state before the current frame is coded, including restoring a reference frame identifier in the linked image list.

S 5 : determining whether a scene is switched, if yes, then going to step S 6 , if no, then going to step S 7 . A determination method for scene switching is: if p i >4 p and p i >10, it is determined that scene switching occurs in the ith frame, wherein p i is an average motion compensation prediction absolute error of the current frame, and p is an a mean value of the average motion compensation prediction absolute error of previous 6 frames.

S 6 : setting a quantization parameter of the current frame as QP i =QP 0 , then performing coding, and outputting the code stream and storing the reconstructed image in a default way of the coder.

S 7 : setting the quantization parameter of the current frame as

wherein QP HM,i is the quantization parameter of the current frame set in the original HEVC coder HM, and Round(⋅) is a rounding operator. It should be noted that the above quantization parameter setting only operates on the coded frames of level 2 and 3, and the quantization parameter of the key frame keeps the setting in the original HEVC coder.

After a frame-level Lagrangian multiplier λ p is calculated through QP i , obtaining a Lagrangian multiplier λ n and a quantization parameter QP n for coding each CTU in the current frame by the following formulas:

wherein M is the number of the 16×16 pixel blocks included in an nth CTU, N is the number of the CTUs in a coded frame. Eventually, for a case where scene switching does not occur, coding each CTU in the current frame by using the above calculated Lagrangian multiplier λ n and quantization parameter QP n , and outputting the code stream and storing the reconstructed image in a default way of the coder.

S 8 , determining whether the coded frame is the last frame of the video sequence. If “yes”, then ending the coding; if “no”, continuing to determine whether the coded frame is the last frame in the current GOP. If “yes”, skipping to step S 2 to read the next GOP data; if “no”, skipping to step S 3 to code the next frame in the current GOP.

The bit stream generated by coding according to the present invention conforms to the syntax format of the HEVC standard, and all generated bit streams can be decoded by a standard HEVC decoder. Coding experiments were conducted according to HEVC general test conditions, and two coder configurations, LDB and LDP, were tested, the reference relationship of which is shown in FIG. 2 . It can be seen from FIGS. 3 and 4 that the frame-level temporal impact factor k i and the block-level temporal impact factor k B,j proposed by the present invention effectively represent the rate distortion dependence.

FIG. 5 shows a comparison of rate distortion curves of a test sequence PartyScene in coding configurations of LDB and LDP. It can be seen that the rate distortion performance at both low and high code rates according to the present invention is superior to the original HEVC coder HM-16.7.

FIG. 6 is a schematic diagram showing coding time increase and code rate savings of the present invention relative to an original HEVC coder HM-16.7. It can be seen that in a case where coding complexity is increased by an average of 23%, average code rate savings of 5.1% and 5.3% are respectively obtained in coding configurations of LDB and LDP according to the present invention, showing remarkable rate distortion performance improvement.