On October 19, the US Patent & Trademark Office published Apple’s patent application titled “Selective reencoding for GOP conformity,” originally filed in April 2005. One aspect of the patent relates to a video editor which can be a nonlinear video editing application such as Final Cut Pro.
For the record: While Apple isn’t officially listed as the assignee on this patent, well known Apple inventors such as Gregory Kent Wallace and Jian Lu are listed herein and are likewise listed in one of Apple’s other patent applications today, titled “Single pass constrained constant bit-rate encoding.”
Apple’s Summary
This disclosure generally describes systems and methods for editing and selectively encoding video. A proposed system can perform video editing in a native format without time-consuming and processor-intensive encoding of video sources into an intermediate editing format. The proposed system can also output video sequences conforming to, for example, the MPEG (Motion Picture Experts Group) standard or other video format having interframe dependencies (e.g., GOPs, or Group Of Pictures, such as long-GOPs) and/or buffer constraints, and the like, without reencoding a greater portion of the video segment timeline than necessary.
In general, in one aspect, a method is provided. The method comprises generating video segments from portions of one or more video sources, at least one video source being encoded to have interframe dependencies; generating a video sequence from the video segments; scanning the generated video sequence to identify invalid interframe dependencies; and selectively reencoding portions of the generated video sequence to generate an output video sequence having valid interframe dependencies.
Particular implementations can include one or more the following features. The method can further comprise generating the output video sequence without reencoding valid portions of the generated video sequence. Valid portions of the generated video sequence can comprise portions lacking invalid interframe dependencies. The at least one video source can comprise an MPEG video clip encoded to have at least one of a B-frame or a P-frame that is dependent on a source frame within the MPEG video clip, and a portion of the MPEG video clip included in the video sequence can be identified as invalid responsive to the source frame not being included in the generated video sequence. The video segment of the MPEG video clip can be reencoded to begin with a closed GOP. The method can further comprise identifying a portion of the generated video sequence as invalid if an interframe dependency relies on a video segment of the at least one video source that is not included in the generated video sequence. The portion of the invalid video sequence can begin or end mid-GOP or with an open GOP. Scanning can comprise scanning a video segment to identify invalid interframe dependencies within the video segment; and rescanning one or more adjoining video segments to identify invalid interframe dependencies outside of the video segment resulting from a reencoding of one or more of the video segments.
The method can further comprise storing a CRC representation of an encoded frame of an adjacent video segment that is conforming to a video segment that has a dependent frame; and verifying that dependency between frames remains valid by comparing the stored CRC with a current CRC of the encoded frame. The method can further comprise identifying a video segment of the generated video sequence as valid if an interframe dependency relies on a video segment of the at least one video source that is included in the generated video sequence. The video segment of the valid video sequence can comprise a closed-GOP. The method can further comprise scanning the video sequence to check the validity of buffer levels associated with decoding the generated video sequence. Scanning the video sequence to check the validity of buffer levels can comprise comparing buffer levels associated with adjoining frames on either side of a video segment boundary for continuity. Scanning the video sequence to check the validity of buffer levels can comprise ensuring that a buffer level remains above an underflow level and below an overflow level.
A video segment can comprise a subset of frames from a video source including a set of frames. Generating the video segments can change at least one of a duration of a video source or contents of the video source. The method can further comprise rendering the output video sequence.
In general, in another aspect, a method is provided. The method comprises generating a video sequence from portions of one or more video sources, at least one video source being encoded to have frames with varying data rates; receiving a buffer size; scanning the video sequence to check validity of buffer levels by comparing buffer levels associated with adjoining frames for continuity; and selectively reencoding video segments of the generated video sequence to generate an output video sequence having continuous buffer levels.
In general, in another aspect, a method is provided. The method comprises receiving one or more video sources, at least one video source being encoded to have interframe dependencies with respect to the at least one video source; generating a video sequence from video segments of one or more video sources, scanning the generated video sequence to check validity of interframe dependencies and buffer levels with respect to the generated video sequence; marking boundaries of the generated video sequence that have invalid interframe dependencies or buffer levels; selectively reencoding video segments of the video sequence to generate an output video having valid interframe dependencies with respect to the generated video sequence and having valid buffer levels.
In general, in another aspect, a computer program product, tangibly stored on a computer-readable medium, is provided. The product can include instructions operable to cause a computer system to perform a method, comprising generating video segments from portions of one or more video sources, at least one video source being encoded to have interframe dependencies; generating a video sequence from the video segments; scanning the generated video sequence to identify invalid interframe dependencies; and selectively reencoding portions of the generated video sequence to generate an output video sequence having valid interframe dependencies.
In general, in another aspect, a system is provided. The system comprises an editing timeline to facilitate generation of video segments from portions of one or more video sources, at least one video source being encoded to have interframe dependencies, the editing timeline generating a video sequence from the video segments; a selection block, in communication with the editing timeline, the selection block to identify invalid interframe dependencies; and an encoder, in communication with the selection block, the encoder to selectively reencode portions of the video sequence to generate an output video sequence having valid interframe dependencies.
Particular implementations can include one or more of the following features. The encoder can generate the output video sequence without reencoding valid portions of the generated video sequence. Valid portions of the generated video sequence can comprise portions lacking invalid interframe dependencies. The at least one video source can comprise an MPEG video clip encoded to have at least one of a B-frame or a P-frame that is dependent on a source frame within the MPEG video clip, and a portion of the MPEG video clip included in the video sequence is identified as invalid responsive to the source frame not being included in the generated video sequence. The video segment of the MPEG video clip is reencoded to begin with a closed GOP. The selection block can further comprise a GOP scanner to identify a portion of the generated video sequence as invalid if an interframe dependency relies on a video segment of the at least one video source that is not included in the generated video sequence. The portion of the invalid video sequence can begin or end mid-GOP or with an open GOP. The selection block can further comprise a GOP scanner to scan a video segment to identify invalid interframe dependencies within the video segment, and the GOP scanner can rescan one or more adjoining video segments to identify invalid interframe dependencies outside of the video segment resulting from a reencoding of one or more of the video segments.
The selection block can store a CRC representation of an encoded frame of an adjacent video segment that is conforming to a video segment that has a dependent frame, and the selection block can verify that dependency between frames remains valid by comparing the stored CRC with a current CRC of the encoded frame. The selection block can identify a video segment of the generated video sequence as valid if an interframe dependency relies on a video segment of the at least one video source that is included in the generated video sequence. The video segment of the valid video sequence can comprise a closed-GOP.
The selection block can comprise a virtual buffer to scan the video sequence to check the validity of buffer levels associated with decoding the generated video sequence. The virtual buffer can compare buffer levels associated with adjoining frames on either side of a video segment boundary for continuity. The virtual buffer can ensure that a buffer level remains above an underflow level and below an overflow level.
A video segment can include a subset of frames from a video source including a set of frames. The editing timeline can change at least one of a duration of a video source or contents of the video source. The system can further comprise a rendering engine, in communication with the selection block, the rendering engine to render the output video sequence.
Selectively Reencoding a Video Sequence
Patent FIG. 1 is a block diagram illustrating a system 100 to selectively reencode a video sequence. The system 100 includes input video sources 110, a video editor 120, and an output video sequence 130. The video editor 120 further includes an editing timeline 122, a selection block 124, and an encoder 126.
Patent FIG. 3 is a block diagram illustrating one implementation of the selection block 124 within the system 100 of FIG. 1. The selection block 124 includes a GOP scanner 310 and a virtual buffer 320.
The GOP scanner 310 identifies violations of interframe dependencies in an edited video sequence. For example, a video segment that begins mid-GOP and has a B-frame (a P-frame) that depends on another frame that is not part of the video segment violates the interframe dependency of the B-frame. The GOP scanner 310 examines frames in each edit, including the preceding video segment and one or more subsequent video segments following the edit. The GOP scanner 310 selects and marks portions of the edits that need reencoding to remove the interframe dependency.
In one implementation, the virtual buffer 320 identifies violations of buffer levels. The virtual buffer 320 can simulate conditions of a decode buffer by tracking a state, or current buffer level, and changes to the buffer level that occur while decoding the video sequence. By doing so, the virtual buffer 320 prevents an overflow condition where the data rate is too low or an underflow condition where the data rate is too high. A decode standard such as MPEG can provide guidelines or restrictions for setting a standard buffer size and data rate for tracking by the virtual buffer 320. In some implementations, the virtual buffer 229 can be a constrained constant bit rate buffer as described in U.S. Pat. No. [Attorney Docket No. P3837US1].
NOTICE: MacNN presents only a brief summary of patents and/or trademarks with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent and/or trademark applications and/or grants should be read in its entirety for further details.
Written and researched by Neo.
