0
As some may know, I've been uploading more than the usual Hello! Morning recently. For a while, I've been wanting to upload more than just Hello! Morning each week, but since I'm on a 768 kbps upload cable connection, I had to do something to get the file sizes down. All of my source files are large MPEG-2 files. I won't say where I get them from, but it's a fairly well known source. Anyway, to get the file sizes down, I decided to play around with video compression.
I've been playing with video compression and processing for years now. I started way back during the DivX 3 days. I still remember the days of MPEG2AVI, FlaskMPEG, and Nandub. Things were much harder and lower quality back then. Nowadays, getting high quality encodes in a low file size is much easier. You just have to find the right format and the right trade-off between quality and file size.
I tried a few different video compression formats: MPEG-4 AVC (x264), VC-1 (WMV9 Advanced Profile), and MPEG-4 ASP (XviD).
MPEG-4 AVC is the "hot" video compression format right now. I delivers amazing quality at lower bitrates than ever before. Unfortunately, the free AVC tools that are currently available don't do what I need them to do well enough. My main two requirements are interlacing support and non-square pixel support on both encoding and playback. x264 does support both interlacing and non-square pixels, and libavcodec-based software applications (ffdshow, VLC, mplayer) also support both features. So, what's the problem? Well, there are a couple... First, AVC takes quite a bit of CPU to decode. I know the playback CPU usage can be drastically reduced by turning off some AVC features like CABAC, but then I might as well be using VC-1. Second, AVC interlacing support is iffy both on encoding and decoding. x264 doesn't let you set the field priority, and libavcodec's interlaced AVC support is buggy. The result... video that looks like a jerky mess. Needless to say, AVC was out of the picture.
VC-1 is basically Microsoft's watered down version of AVC. Almost nobody in the file-sharing world uses it because they all know they can get better quality from AVC. I agree that's technically true, but based on the free tools available right now, AVC's quality advantage only holds true for progressive video. Microsoft's VC-1 encoder has excellent interlacing support. In fact, I think interlaced video was a main focus of the new VC-1 codecs. So, why didn't I choose to use VC-1? Well, support on the decoding end is limited. In Windows Vista, there are no problems. The VC-1 decoder in Windows XP, though, doesn't support hardware deinterlacing on the video card, so all deinterlacing is handled by the decoder's crappy half-rate deinterlacer. Of course, libavcodec can also decode VC-1, but libavcodec's interlaced VC-1 support is almost as bad as its interlaced AVC support. Every interlaced VC-1 file I've thrown at mplayer and VLC results in very jerky video.
MPEG-4 ASP... The most common video compression format used in the file sharing world... This is the format I ended up using. XviD's interlacing support and non-square pixel support are good, and pretty much every MPEG-4 ASP software decoder supports both features. Sure, it doesn't compress as well as AVC and VC-1, but almost everybody can play it with little to no problems. So, in the end, I chose compatibility over bleeding-edge compression technologies. I could probably get better quality out of VC-1 and AVC, but I think ASP's quality is good enough and playback support for ASP is much much better.
Well, now that I've chosen a format, I have to find good settings that have good quality at a low file size. I see so many videos encoded at constant quantizer 4 and lower. Sure, it results in a cleaner picture, but it inflates the file size quite a bit. I may be in the minority here, but I find a small amount of blockiness easy to ignore. So, I'm not afraid to use higher quantizers to compress the video more. Want to see a good example? Look at the Quiz! Hexagon II episode I uploaded. 48 minutes of 720x480 interlaced video in under 500 MB, and the quality is quite good. How did I do it? Well let me show you...
First, I take the source MPEG-2 file and run it through DGIndex. Then, I create an AVISynth script that looks like this:
Yup, that's it. No resizing and no complex noise filters. Just simple MPEG-2 deblocking.
Then, I feed the script to AVS2AVI and tell it to use these XviD settings:
Unrestricted Profile
H.263 Quantization
Interlaced Encoding (Top Field First if source is TFF)
Default B-VOP settings
16:9 NTSC or 4:3 NTSC Pixel Aspect Ratio (depending on source video's aspect ratio)
Single pass encoding
Constant Quantizer: 8
Chroma Optimizer
Motion Search Precision: 6
VHQ mode: 4
Use VHQ for bframes
Use chroma motion
Max I-frame interval: 300
Trellis quantization
I listed a ton of settings, but a lot of them are actually the defaults.
As you can see, there's no special trick I have. I'm just not afraid to use quantizer 8 to compress videos. It lets me retain the interlacing and original resolution without inflating the file size or reducing the picture quality too much. The result is a video with quality that, in my opinion, is actually much better than the constant quant 4 progressive encodes. In my opinion, the video quality improvement brought by interlaced video far outweighs the small amount of increased blockiness.
So, why the long post about video compression? Well, it all kind of points back to the MPEG-2 Hello! Mornings. First, let me say that there are basically two reasons why I upload the MPEG-2 Hello! Mornings: I don't have to do any extra work on them, and it's easy for other people to put them on DVD.
That said, ever since I've been uploading them, there have always been people who have said that they're too big, and the quality isn't worth the space. I'll admit that I agree. MPEG-2 is an outdated video compression format that pales in comparison to MPEG-4 and VC-1, and a 2 GB file size is unreasonable to many people. BUT, the same way that some think that a 2 GB MPEG-2 Hello! Morning is unreasonable, I think 700 MB and 1 GB MPEG-4 Hello! Mornings are also unreasonable. I think that the qualities provided by the 720x404 and 1024x576 MPEG-4 Hello! Mornings are not worth their file sizes. A 700 MB, 45 minute MPEG-4 video should have interlacing, but it doesn't. Instead, you get a 30 progressive frames per second video that could have been easily compressed to half of that size with a minimal loss in quality. Also, a small increase in spatial resolution doesn't warrant a 300 MB increase in file size. Basically, I think that all of the Hello! Mornings, including the MPEG-2 version, are oversized.
Of course, all of us weekly Hello! Morning uploaders just upload files we get from other sources, so we don't really have any control over it anyway. I could re-compress the MPEG-2 Hello! Mornings like I do for other shows, but I don't mind uploading one huge file a week, so I'll continue to upload the MPEG-2 version.
I've been playing with video compression and processing for years now. I started way back during the DivX 3 days. I still remember the days of MPEG2AVI, FlaskMPEG, and Nandub. Things were much harder and lower quality back then. Nowadays, getting high quality encodes in a low file size is much easier. You just have to find the right format and the right trade-off between quality and file size.
I tried a few different video compression formats: MPEG-4 AVC (x264), VC-1 (WMV9 Advanced Profile), and MPEG-4 ASP (XviD).
MPEG-4 AVC is the "hot" video compression format right now. I delivers amazing quality at lower bitrates than ever before. Unfortunately, the free AVC tools that are currently available don't do what I need them to do well enough. My main two requirements are interlacing support and non-square pixel support on both encoding and playback. x264 does support both interlacing and non-square pixels, and libavcodec-based software applications (ffdshow, VLC, mplayer) also support both features. So, what's the problem? Well, there are a couple... First, AVC takes quite a bit of CPU to decode. I know the playback CPU usage can be drastically reduced by turning off some AVC features like CABAC, but then I might as well be using VC-1. Second, AVC interlacing support is iffy both on encoding and decoding. x264 doesn't let you set the field priority, and libavcodec's interlaced AVC support is buggy. The result... video that looks like a jerky mess. Needless to say, AVC was out of the picture.
VC-1 is basically Microsoft's watered down version of AVC. Almost nobody in the file-sharing world uses it because they all know they can get better quality from AVC. I agree that's technically true, but based on the free tools available right now, AVC's quality advantage only holds true for progressive video. Microsoft's VC-1 encoder has excellent interlacing support. In fact, I think interlaced video was a main focus of the new VC-1 codecs. So, why didn't I choose to use VC-1? Well, support on the decoding end is limited. In Windows Vista, there are no problems. The VC-1 decoder in Windows XP, though, doesn't support hardware deinterlacing on the video card, so all deinterlacing is handled by the decoder's crappy half-rate deinterlacer. Of course, libavcodec can also decode VC-1, but libavcodec's interlaced VC-1 support is almost as bad as its interlaced AVC support. Every interlaced VC-1 file I've thrown at mplayer and VLC results in very jerky video.
MPEG-4 ASP... The most common video compression format used in the file sharing world... This is the format I ended up using. XviD's interlacing support and non-square pixel support are good, and pretty much every MPEG-4 ASP software decoder supports both features. Sure, it doesn't compress as well as AVC and VC-1, but almost everybody can play it with little to no problems. So, in the end, I chose compatibility over bleeding-edge compression technologies. I could probably get better quality out of VC-1 and AVC, but I think ASP's quality is good enough and playback support for ASP is much much better.
Well, now that I've chosen a format, I have to find good settings that have good quality at a low file size. I see so many videos encoded at constant quantizer 4 and lower. Sure, it results in a cleaner picture, but it inflates the file size quite a bit. I may be in the minority here, but I find a small amount of blockiness easy to ignore. So, I'm not afraid to use higher quantizers to compress the video more. Want to see a good example? Look at the Quiz! Hexagon II episode I uploaded. 48 minutes of 720x480 interlaced video in under 500 MB, and the quality is quite good. How did I do it? Well let me show you...
First, I take the source MPEG-2 file and run it through DGIndex. Then, I create an AVISynth script that looks like this:
MPEG2Source("whatever.d2v",cpu=4)Yup, that's it. No resizing and no complex noise filters. Just simple MPEG-2 deblocking.
Then, I feed the script to AVS2AVI and tell it to use these XviD settings:
Unrestricted Profile
H.263 Quantization
Interlaced Encoding (Top Field First if source is TFF)
Default B-VOP settings
16:9 NTSC or 4:3 NTSC Pixel Aspect Ratio (depending on source video's aspect ratio)
Single pass encoding
Constant Quantizer: 8
Chroma Optimizer
Motion Search Precision: 6
VHQ mode: 4
Use VHQ for bframes
Use chroma motion
Max I-frame interval: 300
Trellis quantization
I listed a ton of settings, but a lot of them are actually the defaults.
As you can see, there's no special trick I have. I'm just not afraid to use quantizer 8 to compress videos. It lets me retain the interlacing and original resolution without inflating the file size or reducing the picture quality too much. The result is a video with quality that, in my opinion, is actually much better than the constant quant 4 progressive encodes. In my opinion, the video quality improvement brought by interlaced video far outweighs the small amount of increased blockiness.
So, why the long post about video compression? Well, it all kind of points back to the MPEG-2 Hello! Mornings. First, let me say that there are basically two reasons why I upload the MPEG-2 Hello! Mornings: I don't have to do any extra work on them, and it's easy for other people to put them on DVD.
That said, ever since I've been uploading them, there have always been people who have said that they're too big, and the quality isn't worth the space. I'll admit that I agree. MPEG-2 is an outdated video compression format that pales in comparison to MPEG-4 and VC-1, and a 2 GB file size is unreasonable to many people. BUT, the same way that some think that a 2 GB MPEG-2 Hello! Morning is unreasonable, I think 700 MB and 1 GB MPEG-4 Hello! Mornings are also unreasonable. I think that the qualities provided by the 720x404 and 1024x576 MPEG-4 Hello! Mornings are not worth their file sizes. A 700 MB, 45 minute MPEG-4 video should have interlacing, but it doesn't. Instead, you get a 30 progressive frames per second video that could have been easily compressed to half of that size with a minimal loss in quality. Also, a small increase in spatial resolution doesn't warrant a 300 MB increase in file size. Basically, I think that all of the Hello! Mornings, including the MPEG-2 version, are oversized.
Of course, all of us weekly Hello! Morning uploaders just upload files we get from other sources, so we don't really have any control over it anyway. I could re-compress the MPEG-2 Hello! Mornings like I do for other shows, but I don't mind uploading one huge file a week, so I'll continue to upload the MPEG-2 version.
