There is an obscure matter in regard to digital projection and DCP creation, that -one way or another- I came across quite a few times. That being, either when needed to create a DCP for screenings, or when needed to store, send, receive, upload, download, archive such a package. Improbable as it might seem, there are past movie titles of which the best quality surviving material is their digital cinema package. Best overall, or best for cinema screening purposes.
The obscurity of it, is not because the matter wasn't brought up in numerous occasions, it's because not much of concrete, well backed info is provided by reliable sources. We have some established facts, that apply for 2K, 24fps DCPs, regardless of aspect ratio, but keeping in mind that between the two standards (Flat, 1,85:1 and Scope 2,39:1) that are used as containers for the horde of the aspect ratios around, Flat would be the more demanding of the two. So, within those boundaries, we have the following two facts:
- The original bandwidth limit of the media blocks, the one we still apply for anything else than high frame rate, is 250 megabits per second. That was planned to be the maximum to work with 3D movies of the 2K kind, that would be twice the bandwidth of a two-dimension 24 fps, since there is need for two times the images for our eyes to conceive stereoscopy. Also, the 250 Mbits/sec limit is not safe for all media blocks, some were reported to not handle that much, and many avoid that and prefer to go for ~235 Mbits/sec as peak. It's worth mentioning here that after the second generation of DCI approved equipment, media blocks came that support much more bandwidth. Namely, twice that. Yet, good practice forbids content creators to go above for non-HFR, 2D content, even if it is 4K, which is four times the number of the pixels of 2K.
- The JPEG2000 compression that is used for digital cinema purposes is the so called Visually Lossless, which means that even a trained eye wouldn't distinguish the compressed version of a frame. Though, that compression is not mathematically Lossless, where there is not any image information lost and the compressed version is the exact same with the original. The Visually Lossless compression ratio is referred as 10:1 to 20:1
I should direct the reader's attention to the fact that there is a small detail in the phrasing here, that springs questions to the mind. What does it mean, "the trained eye"? Well, it describes a person that knows what to look for and how, in order to recognize compression artifacts. A nice piece of info that -unfortunately- hasn't found its way to the inquisitive audience widely, was that a group of "trained eyes", when looking for degradation on DCPs connected to bandwidth, they didn't find any in the equivalent of 75 Mb/s for 2K 24fps. And -most interesting- they just didn't try to go lower, in order to establish where the turning point was.
And that brings us to a natural question, according to all the above: If that 75 Mb/s is a fair bit rate for a 2K 24fps video, and 125 Mb/s is more like the limit set, to what would those numbers translate, if we were to think 25fps, or 4K.
The answer, concerning 25fps is a rather simple one. Staying in the "fare" bitrate for the sake of being brief (you may do the math for 125Mb/s), we would say that, if for 24fps the bitrate is 75 Mb/s, for 25, it would be one and a 24th times more. That would be 75x25/24 = 78 and 1/8 Mb/s. Let's round it up to 80 Mb/s, making easy to remember.
Yet, what about 4K? There is no simple analogy or formula that will give us a fixed ratio between the JPEG2000 bandwidth of 2K and 4K.
Why isn't there such a fixed ratio?
Because we are talking about a compression, and a very efficient one as well. So, quadrupling the number of pixels does not by any reasonable means translate into increasing fourfold the data. As a matter of fact, it doesn't even mean increasing those twofold, as many would intuitively choose to do (based on the image having twice the width or/and twice the height).
Why doesn't quadrupling the pixels reasonably translates into quadrupling the data necessary for the equivalent image quality of a digital cinema package?
Because of the nature of the compression used. JPEG2000 is compressing the image by creating "layers" of the image, where the next layer is half the width and half the height of the previous (just like 4K and 2K). Additionally to that downscaled image, it holds the data difference that is necessary to recompose the "upper" layer. It holds the differences in each of horizontal, vertical and diagonal directions, creating a so called "decomposition level". And keeps that extra info necessary to recompose the original, compressed and (depending on the compression scheme) quantized. (Quantization is the process of reducing precision of a digital signal. In this case, an image.) Then, the first decomposition level is undergoing the same procedure, in order for the second decomposition level to be created, with half the width and half the height resolution of the first decomposition level. If you wonder how many decomposition levels are created on JPEG2000 compression for making a DCP frame, the answer is five for 2K and six for 4K images. In other words, in each DCP frame, there are six resolution layers, if the image is 2K, and seven, if the image is 4K. With each layer/resolution (but the first), having the so-called high frequency subbands (extra info) compressed (and quantized) accordingly.
Finally, we reached the question about which this text is all about:
So, what is the ratio between a 4K and a 2K DCP?
I won't bore you any more with my fruitless efforts to find that answer online or offline. I will just describe how I went about to figure that out by myself an what the findings were, of the specific search I made. At the end, I will explain why those results are indicative and may serve as a guideline where there wasn't any, but can not fit perfectly every usecase.
The file size ratio between two different DCPs of the same movie in the equivalent quality of 4K and 2K can be given by figuring what the difference of their image files are, if one would discard from the 4K DCP the so-called high frequency subbands. That procedure is common, discarding the higher resolution of a 4K DCP on every screening of such a DCP taking place on a 2K system. The cinema screen server and media block are not decoding the whole 4K image, they just decode the 2K and feed that to the projector, sparing the extra processing and bandwidth. A JPEG2000 characteristic that makes it so useful when one wants to focus on a part of an image or take advantage of its scalability.
Here are the numbers, followed by the method used and reasoning behind the choice of the DCP sample used:
Findings:
The 8.000 4K frames used out of the DCP of choice were exported as .j2c files.
Then, the higher resolution was discarded, without re-encoding the image, downscaling it to 2K.
----------------------------------------------------------------------------------
Original folder (4K):
6,76 GB (7.268.132.125 bytes)
Average frame size: 908.517 bytes
Largest frame size: 910.425 bytes (Frame0200.j2c which is 413.590 bytes when reduced to 2K by 54,572%)
Smallest frame size: 651.496 bytes (Frame4353.j2c which is 645.696 bytes when reduced to 2K by 0,89%)
Reduced JPEG2000 folder (2K):
4,90 GB (5.263.438.590 bytes)
Average frame size: 657.930 bytes
Largest frame size: 876.438 bytes (ReducedFrame1923.j2c which was 910.291 bytes in original 4K before reduced by 3,719%)
Smallest frame size: 407.213 bytes (ReducedFrame0198.j2c which was 910.332 bytes in original 4K before reduced by 55,268%)
2K is 72,418 % of 4K, put otherwise,
2K is 27,582 % smaller than 4K, or,
4K is 38,087 % bigger than 2K
maximum reduction: 55,268 % (Frame0198.j2c 910332 bytes ReducedFrame0198.j2c 407213 bytes)
minimum reduction: 0,775 % (Frame4354.j2c 652422 bytes ReducedFrame4354.j2c 647364 bytes)
average reduction: 27,547 % (between individual files, not in total - differences on filesystem, spreadsheet rounding, or my rounding to the one thousandth of the percentage may account for that total difference of 0,035%)
Resulting DCP video mxf files
4K of 8000 frames 24fps repackaged:
6,76 GB (7.268.397.129 bytes)
different from .j2c files by 265.004 bytes
Overall bit rate (media info): 174 Mb/s
Bits/(Pixel*Frame) (media info): 1.034
Reduced 2K of 8000 frames 24fps packaged:
4,90 GB (5.263.703.594 bytes)
different from .j2c files by 265.004 bytes
Bit rate (media info) : 126 Mb/s
Bits/(Pixel*Frame) (media info) : 2.995
----------------------------------------------------------------------------------
Where did those numbers came from?
a) An excerpt of eight thousand frames were extracted out of a test DCP that was almost three times that length, skipping the first ten thousand. The asdcplib library was used for that purpose.
b) Those frames were stripped of the higher resolution, using the transcode command of the kakadu software and the '-reduce' operation. (Thanks to their support, providing an evaluation license.) I initially tried a number of open source and/or freeware JPEG2000 codecs, the problem I came upon was that none of those I found could encode .j2c files out of .j2c files. Some implementations had the ability to reduce a resolution layer (or more), or, otherwise, retain a chosen number of resolution layers. Yet, only in order to transcode into another image format. Retaining the original file's compression was not possible with any other tested software but Kakadu.
c) Creating a .csv list of the files and file sizes for each version, 4K original and 2K. Importing into a spreadsheet document, in order to document and define maximum, minimum and average values easily.
d) (Re-)packaging the frames into a 24fps DCP MXF file and calculating bitrate of the resulting files. (That could have been done without the repackaging.)
Why not using the whole test DCP?
The DCP included a title and credits. Those were images of low complexity, that resulted in smaller files to start with. To begin with, I wanted to focus on what is the concern of most people in terms of bitrate allocation, higher complexity images that result in bigger image file sizes. Afterwards, I followed the same procedure for the entirety of the file and I want to share those findings in another post. When I do, I will share a link in the comments. The smallest 4K frame file size was 861 bytes and the largest was just 35 bytes bigger.
I do intent to examine other cases and check the results. But I don't expect significant deviations in the results, given that the original DCPs are live action videos.
Why started with this DCP?
This DCP is a short film created by the American Society of Cinematographers Motion Imaging Technology Council designed to be used as both reference material and stress testing for color pipelines, image processing, monitors and projection systems.
I found it would be a generally good start, if I was to take advantage of a video that has a variety of characteristics that may appear in different cases.
On top of that, on the site that is the source of this (48 nits, EOTF 2.6, P3) DCP, it states that it is permitted to use it for educational purposes (I copy the entire notice at the end of this text).
In short and without too much detail, what may someone keep from this post and test?
That between a 2K and a 4K version of the same video encoding, the difference in file size (and therefore bandwidth) is:
In average of the video excerpt:
4K is 38% larger than the 2K version
The least and most of file size difference between 4K and 2K frames:
4K is 1,008 up to 2,236 times the 2K version
I will leave whomever had the patience to read this text with the given numbers, I will promise to share the link to the numbers for the entirety of the test DCP that was used and I will avoid to write here my conclusions about what would I consider a fair bandwidth used for a 4K feature DCP. As a more personal estimation, I will leave that for the comments section.
Feel free to share comments on what would have made the estimations reflect better how the JPEG2000 compression used in digital cinema would be more effectively put into use. The motive behind this effort is and was to figure out what would be the most effective way to balance quality and compression/storage.
I do understand that there is no correct answer, particularities in image complexity will always appear, that would call for weighting up a situation. Yet, I also understand that people tend to think that "one can't go wrong when going for more", and that turns out to be problematic at one or another point of a DCP's lifetime. It is true that a DCP is not made for archiving. On the other hand, it is also true that archives often enough (to take under consideration) find that (4K or 2K) DCPs are the most handy or early-generation export found in digital form.
I would also ask you to let me know in comments if there is any open source, non proprietary software that would do that reduction-without-re-encoding for cinema .j2c files, or (even better), if there are DCP authoring programs that do that. I would sincerely like very much to find that feature available.
I hope you will find this as insightful as I did, and I wish everyone the best for these festive days.
(This post is not primarily intended to be here. This blog is kept for different reasons.
Therefore, it will move, once it find its proper home.)
----------------------------------------------------------------------------------
Terms of the DCP license (December 24, 2024, https://dpel.aswf.io/asc-stem2/):
ASWF Digital Assets License v1.1
StEM2 -- Copyright 2022 -- American Society of Cinematographers -- All rights reserved.
Redistribution and use of these digital assets, with or without modification, solely for education, training, research, software and hardware development, performance benchmarking (including publication of benchmark results and permitting reproducibility of the benchmark results by third parties), or software and hardware product demonstrations, are permitted provided that the following conditions are met:
1. Redistributions of these digital assets or any part of them must include the above copyright notice, this list of conditions and the disclaimer below, and if applicable, a description of how the redistributed versions of the digital assets differ from the originals.
2. Publications showing images derived from these digital assets must include the above copyright notice.
3. The names of copyright holder or the names of its contributors may NOT be used to promote or to imply endorsement, sponsorship, or affiliation with products developed or tested utilizing these digital assets or benchmarking results obtained from these digital assets, without prior written permission from copyright holder.
4. The assets and their output may only be referred to as the Asset Name listed above, and your use of the Asset Name shall be solely to identify the digital assets. Other than as expressly permitted by this License, you may NOT use any trade names, trademarks, service marks, or product names of the copyright holder for any purpose.
