Ultrahigh-Definition Television System With 4000 Scanning Lines F.Okano1, M.Kanazawa1, K.Mitani1, K.Hamasaki1, M.Sugawara1, M.Seino2, A.Mochimaru3 and K.Doi4 1

2

3

4

NHK, Ikegami Tsushinki Co., Ltd., BOSE Japan and Victor Company of Japan

ABSTRACT An ultrahigh-definition television system with 4000 scanning lines has been developed. A camera and a projection display together with a hard disc recorder have been produced as a trial version. Two green channels are arranged by the diagonal-pixel-offset method to achieve the 4000 scanning lines. The 22.2 multichannel audio system is also constructed for reproducing 3-D sound and natural impression. INTRODUCTION The HDTV system has been developed to provide high resolution images, and we can enjoy realistic atmosphere in our home, nowadays. Sensation that observer perceives is affected by many factors, for example, resolution, screen size, colorimetory, smoothness of motion, viewing distance, and etc. Especially, the viewing angle is significant and deeply related to realistic and immersive sensation. If we could use the factors more effectively and establish the new television system beyond HDTV, it can open the new era for the future imagery. Our project has shown that the viewers prefer larger angle with wide-screen and high-resolution images than that with HDTV where the angle of 30 degrees is recommended. The sensation of reality induced to an observer by watching a display screen increases with the viewing angle and saturates at an angle around 100 degrees horizontally [1]. Considering human vision characteristics, the spatial resolution of a display is desired to have 60 pixels per degree [2]. To meet the above conditions, a display with approximately 4k(V)×8k(H) pixels is necessary. Table 1 shows the specifications of the ultra-high definition television system with 4k×8k pixels, comparing with those of HDTV. Figure 1 shows the relationship between human visual field [1] and display areas of HDTV and the 4k×8k system. While the display area of HDTV is covering the effective field [2], that of the 4k×8k system covers the induced field, which is related to the observer’s posture judgment and therefore the sensation of reality. At a viewing angle of 100 degrees under the aspect ratio of 9:16, which corresponds to a viewing distance of 0.75 H (H: picture height), viewers can enjoy much higher sensation of reality with 4k×8k video images than with HDTV. The resolution of the 4k×8k system is 16 times of that of HDTV and is

more than twice of that of 70 mm motion film horizontally and vertically, assuming the resolution of 35mm motion film to be 700TVL [3]. 4k×8k system

HDTV Number of pixels 1080×1920 30 deg. horizontally

Viewing angle (pixel invisible) Comparison with movie

4320×7680 More than 100 deg. horizontally

Equivalent to More than twice of 35mm motion film 70 mm motion film

Table 1 Specifications of ultrahigh-definition television system with 4000 scanning line Effective field

Induced field

20 deg. 85 deg. 30 deg 100 deg.

HDTV 4K×8K system Fig 1 Human visual field and display area

HIGH RESOLUTION TECHNOLOGY At the present, both image sensor and display panel with such a huge number of pixels of 4k×8k are not available. On the other hand, recently, the sensor and panel with 2k×4k pixels have been developed. Therefore, introducing them is the most feasible way to achieve ultrahigh-definition system. We increased the number of pixels in both the horizontal and vertical directions. Both camera and display are applying two channels to green (G1,G2) in the diagonal-pixel-offset method, as shown in the figure 2. In addition, we set one panel of 2048×3840 pixels to red (R) and blue (B) each. The figure 3 shows the equivalent pixel structure for the scheme that is well known as a Green Bayer Pattern [4]. It makes use of human visual characteristics. Specifically, while the

2004 NAB BEC Proceedings — 437

human vision features high spatial frequency response with respect to brightness, it has relatively low response with respect to hue and chroma. Because green makes a bigger contribution than red and blue to brightness components, achieving high resolution even in only green can still be effective.

Number of pixels Method A B C

R Original image

D

4k×8k 2k×4k ×2 diagonal offset 2k×4k ×2 2k×4k diagonal offset 2k×4k

Table 2 Result of subjective evaluation

G1 2160×3840 pixels for each channel

G2 B Fig 2 2G/R/B method 7680 pixels Ｇ１

Red/Blue

Green

Ｒ

green. The image from Method-D has 2k×4k pixels for red, green and blue (half resolution of Method-A). The simulated evaluation was performed by conventional video monitor and its result is shown by figure 4. Because the scores of the Method-B and Method-C were as high as that of Method-A (Original) and the score of the Method-D was very low, the spatial pixel-offset method was proved to be effective. The Method-C is the most cost effective, therefore, this method was adopted to develop the ultrahigh-definition system.

diagonal offset Ｂ

4320 pixels

EXPERIMENTAL EQUIPMENTS

Ｇ2 One pixel interval

G: 4320×7680 pixels equivalently R,B: 2160×3840 pixels Fig 3 Pixel arrangement on 2G/R/B method

SUBJECTIVE EVALUATION The authors conducted a subjective assessment [5] test in order to confirm the picture quality of the pixel-offset method. The picture quality related to pixel structures of the four methods shown in Table 2 was investigated. The image from the Method-A has full 4k×8k pixels for red, green and blue. The Method-B and Method-C are the spatial pixel-offset method, where the image from Method-B has 2k×4k×2 pixels for red, green and blue, and that from Method-C has 2k×4k×2 pixels for only

1) Camera The optical system in this camera is composed of a lens and color separation prism, shown in figure 5. The color separation prism divides an optical image into two G1, G2, R, and B on each 2k×4k-pixel image sensor. The four image sensors are attached to the prism with half-pixel pitch offset so that the image sampling pattern of their pixel is equivalent to that of a single chip color image sensor with 4k×8k pixels [6], as shown in figure 3. B-sensor

G1-sensor

Lens

G2-sensor R- sensor

Opinion scores 5

Fig 5 Color separation prism for 4-image sensors

4 3 2

95% reliable

1 Method A

Method B

Method C

Evaluated method Fig 4 Result of subjective evaluation

438 — 2004 NAB BEC Proceedings

Method D

The first proto-type camera system uses four 2k×4k-pixel CCDs with 60 frames per second progressive operation capability [7]. The 34 mm by 17.2 mm image area has 2048(V)×4046(H) active pixels with 8.4 µm squares. The multiple-output structure with its 16 outputs enables high-data-rate imaging for ultrahigh-resolution moving pictures. By using this camera system, we have performed several field tests to get a various kinds of images for evaluating the system and making demonstration programs. Moreover, aiming at the practical camera system,

we have studied to make a camera head smaller. A 2160×3840-pixel CMOS image sensor with the 16 mm by 9 mm image area has been developed for this system [8]. The main features of CMOS, for example, the low power consumption, the on-chip signal processing circuits and the column parallel structure, become remarkable in this application. The pixel size of the sensor is 4.2×4.2 micron squared, smaller than that of the CCD. A 10-bit analog to digital converter is placed at every two columns on the chip, and the 16-column parallel digital output structure is built in. The authors have developed an experimental image pickup system using four CMOS sensors. Figure 6 shows the external

Fig 6 External views of the first version camera head (left) and the experimental compact camera head (right).

views of the first camera head (80Kg with lens) and this experimental one. The size of the camera head with CMOS image sensors has become almost half that of the first one. The resolution characteristics of the camera is shown in figure 7. The vertical limiting resolution of the system can attain to more than 3200 TV lines successfully. Based on this result, the compact practical camera system will be developed.

resolution of 4k×8k with a wide screen [9]. Here, the relative positioning of the two LCD panels for green has a great effect on resolution characteristics, and these two panels must be accurately offset by 0.5 pixel. To meet this requirement, we employed two projection units, one for color green, where two green panels are installed and the other for colors red and blue, where red and blue panels are installed. Figure 8 shows the optical system for green projection unit [10]. Polarizing beam splitters (PBS) with phase shifting plates are used to separate and combine G1 and G2. A compact stepping motor is attached onto the G2 panel to make fine adjustment of the 0.5-pixel offset between these two panels. The optical output of the display is approximately 5000 lumen, resulting in the peak brightness on the screen of about 50 cd/m2 with a 320-inch screen whose screen gain is 0.9. (about 40 cd/m2 with a 450-inch screen whose screen gain is 1.5 ) Because there are two projection units for dual-G and R/B, the images from these units are not projected on the same position on the screen without any correction. Therefore a convergence error correction scheme was developed, which converts red and blue images so that the convergence error on the screen is corrected.

Projection lens

PBS

PBS G1 panel

Green light

PBS

White light Phase shifting plate

G2 panel (with stepping motors)

MTF [%] 100

Green filter

Fig 8 Optical structure of G projection unit Lens F-number 5.6

80

: Calculated data : Measured value

60 40 20

1000

2000

3000

4000

5000

Spatial frequency [TVL] Fig 7 Vertical resolution characteristics of the experimental compact camera head

2) Display The authors developed a front projection type display using four 1.7-inch LCoS panels (Liquid Crystal on Silicon) with 2048×3840-pixels in order to obtain the

3) Audio system As the sound with an extra large screen is expected to enhance the sensation of reality and presence, the necessary conditions of sound system for ultrahigh definition video system are decided as follows; - Stable localization of a frontal sound image at any point over the screen - Reproduction of a sound image in all directions around a viewer - Reproduction of three dimensional spatial impression with proper “listener’s envelopment” - Wide listening area with adequate quality of sound - Compatibility with existing multichannel sound systems According to these conditions, 22.2 multichannel sound system is developed. This system is proposed as the highest sound system in the hierarchy of compatible

2004 NAB BEC Proceedings — 439

The experimental system is shown in figure 10. The system is to have 4×4 times the number of HDTV pixels, the authors simplified the equipments and interfaces between them by using and synchronising multiple devices based on the HDTV format. The video signal is transmitted through 16 channels of HDTV-SDI (Serial Digital Interface) format signal, in total, approximately 24 Gbps.

(a)Ground plan Screen

(b) Side view

CONCULUTION An ultrahigh-definition television systems have a lot of opportunities for various potential applications. The system with 4k×8k pixels (16 times that of HDTV) was developed to realize nearly the maximum of sensation of reality. There are 4k×8k pixels equivalently for green, while 2k×4k pixels for red and blue. A spatial pixel-offset method was used for green. The system is still at a basic stage. The performances of the devices must be improved. In order to investigate the possibilities for future broadcast services, the authors will make a quantitative evaluation of sensation of reality with the experimental system described in this paper. A demonstration program that uses this ultrahigh-definition television system will be exhibited in the 2005 World Exposition, Aichi, Japan. REFERENCES

Upper layer

Middle layer

Lower layer

LFE

Fig 9 Loudspeakers arrangement for 22.2 multichannel sound system

multichannel sound systems for the theatre reproduction including various multichannel sound systems such as 5.1, 6.1, 7.1 and 10.2. [11][12] The 22.2 multichannel sound system consists of three layers of loudspeakers; upper layer with 9 channels, middle layer of 10 channels, lower layer of 3channels and 2 channel for LFE (Low Frequency Effects). Figure 9 shows detailed arrangement of loudspeakers. 4) Other equipments and interface A disc recorder system was also developed. It consists of 16 of HDTV disc recorder units and can record approximately 18 minutes of the ultrahigh-definition video signal in real time with a capacity of nearly 3.5 TB.

Camera

HD-SDI Camera 16ch Display control processor unit

Hard disk recorder

Projector R,B G1,G2

Video material preservation

Fig 10 Experimental system

440 — 2004 NAB BEC Proceedings

[1]T.Hatada, et al., 1980. "Psychophysical analysis of the sensation of reality induced by a visual wide-field display", SMPTE Journal, Vol.89 pp.560-569 (1980) [2]"High-definition television", Van Nostrand Reinhold、pp.7 1992, [3]A.Kaiser, H.W.Mahler, R.H.McMann, "Resolution requirements for HDTV based upon the performance of 35mm motion-picture films for theatrical viewing", SMPTE Journal, pp.654-659 (1985) [4] B.E Bayer, "Color Imaging Array", US Patent 3971065 (1976) [5]Methodology for the Subjective Assessment of the Quality of Television Pictures, ITU–R Recommendations, BT.500 [6] M. Sugawara et al., "Four-Chip CCD camera for HDTV", SPIE Proceedings, vol.2173, pp.122-129 (1994) [7]K. Mitani, et al., "Ultrahigh-definition color video camera system with 4K-scanning lines", SPIE Proceedings, Sensor and Camera Systems for Scientific, Industrial and Digital Photography Applications IV, pp.5017-18 ( 2003). [8] I. Takayanagi, et al., "A 1 1/4 inch 8.3M pixel digital output CMOS APS for UDTV application", Proceedings of ISSCC2003, pp.216-217 (2003) [9]M.Kanazawa, et al., "An Ultrahigh-Definition Display using the pixel offset method", (under reviewed), 2003. [10] W.P.Bleha, et al., "D-ILA technology for high resolution projection displays", SPIE AeroSense-03, pp.5080-29 (2004) [11] Tomlinson Holman, “5.1 Surround Sound, Up and Running”，Focal Press, Woburn (2000) [12] Francis Rumsey, “Spatial Audio”，Focal Press, Oxford (2001)