Communication device

The present application claims priority from Japanese Patent Application No. JP 2010-070384 filed in the Japanese Patent Office on Mar. 25, 2010, the entire content of which is incorporated herein by reference.

1. Field of the Invention

The present invention relates to a communication device applicable to a picturephone using the Internet.

2. Description of the Related Art

In recent years, various electronic devices including a personal computer have become capable of accessing the Internet. With this trend, picturephones using the Internet have become widespread, and lately, even TV sets have become equipped with a picturephone function (see http://www.skype.com/intl/ja/allfeatures/tv/).

Picturephones are capable of interactively communicating not only sound but image. To acquire a picture, a small camera including a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or the like is generally used, and to display a picture, a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel is generally used. As this camera for the picturephone, one having a small angle of view is usually used. When an angle of view is wide, there is a possibility that a person will appear in a small size on a display panel, so that someone on the other side of the picturephone cannot see the face of the person. In addition, there is also a possibility that the entire room will appear so that untidiness of the room may be transmitted to the person on the other side of the picturephone. Therefore, as the camera for the picturephone, for example, one having around 45 degrees is used.

With the widespread use of the picturephones, the way of using the picturephone has also diversified. In particular, recently, the number of cases in which two or more people use one picturephone simultaneously to communicate with a counterpart has increased. In this case, to arrange all the members in an area that can be photographed by a camera, all the members need to be at positions away from the camera to some extent. In addition, even in a case where there is only one person speaking on the picturephone, the person needs to be at a position away from the camera to some extent when the person wants to move in front of the camera. As a result, the person appears in a small size on a display panel. Thus, the existing picturephones have such a problem that the usability is not so high because of the limitation to the angle of view of the camera.

In view of the foregoing, it is desirable to provide a communication device capable of improving a decline in the usability caused by an angle of view of a camera.

A communication device according to an embodiment of the present invention includes: a camera capturing a picture; and an information processing section detecting a face image from the picture captured by the camera, to generate a partial picture including the detected face image. This communication device further includes: a transmitting section encoding the partial picture to generate encoded data, and transmitting the encoded data to a counterpart; a first receiving section receiving data transmitted from the counterpart, and decoding the received data to generate a picturephone picture; and a display section capable of displaying the picturephone image.

In the communication device according to the embodiment of the present invention, the face image is detected from the picture captured by the camera, and the partial picture including the detected face image is generated. As a result, in the communication device of the counterpart, the partial picture is displayed as the picturephone picture in the display section. In other words, according to the embodiment of the present invention, rather than adjustment of the angle of view itself of the camera, adjustment of the picture size correlated with the angle of view of the camera is performed by the generation of the partial picture (clipping of the partial picture out of the original image).

In the communication device according to the embodiment of the present invention, the camera preferably has an angle of view of 60 degrees or more. Moreover, the communication device according to the embodiment of the present invention may be a television further including a second receiving section that receives a television broadcast signal, and generates a broadcast image by decoding the television broadcast signal. In this case, preferably, the information processing section causes, according to a user instruction, the display section to display either one or both of the picturephone picture and the broadcast picture.

In addition, in the communication device according to the embodiment of the present invention, preferably, the camera also functions as a human sensor. For example, when being unable to detect a face image from the picture captured by the camera, the information processing section may stop picture display in the display section, and subsequently, upon detection of a face image from the picture captured by the camera, the information processing section may cancel the stoppage of the picture display in the display section.

According to the communication device in the embodiment of the present invention, the picture size correlated with the angle of view of the camera is adjusted by the generation of the partial picture (the clipping of the partial picture out of the original picture). Thus, for example, when two or more people communicate with a counterpart by using one picturephone at the same time, or when a user moves in front of the camera, each person can be displayed in a large size on the display panel of the communication device of the counterpart. Therefore, it is possible to improve a decline in the usability due to the angle of view of the camera.

FIG. 1 is a block diagram illustrating an example of the configuration of a picturephone system including a VC (Visual Communication) device according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating an example of the internal configuration of the VC device in FIG. 1 ;

FIG. 3A and 3B are schematic diagrams for explaining an example of the method of generating a partial picture;

FIG. 4A and 4B are schematic diagrams for explaining another example of the method of generating the partial picture;

FIG. 5 is a schematic diagram of a state in which a broadcast picture and a picturephone picture are displayed simultaneously; and

FIG. 6 is a schematic diagram for explaining an example of the method of generating a sensor picture.

An embodiment of the present invention will be described in detail with reference to the drawings. Incidentally, the description will be provided in the following order.

1. Embodiment

Configuration of picturephone system ( FIG. 1 )

Configuration of VC device ( FIG. 2 )

Picturephone function ( FIG. 3A-FIG . 5 )

Effect

2. Modification

Human sensor ( FIG. 6 )

1. Embodiment

[Configuration of Picturephone System]

FIG. 1 is a block diagram illustrating an example of the configuration of a picturephone system 100 including plural VC devices according to an embodiment of the present invention. For example, the picturephone system 100 includes two VC devices 1 and 2 connected to an open network 200 . The open network 200 refers to a network in which communication is performed by using a communication protocol generally used in picturephones. To the open network 200 , other VC devices can be connected at any time. Incidentally, in the present embodiment, the VC device 1 or the VC device 2 corresponds to a specific example of the “communication device” according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating an example of the configuration of the VC device 1 . Incidentally, the VC device 2 includes the same components as those of the VC device 1 . The VC device 1 is, for example, a television that can be connected to the Internet. The VC device 1 includes, for example, an antenna terminal 1 A, a digital tuner 1 B, a demultiplexer 1 C, a decoder 1 D, an image-signal processing circuit 1 E, a graphic generation circuit 1 F, a panel driving section 1 G and a display panel 1 H. The VC device 1 also includes, for example, a sound-signal processing circuit 1 I, an audio amplifier circuit 1 J and a speaker 1 K. The VC device 1 further includes, for example, an internal bus 1 L, a memory 1 M, an arithmetic circuit 1 N, a network interface 1 P, a network terminal 1 Q, a camera 1 R, a microphone 1 S, a decoder 1 T, an encoder 1 U, a remote control receiving circuit 1 V and a remote control transmitter 1 W. Incidentally, alternate long and short dashed line illustrated in FIG. 2 represents, for example, a main unit (enclosure 1 X) of the VC device 1 , and indicates the remote control transmitter 1 W is provided separately from the main unit of the VC device 1 .

The antenna terminal 1 A is a terminal to which a television broadcast signal received by a receiving antenna (not illustrated) is input. The digital tuner 1 B is, for example, configured to process the television broadcast signal input into the antenna terminal 1 A, thereby outputting a predetermined transport stream corresponding to a channel selected by a user. The demultiplexer 1 C is, for example, configured to extract a partial TS (Transport Stream) corresponding to the channel selected by the user, from the transport stream captured by the digital tuner 1 B.

The decoder 1 D is, for example, configured to subject a video PES (Packetized Elementary Stream) packet included in the partial TS obtained by the demultiplexer 1 C to decoding processing, thereby obtaining picture data (broadcast picture data). Further, the decoder 1 D is, for example, configured to subject a sound PES packet included in the partial TS obtained by the demultiplexer 1 C to decoding processing, thereby obtaining sound data.

The image-signal processing circuit 1 E and the graphic generation circuit 1 F are configured, for example, to subject the picture data obtained by the decoder 1 D and the decoder 1 T to multi-graphic processing, graphics-data superposition processing, and the like, as needed. The graphic generation circuit 1 F is, for example, configured to generate a UI (User Interface) screen to be used at the time of screen display. The panel driving section 1 G is, for example, configured to generate a driving signal based on the picture data output by the graphic generation circuit 1 F, and to drive the display panel 1 H by using the driving signal.

The display panel 1 H is configured by, for example, a LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display panel, or a PDP (Plasma Display Panel). The sound-signal processing circuit 1 I is, for example, configured to subject the sound data obtained by the decoder 1 D and the decoder 1 T to processing including D-A conversion and the like. The audio amplifier circuit 1 J is, for example, configured to amplify the sound signal output by the sound-signal processing circuit 1 I and supply the amplified sound signal to the speaker 1 K.

The memory 1 M performs, for example, storage of the setting information of the VC device 1 , storage of various kinds of software, and data management. The memory 1 M includes, for example, a flash ROM (Read Only Memory) (not illustrated) that stores control software for controlling the VC device 1 , and data to be used at the time when this control software is executed. The memory 1 M further includes, for example, a DRAM (Dynamic Random Access Memory) (not illustrated) that forms a work area of the various kinds of software. The memory 1 M further includes, for example, a HDD (Hard Disk Drive) (not illustrated) that stores various kinds of applications. As such applications, an application (not illustrated) that realizes a picturephone function is stored in the HDD by default.

The arithmetic circuit 1 N controls the operation of each part of the VC device 1 and is, for example, configured to develop the software read from the HDD of the memory 1 M in the work area of the memory 1 M and start the software, thereby controlling the operation of each part of the VC device 1 .

The network terminal 1 Q is, for example, a terminal connected to the network (the open network 200 ), and connected to the network interface 1 P. The network interface 1 P is, for example, an interface that communicates with others (the VC device 2 and the like) according to Ethernet (registered trademark) which is one type of standard of LAN (Local Area Network). Incidentally, the Ethernet (registered trademark) is a registered trademark of Fuji Xerox Co., Ltd. The network interface 1 P is, for example, connected to the internal bus 1 L, together with the arithmetic circuit 1 N, the memory 1 M, the camera 1 R, the microphone 1 S, the decoder 1 T and the encoder 1 U.

The camera 1 R takes a picture of scenery in the front of the display panel 1 H, and is disposed, for example, on the fringe of the display panel 1 H. The camera 1 R is, for example, configured to include an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 1 R is equipped with, for example, a wide-angle lens (not illustrated), on its plane of light incidence (not illustrated). The wide-angle lens is shaped to realize, for example, an angle of view of 60 degrees or more.

The microphone 1 S mainly captures the sound in front of and around the display panel 1 H and is disposed, for example, on the fringe of the display panel 1 H.

The decoder 1 T obtains picture data by, for example, performing the decoding processing on the video PES packet included in the data obtained from the network terminal 1 Q via the network interface 1 P. Further, the decoder 1 T also obtains sound data by, for example, performing the decoding processing on the sound PES packet included in the data provided from the network terminal 1 Q via the network interface 1 P.

The encoder 1 U is, for example, configured to subject the picture data (picturephone picture data) obtained from the arithmetic circuit 1 N to encoding processing, thereby obtaining data to be transmitted. The encoder 1 U is also configured, for example, to subject the sound data obtained from the microphone 1 S to the encoding processing, thereby obtaining data to be transmitted.

The remote control receiving circuit 1 V is, for example, configured to receive a remote control signal transmitted from the remote control transmitter 1 W and supply the received signal to the arithmetic circuit 1 N. The arithmetic circuit 1 N is, for example, configured to control each part of the VC device 1 according to the remote control signal.

[Picturephone Function]

Next, a picturephone function in the VC device 1 will be described. In the VC device 1 of the present embodiment, for example, in accordance with a user instruction, the arithmetic circuit 1 N develops the application that implements the picturephone function into the DRAM and starts the application. Subsequently, in accordance with various instructions from the user, the arithmetic circuit 1 N executes the application.

(Transmission)

Subsequently, the arithmetic circuit 1 N instructs the camera 1 R to take a moving picture and also instructs the microphone 15 to pick up the sound. Then, the camera 1 R starts taking the moving picture and transfers obtained picture data to the arithmetic circuit 1 N, and the microphone 1 S starts picking up the sound and transfers obtained sound data to the arithmetic circuit 1 N. The arithmetic circuit 1 N inputs the obtained sound data into the encoder 1 U, and performs predetermined processing (clipping processing that will be described later) on the obtained picture data and then inputs the processed picture data (a partial picture 4 that will be described later) into the encoder 1 U. Then, the encoder 1 U performs the encoding processing on the input sound data and picture data, and inputs the encoded data into the arithmetic circuit 1 N. The arithmetic circuit 1 N transmits the data to the VC device 2 of a communication counterpart through the network interface 1 P and the network terminal 1 Q.

(Reception)

Upon acquisition of data from the VC device 2 of the communication counterpart through the network interface 1 P and the network terminal 1 Q, the arithmetic circuit 1 N inputs the data into the decoder 1 T. Then, the decoder 1 T performs the decoding processing on the input data, generates picture data and sound data from the data, and inputs the generated picture data and sound data into the arithmetic circuit 1 N.

The arithmetic circuit 1 N inputs the obtained picture data into the image-signal processing circuit 1 E via, for example, a High-Definition Multimedia Interface (HDMI). Then, the image-signal processing circuit 1 E performs predetermined processing on the obtained picture data and then, inputs the resultant picture data into the graphic generation circuit 1 F. The graphic generation circuit 1 F performs predetermined processing on the obtained picture data and then, inputs the processed picture data into the panel driving section 1 G as a picture signal. The panel driving section 1 G drives the display panel 1 H based on the picture signal, thereby causing the display panel 1 H to display a picture. As a result, the picture of the counterpart is displayed on the display panel 1 H.

Further, the arithmetic circuit 1 N inputs the obtained sound data into the sound-signal processing circuit 1 I via, for example, the HDMI. Then, the sound-signal processing circuit 1 I performs predetermined processing on the obtained sound data and then, inputs the processed sound data into the speaker 1 K via the audio amplifier circuit 1 J. As a result, the sound of the counterpart is output from the speaker 1 K. In this way, the interactive communication of sound and picture by using the picturephone function is carried out.

(Clipping Processing)

Next, there will be described the clipping processing in the VC devices 1 and 2 .

In the present embodiment, the arithmetic circuit 1 N performs the clipping processing that will be described below on the picture data obtained from the camera 1 R.

When the picture data obtained from the camera 1 R is, for example, a picture as illustrated in FIG. 3A (obtained picture 3 ), the arithmetic circuit 1 N, at first, detects face images f 1 and f 2 in the obtained picture 3 . Subsequently, the arithmetic circuit 1 N sets a clipped region 3 A including the all detected face images f 1 and f 2 in the obtained picture 3 , clips the clipped region 3 A out of the obtained picture 3 , and generates the partial picture 4 as illustrated in FIG. 3B . At the time, the arithmetic circuit 1 N adjusts (reduces or expands) the size of the partial picture 4 as needed.

When, for example, the pixel size of the camera 1 R is large (e.g., 5M pixels), and the size of the partial picture 4 exceeds a HD size, the arithmetic circuit 1 N reduces the size of the partial picture 4 to the HD size. Further, when, for example, the size of the partial picture 4 is less than the HD size, the arithmetic circuit 1 N enlarges the size of the partial picture 4 to the HD size. In this way, the clipping processing in the arithmetic circuit 1 N is carried out.

Incidentally, the clipping processing may be performed by using another method. For example, when the picture data obtained from the camera 1 R is a picture as illustrated in FIG. 4A (obtained picture 3 ), the arithmetic circuit 1 N, at first, detects the face images f 1 and f 2 in the obtained picture 3 . Subsequently, the arithmetic circuit 1 N sets clipped regions 3 B and 3 C including the face image f 1 and the face image f 2 in the obtained picture 3 , respectively, for the individual face images f 1 and f 2 , clips the clipped regions 3 B and 3 C out of the obtained picture 3 , and generates two portrait pictures 3 D and 3 E (see FIG. 4B ). Subsequently, the arithmetic circuit 1 N merges the generated two portrait pictures 3 D and 3 E, thereby producing a partial picture 4 illustrated in FIG. 4B . The merge of the two portrait pictures 3 D and 3 E is performed, as illustrated in FIG. 4B for example, by arranging the two portrait pictures 3 D and 3 E side by side in parallel. At the time, the arithmetic circuit 1 N adjusts (reduces or expands) the size of the partial picture 4 as needed, in a manner similar to the way described above. In this way, the clipping processing in the arithmetic circuit 1 N may be carried out.

Incidentally, the arithmetic circuit 1 N may choose either one of the two methods described above, in accordance with an instruction of the user.

(Concurrent Use of Broadcast and Picturephone)

Next, there will be described the “concurrent use of broadcast and picturephone” in the VC devices 1 and 2 . In the present embodiment, each of the VC devices 1 and 2 is a television with the picturephone function. Therefore, the VC devices 1 and 2 can cause the display panel 1 H to display broadcast picture data while serving as a picturephone, according to an instruction of the user. For example, as illustrated in FIG. 5 , the arithmetic circuit 1 N causes the display panel 1 H to display a broadcast picture 11 and picturephone pictures 12 and 13 at the same time.

The picturephone picture 12 is a picture generated based on the data obtained from the VC device 2 of the counterpart, and the picturephone picture 13 is a picture generated based on the data obtained from the camera 1 R of the user's own VC device 1 . Incidentally, the arithmetic circuit 1 N may cause the display panel 1 H to display either one or both of the picturephone pictures 12 and 13 , depending on an instruction of the user.

Such simultaneous display is enabled by provision of the decoder 1 D for broadcasting and the decoder 1 T for picturephone in each of the VC devices 1 and 2 . Incidentally, the decoder 1 D and the decoder 1 T may be provided in different chips, or may be provided in the same chip. However, when the decoder 1 D and the decoder 1 T are provided in the same chip, the decoder 1 D is, for example, a dual decoder including the decoder 1 T, and the data inputted from the VC device 2 is input into the decoder 1 D via the network interface 1 P, the memory 1 M and the like.

[Effect]

In the present embodiment, in the VC device 1 , the face images f 1 and f 2 are detected from the obtained picture 3 that is captured by the camera 1 R, and the partial picture 4 including the detected face images f 1 and f 2 is generated. As a result, the partial picture 4 is displayed as a picturephone picture on the display panel 1 H in the VC device 2 of the counterpart. In other words, in the present embodiment, rather than adjustment of the angle of view itself of the camera 1 R, adjustment of the picture size correlated with the angle of view of the camera 1 R is performed by the generation of the partial picture 4 (the clipping of the partial picture out of the original picture). As a result, for example, two or more people can communicate with a counterpart by using one VC device 1 at the same time, and even when the user moves in front of the camera 1 R, each person can be displayed in a large size on the display panel 1 H of the VC device 2 of the counterpart. Therefore, it is possible to improve a decline in the usability due to the angle of view of the camera.

2. Modification

In the embodiment described above, the case in which the display panel 1 H and the panel driving section 1 G are built in the main unit (enclosure 1 X) of the VC device 1 has been taken as an example, but the display panel 1 H and the panel driving section 1 G may be provided separately from the main unit (enclosure 1 X) of the VC device 1 . In this case, the display panel 1 H and the panel driving section 1 G are external devices for the VC device 1 and thus, for example, the VC device 1 controls the external devices (the display panel 1 H and the panel driving section 1 G) by cable or radio.

Further, in the embodiment described above, the case in which the VC device 1 is a television has been taken as an example, but the VC device 1 may not be a television. In this case however, for example, the VC device 1 causes the display panel 1 H to display a broadcast picture by using picture data and sound data inputted from a piece of electronic equipment (a tuner) configured to include the digital tuner 1 B, the demultiplexer 1 C, the decoder 1 D and the like, and causes the speaker 1 K to output the sound.

Furthermore, in the embodiment described above, the case in which in the VC device 1 , the part functioning as a picturephone is built in the enclosure 1 X has been taken as an example, but this part may be provided separately from the main unit (enclosure 1 X) of the VC device 1 . In this case, for example, the part functioning as the picturephone is an external device for the VC device 1 and thus, for example, the VC device 1 controls the external device (the part functioning as the picturephone) via USB (Universal Serial Bus).

Moreover, in the embodiment and the modification described above, the camera 1 R may function as a human sensor. At the time, there is no need to clip the partial picture 4 out of the obtained picture 3 captured by the camera 1 R and, the size of the picture may only need to be a size sufficient for use of the obtained picture 3 captured by the camera 1 R for the purpose of the human sensor. When the size sufficient for use for the purpose of the human sensor is, for example, a VGA size, the arithmetic circuit 1 N, for example, adjusts the size of the obtained picture 3 captured by the camera 1 R to the VGA size, and performs one or more of various kinds of processing to be described below, by using the post-adjustment picture data.

(Energy Saving Function)

For example, the arithmetic circuit 1 N may be configured to stop the picture display in the display panel 1 H when the arithmetic circuit 1 N cannot detect a face image from the picture captured by the camera 1 R, and subsequently to cancel the stoppage of the picture display in the display panel 1 H when the arithmetic circuit 1 N detects a face image from the picture captured by the camera 1 R.

(Approach Preventing Function)

Further, for example, the arithmetic circuit 1 N may be configured to derive, when detecting a face image from the picture captured by the camera 1 R, a distance from the camera 1 R to the detected face image or a numerical value corresponding to the distance, thereby performing predetermined processing when the distance or the numerical value exceeds a predetermined threshold. The predetermined processing here includes, for example, generation of an audible alert, reduction in the brightness of the picture displayed on the display panel 1 H, and the like.

(Sound Balance Correction Function)

Furthermore, for example, the arithmetic circuit 1 N may be configured to derive, when detecting a face image from the picture captured by the camera 1 R, an orientation of the detected face image or a numerical value corresponding to the orientation, thereby performing predetermined processing according to the orientation or the numerical value. The predetermined processing here includes, for example, adjustment of the sound balance between right and left.

Incidentally, the angle of view of the camera 1 R is, for example, 60 degrees or more as mentioned above, which is the size sufficient for use as a human sensor. In the present modification, when the camera 1 R is used as a camera for the picturephone, the VC device 1 performs the “clipping processing” described above on the picture taken by the camera 1 R, thereby narrowing the angle of view of the camera 1 R mimetically. Further, when the camera 1 R is used as a camera for the human sensor, the VC device 1 takes a picture of a wide area in front of the display panel 1 H by using the wide angle of view of the camera 1 R, and performs the various kinds of processing described above by using the taken picture. In this way, in the present modification, a camera for a picturephone that requires a narrow angle of view and a camera for a human sensor that requires a wide angle of view are realized by the single camera 1 R. This makes it possible to eliminate an increase in the cost of components due to addition of a human sensor function.

It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.