Apparatus for detecting direction of sound source and turning microphone toward sound source

1. Technical Field of the Invention

The present invention relates to an apparatus for detecting a direction of sound source and an image pick-up apparatus with the sound source detection apparatus, applicable to a video conference and a video phone.

2. Description of the Prior Art

A direction of a narrator in conventional video conference using a plurality of microphones is detected, as disclosed in JP 4-049756 A (1992), JP 4-249991 A (1992), JP 6-351015 A (1994), JP 7-140527 A (1995) and JP 11-041577 A (1999).

The voice from a narrator reaches each of the microphones after each time delay. Therefore, the direction of the narrator or sound source is detected by converting time delay information into angle information.

FIG. 4 is a front view of a conventional apparatus for the video conference, which comprises image input unit 200 including camera lens 103 for photographing a narrator, microphone unit 170 including microphones 110 a and 110 b , and rotation means 101 for rotating image input unit 200 .

The video conference apparatus as shown in FIG. 4 picks up the voice of the narrator and detects the direction of the narrator, thereby turning the camera lens 103 toward the narrator. Thus, the voice and image of the narrator are transmitted to other video conference apparatus.

FIG. 5 is an illustration for explaining a principle of detecting the narrator direction by using microphones 110 a and 110 b . There is a delay between the time when microphone 110 b picks up the voice of the narrator and the time when microphone 110 a picks up the voice of the narrator.

The narrator direction angle θ is equal to sin −1 (V·d/L), where V is speed of sound, L is a microphone distance and “d” is a delay time period, as shown in FIG. 5 .

However, an accuracy of determining the direction θ is lowered, when the delay and θ becomes great.

Further, the voice of the narrator reflected by a floor and walls is also picked up by the microphones. The background noises in addition to the voice are also picked up. Therefore, the narrator direction may possibly be detected incorrectly.

An object of the present invention is to provide an apparatus for detecting a direction of a sound source such as a narrator, thereby turning an image pick-up apparatus toward the sound source.

An another object of the present invention is to provide an apparatus for detecting the direction of sound sources which move quickly or are switched rapidly.

A still another object of the present invention is to provide a sound source detection apparatus which is not easily affected by the reflections and background noises.

The apparatus for detecting the direction of sound source comprises a microphone pair, narrator direction detection means for detecting a delay of sound wave detected by the microphones, rotation means for rotating the microphone pair, driving means for driving the rotation means on the basis of the output from the narrator direction detection means, so that the microphone are equidistant from the sound source.

The apparatus for detecting the sound direction of the present invention may further comprises another fixed microphone pair, for turning quickly the rotatable microphone set toward the direction of the sound source.

The narrator direction detection means may comprises mutual correlation calculation means for calculating a mutual correlation between the signals picked up by left and right microphones of the microphone pair, delay calculation means for calculating the delay on the basis of the mutual correlation. Further, the delay may be calculated in a plurality of frequency ranges and averaged with such weights that the lower frequency components are less effective in the averaged result.

According to the variable gain amplifier of present invention, the first microphone pair is turned toward a narrator, so that the sound wave arrives at the microphones simultaneously. Accordingly, the microphone is directed just in front of the sound source.

Further, according to the present invention, the second fixed microphone pair executes a quick turning of the microphone direction. Furthermore, according to the present invention, the direction of the sound source is quickly detected by directing the second microphone set toward the center of the sound sources, when the sound source such as a narrator is changed.

Furthermore, according to the present invention, the detection result is hardly affected by the reflections from floors and walls in the lower frequency range, because the outputs from a plurality of band-pass filters are averaged such that the lower frequency components are averaged with smaller weight coefficients.

FIG. 1A is a front view of the video conference apparatus of the present invention.

FIG. 1B is a plan view of the video conference apparatus as shown in FIG. 1 of the present invention.

FIG. 1C is a block diagram of the narrator direction detection means and microphone rotating means for the video conference apparatus as shown in FIG. 1 A.

FIG. 2 is a detailed block diagram of the narrator direction detection means as shown in FIG. 1 C.

FIG. 3 is a flow chart for explaining a method for detecting the sound source.

FIG. 4 is a block diagram of a conventional video conference apparatus.

FIG. 5 is an illustration for explaining a principle of detecting a direction of a sound source.

The embodiment of the present invention is explained referring to the drawings.

FIG. 1A is a front view of a video conference apparatus provided with the apparatus for detecting the sound source direction of the present invention. FIG. 1B is a plan view of the video conference apparatus 100 as shown in FIG. 1 A.

The video conference apparatus as shown in FIG. 1A comprises camera lens 103 for photographing the narrator, microphone set 160 including microphones 120 a and 120 b , microphone set 170 including microphones 110 a and 110 b , and rotation means 101 .

Microphones 110 a , 110 b , 120 a and 120 b may be sensitive to the sound of 50 Hz to 70 kHz.

FIG. 1C is a block diagram of a detection system for detecting the direction of narrators. There are shown in FIG. 1C, narrator direction detection means 130 using microphone set 170 , narrator direction detection means 150 using microphone set 160 , driving means 140 for driving rotation means 101 . Driving means 140 feeds information of the narrator direction detected by narrator direction detection means 130 and 150 back to video conference apparatus 100 .

FIG. 2 is a block diagram of microphone set 170 and narrator direction detection means 130 . There are shown in FIG. 2, A/D converters 210 a and 210 b for sampling the voice picked up by microphones 110 a and 110 b under the sampling frequency, for example, 16 kHz, and voice detection means for determining whether or not the signals picked up by microphones 110 a and 110 b are the voice of the narrator.

Further, there are shown in FIG. 2 band-pass filters 220 a , 220 b , 220 a ′, 220 b ′, calculation means for calculating a mutual correlation between the signal from microphone 110 a and the signal from microphone 110 b , integration means 240 and 240 ′ for integrating the mutual correlation coefficients, and detection means 260 and 260 ′ for detecting a delay between microphone 110 a and microphone 110 b which maximizes the integrated mutual correlation coefficients.

Band-pass filters 220 a and 220 b pass, for example, 50 Hz to 1 kHz, while band-pass filters 220 a ′ and 220 b ′ passes, for example, 1 kHz to 2 kHz. Two sets of band-pass filters ( 220 a , 220 b ) and ( 220 a ′, 220 b ′) are shown in FIG. 2. A plurality of more than two sets of band-pass filters, for example, 7 sets, may be included in narrator direction detection means 130 . In this case, each of not-shown band-pass filters passes, 2 kHz to 3 kHz, . . . , 6 kHz to 7 kHz, respectively.

Furthermore, there are shown in FIG. 2 delay calculation means 270 for calculating the delay between microphone 110 a and microphone 110 b on the basis of prescribed coefficients, and conversion means for converting the calculated delay into an angle. Here, the delay is a time difference between a time when said sound wave arrives at a microphone and a time when said sound wave arrives at another microphone in a microphone pair.

Narrator direction detection means 150 is similar to narrator direction detection means 130 .

In the video conference apparatus as shown in FIGS. 1A, 1 B, 1 C and 2 , the voice of the narrator is picked up by microphones 11 a to 120 b and inputted into narrator direction detection means 130 and 150 . The inputted voice is converted into digital signal by A/D converters 210 a and 210 b . The digital signal is inputted simultaneously into voice detection mean 250 , band-pass filters 220 a , 220 b , 220 a ′, 220 b′.

Each of the seven sets of band-pass filters passes only its proper frequency range, for example, 50 Hz to 1 kHz, 1 kHz to 2 kHz, 2 kHz to 3 kHz, . . . , 6 kHz to 7 kHz, respectively.

The outputs from the band-pass filters are inputted into calculation means 230 , 230 ′, . . . In this example, there are seven calculation means for calculating the mutual correlation coefficients between signals inputted into the calculation means. Then, the calculated mutual correlation coefficients are integrated by integration means 240 , 240 ′, . . . .

On the other hand, voice detection means 250 determines whether or not the picked-up sound human voice. The determination result is inputted into integration means 240 , 240 ′, . . . Then, the integration means output the integrated mutual correlation coefficients toward detection means 260 , 260 ′, . . . when the picked-up signal is human voice. On the contrary, the integration means clear the integrated mutual correlation coefficients, when the sound picked-up by microphones 110 a and 110 b.

FIG. 3 is a flow chart for explaining the operation of voice detection means 250 which distinguishes human voices from background noises. Voice detection means 250 measures the signal level of the outputs from A/D converters 210 a and 210 b , during the time period when its timer is set to be zero (step S 1 ). Then, the ratio A (=X/Y) of a signal level X at time “T-1” to a signal level Y at time “T” (step S 2 ).

Then, the ratio A is compared with a prescribed threshold (step S 3 ). When the ratio A is greater than the prescribed level threshold, the step S 4 is selected. On the contrary, when the ratio A is not greater than the prescribed level threshold, step S 8 is selected. The frequency of the signal for the level comparison may be, for example, about 100 Hz for determining whether the signal picked-up by microphones 110 a and 110 b belongs to the frequency range of human voice.

The timer is turned on in step S 4 . The timer measures the time duration of a sound. Then, the time duration is compared with a prescribed time threshold (step S 5 ). The prescribed time threshold may be, for example, about 0.5 second, because the time threshold is introduced for distinguishing the human voice and the noise such as a sound caused by a participant letting documents fall down.

When the measured time duration is greater than the prescribed time threshold, step S 6 is selected. On the contrary, when the measured time duration is not greater than the prescribed time threshold, step S 8 is selected. The sound is determined to be human voice in step S 6 , while the sound id determined not to be human voice in step 8 . Then, step S 7 is executed in order to reset the timer or set the timer to be zero. Thus, voice detection means 250 repeats the steps as shown in FIG. 3 .

There are seven detection means 260 , 260 ′, . . . in an exemplary embodiment as shown in FIG. 2 . The detection means detect delays D 1 to D 7 , respectively, which maximizes the integrated mutual correlation coefficients. then, delays D 1 to D 7 are inputted into delay calculation unit 270 which calculates averaged delay “d”.

d=D

1

·A

1

+D

2

·A

2

+D

3

·A

3

+D

4

·A

4

+D

5

·A

5

+D

6

·A

6

+D

7

·A

7

where A 1 to A 7 are prescribed coefficients which satisfy the following relation; A 1 +A 2 +A 3 +A 4 +A 5 +A 6 +A 7 =1.

It is well known that higher frequency components are diffused by a floor and walls, while the lower frequency components are reflected in such a manner that the incident angle added to the reflected angle approaches to 90°, as the frequency becomes low. Therefore, the detection of the narrator direction is affected by the interference between the direct sound and the reflected sound at lower frequency.

Therefore, A 1 <A 2 <A 3 <A 4 <A 5 <A 6 <A 7 is preferable, where, for example, D 1 is a delay for 50 Hz to 1 kHz, D 2 is a delay for 1 kHz to 2 kHz, D 3 is a delay for 2 kHz to 3 kHz, D 4 is a delay for 3 kHz to 4 kHz, D 5 is a delay for 4 kHz to 5 kHz, D 6 is a delay for 5 kHz to 6 kHz,and D 7 is a delay for 6 kHz to 7 kHz.

Thus, the calculation of the averaged delay “d” is not so much by the interference between the direct sound and the sound reflected by the floor and walls in the lower frequency region.

The averaged delay “d” is inputted into conversion means 280 for converting the averaged delay “d” into the angle of the narrator direction.

The angle of the narrator directionangle θ is equal to sin −1 (V·d/L), where V is speed of sound, L is a microphone distance and “d” is the averaged delay. The angle θ is inputted into driving means 140 . Driving means selects either of the output from narrator direction detection means 130 or the output from narrator direction detection means 150 in order to drive rotation means 101 .

Rotation means 101 rotates microphone set 160 so that the narrator becomes substantially equidistant from microphones 120 a and 120 b . In other words, rotation means 101 turns microphone set 160 toward the sounds source so that the time difference tends to zero. Thus, the microphone set is directed precisely to the direction of the sound source. Therefore, conversion means 280 in microphone set 160 are not always required.

Further, the distances are adjusted more precisely on the basis of the output from narrator direction detection means 150 .

Microphone set 170 may be directed to the center of the attendants to the conference, so as to turn microphones quickly, when the narrator is changed. In other words, fixed microphone set 170 is used for turning the rotatable microphone set 160 toward the direction angle θ of the sound source. Therefore, the conversion means is indispensable for microphone set 170 .

Video conference apparatus as shown in FIG. 1A may further comprises speakers and display monitors for the voices and images through the other end of the communication lines such as Japanese integrated services digital network (ISDN).

Further, video conference apparatus as shown in FIG. 1A may be used for a video telephone and other image pick-up apparatus for photographing images of sound sources in general.