Self-equalizing loudspeaker system

This non-provisional patent application claims the benefit of the earlier filing date of U.S. provisional application No. 62/739,051 filed Sep. 28, 2018.

This disclosure relates to the field of digital signal processing systems for audio signals produced by microphones in acoustic environments; and more specifically, to processing systems designed to adjust the tonal balance of a loudspeaker in a room or other acoustic space it is placed in, to improve a listeners experience. Other aspects are also described.

The sound quality of loudspeakers (as perceived by a listener) is known to be affected by the room or other acoustic space or environment (e.g., vehicle cabin) in which they are placed. A reverberant room will cause the level of a certain frequency band (depending on the acoustic characteristics of the room) to increase in such a way that timbral character is deteriorated.

In accordance with various aspects of the disclosure here, digital equalization or spectral shaping is performed by an equalization filter, upon an audio signal that is driving a loudspeaker that is in a loudspeaker enclosure or cabinet. The spectral shaping may be able to compensate for deleterious effects of the acoustic environment. The effect of the acoustic environment on reverberation of the sound from the loudspeaker is measured and on that basis the equalization filter is determined. In particular, a sound measurement is made in the environment that is not at a usual listener's location in the environment. Rather, the measurement is made using one or more microphones that are integrated into the loudspeaker cabinet. In this manner, a neutral or more balanced frequency response is delivered by the loudspeaker which may be more pleasing to a listener, where this effect can adapt automatically to the ambient environment of the loudspeaker cabinet. For example, consider a smart speaker that has been placed in a reverberant bathroom. In a typical case, the smart speaker would sound louder and perhaps a little harsher than when it was in a furnished living room; the disclosed system would automatically adjust the tonal balance to make the sound less harsh and not appear unduly loud in that case. This process may be viewed as “automatic” in that no specific user intervention is required.

The above summary does not include an exhaustive list of all aspects of the present invention. It is contemplated that the invention includes all systems and methods that can be practiced from all suitable combinations of the various aspects summarized above, as well as those disclosed in the Detailed Description below and particularly pointed out in the claims filed with the application. Such combinations have particular advantages not specifically recited in the above summary.

Several aspects of the disclosure here are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” or “one” aspect in this disclosure are not necessarily to the same aspect, and they mean at least one. Also, in the interest of conciseness and reducing the total number of figures, a given figure may be used to illustrate the features of more than one aspect of the disclosure, and not all elements in the figure may be required for a given aspect.

FIG. 1 is a block diagram of an audio system that generates an equalization filter for filtering an audio signal that is driving a loudspeaker.

FIG. 2 is a block diagram of a beamforming audio system with equalization filters.

FIG. 3 shows how an EQ filter can be determined using a loudspeaker enclosure and a microphone that is outside of the loudspeaker enclosure.

FIG. 4 illustrates a plot of reverberant sound field measurement versus a parameter that indicates the room gain, at three different distances from the loudspeaker enclosure.

In the following description, numerous details are set forth. However, it is understood that aspects of the disclosure here may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure a rapid understanding of this description.

As used herein, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms “comprises” and “comprising” specify the presence of stated features, acts, operations, elements, or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, or groups thereof.

The terms “or” and “and/or” as used herein are to be interpreted as inclusive or meaning any one or any combination. Therefore, “A, B or C” or “A, B and/or C” mean any of the following: A; B; C; A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, functions, steps or acts are in some way inherently mutually exclusive.

FIG. 1 is a block diagram of one aspect of the disclosure here, as a digital audio system having a filter generator 2 and associated memory (not shown) having stored therein instructions that when executed by the processor perform the following operations that may enhance human listening experience during playback of an audio signal. Playback is through a loudspeaker 4 that is integrated in a loudspeaker enclosure 6 (cabinet), that can for example be part of a smart speaker, a laptop computer, or a tablet computer. An audio signal that may originate from a variety of different sources, e.g., a movie, music or podcast file streaming directly from a remote server or via a network appliance media player, a telephony communications downlink signal, a locally stored audio file, etc. is fed through an audio signal enhancement 8 , e.g., noise reduction, dynamic range control, loudness normalization, automatic gain control, in addition to an equalization EQ filter 9 , before driving the loudspeaker 4 through a power amplifier, PA. There may be multiple components in the audio signal and in that case each could be output through its respective audio signal processing and loudspeaker chain shown in the figure (in the case where there is more than one loudspeaker 4 ). There is also a microphone 7 integrated in the enclosure 6 . The microphone 7 is arranged and designed to pick up sound in the ambient environment outside of the enclosure 6 . There may be additional loudspeakers and microphones as depicted by the dotted lines, which may be used in various aspects of the equalization schemes described here.

It should be noted that depending on the particular consumer electronic product in which the aspects described here are being implemented, the digital signal processing operations described may be performed by one or more microprocessors or equivalents which are generically referred to here as “a processor”, executing instructions that are stored in various types of digital storage (referred to generically here as “memory”). In fact, in one instance, the audio signal enhancement 8 , the EQ filter 9 and the EQ filter generator 2 may be implemented by the processor 2 executing instructions stored in its associated memory. In other instances, certain operations may be performed by dedicated digital logic circuits, e.g. for faster response to achieve real-time adjustments in the EQ filter 9 , or they may be off-loaded to a different microprocessor for example in a remote server in the case of compute-intensive signal processing tasks. Also, in one instance, all of the elements shown in FIG. 1 are implemented inside the loudspeaker enclosure 6 (e.g., as a smart speaker, a laptop computer, a smartphone, or a tablet computer), while in other instances the filter generator 2 could be implemented in a separate device such as a laptop or desktop computer and could for example send its control output signal to adjust the EQ filter 9 over a wireless communication link with a smart speaker.

The filter generator 2 computes an impulse response or equivalently a transfer function, between i) an audio signal that is being output as sound by the loudspeaker 4 , and ii) a microphone signal from the microphone 7 that is recording the output by the loudspeaker 4 . The stimulus audio signal may be a test tone (e.g., as part of sine sweep) or it may be user program audio signal containing for example music. The impulse response may be computed using for example an echo canceller that estimates the impulse response in real-time.

The filter generator 2 analyzes this measured impulse response to extract a reverberation level at each of a number of frequency bands of interest (e.g., frequency bins), to yield a reverberation spectrum P_rev0(f). This may be done by extrapolating the slope sound decay (decay curve) back to the beginning of the impulse response, while ignoring the direct sound and early reflections that are also present in the impulse response. The reverberation spectrum P_rev0(f) is obtained by collecting the extracted reverberation levels of the different frequency bands.

Next, a reverberation spectrum P_rev(f, r) at a listening distance r from the loudspeaker 4 is estimated, based on the reverberation spectrum P_rev0(f). This may be based on knowledge of attenuation of sound in a room, over distance. The following assumptions may be made for making this estimation. In a perfectly diffuse sound field, the reverberant sound field does not change as a function of distance in the room, and so P_rev(f,r)=P_rev0(f). Here, an empirical attenuation can be chosen that represents a central tendency of a population of typical rooms, e.g., an average. For instance, let P_rev(f,r)=P_rev0(f)/sqrt(r). Or, more generically, one can write

P _rev( f,r )= a*P _rev0( f )/ r{circumflex over ( )}b   (eq. 1)

where a and b are estimated from a population of typical rooms. Note here that the parameters a and b may be further tuned based on knowledge of the room type (e.g., bathroom vs. living room vs. bed room vs. kitchen vs. garage) and/or based on distance between the loudspeaker and nearby acoustic boundaries (e.g., floor, walls, book, table top). It has been discovered that the reverberant sound field decreases more steeply as a function of distance if the loudspeaker is close to a corner of the room, whereas it does not decrease as much (as a function of distance) if the loudspeaker is in the middle of the room.

Next, the sound power spectrum at the listening distance is estimated, based on the estimated reverberation at the listening distance r. For example, the total sound power spectrum at a given distance r from the loudspeaker 4 can be estimated (reconstructed) by combining i) the direct sound (which may be based on a known on-axis response of the loudspeaker 4 ) and ii) the reverberant sound estimated above, using the following equation:

And finally, the EQ filter 9 is determined (e.g., its transfer function is computed, its digital filter coefficients are computed, or a table look up is performed to select one of several previously computed digital filters) based on i) the estimated sound power spectrum and ii) a desired frequency response at the listening distance r. For instance, the transfer function H_eq(f) of the EQ filter 9 may be calculated to satisfy the following equation

Sqrt( P _total( f ))* H _ eq ( f )= H _target( f )  (eq. 3)

where H_target(f) is the desired frequency response at the listening distance r (e.g., listener location).

So configured, the EQ filter 9 can then filter any user audio program signal for output by the loudspeaker 4 , in a way that is more acoustically pleasing for a user or listener in the present ambient environment of the enclosure 6 , at least near the listening distance r from the loudspeaker 4 .

Referring now to FIG. 2 , if the loudspeaker system is as shown there, able to produce a number of beamformer input signals (e.g., ambient audio content and direct audio content) for driving a loudspeaker array 10 to produce a number of output sound beams, respectively, with at least two different directivity indices, then each beamformer input signal may be filtered by a different instance of the EQ filter 9 . In particular, for the ambient audio content that will be reproduced in the two ambient content beams shown that are directed away from the listening position (in contrast to the direct content beam which may be aimed at the listening position), the above calculations for determining its respective EQ filter 9 a may be modified by omitting P_direct in eqs. 2 as the EQ filter 9 a in that instance only accounts for the diffuse sound field intended for the ambient audio signal.

While the above description refers to a microphone signal from the microphone 7 to compute P_rev, it is also possible to take multiple N>=2 microphone signals from N microphones, respectively, that are also integrated in the enclosure 6 , to compute N impulse responses, respectively. In that case, N reverberation spectra would be computed, and then a single reverberation spectrum P_rev may be derived, e.g., as an average of the N spectra.

In another aspect of the disclosure, at least two sound output beams (with different directivity indices and/or in different directions, e.g. as in FIG. 2 ) may be used to more robustly estimate a room gain property (which is a function of frequency). This room gain property may be denoted as C(f) and is independent of the loudspeaker directivity. Since the latter is known (for the known beams that are being produced), it can be accounted for in the following relation for total sound power:

P _total( f,r )= P _onaxis( f )*[1/ r{circumflex over ( )} 2+ C ( f,r )/ D ( f )]

where D(f) is the directivity gain of the loudspeaker beam.

In another instance, where there are several of the loudspeaker systems shown in FIG. 1 that are placed in the same room (several loudspeaker enclosures 6 ), where each one can compute its respective instance of the EQ filter 9 , the so-called self-measurements determined by each of these systems can be shared amongst them, e.g., over wireless communication links that connect them for example as part of a computer network. This enables for example comparisons to be made to verify the likelihood that an EQ filter determination is accurate, or an average of the several self-measurements can be used to compute the individual EQ filter 9 for each system.

In another instance, referring now to FIG. 3 where there are several of the loudspeaker systems of FIG. 1 placed in the same room, one of the loudspeaker enclosures can be used as a source (stimulus) and another can be used as a measurement device to measure the impulse response. In other words, a processor computes an impulse response between i) an audio signal that is being output as sound by a first loudspeaker that is integrated in a first loudspeaker enclosure 6 a , and ii) a microphone signal from a microphone that is recording the output by the first loudspeaker, wherein the microphone is separate from the first loudspeaker enclosure, e.g., it is integrated in another loudspeaker enclosure 6 b in the room or is otherwise located outside of the loudspeaker enclosure 6 b . In this way, the extrapolation operation described above that is part of analyzing the impulse response to extract therefrom the reverberation spectrum, is made easier since the direct sound from one loudspeaker enclosure that is arriving at the microphone which is now in another enclosure is not as dominant as in the single loudspeaker enclosure case illustrated in FIG. 1 where the microphone 7 is close to the loudspeaker.

It should also be noted that the EQ filter 9 as described above may be restricted to operate in a certain frequency range, e.g., affecting its input audio signal only at 1 kHz and above. It may also be combined with another spectral shaping (equalization) filter that operates at lower frequencies, e.g., below 1 kHz. Also, the processors determination of the EQ filter 9 may be updated or repeated, whenever the computed impulse response changes more than a threshold amount, and/or it may be computed during a setup phase, e.g., upon each power up event, or waking from a sleep state.

In yet another instance, the processor could apply the EQ filter 9 , to filter the user audio program signal being output by the loudspeaker 4 , in response to a user volume setting of the loudspeaker system being changed.

In still another instance, a broad-band room gain property (e.g. covering the entire range between 1 and 8 kHz) is computed and is then used to either 1) change the sound output gain (that is applied to the audio signal during playback), in such a way that the loudspeaker 4 outputs sound at the same level in different rooms, or 2) perform a more informed loudness compensation (e.g., using a Fletcher-Munson curve), by taking into account a corrected loudspeaker sensitivity (that includes the room gain). Although the broad-band room gain property (“room gain”) is estimated at higher frequencies (e.g. it is undefined below 1 kHz, and defined between 1 kHz to 8 kHz or perhaps even higher), it has been determined that in most rooms the frequency-dependency of the room gain is not strong. This suggests that the broad-band room gain would also be valid when applied to the lower frequency range of the audio signal that is driving the loudspeaker, e.g., lower than 1 kHz.

Note that one reason why the measurement of the broad band room gain may have a high pass characteristic is that it is easier to remove the near-field effects of reflections at high frequencies. Near-field low-frequency measurements do not translate well to the estimation of the far-field room-gain.

Applying such loudness compensation (using the computed broad band room gain) results in a more appropriate spectral balance. For instance, in a bathroom the room gain could be around 10 dB; the loudness compensation will in that case result in a bass cut of around 4 dB. These numbers of course are just one example of the loudness compensation described here.

The room gain may be computed as follows. Note that this process which also estimates a reverberation level is less complex than the one mentioned above (the decay curve analysis.) In such a process, the direct sound and early reflections in the impulse response are windowed out (e.g., the first ten milliseconds are cut out.) This is then band-pass or high pass filtered, e.g., as a second order Butterworth-type high pass filter having a cutoff at 400 Hz, and then the RMS level of the filtered signal is calculated. That RMS level represents the measured or estimated level of the reverberant sound field observed by the device, also referred to here as Lrev0. A mapping is then performed using a predetermined relationship that relates reverberant sound field levels to predicted room gains (spectra), at a given distance in the room from the loudspeaker 4 . One solution is to use equations 1 and 2, where Prev0 is equivalent to 10{circumflex over ( )}(Lrev0/10). Another solution is to use a pre-calculated mapping curve. For example, FIG. 4 illustrates a mapping of reverberant sound field measurements Lrev0 to the estimated far-field level Lfield, at three different distances from the loudspeaker enclosure (in this example, 1 meter, 2 meters and 3 meters.) If the actual listening distance is not known or cannot be estimated reliably, then a default distance may be selected, e.g., 2 meters, and its associated mapping curve is selected. Room gain is then calculated as the difference (in dB) between Lfield and a reference level (which could come from a measurement for a specific loudspeaker system at an ideal listening distance in a reference room.)

While certain exemplary instances have been described and shown in the accompanying drawings, it is to be understood that these are merely illustrative of and not restrictive on the broad invention, and that this invention is not limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those of ordinary skill in the art. For example, the listening distance r may be entered manually by a user, or it may be estimated by the processor using proximity sensing, voice analysis, or camera image analysis, or it may be set to a default fixed value, e.g., three meters. The description is thus to be regarded as illustrative instead of limiting.