Cue-aware privacy filter for participants in persistent communications

The present application is related to and claims the benefit of earliest available effective filing date(s) from the following listed application(s) (the “Related Applications”) (e.g., claims earliest available priority dates for other than provisional patent applications; claims benefits under 35 USC §119(e) for provisional patent applications), and incorporates by reference in its entirety all subject matter of the following listed application(s); the present application also claims the earliest available effective filing date(s) from, and also incorporates by reference in its entirety all subject matter of any and all parent, grandparent, great-grandparent, etc. applications of the Related Application(s) to the extent such subject matter is not inconsistent herewith:

1. For purposes of the USPTO extra-statutory requirements, the present application constitutes a continuation-in-part of United States Patent Application entitled, CUE-AWARE PRIVACY FILTER FOR PARTICIPANTS IN PERSISTENT COMMUNICATIONS naming Edward K.Y. Jung; Royce A. Levien; Mark A. Malamud; John D. Rinaldo, Jr.; and Paul G. Allen as inventors, filed Jul. 30, 2004, application Ser. No. 10/909,962, which is currently co-pending, or is an application of which a currently co-pending application is entitled to the benefit of the filing date.

The United States Patent Office (USPTO) has published a notice to the effect that the USPTO's computer programs require that patent applicants reference both a serial number and indicate whether an application is a continuation or continuation-in-part. Stephen G. Kunin, Benefit of Prior - Filed Application , USPTO Official Gazette Mar. 18, 2003, available at http://www.uspto.gov/web/offices/com/sol/og/2003/week11/patbene.htm. The present Applicant has provided above a specific reference to the application(s)from which priority is being claimed as recited by statute. Applicant understands that the statute is unambiguous in its specific reference language and does not require either a serial number or any characterization, such as “continuation” or “continuation-in-part,” for claiming priority to U.S. patent applications. Notwithstanding the foregoing, Applicant understands that the USPTO's computer programs have certain data entry requirements, and hence Applicant is designating the present application as a continuation of its parent applications as set forth above, but expressly points out that such designations are not to be construed in any way as any type of commentary and/or admission as to whether or not the present application contains any new matter in addition to the matter of its parent application(s).

All subject matter of the Related Application and of any and all parent, grandparent, great-grandparent, etc. applications of the Related Applications is incorporated herein by reference to the extent such subject matter is not inconsistent herewith.

The present disclosure relates to inter-device communication.

Modern communication devices are growing increasingly complex. Devices such as cell phones and laptop computers now often are equipped with cameras, microphones, and other sensors. Depending on the context of a communication (e.g. where the person using the device is located and to whom they are communicating, the date and time of day, among possible factors), it may not always be advantageous to communicate information collected by the device in its entirety, and/or unaltered.

The following summary is intended to highlight and introduce some aspects of the disclosed embodiments, but not to limit the scope of the invention. Thereafter, a detailed description of illustrated embodiments is presented, which will permit one skilled in the relevant art to make and use aspects of the invention. One skilled in the relevant art can obtain a full appreciation of aspects of the invention from the subsequent detailed description, read together with the figures, and from the claims (which follow the detailed description).

A device communication is filtered according to an identified cue. The cue can include at least one of a facial expression, a hand gesture, or some other body movement. The cue can also include at least one of opening or closing a device, deforming a flexible surface of the device, altering an orientation of the device with respect to one or more objects of the environment, or sweeping a sensor of the device across the position of at least one object of the environment. Filtering may also take place according to identified aspects of a remote environment.

Filtering the device communication can include, when the device communication includes images/video, at least one of including a visual or audio effect in the device communication, such as blurring, de-saturating, color modification of, or snowing of one or more images communicated from the device. When the device communication includes audio, filtering the device communication comprises at least one of altering the tone of, altering the pitch of, altering the volume of, adding echo to, or adding reverb to audio information communicated from the device.

Filtering the device communication may include substituting image information of the device communication with predefined image information, such as substituting a background of a present location with a background of a different location. Filtering can also include substituting audio information of the device communication with predefined audio information, such as substituting at least one of a human voice or functional sound detected by the device with a different human voice or functional sound.

Filtering may also include removing information from the device communication, such as suppressing background sound information of the device communication, suppressing background image information of the device communication, removing a person's voice information from the device communication, removing an object from the background information of the device communication, and removing the image background from the device communication.

The headings provided herein are for convenience only and do not necessarily affect the scope or meaning of the claimed invention.

In the drawings, the same reference numbers and acronyms identify elements or acts with the same or similar functionality for ease of understanding and convenience. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced.

FIG. 1 is a block diagram of an embodiment of a device communication arrangement.

FIG. 2 is a block diagram of an embodiment of an arrangement to produce filtered device communications.

FIG. 3 is a block diagram of another embodiment of a device communication arrangement.

FIG. 4 is a flow chart of an embodiment of a method of filtering device communications according to a cue.

FIG. 5 is a flow chart of an embodiment of a method of filtering device communications according to a cue and a remote environment.

The invention will now be described with respect to various embodiments. The following description provides specific details for a thorough understanding of, and enabling description for, these embodiments of the invention. However, one skilled in the art will understand that the invention may be practiced without these details. In other instances, well known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the invention. References to “one embodiment” or “an embodiment” do not necessarily refer to the same embodiment, although they may.

FIG. 1 is a block diagram of an embodiment of a device communication arrangement. A wireless device 102 comprises logic 118 , a video/image sensor 104 , an audio sensor 106 , and a tactile/motion sensor 105 . A video/image sensor (such as 104 ) comprises a transducer that converts light signals (e.g. a form of electromagnetic radiation) to electrical, optical, or other signals suitable for manipulation by logic. Once converted, these signals may be known as images or a video stream. An audio sensor (such as 106 ) comprises a transducer that converts sound waves (e.g. audio signals in their original form) to electrical, optical, or other signals suitable for manipulation by logic. Once converted, these signals may be known as an audio stream. A tactile/motion sensor (such as 105 ) comprises a transducer that converts contact events with the sensor, and/or motion of the sensor, to electrical, optical, or other signals suitable for manipulation by logic. Logic (such as 116 , 118 , and 120 ) comprises information represented in device memory that may be applied to affect the operation of a device. Software and firmware are examples of logic. Logic may also be embodied in circuits, and/or combinations of software and circuits.

The wireless device 102 communicates with a network 108 , which comprises logic 120 . As used herein, a network (such as 108 ) is comprised of a collection of devices that facilitate communication between other devices. The devices that communicate via a network may be referred to as network clients. A receiver 110 comprises a video/image display 112 , a speaker 114 , and logic 116 . A speaker (such as 114 ) comprises a transducer that converts signals from a device (typically optical and/or electrical signals) to sound waves. A video/image display (such as 112 ) comprises a device to display information in the form of light signals. Examples are monitors, flat panels, liquid crystal devices, light emitting diodes, and televisions. The receiver 110 communicates with the network 108 . Using the network 108 , the wireless device 102 and the receiver 110 may communicate.

The device 102 or the network 108 identify a cue, either by using their logic or by receiving a cue identification from the device 102 user. Device 102 communication is filtered, either by the device 102 or the network 108 , according to the cue. Cues can comprise conditions that occur in the local environment of the device 102 , such as body movements, for example a facial expression or a hand gesture. Many more conditions or occurrences in the local environment can potentially be cues. Examples include opening or closing the device (e.g. opening or closing a phone), the deforming of a flexible surface of the device 102 , altering of the device 102 orientation with respect to one or more objects of the environment, or sweeping a sensor of the device 102 across at least one object of the environment. The device 102 , or user, or network 108 may identify a cue in the remote environment. The device 102 and/or network 108 may filter the device communication according to the cue and the remote environment. The local environment comprises those people, things, sounds, and other phenomenon that affect the sensors of the device 102 . In the context of this figure, the remote environment comprises those people, things, sounds, and other signals, conditions or items that affect the sensors of or are otherwise important in the context of the receiver 110 .

The device 102 or network 108 may monitor an audio stream, which forms at least part of the communication of the device 102 , for at least one pattern (the cue). A pattern is a particular configuration of information to which other information, in this case the audio stream, may be compared. When the at least one pattern is detected in the audio stream, the device 102 communication is filtered in a manner associated with the pattern. Detecting a pattern can include detecting a specific sound. Detecting the pattern can include detecting at least one characteristic of an audio stream, for example, detecting whether the audio stream is subject to copyright protection.

The device 102 or network 108 may monitor a video stream, which forms at least part of a communication of the device 102 , for at least one pattern (the cue). When the at least one pattern is detected in the video stream, the device 102 communication is filtered in a manner associated with the pattern. Detecting the pattern can include detecting a specific image. Detecting the pattern can include detecting at least one characteristic of the video stream, for example, detecting whether the video stream is subject to copyright protection.

FIG. 2 is a block diagram of an embodiment of an arrangement to produce filtered device communications. Cue definitions 202 comprise hand gestures, head movements, and facial expressions. In the context of this figure, the remote environment information 204 comprise a supervisor, spouse, and associates. The filter rules 206 define operations to apply to the device communications and the conditions under which those operations are to be applied. The filter rules 206 in conjunction with at least one of the cue definitions 202 are applied to the local environment information to produce filtered device communications. Optionally, a remote environment definition 204 may be applied to the filter rules 206 , to determine at least in part the filter rules 206 applied to the local environment information.

Filtering can include modifying the device communication to incorporate a visual or audio effect. Examples of visual effects include blurring, de-saturating, color modification of, or snowing of one or more images communicated from the device. Examples of audio effects include altering the tone of, altering the pitch of, altering the volume of, adding echo to, or adding reverb to audio information communicated from the device.

Filtering can include removing (e.g. suppressing) or substituting (e.g. replacing) information from the device communication. Examples of information that may suppressed as a result of filtering include the background sounds, the background image, a background video, a person's voice, and the image and/or sounds associated with an object within the image or video background. Examples of information that may be replaced as a result of filtering include background sound information which is replaced with potentially different sound information and background video information which is replaced with potentially different video information. Multiple filtering operations may occur; for example, background audio and video may both be suppressed by filtering. Filtering can also result in application of one or more effects and removal of part of the communication information and substitution of part of the communication information.

FIG. 3 is a block diagram of another embodiment of a device communication arrangement. The substitution objects 304 comprise office, bus, and office sounds. The substitution objects 304 are applied to the substitution rules 308 along with the cue definitions 202 and, optionally, the remote environment information 204 . Accordingly, the substitution rules 308 produce a substitution determination for the device communication. The substitution determination may result in filtering.

Filtering can include substituting image information of the device communication with predefined image information. An example of image information substitution is the substituting a background of a present location with a background of a different location, e.g. substituting the office background for the local environment background when the local environment is a bar.

Filtering can include substituting audio information of the device communication with predefined audio information. An example of audio information substitution is the substituting at least one of a human voice or functional sound detected by the device with a different human voice or functional sound, e.g. the substitution of bar background noise (the local environment background noise) with tasteful classical music.

FIG. 4 is a flow chart of an embodiment of a method of filtering device communications according to a cue. At 402 it is determined that there is a cue. If at 404 it is determined that no filter is associated with the cue, the process concludes. If at 404 it is determined that a filter is associated with the cue, the filter is applied to device communication at 408 . At 410 the process concludes.

FIG. 5 is a flow chart of an embodiment of a method of filtering device communications according to a cue and a remote environment. At 502 it is determined that there is a cue. At 504 at least one aspect of the remote environment is determined. If at 506 it is determined that no filter is associated with the cue and with at least one remote environment aspect, the process concludes. If at 506 it is determined that a filter is associated with the cue and with at least one remote environment aspect, the filter is applied to device communication at 508 . At 510 the process concludes.

Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to.” Words using the singular or plural number also include the plural or singular number respectively. Additionally, the words “herein,” “above,” “below” and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application. When the claims use the word “or” in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.