Method and apparatus for detecting data

The present invention relates to methods and apparatus for detecting the presence or absence of a block of data which may have been embedded repeatedly in material. Correspondingly, the present invention also relates to data embedding apparatus and methods for embedding data in material.

Material as used herein refers to and includes any form of information material such as, for example, video material, audio material and material for conveying data.

A watermark is a term generally used to describe data or information, which has been embedded in material in some way. A watermark may be imperceptible or perceptible in the material.

A watermark may be used for various purposes. It is known to use watermarks for the purpose of protecting the material against, or trace, infringement of the intellectual property rights of the owner(s) of the material. For example, a watermark may identify the owner of the material. The owner is thereby provided with a facility for proving unauthorised copying of the material. A system for introducing and detecting a watermark embedded in material can therefore provide a facility for copyright protection, so that the copyright in the material can be asserted.

There are generally three statistics which are used to measure the performance of copyright protection systems. These are:

1. False positive: This is the probability of deciding that a part of the material is watermarked, even though it is actually un-watermarked. For video material the European Broadcasting Union (EBU) requires the probability of a false positive to be at most 10 −8 .

2. False negative: This is the probability of deciding that a part of the material is not watermarked, even though it is actually watermarked. The EBU requires the probability of a false negative to be at most 0.05.

3. Error free payload: This is the probability of finishing with an error free payload (after error-correction) if the watermark has been detected. The EBU requires the probability of having errors in the decoded payload to be at most 10 −8 .

International patent application number WO 99/12347 discloses an apparatus for detecting a watermark embedded in material. The apparatus is provided with a copy of the watermark to be detected. The apparatus correlates the copy of the watermark with respect to data recovered from the material. From the correlation the watermark is declared as being present or absent.

According to the present invention there is provided an apparatus for detecting the presence or absence of a data block of data bits, which may have been embedded repeatedly a predetermined number of times in material, the presence or absence of said data block being detected from data representative of versions of the data block recovered from said material, the apparatus comprising an accumulator operable to combine the value of the bit at each corresponding position within each recovered version of the data block to generate for each bit an accumulated score, and a detection processor operable to compare the value of the accumulated scores for the bits of the block with at least one threshold, and from the comparison to determine whether each of the bits of the block has been detected, and to declare the data block as being present or absent in dependence upon a total number of detected bits.

Embodiments of the present invention can provide an improvement in the likelihood of false detection and the false negative detection. The improvement can be provided to a watermarking system without adaptation of the system or at least requiring only a minimum adaptation of the system. Embodiments of the invention can be used, to provide an improvement in copyright protection. The copyright protection is generally performed at a higher level to the watermarking system, so that by providing an additional copyright protection layer on top of an existing watermarking layer, copyright protection of material can be facilitated.

The present invention utilises a probability distribution of the value of each bit of the data block, which has been embedded repeatedly in material. By adding the value of the same bit in each version of the data block recovered from the material, an accumulated score is formed for each bit. In one embodiment the accumulated score is compared with two threshold values, which define three regions for possible accumulated scores between zero and a maximum corresponding to the predetermined number of times the same data block has been embedded in the material. The first region, below a first of the thresholds, defines values, which are associated with the bit being detected as a zero. The second region formed between the value of the two thresholds defines values for which the bit is declared as not being detected. The third region, above a second of the two thresholds defines values which are associated with the bit being detected as a one. The detection processor then determines how many of the bits in the data block have been detected as zeros or ones. By comparing the number of detected bits of the block with a third threshold, a data block can be declared as being present in or absent from the material.

Comparing an accumulated score for each bit with respect to, for example, first and second thresholds to determine whether or not this bit has been detected, in combination with comparing the number of detected bits of the block with respect to a third threshold, forms a two-stage detection process. The two-stage process has been found to provide improved false positive and false negative detection probabilities, which addresses the EBU requirements indicated above.

Furthermore an advantage is provided because the detection process does not require knowledge of the content of the embedded data. Unlike the watermark detection system disclosed in WO 99/12347 referred to above, a detector for the watermark does not require a priori knowledge of the watermark to determine the presence or absence of the watermark.

As will be appreciated an aspect of the present invention provides a data embedding apparatus for embedding data in material to the effect that the presence or absence of the data can be detected. The data embedding apparatus comprises a data formatter for arranging the data into at least one data block, and a unit repeating processor operable to receive the material and to identify units of the material. In combination with the embedding processor, the repeating processor embeds the data block repeatedly in each of the units for a plurality of the units in accordance with a predetermined number of times the data block is repeatedly embedded.

In preferred embodiments, the embedding apparatus includes a shuffling processor coupled to the data formatter and to the unit repeating processor and operable to shuffle the order of the bits of the block to be embedded in the material. Shuffling the bits of the data block has the effect of ensuring the statistical independence of the decoded versions of each payload bit. This provides an advantage to the copyright protection layer, because the independence of the decoded versions of the payload bits reduces the likelihood of the payload being incorrectly detected as being present. Shuffling also provides an advantage of reducing the likelihood of fixed pattern noise, which may be induced as a result of embedding the data.

In order to meet the EBU requirements for providing as far as possible an error free payload (or at least a payload having a small number of errors), the embedding apparatus may include an error correction encoder. The error correction encoder may be coupled to the data formatter and operable to encode the data formed by the data formatter to produce an error correction encoded data block before the data block is embedded in the material.

As will be appreciated, correspondingly the apparatus for detecting the presence or absence of the data block may include a de-shuffling processor operable to change the order of the bits of the data blocks recovered from each unit to the effect of reversing the order of the predetermined shuffling pattern. The apparatus for detecting the data block may also include an error correction decoding processor.

In some embodiments the material may be video material and the units may be image frames represented by the video material. The data block may be formed from meta data describing the content of the material such as, for example, a Universal Material Identifier or the like. In other embodiments the data block may be an owner unique or at least specific identifier.

According to a further aspect of the present invention there is provided a copyright protection system for detecting the presence or absence of data embedded in material, the system comprising a de-embedding processor operable to recover data representative of each version of the data block from the material, and an apparatus for detecting the data block from the recovered versions as claimed in claims 1 to 12 .

In some circumstances, data which has not been embedded in material may be falsely recovered as an all zeros or all ones sequence. In preferred embodiments, in order to address a technical problem of correctly detecting an all zeros or all ones sequence as not present in material, the data to be embedded may be combined with a Pseudo Random Bit Sequence (PRBS), or part thereof, before the data is embedded. To remove the effect of the PRBS, in preferred embodiments, the de-embedding processor may be operable to combine the data representative of the recovered versions of the data block with the predetermined PRBS to the effect of recovering the versions of the data blocks. For example the combination of the data with the PRBS may be provided by logically XOR-ing the data and the PRBS. Therefore combining the recovered data again with the PRBS will remove the effect of the PRBS from the embedded data.

Data recovered from the material which would cause the detection processor to indicate falsely that the data was embedded as an all ones or zeros sequence will appear as random data according to the PRBS and so will have a reduced likelihood of being falsely detected.

Various further aspects and features of the present invention are defined in the appended claims.

Embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings wherein:

FIG. 1 is a schematic block diagram of a copyright protection system according to an example embodiment of the present invention;

FIG. 2 is a schematic block diagram of the copyright protection system of FIG. 1 , showing the data embedding processor which appears in FIG. 1 in more detail;

FIG. 3 provides seven schematic representation of the versions of the data block at different stages ST. 1 , ST. 2 , ST. 3 , ST. 4 , ST. 5 , ST. 5 . 1 , ST. 6 , ST. 7 as produced by the data embedding processor and the data detecting processor;

FIG. 4 is a schematic block diagram of an apparatus for detecting the presence or absence of a data block appearing in FIG. 1 ;

FIG. 5 provides graphical representations of the probability distributions of the accumulated score for an embedded bit being a zero, a one and detecting a bit when a bit is not present in the material, as formed by the detecting apparatus shown in FIG. 4 ;

FIG. 6 provides graphical representations of the probability distributions of the number of detected bits for watermarked and not watermarked material;

FIG. 7 is a table providing the false positive, false negative and errors in the decoded payload for the copyright protection system according to the example embodiment in comparison to the EBU requirements; and

FIG. 8 is a table illustrating the relative performance of the copyright protection system according to the example embodiment of the invention after the material has undergone different forms of compression.

Although it will be appreciated that the material could be audio material, video material, audio/video material or data in any form, in the example embodiment which will now be described, the material is video material. In the following description it will be assumed that a watermark is formed in video material which comprises a plurality of frames. However, it will be appreciated that this is just an illustrative example and that the data may be embedded in accordance with any unit of the video material, such as for example a field of the video material.

FIG. 1 provides a general representation illustrating a process in which data is embedded into video material. The video material is divided into image frames I. The data embedded into these image frames I forms a watermark. The data is embedded by a watermarking layer WM. As shown in FIG. 1 , a copyright protection processing layer CP provides a facility for forming a payload of data into a watermark for embedding into material and for detecting whether the payload data has been embedded in the material. An improvement in the probability of false positive and false negative detection is provided by the copyright protection processing layer CP.

As shown in FIG. 1 , the image frames I are received within the watermarking layer WM by a data embedder 2 . The data embedder 2 is operable to embed data received from an encoding processor 4 from the copyright protection layer CP. Embedding may be performed in various ways. One example arrangement for embedding data is disclosed in our co-pending patent application serial number 0029854.7 and UK patent application serial number 0029856.2. In these co-pending patent applications, a watermarking system for embedding data in material is disclosed. The data is embedded by modulating a pseudo random bit sequence and adding the modulated bit sequence to a transform domain version of the data.

The encoding processor 4 is arranged to process the data to be embedded within the images I in order to effect the improvement in the false positive and false negative detection probabilities. The data embedder 2 produces watermarked images I′ which are then further processed. The further processing might be for example that the watermarked images are communicated, recorded, broadcast or reproduced in some way.

In order to recover the data from the watermarked images I′, a data de-embedder 6 is provided which is arranged to recover the data from the image. The embedded data recovered from the image is then fed to the decoding processor 10 , via a connecting channel 7 , which processes the recovered data in order to improve the false positive and false negative detection probabilities. The encoding processor 4 which forms part of the copyright processing layer CP is illustrated in FIG. 2 where parts also appearing in FIG. 1 have the same numerical references.

In FIG. 2 the encoding processor 4 comprises a data formatter 18 for forming the data blocks DB of the payload PAY and an error correction encoder 20 which is operable to encode data received from the data input channel 5 in accordance with an error correction code. For the illustrative example embodiment the encoder operates to encode 64 bits of data which are referred to in the following description as the pay load. The pay load could be for example part of an UMID or other types of meta data which describe the content of the video material. In other embodiments the pay load could represent an identifier specific to the copyright owner or provider. This identifier can therefore be used by the owner or provider to conform ownership and prove copying of the video material.

In accordance with the example embodiment the encoder 20 operates to encode the data using a BCH code to generate 127 error correction bits from the 64 payload bits, which can correct up to 10 bits.

Processing performed by the encoding data processor 4 is illustrated by diagrams labelled ST. 1 , ST. 2 , ST. 3 , ST. 4 , ST. 5 , ST. 5 . 1 , ST. 6 and ST. 7 in FIG. 3 . Each of the diagrams ST. 1 , ST. 2 , ST. 3 , ST. 4 , ST. 5 , ST. 5 . 1 , ST. 6 and ST. 7 represents a form of the 64-bit pay load after processing performed by parts of the encoding processor 4 and the decoding processor 10 .

The payload of 64-bits PAY are represented in diagram ST. 1 in FIG. 3 . The 64-bit payload is received by the encoder 20 of FIG. 2 and encoded using the BCH code. The encoded bits produced by the encoder 20 , presented at the output of the encoder 20 , are represented in diagram ST. 2 in FIG. 3 as 127 error correction encoded bits ENC. The error correction encoded bits ENC are fed to a data expander 30 which adds extra bits to the encoded data word to produce 130 bits as formed at an output of the expander 30 . The data word at the output of the expander EXP is represented as diagram ST. 3 .

The 130 bits EXP from the output of the expander 30 are then fed to a cyclic shuffling processor 40 . The cyclic shuffling processor 40 divides the 130 expanded bits into four blocks of 30 bits and one of 10 bits. Correspondingly this is represented at a first step as diagram ST. 4 in FIG. 3 as the data blocks B 1 , B 2 , B 3 , B 4 , B 5 .

The cyclic shuffling processor 40 is operable to shuffle the bits within each block B 1 , B 2 , B 3 , B 4 , B 5 of the processed bits represented in diagram ST. 4 in FIG. 3 . The shuffling processor 40 shuffles the bits within each block B 1 , B 2 , B 3 , B 4 , B 5 in accordance with a predetermined pattern which repeats cyclically after a predetermined number of frames.

For the present example embodiment it is assumed that the copyright protection processing and embedding is performed on a cycle of 30 frames. As such, the same data is embedded into each of the 30 frames as will be explained shortly. Accordingly, each of the blocks of 30 bits and 10 bits is shuffled so that in each frame the bits are in a different positions. The shuffling is represented as lines SHUF between the un-shuffled blocks B 1 , B 2 , B 3 , B 4 , B 5 of diagram ST. 4 and the shuffled blocks B 1 ′, B 2 ′, B 3 ′, B 4 ′, B 5 ′ of diagram ST. 5 . Shuffling the bits of the data block has the effect of ensuring the statistical independence of the decoded versions of each payload bit. This provides an advantage to the copyright protection layer, because the independence of the decoded versions of the payload bits reduces the likelihood of the payload being incorrectly detected as being present. Shuffling also provides an advantage of reducing the likelihood of fixed pattern noise, which may be induced as a result of embedding the data.

After the 130 bits have been shuffled by the cyclic shuffling processor 40 the bits are received at a frame processor 50 . The frame processor 50 operates to generate replicas of each of the shuffled data blocks CYC_SHUF for repeated embedding of each data block into each frame. For the present example embodiment, the same shuffled data block CYC_SHUF is embedded four times into each frame. The frame processor 50 therefore serves to generate four replicas of the data block for repeated embedding in each frame.

In the present example embodiment the 130-bit block is divided into sub-blocks. The block of shuffled bits CYC_SHUF comprising the shuffled sub-blocks B 1 ′, B 2 ′, B 3 ′, B 4 ′, B 5 ′ are embedded four times into each frame as represented by step ST. 5 . 1 . Furthermore, as indicated each shuffled block is embedded both repeatedly within a frame and repeatedly in a different shuffled form throughout a number of frames. For the illustrative embodiment the number of frames in which the same shuffled data blocks B 1 ′, B 2 ′, B 3 ′, B 4 ′, B 5 ′ are repeated is thirty although it will be understood that any number of frames could be used. To provide a better understanding of the present invention it is assumed that the 130 encoded and shuffled pay load bits are simply repeated four times in each frame, for each of 30 frames.

Returning to FIG. 1 the embedded data is recovered from the watermarked image data I′ by the data de-embedder 6 . The de-embedder 6 recovers versions of the data blocks from the image data I′. The recovered data blocks are fed to the decoding processor 10 , via the connecting channel 7 . The decoding processor 10 is shown in more detail in FIG. 4 where parts also appearing in FIG. 1 bear the same numerical designations.

In FIG. 4 each of the image frames I′ is received by the de-embedding processor 6 which recovers the data embedded in each frame. The data which has been recovered from each frame is fed to a first accumulator 100 in the decoding processor 10 .

The watermark de-embedding processor 6 is operable to recover each of four versions of the 130 embedded bits from each frame. The first accumulator 100 accumulates the value of the bit in each corresponding position in the 130 bit block by adding these values to form an accumulated score from the four recovered versions. This is represented in FIG. 3 by step ST. 6 . The accumulated data block recovered from each frame is then fed to a de-shuffling processor 120 which is operable to reverse the shuffling process performed by the shuffling processor 40 shown in FIG. 2 . The effect of the de-shuffling processor 120 is to produce for each frame a de-shuffled version of the 130 bits. This is represented at step ST. 7 in FIG. 3 . Each version recovered from each of the thirty frames should therefore correspond with one another. Accordingly, a second accumulator 130 receives each accumulated and de-shuffled block and again combines the value of each bit in the corresponding position in each of the recovered versions of the 130 bit block by adding the bit value for each corresponding bit within the version of the block for each corresponding version recovered from each of the 30 frames.

The second accumulator 130 forms an estimate of the value of each bit within the 130 bit block by combining each of the (4×30=120) versions of the block which were embedded within the 30 frames. In combination with the first accumulator 100 , the value of each bit in the block is determined by summing the value of each bit at the corresponding bit position in each of the 120 versions formed by embedding the block four times within each of 30 frames repeatedly. Effectively, therefore summing each of the bits produces a score between 0 and 120 depending on whether the value of each bit is a ‘0’ or a ‘1’. Accordingly, all values between 0 and 120 are possible, with the value 60 corresponding to the case where exactly half the bits (for a given bit position in the encoded block) are 0 and a half are 1.

For each bit position in the 130-bit block, the second accumulator provides an accumulated score to a detection processor 135 . The detection processor 135 receives two thresholds S 1 , S 2 which are set to define a boundary between an accumulated bit being declared as a 1, and accumulated bit being declared as a 0. A region formed between the two thresholds S 1 and S 2 corresponds to a bit being declared as not being present. This is represented graphically in FIG. 5 .

In FIG. 5 a probability distribution of the accumulated score for a given bit is shown for each of three cases. The probability distribution represents the probability of correctly recovering each bit from the watermarked image. The three cases are that the embedded bit in the block is actually a ‘0’ PD 1 , that no bit has actually been embedded, that is to say a probability distribution function (PDF) produced by random data PD 2 and the probability distribution if the encoded bit was a “1” PD 3 . The thresholds S 1 , S 2 are set so that, as far as possible, if a data bit has been embedded, then this is detected, thereby reducing the false negative detection. Similarly the thresholds S 1 , S 2 , are set to ensure that as far as possible a bit which has not been embedded is detected as not being present, thereby reducing false positive detection.

As shown in FIG. 5 , the first and second thresholds (S 1 and S 2 ) may be set with respect to an integer variable S, which is set to a value less than N/2 (where N is the predetermined number of times a data block is repeatedly embedded—in this case 120), the first threshold being set in accordance with S, (S 1 =S57 in this example), and the second threshold being set in accordance with N−S, (S 2=N−S= 63 in this example).

As will be seen from FIG. 5 an advantage of repeatedly embedding the same bit within each frame, for each of a plurality of frames, is to produce a statistical distribution for decoding 0s and 1s which are easily separable. By setting the threshold S 1 and S 2 accordingly, the detection processor 135 can detect the presence of a 0 or 1 or declare a bit as not being detected.

Returning to FIG. 4 , the detection processor 135 is also arranged to receive a third threshold parameter T. Threshold parameter T is provided in order to determine whether or not the detection processor 135 declares that data has or has not been embedded within the video material. The detection processor 135 is arranged to count the number of 0s and 1s which have been declared as being detected. If this number is above the threshold T then it is declared that data has been embedded into the video material and that therefore this material has been watermarked. Correspondingly, if this number is below the threshold, then it is declared that data has not been detected. As will be appreciated a suitable output channel (not shown) can be provided to communicate a signal representative of this declaration. This process is represented graphically in FIG. 6 .

As can be seen in FIG. 6 for the present example embodiment the threshold T is set so that it is above 120. Setting the threshold T at 120 provides an advantage because the number of bits declared as not being detected within the 127 bit encoded block (ignoring the expanded bits) is well within the 10 bit error correction capacity of the BCH code. The payload can therefore be recovered without these 10 bits being detected (140). As illustrated in FIG. 6 , a graphical representation of a probability distribution for watermarked material exhibits a sharp peak about 127-bits, which illustrates that the copyright protection layer according to the example embodiment of the invention provides a robust false negative protection. Also shown FIG. 6 is a graphical representation of a probability distribution for a number of detected bits from material which has not been watermarked. A “hump” produced at around 65 detected bits corresponds to approximately half the bits being detected at being present at random. As will be appreciated, the low value of the probability of a number of detected bits in the range represented by the region of the “hump”, and the relative separation of this range from the detection threshold T provides a robust false positive protection provided by the example embodiment.

An illustration of the effectiveness of the copyright protection process is provided by a table shown in FIG. 7 . FIG. 7 provides a table of values for false positive, false negative and decoded payload errors. In a first row, the probability values set by the European Broadcasting Union (EBU) are presented. In the second row, the probability values which are produced by the example embodiment of the present invention are presented for a given probability p of 0.61 that an embedded bit is correctly recovered by the underlying watermarking layer from the material. As illustrated in FIG. 7 the probability of false positive detection, false negative detection and errors in the decoded payload for a detected watermark are all better than those required by the EBU.

FIG. 8 provides a further illustration of the relative robustness for copyright protection processing. In FIG. 8 in the first column the entries represent processing performed on the watermarked video material. These include low pass filtering, JPEG encoding at 20% quality, JPEG 2000 encoding at 10 megabits per second, JPEG 2000 encoding at 7.5 megabits per second, JPEG 2000 at 5 megabits per second and MPEG2 encoding at 5 megabits per second. In all cases except the MPEG2 encoding these results were obtained for 73 standard definition frames. For MPEG2 processed images the encoding was performed with 51 sequences each sequence consisting of 29 standard definition frames with MPEG2 encoding on the basis of a 12 frame GOP so at least some of the GOPs involved scene changes.

As will be appreciated it is important that synchronisation is maintained between the encoder and the decoder. Accordingly in some circumstances it may not be possible to determine the relative synchronisation position of the frames to which the same encoded data has been added. However, without prior knowledge of the relative synchronisation between the bits embedded by the encoder and the bits recovered by a decoder, the decoder can be arranged to detect in parallel 30 versions of the embedded data. For each of the 30 frames an offset of one frame is introduced, so that all possible combinations of successive frames are considered.

An ancillary technical problem subsists in a particular case where the decoding processor 10 of the copyright protection layer receives an image which is entirely black, and in which no data block has been embedded. For this example, the decoding processor 10 will recover data representative of versions of the data blocks, which may contain all zeros. This will have the effect that the accumulated score formed by the second accumulator 130 will indicate a value of “0”, which will be determined with respect to the first detection S 1 threshold as being a “0”, with high confidence. As a result the detection processor will erroneously determine that 130 bits have been detected and indicate correspondingly that the data block representing a watermark is present.

In order to address this technical problem, the data embedder 2 is arranged to form a logical XOR combination of the versions of the data blocks to be embedded with a Pseudo Random Bit Sequence (PRBS). As those skilled in the art will know, the XOR combination is formed by XOR-ing each bit of the data block with a corresponding bit of the PRBS. By repeating the XOR combination of the versions of the data block recovered from the material at the de-embedding processor 6 , the effect of the PRBS will be removed from the recovered embedded data. However, if no data block has been embedded then the XOR combination at the de-embedder will have an effect of generating random data, which will produce an accumulated score falling between the first and second detection thresholds S 1 , S 2 . The detection processor 135 will therefore correctly detect that no data block has been embedded in the material. In particular, an all zeros sequence, which may be recovered from a black image, will produce a random sequence according to the PRBS with which the sequence is combined at the de-embedding processor. The detection processor 135 will correctly determine that no embedded data block has been embedded.

Although the example embodiment has been described as embedding the 130 coded bits within the same frame by repeating this four times within a frame, in other embodiments the encoded data block may not be repeated but may be embedded using a spreading code having a greater length. As disclosed in our co-pending patent application number 0029854.7 and UK patent application number 0029856.2, each bit of data to be embedded within a frame may be arranged to modulate a pseudo random bit sequence. This modulated bit sequence is added to an image frame in the wavelet transform domain. Accordingly, by using a longer spreading code, each bit of the 130 bit block to be embedded may be arranged to modulate the longer spreading code. The longer spreading code therefore has the effect of utilising four times the capacity which would be used if the data block were to be embedded only once.

Various modification may be made to the embodiments herein before described without departing from the scope of the present invention. Although the encoded data blocks have been added repeatedly to each of a plurality of frames of video material, it will be appreciated that in other embodiments data can be added repeatedly on the basis of other units which may or may not depend on the type of data. Although in the example embodiment the copyright protection layer includes an error correction encoder and decoder, it will be appreciated that in other embodiments the data blocks may not be error correction encoded. Similarly, in some embodiments the process of shuffling the data bits within the block embedded into the frame or other data unit may not be applied.