WO2008107001A1

WO2008107001A1 - Document with encoded portion

Info

Publication number: WO2008107001A1
Application number: PCT/EP2007/006126
Authority: WO
Inventors: Taswar Iqbal; Walter Geisselhardt
Original assignee: Universität Duisburg-Essen
Priority date: 2007-03-02
Filing date: 2007-07-11
Publication date: 2008-09-12
Also published as: EP2119217A1

Abstract

A document comprises an encoded portion with symbols which are invisible or unreadable to human eyes but are machine readable. Each symbol comprises a small number of dots which can be placed in a small number of locations in a data region. The data region is adjacent to a synchronization region populated by one or a small number of synchronization dots. A method of producing such a document is also provided.

Description

Document with Encoded Portion

Field of Invention

5

The present invention relates to a document comprising an encoded portion with symbols consisting of preferably printed dots, and to a method for generating such a document. In particular, the present invention relates to a high capacity invisible data stripe for digital authentication of hardcopy documents. t o

This invention relates to the digital authentication of printed documents regardless of the underlying textual-content nature (i.e. alphanumeric characters type and size) by using high-quality superposed background image for machine-readable data encoding. The superposed background image forming or

15 containing the encoded portion does not affect the aesthetic appearance of the document while offering much higher data encoding capacity. Furthermore, the invention facilitates optical character recognition (OCR) technology and here it eliminates the challenges encountered from the languages with complex writing structures due to non-Roman alphanumeric characters, tables,0 figures, equation symbols etc. because the invention offers higher capacity that allows foreground contents to be encoded in particular as doc file. Another aspect is original quality fax document transmission. Finally, the invention has applications for military communications, where conventional ways of communication are not applicable (e.g. data encryption makes communica-5 tion suspicious, and digital communication might not be applicable always).

Background of Invention

The literature dealing with digital authentication of hardcopy text documents0 can be divided into two categories. In first category digital watermarking techniques are used to embed authenticity verification related data in the textual contents by slightly modifying some selected features of text such as words, paragraphs, lines etc. In the authenticity verification process selected modified features are checked in the scanned/digitized image. The limitation5 of the watermarking approach is its low capacity and it is attributed to con- tents dependence of the watermarking process. Such watermarking is described e.g. in A. M. Alattar, and O. M. Alattar, ''Watermarking electronic text documents containing justified paragraphs and irregular line spacing," SPIE Vol. 5306, 2004, Security and Watermarking of Multimedia Contents VI, San Jose, CA USA, J. T. Brassil, S. Low, N. F. Maxemchuk, and L. O'Gorman, "Electronic marking and identification techniques to discourage document copying," IEEE Journal on Selected Areas in Communication, vol. 13, no. 8, pp. 1495-1504, October 1995, J. T. Brassil, S. Low, and N. F. Maxemchuk, "Copyright protection for the electronic distribution of text documents," Proc. of the IEEE, vol. 87, no. 7, July 1999, pp. 1 181-1 196, S. H. Low and N. F. Maxemchuk, "Performance comparison of two text marking methods," IEEE Transaction on Selected Areas in Communication, 1998, S. H. Low and N. F. Maxemchuk, "Capacity of text marking channel," IEEE Signal Processing Letters, vol. 7, no. 12, pp. 345-347, December 2000. Furthermore, not all the contents can be used for embedding process due to the fact that the neighboring contents are left unchanged in order to be used as a reference in the watermark recovery process. Otherwise original contents would be required (non-blind mode) for watermark recovery, which is not desirable as the original contents might not always be available.

In order to overcome the lower capacity problem that is attributed to contents dependence there has been used another approach in which superposed background image is used to carry authenticity verification related data. This approach is similar to two-dimensional barcodes except that here aesthetic ap- pearance is focused upon to make the superposed background image visually pleasing. The well-known technique belonging to this category is called DataGlyphs and described in Motwani, R.; Breidenbach J. A; Black, J. "Collocated Dataglyphs for Large Message Storage and Retrieval", SPIE Vol. 5306, 2004, Security and Watermarking of Multimedia Contents VI, San Jose, CA USA. DataGlyphs that offer much higher data encoding capacity per unit area than text watermarking techniques, are more pleasing than the barcode counterparts. However, its aesthetic appearance is not satisfactory and this is due to larger (visually perceptible) data encoding symbol size as well as the synchronization marks. Whereas the smaller symbol size poses challenge in information recovery process due to the noise encountered from print and scan process. The DataGlyphs technique is used in G. T. Brewington, "System and method for providing hardcopy secure documents and for validation of such documents" Xerox Corporation, and in US 2004/01 17627 A l for forgery and counterfeit detection in passports, bank checks etc.. claiming that the resulting document is difficult to counterfeit.

Another technology known as SecureSeal ^{I M} from EnSeal systems is described in D. Hilton, W. Tan and P. Wells, "Document printed with graphical symbols which encode information" Enseal Systems Limited (GB), and in US 6,871,789 B2. This technology focuses on aesthetic appearance of the data en- coding symbols. The improvement is achieved by reducing symbol size (but it is still visible) at a cost of lower capacity than DataGlyphs technology. Superposed background image given in M. Suzaki, Y. Mitsui, and M. Suto, "New alteration detecting technique for printed documents using dot pattern watermarking", SPIE 2003, Security and Watermarking of Multimedia Contents V, San Jose, CA USA differs from the others discussed above that it results in very smooth image. In this technique imperceptibility constraint is achieved by using two different data encoding patterns, each one consisting of number of very small size dots that are arranged in a specific order so that the resulting symbol remains invisible and can be identified from printed and scanned process. This technique also results in lower capacity than DataGlyphs. One common weakness of above techniques is that none of these allows full document contents to be encoded into the background image. Consequently they do not allow one-to-one contents authentication that is necessary to check each and every modification as well as some other limitations to be discussed in the following while reviewing another technique given for digital authentication of hardcopy text documents.

In J. Zhao, "Digital authentication with digital and analog documents" Medi- aSec Technologies GmbH (DE), US Patent No. 6,751,336, Jun. 2004 while focusing on digital authentication, digital signatures of semantic information using OCR-technology are computed and encoded into a barcode or digital watermark, which are added to the background image in such a way that it does not affect the extraction of original semantic information (e.g. IR, UV, magnetic, flourescent inks, security patterns) to make the information embed- ded in watermark robust against high quality photocopying attacks). This technique has following weaknesses: OCR dependence and the need for non- conventional expensive ways (i.e. invisible inks) are used to make the digital signatures invisible as well as robustness against copying attacks of two- dimensional barcodes. Also the given message digest computation algorithm has limitations due to 99% rather than 100% performance of OCR technology in ideal cases. It is mentionable that this technique can be applied only with drawbacks, in particular, to documents in which OCR-technology is very difficult due to complex writing structure of the language alphanumeric characters. One example of such languages is Arabic, a widely used language that is very attractive from commercial point of view.

US 2006/0147082 Al discloses marking of a document with invisible marks. Each mark is preceded by a marker to indicate to the scanner the beginning of a replication of a unique identifying code pattern. The code pattern is formed by dots forming a series of binary coded decimal numbers. So that the number of dots depends on the coded number. The distributed marks do not allow an optimized capacity. Further, the system is sensible regarding scanning errors and dirt or other disturbances and influences.

Summary of the Invention

Object of the present invention is to provide a document with a preferably invisible encoded portion and a method for generating such document, wherein the encoded portion allows an optimized high capacity of data and/or is easy to scan and/or read and/or can be read with high security or only few errors.

The above object is achieved by a document according to claim 1 or by a method according to claim 28. Preferred embodiments are subject of the sub- claims.

The encoded portion comprises multiple symbols which encode information as data bits. According to one aspect of the present invention, each symbol comprises at most two spatially spaced dots, and/or comprises at least one data bit and at least one synchronization bit or dot. This allows a very dense arrangement of the symbols with high data capacity and fewer reading errors due to the synchronization. An additional or alternative aspect is that the sym- bols with different data bit values consist of the same number of dots. This facilitates reading and/or error correction.

According to another aspect, the synchronization dots are regularly arranged and interleaved with data dots. This facilitates reading and/or error correction.

In the present invention, the term "symbol" shall be understood in particular as a pattern of the encoded portion which pattern is repeated and/or forms a unit containing one data bit or multiple data bits.

In the present invention, the term "synchronization" means a pattern or dot arrangement in the space domain that is used for calibrating or reading or sensing the special location of encoded information or data dots.

Further aspects, features, advantages and uses of the present invention will be apparent from the claims and the following description with reference to the drawings.

Brief Description of Drawings

Fig. 1 shows a document with an encoded portion as constant greyscale background;

Fig. 2 shows another document according to the present invention;

Fig. 3a to 3c shows schematic diagrams of preferred constructions of the encoded portion according to the present invention;

Fig. 4 shows a schematic flow chart representing a preferred method for generating a document according to the present invention.

Detailed Description of Preferred Embodiments of the Present Invention

Fig. 1 shows an example of a document 1 according to the present invention. The shown document 1 comprises text 2 and an encoded portion 3. The en- coded portion 3 preferably forms a background, in particular an at least substantially constant greyscale background for the human eye.

The document 1 is preferably a paper printout. In particular, the text 2 and the encoded portion 3 are printed, in particular with a laser printer (not shown).

In the shown embodiment, the text 2 is superposed onto the encoded background, i.e. encoded portion 3. In particular, the encoded portion 3 is interleaved between the text 2. It is possible to provide for example rectangular ar- eas around the letters and/or words, sentences and/or lines of the text 2 without encoded portion within these areas. Alternatively it is also possible that the text is superposed onto the encoded portion 3 without any consideration of covering of parts of the encoded portion 3.

Alternatively or additionally, it is also possible to place the text 2 beside the encoded portion 3 or to omit the text 2.

Fig. 2 schematically shows another document 1 according to the present invention. In particular, this document 1 is an identification (ID) card, driver Ii- cense or the like. In this example, the document 1 may contain an area 4 for a the picture or drawing, an area 5 e.g. for a signature, and/or an area 6 e.g. for biographical data, personal data, text or the like. The area(s) 4, 5, 6 may be surrounded or embedded or superposed by or on the encoded portion 3.

However, there are multiple other possibilities to use the encoded portion 3. For example, the encoded portion 3 can be located on the front side and/or on the backside of the document 1 and/or can be combined with printed text, handwriting, images, holograms and/or other optionally encoded patterns.

Fig. 3a shows a schematic representation of a preferred construction of the encoded portion 3.

The encoded portion 3 comprises or consists of symbols 7. The symbols 7 and, thus, the encoded portion 3, are preferably unreadable or invisible for human eyes. However, the symbols 7 are machine-readable and encode information as data bits. Fig. 3a shows four symbols 7. Each symbol 7 forms at least one data bit or consists of preferably only spatially spaced dots 8, 9. Preferably, each symbol 7 comprises at most three or two spatially spaced dots 8, 9.

Alternatively or additionally, each symbol 7 preferably comprises at least one data bit and at least one synchronization bit or dot 9.

Preferably, at most three spaced dots 8 of one symbol 7 form multiple data bits, preferably two data bits.

More preferably, only one or two spaced dots 8 of one symbol 7 form the at least one data bit, preferably two data bits.

In the present embodiment, only one or each data dot 8 of one symbol 7 has n different possible positions 12 and preferably forms FIX (Iog2(n)) data bits, preferably wherein n=4.

The symbol 7 may comprise only one data bit. In the present embodiment, each symbol 7 comprises multiple data bits, preferably two.

In the present embodiment, each symbol 7 comprises only one data dot 8, and/or preferably only one synchronization dot 9.

Each symbol 7 comprises a synchronization region 10 and a data region 1 1.

Preferably only one dot 8, 9 is located in the synchronization region 10 and/or in the data region 1 1.

The synchronization region 10 may be restricted to a space for the only one dot 9 or may cover a wider space, e.g. a stripe with multiple potential dot positions or any other area with multiple potential dot positions.

In the present embodiment, the preferably only one synchronization dot 9 per symbol 7 is located at the same position within each symbol 7. The synchronization dots 9 of multiple or all symbols 7 are preferably regularly arranged. This facilitates scanning and/or reading and decreases errors. Further, this supports the preferably desired constant grey appearance of the encoded portion 3, in particular if it is used as background.

According to one embodiment, the number of synchronization dots 9 per symbol 7 can be changed in order to vary the grey level of the encoded portion 3.

In the embodiment according to Fig. 3a, the data region 1 1 comprises four potential positions 12 for the data dot 8. Each of the positions 12 is preferably spaced to the other positions and/or to the synchronization dot 9 or potential positions of synchronization dot(s) 9 and/or to the synchronization region 10. This facilitates correct reading with reduced error rate.

In the present case, each symbol 7 has preferably the form of a square, in particular of 6x6 units corresponding to the minimum dot size, as shown in Fig. 3a. Depending on the printing or printer resolution, the unit size is preferably about 1/300 inch at 300 dpi and at about 1/600 inch at 600 dpi.

However, each symbol 7 can also have any other suitable form, e.g. a rectangular form or the like.

In the present embodiment, the preferably single data dot 8 can have four dif- ferent positions 12 (n=4).

A pair of two positions may form one bit. Then, the bit value depends whether the data dot 8 is in one of the positions. With two such pairs and two data dots 8, two data bits can be formed.

In the present embodiment, preferably only one data dot 8 is provided which is located in one of the four positions 12 so that a value range of two data bit is also achieved. Further, this reduces the grey level.

From the forgoing, it is obvious that the data dots 8 of the symbols 7 are interleaved between the synchronization dots 9. This allows a very dense arrange- ment with high data capacity. The preferably regular arrangement of the synchronization dots 9 facilitates error correction and/or reading and/or decoding of the encoded portion 3.

Fig. 3b shows in a schematic representation another embodiment with different symbol size. Here, the symbol 7 is a square of 8x8 units.

The symbol 7 may have multiple synchronization dots 9 in the synchronization region 10. In the example, three synchronization dots 9 are shown.

The data region 1 1 provides e.g. nine positions 12 (n=9) for the data dot 8. However, also multiple data dots 8, e.g. two or three data dots 8, could be used.

The symbol 7 according to Fig. 3b can also be combined to form an encoded portion 3 as described above.

Fig. 3c shows an other similar embodiment of the symbol 7. Here, the symbol 7 consists of 10x10 units and comprises e.g. only one synchronization position and synchronization dot 9 and e.g. 24 positions 12 (n=24) for one or multiple data dots 8, e.g. three data dots 9.

In general, the following aspects may apply for the different symbols 7 according to Fig. 3a to 3c or to similar symbols.

Preferably, the synchronization regions 10 and/or the data regions 1 1 of multiple or all symbols 7 are regularly arranged, in particular grid-like.

Preferably, each symbol 7 comprises only one synchronization dot 9 and the synchronization dots 9 of the symbols 7 are regularly arranged, in particular grid-like.

Preferably, the data dots 8 of the symbols 7 are interleaved between the synchronization dots 9. Interleaving means here that the data dots 8 are spatially arranged between the synchronization dots 9. Preferably, the symbols 7 are arranged one adjacent the other.

Preferably, the encoded portion 3 forms a uniform region of greyscale or halftone.

Preferably, the encoded portion 3 forms a background of at least part or the document 1 , preferably a constant greyscale background image of at least part of the document 1.

Preferably, the optional text 2 is superposed on the encoded portion 3. In particular, the encoded portion 3 can be interleaved between text lines. Preferably, the background portion visible within letters or words of the text 2 is also grey, but can or can not contain symbols 7, i.e. encoded data.

Preferably, the encoded portion 3 encodes the complete text 2, i.e. contents of the text 2, or contents of the document 1.

When the document 1 comprises a page, the encoded portion 3 preferably encodes the complete text 2 on this page.

When the document 1 comprises multiple pages, the encoded portion 3 of one or each page preferably encodes the complete text 2 of some, e.g. adjacent, or all pages.

The maximum size of the dots 8, 9 forming the symbol 7 is preferably at most 1/300 inch, more preferably 1/600 inch or less, in particular depending on printer resolution.

Preferably, all dots 8, 9 of one symbol 7 and of all adjacent symbol 7 are spa- tially spaced from each other.

Preferably, some or all symbols 7 having different data bit values consist of the same number of synchronization dots 9 and/or data dots 8. Preferably, the encoded portion 3 comprises a fingerprint (e.g. for watermarking and/or biometric aspects) and/or a digital signature and/or the content of the text 2 of the document 1.

Preferably, the documents contains more than 200.000 symbols 7 per page when printed at 300 dpi, or more than 800.000 symbols 7 per page when printed at 600 dpi.

Preferably, the size of each symbol 7 (in one direction or each direction) is at most 10/300 inch, preferably 6/300 inch, in particular about 6/600 inch to 10/600 inch or less. However, the symbol size can be further decreased or increased while relaxing per unit data encoding capacity.

Preferably, the data capacity of the encoded portion 3 is at least 0.6 kbyte per square inch at 300 dpi or at least 2 kbyte per square inch at 600 dpi, preferably about 2.5 kbyte per square inch or more.

In particular, the data capacity can be calculated as follows:

For the data encoding symbol of size N by A (without taking into account synchronization region) where A is a row of the positions 12 that can be used to put data encoding black dots 8 and N is the number of rows of size A. Note in each position only one dot can be put and only black dots are being considered for data encoding. Now, data encoding capacity can be described as follows:

Bit encoding capacity per symbol= N times (FIX (log2(X)))

where the FIX operation rounds real value to integer value towards zero.

Here, X is the number of unique arrangements that can be achieved for the given number of dots to be used in single row of size (A). For instance for a row of size 3 and using 1 , 2 and 3 dots, the possible unique arrangements are shown below (Black dot B, White dot W):

Total cases: using 1 black dot using 2 black dot using 3 black dots

In above scenario using 1 , 2 and 3 black dots for data encoding, the number of unique arrangements are, 3, 3 and 1, respectively. The capacity for this scenario using N=5, X=3, 3, 1 shall be: Capacity^ 5 times fix (log 2(3))= 5« 1 =5 bits (for 1 dot)

= 5 times fix (Iog2(3)) =5- 1 =5 bits (for 2 dots)

= 5 times fix (Iog2( I)) =5-0 =0 bits (for 3 dots)

Similarly, it can be extended when size of A and N is changed. Of course for the scenario using three black dots resulting in zero bit capacity can be changed considering all rows of size A as a single row (or an array of size 1 by N. A) and using the number of dots 8 that result in maximum capacity. For instance, using only one dot 8 for symbol with A=3, and N=3, results in X=9 and offers, capacity = l *FIX(log2(9))=3 bits.

Alternatively or additionally, the data capacity can be increased by using dots of different colours.

Fig. 4 shows a preferred process for generating a document 1 according to the present invention. The original electronic document, e.g. a file readable by a word processing program or the like, is provided in step Sl .

This electronic document is converted into a graphic (printable) image, e.g. in the format tiff, in step S2 and provided as the foreground contents, in particular the text 2 or any image or the like, in step S3. Preferably simultaneously, the electronic document provided in step 1 is compressed in order to reduce the data in step S4. Before data compression, the data of the document for encoding may be or are interpreted as or converted into a bit stram.

Then, the data are encrypted in step S5. An optional error correction coding step S6 and a data scrambling step S7 may follow.

Then, a data encoding process step S8 follows to provide a background image, in particular the encoded portion 3, in step S9.

In step S lO the foreground content and the background image are superposed. The superposed image or data are then printed in step S 11 so that the document 1 according to the present invention is produced, in particular printed. Preferably, a dye sublimation printer or ink jet printer or laser printer (not shown) is used for the printing process or step S I l .

To recover the data from the encoded portion 3, the document 1 or at least the encoded portion 3 is scanned preferably with twice the resolution as printed. Thus, printing, scanning or reading errors can be avoided or at least minimized.

When reading the encoded data, the initial encoded content can be decoded. This information can be used e.g. for OCR or authentification or other pur- poses as explained in more detail in the following.

This invention enables one-to-one basis digital content integrity authentication of valuable hardcopy documents (e.g. contracts, official letters etc.) with large contents. It is independent of content size and can be extended to other appli- cations (e.g. as discussed in the following) as well. Digital authentication process allows a secure document production process (compare Fig. 4), which allows full-contents of the foreground text to be encoded into the superposed background image in machine-readable format. Before encoding the contents each of the following operations: - data compression,

- error correction coding (ECC) against unavoidable errors, data scrambling,

- data encryption, can be applied on the data that is to be encoded into the background image (encoded portion 3).

In digital authentication process the document 1 to be authenticated is scanned at sufficiently higher over-sampling rate and then data-reading technique is applied to decode the contents encoded in the background image. On the recovered data all the operations performed in data encoding process are performed in reverse order and the resulting contents are output e.g. as a doc file, which can be printed or shown on the computer screen for contents integrity verification. At this stage human interaction based authentication can be en- sured.

The present invention is applicable for an automatic digital authentication or verification process in which contents decoded from the superposed background image are compared with the digitized image (consisting of super- posed background image and foreground text, means digitized image used as input. For comparison purpose, background image is superposed on the graphic image of decoded contents and then comparison is made. To differentiate between the intentional (forgery) and unintentional (wear-and-tear) modifications following approach is used. The superposed image with foreground contents is divided into two types of regions: 1) consisting of lines of text (with bounding rectangle) and 2) lines without text. Any modification encountered between two text lines is considered noise, whereas the region of text line is defined by the rectangle (with smallest area) bounding the text line. Within the rectangle (bounding text line) if only few (2-3) changes are en- countered in given character bounding rectangle then it is assumed as the noise from wear-and-tear effects and otherwise a character modification attack (forgery). It is mentionable that due to the small size of data encoding symbol relatively more changes are required as compared with negligible unavoidable changes/errors encountered from print-and-scan process to forge/modify a single character and consequently minor changes are considered as noise. If the noise in region bounding a character is much higher, then this region can be passed to OCR-technology and the resulting character ASCII code can be compared with the ASCII code of the original character. As this is a well- addressed problem, the application of the OCR-technology is not addressed and left for the application developer.

Digital Authentication of languages with complex language structures is possible. For languages having complex writing structures it is very difficult to develop OCR-technology and this is evident from the performance of the existing OCR-software for Arabic language. Such behavior would exist in future as well. It is mentionable that more number of characters used by the Arabic, Urdu and, Persian languages demand (in absence of some special coding technique for data compression) much higher capacity as compared with languages using Roman characters. In this scenario the capacity offered by the superposed background image is at least few copies and it shall be obviously more while using efficient coding techniques. Consequently, the technique is not dependent on hash functions due to large data encoding capacity. The authentication process is the same as discussed above.

Image Quality can be improved. The quality of the superposed background image (encoded portion 3) is higher than in the prior art. The individual data encoding symbols are completely imperceptible and do not affect the aesthetic appearance of the document 1 and even make the document visually more attractive than those without superposed background image. Any visual inspection of the superposed background image (encoded portion 3) does not give any indication about the existence of encoded data in background image.

Multiple grey levels for the superposed background image (encoded potion 3) can be achieved.

The present invention allows many pages of foreground text to be encoded, consequently enabling one-to-one basis contents integrity verification (as mentioned above). The higher data encoding capacity and visual quality are attributed to: smaller data encoding symbol size, data encoding symbol pattern mechanism, synchronization recovery mechanism and/or the data-reading technique.

The data-reading technique takes the scanned image, which is sufficiently over-sampled in the scanning process, preferably at least twice the printing resolution, as input and recovers the encoded contents from the digitized document image. It handles intentional, unintentional skewing distortion and noise encountered from the print-and-scan process. Almost all existing scanning devices in market satisfy the over-sampling constraint.

While using the new technology in ID cards, driver's licenses or the like, it provides the following benefits:

- It acts as invisible data stripe with higher capacity that is sufficient for encoding photographs, textual information, other information such as biometrics templates. A background image of size 5 x 8 cm or 2 x 3.1 inch offers 15 KB raw data encoding capacity that is sufficient for the requirements of the ID cards and much higher than the existing ones

(e.g. DataGlyphs).

- The higher capacity is achieved by the higher data encoding rate (as discussed before) as well as by utilizing the flexibility that is attributed to the background image size. The conventional data strips, two- dimensional barcodes or the like have constraints imposed by the aesthetic appearance that limits them to a fixed size and consequently to a fixed (and less) capacity that is conventionally not sufficient. On the other hand underlying nature of the proposed Superposed Background Image along with higher data encoding rate results in (-12.5 KB without

ECC and excluding 1.1 x 0.91 inch space spared for the photograph) more capacity than the usual cards having 2.8 Kbytes or 3.5 Kbytes raw data encoding capacity for 0.75 x 3 inch data stripe size. - Existing portable card readers, e.g. from DATASTRIP Inc., offer the necessary over-sampling rate along with biometrics matching capability, so the product can be launched immediately. - The higher capacity for biometrics data storage would result in stronger identity verification techniques by using multiple biometrics characteristics for identity verification.

The present invention can be used for bank checks. It allows all the foreground textual contents to be encoded (after encryption) into the background image that is already there for aesthetic appearance of the document (as in [BreO4]). However, the novel technology offers more resistance than the prior art against the counterfeiting attacks due to its nature of data encoding sym- bols. The resulting bank-checks are inexpensive as compared with existing bank-checks, which are expensive due to sophisticated security printing technologies usage for document protection.

Conventionally documents having contents of mixed nature (e.g. mathemati- cal equations, symbols, tables, figures/graphs, text with different fonts and types) result in large file size after digitization process at optimal scanning parameters when document is need to be stored in digital form. This is attributed to mixed nature of contents and limits the compression gain. Any such document produced using this new technique results in:

- Smaller data size due to the fact that the original information (e.g. Doc file) encoded in superposed background image can be decoded and stored as digital file. The data size can be further reduced by using appropriate compression technique on the original contents (decoded from superposed background image).

- Challenges faced by OCR-technology against smaller font sizes, special font types, symbols, equations, tables and graphs, recovery from the printed documents can also be tackled by using the decoded information.

- Logical document template of the original digital document is also available now.

- Above mentioned capabilities would open a new dimension for printed document processing (called as Smart Document Processing).

It has to be noted that the quality of natural and human images might be slightly degraded, as they usually demand for much higher capacity that might not be available sometimes. Such images are assigned lower priority and are not encoded in the background image (encoded portion 3) if capacity is not sufficient. In such scenarios only scanned image at optimal conditions will be stored in digital file and this part (e.g. natural images) of the printed document suffers from some quality degradation, consequently, contributing to compressed image file size.

Another application of the proposed technology is in faxed documents. The existing fax document quality is poor and is mainly constrained by the trans- mitted data size (assuming that high quality equipment/sensors is used for scanning purpose). To improve the visual quality of the faxed document requires more data to be transmitted. In this scenario a document produced with superposed image (encoded portion) results in: - Original quality rather than high quality document to be faxed.

- Less data is to be transmitted as compared with conventional fax transmission process. - Above benefits are attributed to the fact that original digital data decoded from printed and scanned document can be transmitted after necessary processing (e.g. lossless data compression).

- No special devices or device modifications are needed and existing all- in-one (fax, photo-copy, printing and scanning devices) can be used by loading the data-reading software that allows machine-readable data recovery.

Telefax machines characterized by the above scenario may be called Smart Telefax Machines in the future.

Potential setups for such a Smart Telefax Document transmission may be as follows:

1. The document 1 is scanned, and the encoded portion 3 is decoded to obtain the digital data. The digital data are then transmitted to a receiver. There, the original document is printed, in particular together with the encoded portion. Thus, a transmission similar to a telefax can be achieved, however, with much better transmission characteristics. In particular, this arrangement and method allows an optimal quality of the document 1 produced at the receiver side.

2. The encoded portion 3 can also be used to improve telefax transmission or legibility of a document 1 sent by telefax or the like.

For example, the document 1 is transmitted by telefax and contains the encoded portion 3 with the full foreground contents encoded. At the re- ceiver side, either the telefax data or the telefax is received. The telefax data can be used directly by a suitable hardware / software arrangement running a data decoding routine to recover data from the encoded portion 3 to recover or correct foreground data, text images or the like. If necessary, the received telefax will be digitized or scanned (after it has been printed) in order to obtain the respective digital data for running the data decoding routine to recover data from the encoded potion 3 and allow the respective correction if necessary. Afterwards, the corrected data are used to print a corrected document 1 which resembles the original document 1 much better than usual telefaxes.

The present invention is applicable for secret or military communication. The underlying characteristics: invisibility and higher data encoding capacity, and blind-data decoding capability of proposed technique from superposed background image make it very attractive for secret or military communications and also for other government departments due to the fact that digital communication is not applicable in all scenarios (e.g. the conventional cryptographic techniques makes the digital communication suspicious, even desired digital link might not be available or possible at all).

Further or important aspects of the invention are:

A high capacity data communication technique using a high quality superposed background image for contents integrity verification of printed documents is proposed. The superposed background image (encoded portion 3) can be used as channel to encode digital data of any type (e.g. text, graphic, audio/video clip etc.) without affecting the aesthetic appearance of the underlying document.

Preferably the superposed background image (encoded portion 3) is a halftone image that is obtained by the repeated application of especially designed symbols to be called from hereafter data encoding symbols. These symbols are used to encode digital data into the superposed background image in data encoding process.

A data encoding symbol has preferably one or more of the following characteristics:

- visually imperceptible; - encodes multiple data bits per symbol;

- integration of the synchronization recovery unit and information encoding parts in a single symbol;

- multiple grey levels;

- robustness against dot gain effects.

The imperceptibility constraint is achieved (without compromising capacity) in particular by partitioning the symbol 7 into parts and decreasing the size (1/600 of an inch) of the basic elements of the data encoding symbol 7; unlike Suzaki et al. in who have done so at the cost of capacity decrease. The present invention use only one third of the size for the symbol, which immediately results in 9 times higher capacity per square unit.

Apart from the reduced data encoding symbol size another factor contributing to the capacity gain is integration of synchronization and data encoding sym- bols into one symbol that is being used for data encoding. Here no separate symbols are used for synchronization recovery and data encoding.

Conventionally two separate symbols are used for binary data (0 and 1 bits) encoding, however, in new data encoding symbol, position of the one single element (black dot measuring 1/600 of an inch) within very small fixed region in the symbol is changed for encoding multiple data bits per symbol. Here using four different locations two data bits are encoded per symbol.

The data encoding symbol 7 is optimized against dot gain effects (encountered from printing process), imperceptibility and capacity. This means that for the given experimental set up (for laser printing technology) if capacity is further increased it results in higher error rate whereas at less data capacity channel would be underutilized.

The data encoding symbol 7 allows multiple grey levels to be achieved. In order to achieve different grey levels the data encoding symbol is partitioned into two parts in which one part deals with data encoding and the other one with synchronization recovery. Multiple grey levels may be obtained by varying the number of black dots in synchronization recovery region under the constraints: synchronization recovery process is not affected, data decoding process is not affected due to the dot gain effects caused by the additional dots that are added for different grey levels.

The process of different grey levels could be extended to color background images by using color dots (Cyan, Magenta, Yellow). In this case in synchronization recovery process image is to be scanned as greyscale image as the additionally introduced color dots have no impact on the capacity.

However, it is also possible to consider the color of the dots when scanning the encoded portion 3 and during decoding. Thus, it is possible to increase the data capacity and/or number of data bits per symbol.

In particular, the use of colored dots 8, 9 does also allow to obtain more different grey levels or different appearances.

For example, four different colors can be used for the dots 8, 9, such as black, cyan, magenta and yellow. Consequently, this means that at a single location two bits can be encoded by using a colored dot (four different colors). By additionally changing the position, two more bits could be encoded, resulting in a total of four bits per symbol for example. Of course, instead of using four colors less or more colors can be used, depending on the quality of the printing possibilities and/or scanning possibilities.

Alternatively or additionally, larger dot sizes (300 dpi or less) would facilitate the use of ink jet and dye sublimentation printing technologies. In particular, the latter one offers more colors for printed dots.

It is also possible to use ultraviolet (UV), infrared (IR) or magnetic inks with suitable measures (e.g. appropriate dot size) for printing data encoding sym- bols 7.

The data encoding process takes foreground contents (doc file) and converts it into a graphic image. Next, the doc file is converted into binary data stream on which following operations: lossless data compression, data encryption, error correction coding and data scrambling, are performed respectively. The resulting binary data stream is encoded into a superposed background image using the data encoding symbol 7. Finally, the background image is superposed to the foreground contents (graphic image).

The superposition process consists of two stages: 1) selection of suitable data encoding region, and 2) elimination of artifacts caused by data encoding symbols on the foreground contents in overlapping regions, whereas the later stage is common and handled in same way in both cases.

According to first method for data encoding region selection, data is encoded over the entire background image uniformly and the errors caused by the overlapping of foreground contents are compensated by the data scrambling and error correction coding (ECC). This approach results in lower capacity due to the higher overhead for ECC.

In second method graphic image file of foreground contents is processed to look for the regions that do not overlap the background image and these regions in background image are used for data encoding. Such a region in text document is defined by the rectangle of minimum area surrounding a text line and the x, y coordinates of four points of all such rectangular regions encountered in the entire image are encoded separately into the background image with higher overhead for ECC to make it more robust against the errors. Finally, only these non-overlapping regions (decoded in the beginning from background ima^e) are used to recover data from scanned imaee. The ECC and data scrambling operations are still performed in data encoding process. This approach is good for documents with only text and results in higher capacity.

The compensation process to count artefacts caused by data encoding symbols on foreground contents works as follows. By denoting a pixel value at posi- tion (i,j) for the foreground text image, superposed background image and the resulting image after superposition by X(i,j), Y(i,j) and Z(i,j), respectively, the process to count for the artifacts caused by data encoding symbols on foreground contents works as follows.

Z(ij)=X(ij) if X(ij)=black Z(i,j)=Y(i,j) if X(i,j)=white

According above rule, the given pixel (Lj) of resulting image Z(i,j) takes the pixel value of foreground text image X(i,j) when X(i,j) has black pixel, other- wise always the pixel value of the resulting image is the value of the superposed background image Y(i,j) regardless of its value, black or white.

The above process for artifacts compensation can be implemented efficiently by using single logical and operation as follows: Z(i, j)= X(i, j) and Y(i, j). Here, X and Y are two binary matrices having same size and black and white pixels represented by binary values "0" and "1 ", respectively.

In the data reading procedure the printed document 1 is scanned at sufficiently higher over-sampling (at least twice) rate and then the data reading technique is applied. The data reading technique applies different filters that deal with synchronization recovery from noisy environment, noise elimination from data encoding region and identification of information encoding dots.

The objective of synchronization recovery filter is to identify and accurately locate the dot 9 used for the synchronization recovery. A synchronization recovery dot suffers from dot gain effects caused by the up to four neighbouring data encoding dots 8 separated by 1/600 of an inch. The accuracy of the located position is 1/600 of an inch. If synchronization error is encountered (for instance due to overlapped symbols) then it is taken the average value of next immediate neighbouring synchronization dots 9.

The filter dealing with information decoding process identifies and locates the position of data encoding dot in the region, defined by the synchronization dots of three neighbouring data encoding symbols (and fourth one for symbol under consideration), while taking into account the noise from the print and scan process. It has to locate one position among different positions that are used by the data encoding symbol for encoding multiple data bits per symbol. It also takes measures (e.g. using slope of greyscale region) to distinguish if given position lies at the critical region resulting from dot gain effects of the PS process. The accuracy needed for successful data decoding for a given symbol is again 1 /600 of an inch.

Once the information from all the data encoding symbols have been decoded, then the resulting data is converted to the bit stream corresponding to the doc file (encoded into the background image), while performing the operations (data compression, data encryption, error correction coding and data scrambling) in reverse order.

In one application, dealing with authentication of printed documents with large contents size (e.g. contracts, official letters etc.), the data encoding and reading technique serve the following goals:

- sufficient capacity to encode multiple copies of full foreground contents in the superposed background image, - superposed background image does not degrade the aesthetic appearance of the document,

- one-to-one basis contents integrity authentication by showing both decoded and digitized version of printed document side by side on the screen or in printed form for human-based authentication,

- potential for automatic contents integrity verification,

- independence from the OCR technology, - independence from message digest computation (a constraint from lower capacity) that is dependent on the OCR technology, - less-expensive as compared with the security printing technologies using special inks, patterns etc. while more efficient due to its ability one-to- one basis contents integrity authentication.

The higher capacity can be utilized to lock the document (contracts, official letters, educational certificates, bank check) with digital signatures, and/or some desirable biomedical characteristics (fingerprints, voice print, iris scan, signatures, etc.). In this case in addition to the doc file, digitized signatures and fingerprints can also be encoded into the superposed background image and the resulting printed document would offer the same characteristic as the conventional contract writing process where involved parties sign the document (as a proof of willingness).

Another application of the present invention is an extension of the technique to document with text consisting of non-Roman letters. Here the same type of documents (e.g. contracts, official letters etc.) is addressed with one constraint that for such documents OCR technology either does not exist yet or the underlying nature (complex language structure) of the contents make it highly difficult to develop such technology (with very small error rate). Examples are Arabic, Persian, Chinese, Urdu languages with complex language structure, using more or less same alphabets set (except Chinese language). In this scenario authentication is again done on one-to-one basis using the contents recovered from the superposed background image.

For ID-cards application the technique using superposed background image can result in more secure identity verification process by offering more capacity than conventional 2-D barcodes etc. for biometrics data storage, allowing multiple biometrics characteristics to be used. It also eliminates the need for central database system for record keeping, as the full contents can be encoded into the document.

In this scenario aesthetic appearance of the superposed background image allows the whole ID-card background to be used as superposed background im- age, unlike 2-D barcodes where aesthetic appearance of the barcode restricts it to a fixed size.

The backside of ID-card could be a better choice for the superposed back- ground image as usually it has not much foreground contents.

An extension could be to use non-constant greyscale image as a superposed background image. For this purpose the desired graphic image is to be transformed into a very light image (e.g. varying in 210-220 grey levels) and this image would be halftoned using some of the available grey levels from the data encoding technique. In halftoning, data encoding symbol would be used for screening process.

Fax documents quality can be improved up to the originally printed document quality. In this case quality of transmitted document can be improved together with the decrease in size of transmitted data that is conventionally transmitted.

One of the applications of new technique is in data compression for scanned documents. Here contents decoded from the superposed background image demand for much lower data storage capacity than the corresponding conventionally scanned document. The small font size, equation symbol and in general existence of complex structures arising from different languages, pose challenge to get compression gain in scanned documents, however, the new technique can solve this problem as discussed.

Conventionally, it is not possible to process the scanned documents except to some extent for Optical Character Recognition and there are still many challenges (e.g. logical template extraction, tables extraction etc.) that are being addressed by applying image processing and pattern recognition techniques. The new technique eliminates the need for such image processing on the document, as all the information is now available as a doc file. In this invention it is not claimed that all problems in this regards are solved, however, this technique gives a direction for the potential solution.

A lot of research has been focused upon OCR-technology development and this problem is solved (for the documents produced according to this tech- nique) in this invention by encoding fxill contents into superposed background image.

The data hiding techniques for printed documents offer very less capacity (maxim several hundred bits). This lower capacity is not suitable for many applications (e.g. secret communication). Very high capacity, blind contents decoding and foreground contents independence, makes it very attractive for such applications. As author is not familiar with existing techniques in this context, so extension and new applications are left for specialist in this field.

For automatic contents integrity verification, a potential technique is given and this technique can be modified for contents integrity verification of handwritten printed documents by counting the number of overlapping(s) for each line of data encoding symbols (or per unit area of size X by Y symbols 7) and encoding this information in the background image separately with suitable measures against wear-and-tear effects.

The higher capacity can also be utilized for computation gain and here the possibility for encoding multiple copies of foreground contents can be ar- ranged in such a way so that first copy can be recovered very quickly without reading the data from whole background image. If first copy is not recovered successfully then the next copy can be tried.

The printed document with superposed background image can be used for counterfeiting and copy detection purpose. Here very large size of the superposed background image requires about 800,000 data encoding symbols to be identified correctly from scanned document in order to launch a successful counterfeiting attack. To make counterfeiting detection process quick only predetermined pseudorandom locations can be selected and used for counter- feiting detection, which would still demand for the same efforts from the counterfeiter as before (means almost all symbols to be decoded correctly).

The copy detection attack can easily be detected due to the fact that regardless the quality of copying device and application of over-sampling in scanning process, a document printed from scanned copy cannot produce the same vis- ual quality as the originally printed document. Furthermore, encoded information cannot be recovered from copied document.

To count for skewing distortion, the synchronization dots 9 may be used.

The present invention may also be used in the opposite case where the dots 8, 9 are white and/or colored and the other area or background is black or colored. This scenario is similar to the negative of an image and can be used for the same dot size as described or for larger dot sizes ( 1 : 300, 1 : 200, 1 : 150 or 1 : 75 inch).

Claims

Claims:

1. Document ( 1 ) comprising an encoded portion (3) with symbols (7), the symbols (7) being unreadable or invisible for human eyes, the symbols (7) being machine readable and encode information as data bits, each symbol (7) forming at least one data bit or consisting of only spatially spaced dots (8, 9), each symbol (7) comprising at least one of: at most two spatially spaced dots (8, 9), and/or the at least one data bit and at least one synchronization bit or dot (9).

2. Document according to claim 1 , wherein at most three spatially spaced dots (8, 9) of one symbol (7) form multiple data bits, preferably two data bits.

3. Document according to claim 1 , wherein only one or two spatially spaced dots (8, 9) of one symbol (7) form the at least one data bit, preferably two data bits.

4. Document according to claim 1, wherein only one or each data dot (8) of one symbol (7) has n different possible positions ( 12) and forms FIX (Iog2(n)) data bits, preferably wherein n=4.

5. Document according to any one of the preceding claims, wherein each symbol (7) comprises only one data bit.

6. Document according to any one of claims 1 to 4, wherein each symbol (7) comprises multiple data bits, preferably only two data bits.

7. Document according to any one of the preceding claims, wherein each symbol (7) comprises a synchronization region (10) and a data region (1 1).

8. Document according to claim 7, wherein only one dot (8, 9) is located in the synchronization region ( 10) and/or in the data region ( 1 1 ).

9. Document according to claim 7 or 8, wherein the synchronization regions ( 10) and/or the data regions ( 1 1 ) of the symbols (7) are regularly arranged, in particular grid-like.

10. Document according to any one of the preceding claims, wherein each symbol (7) comprises preferably only one synchronization dot (9) and the synchronization dots (9) of the symbols are regularly arranged, in particular grid-like.

1 1. Document according to claim 10, wherein data dots (8) of the symbols (7) are interleaved between the synchronization dots (9) and/or synchronization regions (10) of the symbols (7).

12. Document according to any one of the preceding claims, wherein the symbols (7) are arranged one adjacent the other.

13. Document according to any one of the preceding claims, wherein the encoded portion (3) forms a uniform region of grey scale or halftone.

14. Document according to any one of the preceding claims, wherein the encoded portion (3) forms a background of at least part of the document (1), preferably a constant greyscale background image of at least part of the document (1).

15. Document according to any one of the preceding claims, wherein the document (1) further comprises preferably printed text (2).

16. Document according to claim 15, wherein the encoded portion (3) is interleaved between text lines.

17. Document according to claim 15 or 16, wherein the encoded portion (3) encodes the complete text (2).

18. Document according to claim 15 or 16, wherein the document (1 ) comprises a page and the encoded portion (3) encodes the complete text (2) on the page.

19. Document according to claim 15 or 16, wherein the document (1 ) comprises multiple pages and the encoded portion (3) of one or each page encodes the complete text (2) of all pages.

20. Document according to any one of the preceding claims, wherein the document (1) or at least the encoded portion (3) is printed, preferably laser printed.

21. Document according to any one of the preceding claims, wherein the document (1) is a paper printout or an identification card.

22. Document according to any one of the preceding claims, wherein the maximum size of the dots (8, 9) forming the symbols (7) is at most 1/300 inch, preferably 1/600 inch or less, or is the minimum printable dot size.

23. Document according to any one of the preceding claims, wherein all dots (8, 9) of one symbol (7) and of all adjacent symbols (7) are spatially spaced from each other.

24. Document according to any one of the preceding claims, wherein the symbols (7) having different data bit values consist of the same number of dots (8, 9).

25. Document according to any one of the preceding claims, wherein the encoded portion (3) comprises a fingerprint and/or a digital signature and/or the content of text (2) of the document.

26. Document according to any one of the preceding claims, wherein the document (1) contains more than 200.000 symbols (7) per page when printed at 300 dpi or more than 800.000 symbols (7) per page when printed at 600 dpi.

27. Document according to any one of the preceding claims, wherein the size of each symbol (7) is at most 10/300 inch, preferably 6/300 inch, in particular about 6/600 to 10/600 inch or less.

28. Method for generating a document (1 ) comprising an encoded portion (3) with encoded information, the encoded portion (3) comprising synchronization dots (9) and data dots (8), wherein the synchronization dots (9) are regularly arranged and interleaved with the data dots (9).

29. Method according to claim 28, wherein the encoded portion comprises printed symbols (7), the symbols (7) being unreadable or invisible for human eyes, the symbols (7) being machine readable and encode information as data bits, each symbol (7) forming at least one data bit or consisting of only spatially spaced dots (8, 9).

30. Method according to claim 29, wherein each symbol (7) comprises at least one of: at most two spatially spaced dots (8, 9); and/or the at least one data bit or dot (8) and at least one synchronization bit or dot (9).

31. Method according to claim 29 or 30, wherein only one spatially spaced dot (8, 9) of one symbol (7) has n different possible positions (12) and, thus, forms FIX (Iog2(n)) data bits, preferably wherein n=4.

32. Method according to any one of claims 29 to 30, wherein each symbol (7) comprises multiple data bits, preferably only two data bits.

33. Method according to any one of claims 29 to 32, wherein each symbol (7) comprises a synchronization region (10) and a data region (1 1 ).

34. Method according to claim 33, wherein only one dot is located in the syn- chronization region (10) and/or in the data region (1 1).

35. Method according to claim 33 or 34, wherein the synchronization regions (10) and/or the data regions (1 1 ) of the symbols (7) are regularly arranged, in particular grid-like.

36. Method according to any one of claims 28 to 35, wherein the encoded portion (3) forms a uniform region of grey scale or halftone.

37. Method according to any one of claims 28 to 36, wherein the encoded portion (3) forms a background of at least part of the document (1), preferably a constant grey scale background image of at least part of the document (1).

38. Method according to any one of claims 28 to 37, wherein the document (1) further comprises preferably printed text (2).

39. Method according to claim 38, wherein the encoded portion (3) is interleaved between text lines.

40. Method according to claim 38 or 39, wherein the encoded portion (3) encodes the complete text (2).

41. Method according to claim 38 or 39, wherein the document (1 ) comprises one page and the encoded portion (3) encodes the complete text (2) on the page.

42. Method according to claim 38 or 39, wherein the document comprises multiple pages and the encoded portion (3) of one or each page encodes the complete text (2) of all pages.

43. Method according to any one of claims 28 to 42, wherein the document (1) or at least the encoded portion (3) is printed, preferably laser printed.

44. Method according to any one of claims 28 to 43, wherein the maximum size of printed dots forming the symbols (7) is at most 1/300 inch, preferably 1/600 inch or less.

45. Method according to any one of the claims 28 to 44, wherein all dots (8, 9) of one symbol (7) and of all adjacent symbols (7) are spatially spaced from each other.

46. Method according to any one of claims 28 to 45, wherein the symbols (7) having different data bit values consist of the same number of dots (8, 9).

47. Method according to any one of claims 28 to 46, wherein the document (1) is a paper printout or an identification card.

48. Method according to any one of claims 28 to 47, wherein the encoded portion (3) comprises a fingerprint and/or a digital signature and/or the content of text (2) of the document.

49. Method according to any one of claims 28 to 48, wherein the document (1 ) contains more than 200.000 symbols (7) per page when printed at 300 dpi or more than 800.000 symbols (7) per page when printed at 600 dpi.

50. Method according to any one of claims 28 to 49, wherein the size of each symbol (7) is at most 10/300 inch, preferably 6/300 inch, in particular about

6/600 to 10/600 inch or less.

51. Method according to any one of claims 28 to 50, wherein only black dots (8, 9) are used.

52. Method according to any one of claims 28 to 50, wherein colored dots (8, 9) are used.