Method for analyzing structure of a treatise type of document image

The present invention relates to a method for processing a document image; and, more particularly, to a method for analyzing structure of a treatise type of document image in order to detect a title, an author and an abstract region and recognize the content in each of the regions.

There are many techniques for processing a document image to construct a database system. One of techniques is a document image structure analysis(see, ChunChen Lin, “Logical Structure Analysis of Book Document Image using Contents Information”, ICDAR 97, Vol. II, pp. 1048-1054, August, 1997). According to the document structure analysis, a character recognition process is performed on a table of contents of a book so that the entire logical structure of the book is analyzed. Since, however, in order to utilize this technique, there must be provided a table of the contents of book, therefore, it is impossible to construct a database system of treatise typed of document image.

In order to construct a database system providing a portion of or an entire of treatises contained at each of journal in a form of document image or a hypertext file format, a table of contents having a title, an author and an abstract information has to be generated.

Hitherto, the table of contents having a title, an author and an abstract information is made by a human being. One reason is that a multi-language recognition is very difficult. Generally, the title and the author are represented on two languages. Another reason is that each position of the title, the author and the abstract is different according to each of the journals. Thus, it is difficult to detect of the position thereof. The other reason is that there is not a distinct difference between the title and the author.

Therefore, it is required to automatically detect title, author and abstract regions and recognize the content in each of the regions so as to make a table of the contents of the treatise in the journals.

It is, therefore, a primary object of the invention to provide a method for automatically detecting title, author and abstract regions in document image and recognize the content in each of the regions so as to make a table of the contents of the treatise in the journals.

In accordance with the present invention, there is provided a method for analyzing a structure of a treatise type of document image to make a table of contents having a title, an author and an abstract information, comprising the steps of: dividing the document image into a number of regions and classifying the divided regions into text regions and non-text regions according to attributes of the regions; selecting candidate regions representing an abstract and an introduction, extracting word regions from the candidate regions, and determining an abstract content portion; separating the title and the author using the basic form and the type definition representing an arrangement of each of journals; and recognizing the content of the separated regions to generate said table of contents.

In accordance with another aspect of the present invention, there is provided a computer readable media containing the program, the program having functions of: dividing the document image into a number of regions and classifying the divided regions into text regions and non-text regions according to attributes of the regions; selecting candidate regions representing an abstract and an introduction, and finding word regions from the candidate regions to determine the position of an abstract content portion; separating the title and the author using the basic form and the type definition representing an arrangement of each of journals; and recognizing the content of the separated regions to generate said table of contents.

These and other features of the present invention are more fully shown and described in the drawings and detailed description of this invention. It is to be understood, however, that the description and drawings are for the purpose of illustration and should not be read in a manner that would unduly limit the scope of this invention.

The above and other objects and features of the instant invention will become apparent from the following description of preferred embodiments taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram of an auto tagging system of a treatise type of document image;

FIGS. 2A and 2B show exemplary flowcharts for extracting a word region in a process of analyzing structure of treatise typed of document image in accordance with the present invention;

FIG. 3 describes an exemplary flowchart of determining an abstract content portion in case that two predetermined words representing ‘abstract’ and ‘introduction’ exist and the abstract content portion is positioned at one column;

FIG. 4 depicts a flowchart of searching an abstract content portion in case that two predetermined words representing ‘abstract’ and ‘introduction’ exist and the abstract content portion is positioned at a left column of two columns;

FIG. 5 is a flowchart of searching an abstract content portion in case that two predetermined words representing ‘abstract’ and ‘introduction’ exist and the abstract content portion is positioned at two columns;

FIG. 6 describes a flowchart of searching an abstract content portion in case that a predetermined word representing ‘abstract’ exists and a predetermined word representing ‘introduction’ does not exist and the abstract content portion is positioned at one column;

FIG. 7 shows a flowchart of searching an abstract content portion in case that a predetermined word representing ‘abstract’ exists and a predetermined word representing ‘introduction’ does not exist and the abstract content portion is positioned at two columns;

FIGS. 8A and 8B depict flowcharts of separating a title and an author regions in a process of analyzing structure of treatise typed of document image in accordance with present invention; and

FIG. 9 is an exemplary result screen after analyzing structure of treatise typed of document image in accordance with the present invention.

Referring to FIG. 1, there is shown a block diagram of an auto tagging system of treatise typed of document image in accordance with the present invention.

The auto tagging system recognizes a text, a picture, and a chart from an inputted document image. And then the auto tagging system determines the structure of the inputted document image to automatically generate an HTML(Hyper Text Markup Language) file.

As shown in FIG. 1, a treatise type of document image (300˜400 dpi of TIFF image) is inputted to a treatise document image input unit 11 . In a document image division and attribute classification unit 12 , the inputted image provided from the treatise document image input unit 11 is divided by using a region expansion and merging technique which connects connected components, and also, the attributes of each of the regions is classified based on a black pixel density and the average and the deviation of region spaces. The attributes are text, picture and chart.

And then, the characters contained at the text or chart regions are recognized at a character recognition unit 13 .

A chart region processing unit 14 recognizes the line of the chart and the characters contained at the chart.

On the other hand, in a picture region processing unit 15 , a picture portion is converted to GIF format which is used at the HTML file.

And, a treatise structure detection unit 16 finds the title of a treatise, the author and abstract, and analyzes the structure of the treatise to generate a table of the contents of the treatise. The output from the treatise structure detection unit 16 is used at the HTML file.

A table contents generation and treatise form construction unit 17 utilizes the above described outputs to make the table of contents and to convert the inputted image into the HTML files.

Hereinafter, the process of detecting the position of the abstract, title and author; recognizing the characters corresponding to the abstract, title and author, and generating the table of the contents is described. Since the other process described in FIG. 1 is common technique, a detailed description is omitted.

The inventive process includes five processes as follows:

[1] first process for selecting candidate regions;

[2] second process for detecting distinct word regions;

[3] third process for determining abstract content portions;

[4] fourth process for separating the title and the author; and

[5] fifth process for recognition the characters.

In a first process, a set of candidate regions is selected. If distinct words representing ‘abstract’ and ‘introduction’ are searched at the candidate regions, the searching time can be reduced. In order to select the candidate regions, an average length of text regions is calculated by using Eq. (1) as follows: Ave_L x = ∑ cond  ( Attr  ( R i ) = TEXT )  L x  ( R i ) n  ( R i , cond  ( Attr  ( R i ) = TEXT ) )

In Eq. (1), Lx(R i ) is a horizontal length of ?th region, Attr(R i ) is an attribute of ith region, and n(R i , cond(Attr(R i )=TEXT)) is the number of text regions.

After calculating the average length of the text regions, the regions each of which has the length being smaller than 8 times of the average length is determined to the candidate regions. This is programmed as follows:

Second, a distinct word region is detected. By using the character recognition unit 13 , a distinct word region such like [YOYAK] ([??]) (‘YOYAK(??)’ means the abstract in Korean language), [Abstract], [Summary], [SEORON] ([??]) (‘SEORON([??])’ means the introduction in Korean language), [Introduction] is detected according to following probability standards: in case that the region corresponding to [YOYAK] or [SEORON] is detected, the detection probability must be 100%; when the region corresponding to [Abstract] or [Summary] is detected, the detection probability must be over 80%; and if the region corresponding to the distinct word [Introduction] is detected, the detection probability must be over 70%.

After detecting the distinct word region, the distinct word region is extracted. This process is described with reference to FIGS. 2A and 2B. FIGS. 2A and 2B show exemplary flowcharts for extracting a word region within a process of analyzing structure of treatise typed of document image.

As shown in FIGS. 2A and 2B, in step 201 , a candidate region is inputted. In case that the candidate region is inputted, in step 202 , the number of the candidate regions is checked. And then, in step 203 , the number of characters in the candidate region is obtained.

Thereafter, in steps 204 to 211 , each of the characters in the candidate region is compared to each of the characters in the distinct word to determine how many characters are coincided.

And then, in steps 213 to 215 , the ratio of a numerator to a denominator is calculated. The denominator is the bigger one between the number of characters of each of the candidate regions and the number of characters of each of the predetermined words. The numerator is the number of the coincided characters. After calculating the ratio, in steps 216 and 217 , if the calculated ratio is larger than the predetermined determination probability, the candidate region is determined to be the word region.

The process of detecting the distinct word region is programmed as follows:

Wherein, a distinct word and DB_Table[ ] are shown at Table I.

In third process, the abstract content portion is determined.

The region between the distinct word [YOYAK], [Abstract], [Summary] and another distinct word [SEORON], [Introduction] detected at the first process, is the portion of the abstract content. The abstract content portion is classified with five types. The abstract content portion is detected according to above described second process.

First type is in case that two predetermined words representing ‘abstract’ and ‘introduction’ are existed and the abstract content portion is positioned at one column. The process for searching the abstract content portion in the first type is described with reference to FIG. 3 .

As shown in FIG. 3, in steps 301 to 303 , in case that two words representing the abstract and the introduction exist and the abstract content portion is made by one column in the document image, a region of which vertical direction start coordinate is equal to or larger than that of a predetermined region corresponding to the abstract region and is smaller than that of a predetermined region corresponding to the introduction region is selected.

And then, in steps 307 to 309 , a minimum value(min) and a maximum value(max). In case that the vertical direction last coordinate of the elected region is larger than that of the comparing region, the minimum value is replaced with the vertical region start coordinate of the selected region and the maximum value is replaced with the vertical direction last coordinate of the comparing region. On the other hand, in case that the vertical direction last coordinate of the elected region is less than that of the comparing region, the maximum value is replaced with the vertical region start coordinate of the selected region and the minimum value is replaced with the vertical direction last coordinate of the comparing region.

Thereafter, in steps 310 and 312 , it is determined whether the selected region is the abstract content region or not by using both the maximum and the minimum values. For example, in case that ‘max−min<0’, the selected region is determined to the abstract content portion.

The above described process for searching the abstract content portion may be programmed as follows:

wherein,

SY(R i ) is the start Y coordinate of ith region, and

SY(YOYAK or Abstract or Summary) is the start Y coordinate of region corresponding to the distinct word abstract.

Second type of the five types is in case that two predetermined words representing ‘abstract’ and ‘introduction’ are existed and the abstract content portion is positioned at a left column of two columns. The process for searching the abstract content portion in the second type is described with reference to FIG. 4 .

As shown in FIG. 4, in step 403 , in case that two words representing the abstract and the introduction exist and the abstract content portion is positioned at the left column of two columns in the document image, a region of which the horizontal direction last coordinate is located at the left of the center of the image document, is selected.

Thereafter, in step 404 to 406 , there is selected a region, among the selected regions in step 403 , of which the vertical direction start coordinate is equal or larger than that of a predetermined region corresponding to the abstract region and is smaller than that of another predetermined region corresponding to the introduction region.

The above described process for searching the abstract content portion may be programmed as follows:

Third type of the five types is in case that two predetermined words representing ‘abstract’ and ‘introduction’ are existed and the abstract content portion is positioned at both columns. The process for searching the abstract content portion in the third type is described with reference to FIG. 5 .

As shown in FIG. 5, in steps 503 and 504 , in case that two words representing the abstract and the introduction exist and the abstract content portion is positioned the left and the right columns in the document image, there is selected a region of which the horizontal direction last coordinate is located at the left of the center of the document image and the vertical direction start coordinate is equal to or larger than that of a predetermined region corresponding to the abstract region.

Thereafter, in steps 505 to 507 , there is selected a region, among the selected regions in steps 503 and 504 , of which the horizontal direction start coordinate is located at the right-hand of the center of the document image and the vertical direction last coordinate is smaller than that of the region corresponding to the introduction.

The above described process for searching the abstract content portion may be programmed as follows:

Fourth type of the five types is in case that the distinct word representing ‘abstract’ exists and the distinct word representing ‘introduction’ does not exist and the abstract content portion is positioned at one column. The process for searching the abstract content portion in the fourth type is described with reference to FIG. 6 .

As shown in FIG. 6, in steps 601 and 602 , in case that the word representing the abstract exists and the word representing the introduction does not exist and the abstract content portion is made by one column in the document image, there is selected a region of which vertical direction start coordinate is equal to or larger than that of a predetermined region corresponding to the abstract region.

And then, in steps 603 to 607 , a minimum value(min) and a maximum value(max) is obtained. In case that the vertical direction last coordinate of the selected region is larger than that of the comparing region, the minimum value is replaced with the vertical region start coordinate of the selected region and the maximum value is replaced with the vertical direction last coordinate of the comparing region. Otherwise, in case that the vertical direction last coordinate of the elected region is less than that of the comparing region, the maximum value is replaced with the vertical region start coordinate of the selected region and the minimum value is replaced with the vertical direction last coordinate of the comparing region.

Thereafter, in step 610 , it is determined whether the selected region is the abstract content region or not by using both the maximum and the minimum values. For example, if ‘max−min>=0’, the selected region is determined the abstract content portion.

The above described process for searching the abstract content portion may be programmed as follows:

The last of the five types is in case that the distinct words representing ‘abstract’ and ‘introduction’ do not exist and the abstract content portion is positioned at two columns. The process for searching the abstract content portion in the fifth type is described with reference to FIG. 7 .

As shown in FIG. 7, in steps 701 to 706 , in case that the word representing the introduction does not exist and the word representing the abstract exists and the abstract content portion is positioned at two columns in the document image, there is selected a region of which the horizontal direction last coordinate is located at the left of the center of the image document and the vertical direction start coordinate is equal to or larger than that of the region corresponding to the abstract region.

And then, there is selected a region of which the horizontal direction start coordinate is located at the right-hand of the center of the document image. The finally selected region is determined to the abstract content portion.

The above described process for searching the abstract content portion may be programmed as follows:

After detecting the abstract content portion, the title and author is separated.

The separation is classified into three types according to the number of the regions. One of the regions represents volume, number(No.) and the other region represents the title and author.

First, in case that the number of regions containing the title and author is two, the upper region is title and the lower region is the author.

Second, in case that the number of regions containing the title and author is four, a font size of the characters in each of the regions is compared to separate the title and author. For example, first and second, or first and third regions are title and the other regions are author.

Third, in case that the title and author is separated by the structure form, it is determined whether the number of the regions is equal to the number of items of the type definition(TD) and whether the region arrangement structure is identical to the item arrangement structure. Type definition(TD) is a structure form of each of the journals. Table II represents example of TD of Korean published science journals.

Hereinafter, the process of separating the title and the author is described in reference with FIGS. 8A and 8B.

As shown in FIGS. 8A and 8B, in steps 801 to 803 , in case that the number of regions containing the title and author is two and the size of the characters of a second region less than a predetermined value(i.e., 0.9), the first region is set to the title and the second region is set to the author.

And then, in steps 805 to 812 , in case that the number of regions containing the title and author is four and the sizes of the characters of a first and a second regions is larger than a predetermined time(i.e., 0.9) of the size of the other regions, the first and second regions are set to the title and the third and fourth regions are set to the author.

Thereafter, in steps 814 to 816 , in case that the number of regions containing the title and author is four and the sizes of the characters of a first and a third regions are larger than a predetermined time(i.e., 0.9) of the size of the other regions, the first and third regions are set to the title and the second and fourth regions are set to the author.

In steps 818 to 821 , in case that the number of regions containing the title and author is not two or four, the title and the author is separated by determining whether the number of the regions is equal to the number of items of the type definition and whether the region arrangement structure is identical to the item arrangement structure.

The above described process for searching the abstract content portion may be programmed as follows:

R[I]:ith region

After separating the title and the author, the content contained at each of the regions is recognized by using a common character recognition system. The regions are the title, the author and the abstract regions. Therefore the table of contents is generated as shown in FIG. 9 . FIG. 9 show an exemplary result screen after analyzing structure of treatise typed of document image in accordance with the present invention.

Although the preferred embodiments of the invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.