Unlocking Text from Embedded-Font PDFs: A pytesseract OCR Tutorial

Extracting text from a PDF is usually straightforward when it’s in English and doesn’t have embedded fonts. However, once those assumptions are removed, it becomes challenging to use basic python libraries like pdfminer or pdfplumber. Last month, I was tasked with extracting text from a Gujarati-language PDF and importing data fields such as name, address, city, etc., into JSON format.

If the font is embedded in the PDF itself, simple copy-pasting won’t work, and using pdfplumber will return unreadable junk text. Therefore, I had to convert each PDF page to an image and then apply OCR using the pytesseract library to “scan” the page instead of just reading it. This tutorial will show you how to do just that.

Things you will need

pdfplumber (Python library)
pdf2image (Python library)
pytesseract (Python library)
tesseract-ocr

You can install the Python libraries using pip commands as shown below. For Tesseract-OCR, download and install the software from the official site. pytesseract is just a wrapper around the tesseract software.


pip <span>install </span>pdfplumber
pip <span>install </span>pdf2image
pip <span>install </span>pytesseract
pip <span>install </span>pdfplumber
pip <span>install </span>pdf2image
pip <span>install </span>pytesseract
pip install pdfplumber
pip install pdf2image
pip install pytesseract

Enter fullscreen mode Exit fullscreen mode

Converting the PDF page to an image

The first step is to convert your PDF page to an image. This extract_text_from_pdf() function does exactly that-you pass the PDF path and the page_num (zero indexed) as parameters. Note that I’m converting the page to black and white first for clarity, this is optional.


<span># Extract text from a specific page of a PDF </span><span>def</span> <span>extract_text_from_pdf</span><span>(</span><span>pdf_path</span><span>,</span> <span>page_num</span><span>):</span>
    <span># Use pdfplumber to open the PDF </span>    <span>pdf</span> <span>=</span> <span>pdfplumber</span><span>.</span><span>open</span><span>(</span><span>pdf_path</span><span>)</span>
    <span>print</span><span>(</span><span>f</span><span>"</span><span>extracting page </span><span>{</span><span>page_num</span><span>}</span><span>..</span><span>"</span><span>)</span>
    <span>page</span> <span>=</span> <span>pdf</span><span>.</span><span>pages</span><span>[</span><span>page_num</span><span>]</span>
    <span>images</span> <span>=</span> <span>convert_from_path</span><span>(</span><span>pdf_path</span><span>,</span> <span>first_page</span><span>=</span><span>page_num</span><span>+</span><span>1</span><span>,</span> <span>last_page</span><span>=</span><span>page_num</span><span>+</span><span>1</span><span>)</span>
    <span>image</span> <span>=</span> <span>images</span><span>[</span><span>0</span><span>]</span>
    <span># Convert to black and white </span>    <span>bw_image</span> <span>=</span> <span>convert_to_bw</span><span>(</span><span>image</span><span>)</span>
    <span># Save the B&W image for debugging (optional) </span>    <span>#bw_image.save("bw_page.png") </span>    <span># Perform OCR on the B&W image </span>    <span>e_text</span> <span>=</span> <span>ocr_image</span><span>(</span><span>bw_image</span><span>)</span>
    <span>open</span><span>(</span><span>'</span><span>out.txt</span><span>'</span><span>,</span> <span>'</span><span>w</span><span>'</span><span>,</span> <span>encoding</span><span>=</span><span>'</span><span>utf-8</span><span>'</span><span>).</span><span>write</span><span>(</span><span>e_text</span><span>)</span>
    <span>#print("output written to file.") </span>    <span>try</span><span>:</span>
        <span>process_text</span><span>(</span><span>page_num</span><span>,</span> <span>e_text</span><span>)</span>
    <span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span>
        <span>print</span><span>(</span><span>"</span><span>Error occurred:</span><span>"</span><span>,</span> <span>e</span><span>)</span>
    <span>print</span><span>(</span><span>"</span><span>done..</span><span>"</span><span>)</span>
<span># Convert image to black and white </span><span>def</span> <span>convert_to_bw</span><span>(</span><span>image</span><span>):</span>
    <span># Convert to grayscale </span>    <span>gray</span> <span>=</span> <span>image</span><span>.</span><span>convert</span><span>(</span><span>'</span><span>L</span><span>'</span><span>)</span>
    <span># Apply threshold to convert to pure black and white </span>    <span>bw</span> <span>=</span> <span>gray</span><span>.</span><span>point</span><span>(</span><span>lambda</span> <span>x</span><span>:</span> <span>0</span> <span>if</span> <span>x</span> <span><</span> <span>128</span> <span>else</span> <span>255</span><span>,</span> <span>'</span><span>1</span><span>'</span><span>)</span>
    <span>return</span> <span>bw</span>
<span># Perform OCR using Tesseract on a given image </span><span>def</span> <span>ocr_image</span><span>(</span><span>image_path</span><span>):</span>
    <span>try</span><span>:</span>
        <span># Perform OCR </span>        <span>custom_config</span> <span>=</span> <span>r</span><span>'</span><span>--oem 3 --psm 6 -l guj+eng</span><span>'</span>
        <span>text</span> <span>=</span> <span>pytesseract</span><span>.</span><span>image_to_string</span><span>(</span><span>image_path</span><span>,</span> <span>config</span><span>=</span><span>custom_config</span><span>)</span>  <span># --psm 6 treats the image as a block of text </span>        <span>return</span> <span>text</span>
    <span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span>
        <span>print</span><span>(</span><span>f</span><span>"</span><span>Error during OCR: </span><span>{</span><span>e</span><span>}</span><span>"</span><span>)</span>
        <span>return</span> <span>None</span>
<span># Extract text from a specific page of a PDF </span><span>def</span> <span>extract_text_from_pdf</span><span>(</span><span>pdf_path</span><span>,</span> <span>page_num</span><span>):</span>
    <span># Use pdfplumber to open the PDF </span>    <span>pdf</span> <span>=</span> <span>pdfplumber</span><span>.</span><span>open</span><span>(</span><span>pdf_path</span><span>)</span>
    <span>print</span><span>(</span><span>f</span><span>"</span><span>extracting page </span><span>{</span><span>page_num</span><span>}</span><span>..</span><span>"</span><span>)</span>
    <span>page</span> <span>=</span> <span>pdf</span><span>.</span><span>pages</span><span>[</span><span>page_num</span><span>]</span>
    <span>images</span> <span>=</span> <span>convert_from_path</span><span>(</span><span>pdf_path</span><span>,</span> <span>first_page</span><span>=</span><span>page_num</span><span>+</span><span>1</span><span>,</span> <span>last_page</span><span>=</span><span>page_num</span><span>+</span><span>1</span><span>)</span>
    <span>image</span> <span>=</span> <span>images</span><span>[</span><span>0</span><span>]</span>
    <span># Convert to black and white </span>    <span>bw_image</span> <span>=</span> <span>convert_to_bw</span><span>(</span><span>image</span><span>)</span>
    <span># Save the B&W image for debugging (optional) </span>    <span>#bw_image.save("bw_page.png") </span>    <span># Perform OCR on the B&W image </span>    <span>e_text</span> <span>=</span> <span>ocr_image</span><span>(</span><span>bw_image</span><span>)</span>
    <span>open</span><span>(</span><span>'</span><span>out.txt</span><span>'</span><span>,</span> <span>'</span><span>w</span><span>'</span><span>,</span> <span>encoding</span><span>=</span><span>'</span><span>utf-8</span><span>'</span><span>).</span><span>write</span><span>(</span><span>e_text</span><span>)</span>
    <span>#print("output written to file.") </span>    <span>try</span><span>:</span>
        <span>process_text</span><span>(</span><span>page_num</span><span>,</span> <span>e_text</span><span>)</span>
    <span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span>
        <span>print</span><span>(</span><span>"</span><span>Error occurred:</span><span>"</span><span>,</span> <span>e</span><span>)</span>
    <span>print</span><span>(</span><span>"</span><span>done..</span><span>"</span><span>)</span>

<span># Convert image to black and white </span><span>def</span> <span>convert_to_bw</span><span>(</span><span>image</span><span>):</span>
    <span># Convert to grayscale </span>    <span>gray</span> <span>=</span> <span>image</span><span>.</span><span>convert</span><span>(</span><span>'</span><span>L</span><span>'</span><span>)</span>
    <span># Apply threshold to convert to pure black and white </span>    <span>bw</span> <span>=</span> <span>gray</span><span>.</span><span>point</span><span>(</span><span>lambda</span> <span>x</span><span>:</span> <span>0</span> <span>if</span> <span>x</span> <span><</span> <span>128</span> <span>else</span> <span>255</span><span>,</span> <span>'</span><span>1</span><span>'</span><span>)</span>
    <span>return</span> <span>bw</span>

<span># Perform OCR using Tesseract on a given image </span><span>def</span> <span>ocr_image</span><span>(</span><span>image_path</span><span>):</span>
    <span>try</span><span>:</span>
        <span># Perform OCR </span>        <span>custom_config</span> <span>=</span> <span>r</span><span>'</span><span>--oem 3 --psm 6 -l guj+eng</span><span>'</span>
        <span>text</span> <span>=</span> <span>pytesseract</span><span>.</span><span>image_to_string</span><span>(</span><span>image_path</span><span>,</span> <span>config</span><span>=</span><span>custom_config</span><span>)</span>  <span># --psm 6 treats the image as a block of text </span>        <span>return</span> <span>text</span>
    <span>except</span> <span>Exception</span> <span>as</span> <span>e</span><span>:</span>
        <span>print</span><span>(</span><span>f</span><span>"</span><span>Error during OCR: </span><span>{</span><span>e</span><span>}</span><span>"</span><span>)</span>
        <span>return</span> <span>None</span>
# Extract text from a specific page of a PDF def extract_text_from_pdf(pdf_path, page_num):
    # Use pdfplumber to open the PDF     pdf = pdfplumber.open(pdf_path)
    print(f"extracting page {page_num}..")
    page = pdf.pages[page_num]
    images = convert_from_path(pdf_path, first_page=page_num+1, last_page=page_num+1)
    image = images[0]
    # Convert to black and white     bw_image = convert_to_bw(image)
    # Save the B&W image for debugging (optional)     #bw_image.save("bw_page.png")     # Perform OCR on the B&W image     e_text = ocr_image(bw_image)
    open('out.txt', 'w', encoding='utf-8').write(e_text)
    #print("output written to file.")     try:
        process_text(page_num, e_text)
    except Exception as e:
        print("Error occurred:", e)
    print("done..")

# Convert image to black and white def convert_to_bw(image):
    # Convert to grayscale     gray = image.convert('L')
    # Apply threshold to convert to pure black and white     bw = gray.point(lambda x: 0 if x < 128 else 255, '1')
    return bw

# Perform OCR using Tesseract on a given image def ocr_image(image_path):
    try:
        # Perform OCR         custom_config = r'--oem 3 --psm 6 -l guj+eng'
        text = pytesseract.image_to_string(image_path, config=custom_config)  # --psm 6 treats the image as a block of text         return text
    except Exception as e:
        print(f"Error during OCR: {e}")
        return None

Enter fullscreen mode Exit fullscreen mode

The ocr_image() function uses pytesseract to extract text from the image through OCR. The technical parameters like --oem and --psm control how the image is processed, and the -l guj+eng parameter sets the languages to be read. Since this PDF contained occasional English text, I used guj+eng.

Processing the text

Once you’ve imported the text using OCR, you can parse it in the format you want. This works similarly to other PDF libraries like pdfplumber or pypdf2.


<span>nums</span> <span>=</span> <span>[</span><span>'</span><span>0</span><span>'</span><span>,</span> <span>'</span><span>૧</span><span>'</span><span>,</span> <span>'</span><span>૨</span><span>'</span><span>,</span> <span>'</span><span>૩</span><span>'</span><span>,</span> <span>'</span><span>૪</span><span>'</span><span>,</span> <span>'</span><span>૫</span><span>'</span><span>,</span> <span>'</span><span>૬</span><span>'</span><span>,</span> <span>'</span><span>૭</span><span>'</span><span>,</span> <span>'</span><span>૮</span><span>'</span><span>,</span> <span>'</span><span>૯</span><span>'</span><span>]</span>
<span>def</span> <span>process_text</span><span>(</span><span>page_num</span><span>,</span> <span>e_text</span><span>):</span>
    <span>obj</span> <span>=</span> <span>None</span>
    <span>last_surname</span> <span>=</span> <span>None</span>
    <span>last_kramank</span> <span>=</span> <span>None</span>
    <span>print</span><span>(</span><span>f</span><span>"</span><span>processing page </span><span>{</span><span>page_num</span><span>}</span><span>..</span><span>"</span><span>)</span>
    <span>for</span> <span>line</span> <span>in</span> <span>e_text</span><span>.</span><span>splitlines</span><span>():</span>
        <span>line</span> <span>=</span> <span>line</span><span>.</span><span>replace</span><span>(</span><span>'</span><span>|</span><span>'</span><span>,</span> <span>''</span><span>).</span><span>replace</span><span>(</span><span>'</span><span>[</span><span>'</span><span>,</span> <span>''</span><span>).</span><span>replace</span><span>(</span><span>'</span><span>]</span><span>'</span><span>,</span> <span>''</span><span>)</span>
        <span>parts</span> <span>=</span> <span>[</span><span>word</span> <span>for</span> <span>word</span> <span>in</span> <span>line</span><span>.</span><span>split</span><span>(</span><span>'</span><span> </span><span>'</span><span>)</span> <span>if</span> <span>word</span><span>]</span>
        <span>if</span> <span>len</span><span>(</span><span>parts</span><span>)</span> <span>==</span> <span>0</span><span>:</span> <span>continue</span>
        <span>new_rec</span> <span>=</span> <span>True</span>
        <span>for</span> <span>char</span> <span>in</span> <span>parts</span><span>[</span><span>0</span><span>]:</span>
            <span>if</span> <span>char</span> <span>not</span> <span>in</span> <span>nums</span><span>:</span>
                <span>new_rec</span> <span>=</span> <span>False</span>
                <span>break</span>
        <span>if</span> <span>len</span><span>(</span><span>parts</span><span>)</span> <span><</span> <span>2</span><span>:</span> <span>continue</span>
        <span>if</span> <span>new_rec</span> <span>and</span> <span>len</span><span>(</span><span>parts</span><span>[</span><span>0</span><span>])</span> <span>>=</span> <span>2</span><span>:</span> <span># numbered line </span>            <span>if</span> <span>len</span><span>(</span><span>parts</span><span>)</span> <span><</span> <span>9</span><span>:</span> <span>continue</span>
            <span>if</span> <span>obj</span><span>:</span> <span>records</span><span>.</span><span>append</span><span>(</span><span>obj</span><span>)</span>
            <span>obj</span> <span>=</span> <span>{}</span>
            <span>last_surname</span> <span>=</span> <span>parts</span><span>[</span><span>1</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>kramank</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>0</span><span>]</span>
            <span>last_kramank</span> <span>=</span> <span>parts</span><span>[</span><span>0</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>full_name</span><span>'</span><span>]</span> <span>=</span> <span>'</span><span> </span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>1</span><span>:</span><span>4</span><span>])</span>
            <span>obj</span><span>[</span><span>'</span><span>surname</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>1</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>pdf_page_num</span><span>'</span><span>]</span> <span>=</span> <span>page_num</span> <span>+</span> <span>1</span>
            <span>obj</span><span>[</span><span>'</span><span>registered_by</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>4</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>village_vatan</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>5</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>village_mosal</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>6</span><span>]</span>
            <span>if</span> <span>parts</span><span>[</span><span>8</span><span>]</span> <span>==</span> <span>'</span><span>વર્ષ</span><span>'</span><span>:</span>
                <span>idx</span> <span>=</span> <span>7</span>
                <span>obj</span><span>[</span><span>'</span><span>dob</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>]</span> <span>+</span> <span>'</span><span> વર્ષ</span><span>'</span>
                <span>idx</span> <span>+=</span> <span>1</span>
            <span>elif</span> <span>len</span><span>(</span><span>parts</span><span>[</span><span>7</span><span>])</span> <span>==</span> <span>8</span> <span>and</span> <span>parts</span><span>[</span><span>7</span><span>][</span><span>2</span><span>]</span> <span>==</span> <span>'</span><span>-</span><span>'</span><span>:</span>
                <span>idx</span> <span>=</span> <span>7</span>
                <span>obj</span><span>[</span><span>'</span><span>dob</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>]</span>
            <span>else</span><span>:</span>
                <span>print</span><span>(</span><span>"</span><span>warning: no date</span><span>"</span><span>)</span>
                <span>idx</span> <span>=</span> <span>6</span>
            <span>obj</span><span>[</span><span>'</span><span>marital_status</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>+</span><span>1</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>extra_fields</span><span>'</span><span>]</span> <span>=</span> <span>'</span><span>::</span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>idx</span><span>+</span><span>2</span><span>:</span><span>-</span><span>2</span><span>])</span>
            <span>obj</span><span>[</span><span>'</span><span>blood_group</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>-</span><span>1</span><span>]</span>
        <span>elif</span> <span>parts</span><span>[</span><span>0</span><span>]</span> <span>==</span> <span>last_surname</span><span>:</span> <span># new member in existing family </span>            <span>if</span> <span>obj</span><span>:</span> <span>records</span><span>.</span><span>append</span><span>(</span><span>obj</span><span>)</span>
            <span>obj</span> <span>=</span> <span>{}</span>
            <span>obj</span><span>[</span><span>'</span><span>kramank</span><span>'</span><span>]</span> <span>=</span> <span>last_kramank</span>
            <span>obj</span><span>[</span><span>'</span><span>surname</span><span>'</span><span>]</span> <span>=</span> <span>last_surname</span>
            <span>obj</span><span>[</span><span>'</span><span>full_name</span><span>'</span><span>]</span> <span>=</span> <span>'</span><span> </span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>0</span><span>:</span><span>3</span><span>])</span>
            <span>obj</span><span>[</span><span>'</span><span>pdf_page_num</span><span>'</span><span>]</span> <span>=</span> <span>page_num</span> <span>+</span> <span>1</span>
            <span>obj</span><span>[</span><span>'</span><span>registered_by</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>3</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>village_vatan</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>4</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>village_mosal</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>5</span><span>]</span>
            <span>if</span> <span>len</span><span>(</span><span>parts</span><span>)</span> <span><=</span> <span>6</span><span>:</span> <span>continue</span>
            <span>if</span> <span>parts</span><span>[</span><span>7</span><span>]</span> <span>==</span> <span>'</span><span>વર્ષ</span><span>'</span><span>:</span> <span># date exists </span>                <span>idx</span> <span>=</span> <span>6</span>
                <span>obj</span><span>[</span><span>'</span><span>dob</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>]</span> <span>+</span> <span>'</span><span> વર્ષ</span><span>'</span>
                <span>idx</span> <span>+=</span> <span>1</span>
            <span>elif</span> <span>len</span><span>(</span><span>parts</span><span>[</span><span>6</span><span>])</span> <span>==</span> <span>8</span> <span>and</span> <span>parts</span><span>[</span><span>6</span><span>][</span><span>2</span><span>]</span> <span>==</span> <span>'</span><span>-</span><span>'</span><span>:</span>
                <span>idx</span> <span>=</span> <span>6</span>
                <span>obj</span><span>[</span><span>'</span><span>dob</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>]</span>
            <span>else</span><span>:</span>
                <span>print</span><span>(</span><span>"</span><span>warning: no date</span><span>"</span><span>)</span>
                <span>idx</span> <span>=</span> <span>5</span>
            <span>obj</span><span>[</span><span>'</span><span>marital_status</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>+</span><span>1</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>extra_fields</span><span>'</span><span>]</span> <span>=</span> <span>'</span><span>::</span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>idx</span><span>+</span><span>2</span><span>:</span><span>-</span><span>2</span><span>])</span>
            <span>obj</span><span>[</span><span>'</span><span>blood_group</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>-</span><span>1</span><span>]</span>
        <span>elif</span> <span>obj</span><span>:</span> <span># continuation lines </span>            <span>if </span><span>(</span><span>"</span><span>(</span><span>"</span> <span>in</span> <span>line</span> <span>and</span> <span>"</span><span>)</span><span>"</span> <span>in</span> <span>line</span><span>)</span> <span>or</span> <span>"</span><span>મો.ઃ</span><span>"</span> <span>in</span> <span>line</span><span>:</span>
                <span>obj</span><span>[</span><span>'</span><span>extra_fields</span><span>'</span><span>]</span> <span>+=</span> <span>'</span><span> </span><span>'</span> <span>+</span> <span>'</span><span>::</span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>0</span><span>:])</span>
    <span>if</span> <span>obj</span><span>:</span> <span>records</span><span>.</span><span>append</span><span>(</span><span>obj</span><span>)</span>        
    <span>jstr</span> <span>=</span> <span>json</span><span>.</span><span>dumps</span><span>(</span><span>records</span><span>,</span> <span>indent</span><span>=</span><span>4</span><span>)</span>
    <span>open</span><span>(</span><span>"</span><span>guj.json</span><span>"</span><span>,</span> <span>'</span><span>w</span><span>'</span><span>,</span> <span>encoding</span><span>=</span><span>'</span><span>utf-8</span><span>'</span><span>).</span><span>write</span><span>(</span><span>jstr</span><span>)</span>
    <span>print</span><span>(</span><span>f</span><span>"</span><span>written page </span><span>{</span><span>page_num</span><span>}</span><span> to json..</span><span>"</span><span>)</span>
<span>nums</span> <span>=</span> <span>[</span><span>'</span><span>0</span><span>'</span><span>,</span> <span>'</span><span>૧</span><span>'</span><span>,</span> <span>'</span><span>૨</span><span>'</span><span>,</span> <span>'</span><span>૩</span><span>'</span><span>,</span> <span>'</span><span>૪</span><span>'</span><span>,</span> <span>'</span><span>૫</span><span>'</span><span>,</span> <span>'</span><span>૬</span><span>'</span><span>,</span> <span>'</span><span>૭</span><span>'</span><span>,</span> <span>'</span><span>૮</span><span>'</span><span>,</span> <span>'</span><span>૯</span><span>'</span><span>]</span>

<span>def</span> <span>process_text</span><span>(</span><span>page_num</span><span>,</span> <span>e_text</span><span>):</span>
    <span>obj</span> <span>=</span> <span>None</span>
    <span>last_surname</span> <span>=</span> <span>None</span>
    <span>last_kramank</span> <span>=</span> <span>None</span>
    <span>print</span><span>(</span><span>f</span><span>"</span><span>processing page </span><span>{</span><span>page_num</span><span>}</span><span>..</span><span>"</span><span>)</span>
    <span>for</span> <span>line</span> <span>in</span> <span>e_text</span><span>.</span><span>splitlines</span><span>():</span>
        <span>line</span> <span>=</span> <span>line</span><span>.</span><span>replace</span><span>(</span><span>'</span><span>|</span><span>'</span><span>,</span> <span>''</span><span>).</span><span>replace</span><span>(</span><span>'</span><span>[</span><span>'</span><span>,</span> <span>''</span><span>).</span><span>replace</span><span>(</span><span>'</span><span>]</span><span>'</span><span>,</span> <span>''</span><span>)</span>
        <span>parts</span> <span>=</span> <span>[</span><span>word</span> <span>for</span> <span>word</span> <span>in</span> <span>line</span><span>.</span><span>split</span><span>(</span><span>'</span><span> </span><span>'</span><span>)</span> <span>if</span> <span>word</span><span>]</span>
        <span>if</span> <span>len</span><span>(</span><span>parts</span><span>)</span> <span>==</span> <span>0</span><span>:</span> <span>continue</span>
        <span>new_rec</span> <span>=</span> <span>True</span>
        <span>for</span> <span>char</span> <span>in</span> <span>parts</span><span>[</span><span>0</span><span>]:</span>
            <span>if</span> <span>char</span> <span>not</span> <span>in</span> <span>nums</span><span>:</span>
                <span>new_rec</span> <span>=</span> <span>False</span>
                <span>break</span>
        <span>if</span> <span>len</span><span>(</span><span>parts</span><span>)</span> <span><</span> <span>2</span><span>:</span> <span>continue</span>

        <span>if</span> <span>new_rec</span> <span>and</span> <span>len</span><span>(</span><span>parts</span><span>[</span><span>0</span><span>])</span> <span>>=</span> <span>2</span><span>:</span> <span># numbered line </span>            <span>if</span> <span>len</span><span>(</span><span>parts</span><span>)</span> <span><</span> <span>9</span><span>:</span> <span>continue</span>
            <span>if</span> <span>obj</span><span>:</span> <span>records</span><span>.</span><span>append</span><span>(</span><span>obj</span><span>)</span>
            <span>obj</span> <span>=</span> <span>{}</span>
            <span>last_surname</span> <span>=</span> <span>parts</span><span>[</span><span>1</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>kramank</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>0</span><span>]</span>
            <span>last_kramank</span> <span>=</span> <span>parts</span><span>[</span><span>0</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>full_name</span><span>'</span><span>]</span> <span>=</span> <span>'</span><span> </span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>1</span><span>:</span><span>4</span><span>])</span>
            <span>obj</span><span>[</span><span>'</span><span>surname</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>1</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>pdf_page_num</span><span>'</span><span>]</span> <span>=</span> <span>page_num</span> <span>+</span> <span>1</span>
            <span>obj</span><span>[</span><span>'</span><span>registered_by</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>4</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>village_vatan</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>5</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>village_mosal</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>6</span><span>]</span>
            <span>if</span> <span>parts</span><span>[</span><span>8</span><span>]</span> <span>==</span> <span>'</span><span>વર્ષ</span><span>'</span><span>:</span>
                <span>idx</span> <span>=</span> <span>7</span>
                <span>obj</span><span>[</span><span>'</span><span>dob</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>]</span> <span>+</span> <span>'</span><span> વર્ષ</span><span>'</span>
                <span>idx</span> <span>+=</span> <span>1</span>
            <span>elif</span> <span>len</span><span>(</span><span>parts</span><span>[</span><span>7</span><span>])</span> <span>==</span> <span>8</span> <span>and</span> <span>parts</span><span>[</span><span>7</span><span>][</span><span>2</span><span>]</span> <span>==</span> <span>'</span><span>-</span><span>'</span><span>:</span>
                <span>idx</span> <span>=</span> <span>7</span>
                <span>obj</span><span>[</span><span>'</span><span>dob</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>]</span>
            <span>else</span><span>:</span>
                <span>print</span><span>(</span><span>"</span><span>warning: no date</span><span>"</span><span>)</span>
                <span>idx</span> <span>=</span> <span>6</span>
            <span>obj</span><span>[</span><span>'</span><span>marital_status</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>+</span><span>1</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>extra_fields</span><span>'</span><span>]</span> <span>=</span> <span>'</span><span>::</span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>idx</span><span>+</span><span>2</span><span>:</span><span>-</span><span>2</span><span>])</span>
            <span>obj</span><span>[</span><span>'</span><span>blood_group</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>-</span><span>1</span><span>]</span>
        <span>elif</span> <span>parts</span><span>[</span><span>0</span><span>]</span> <span>==</span> <span>last_surname</span><span>:</span> <span># new member in existing family </span>            <span>if</span> <span>obj</span><span>:</span> <span>records</span><span>.</span><span>append</span><span>(</span><span>obj</span><span>)</span>
            <span>obj</span> <span>=</span> <span>{}</span>
            <span>obj</span><span>[</span><span>'</span><span>kramank</span><span>'</span><span>]</span> <span>=</span> <span>last_kramank</span>
            <span>obj</span><span>[</span><span>'</span><span>surname</span><span>'</span><span>]</span> <span>=</span> <span>last_surname</span>
            <span>obj</span><span>[</span><span>'</span><span>full_name</span><span>'</span><span>]</span> <span>=</span> <span>'</span><span> </span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>0</span><span>:</span><span>3</span><span>])</span>
            <span>obj</span><span>[</span><span>'</span><span>pdf_page_num</span><span>'</span><span>]</span> <span>=</span> <span>page_num</span> <span>+</span> <span>1</span>
            <span>obj</span><span>[</span><span>'</span><span>registered_by</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>3</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>village_vatan</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>4</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>village_mosal</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>5</span><span>]</span>
            <span>if</span> <span>len</span><span>(</span><span>parts</span><span>)</span> <span><=</span> <span>6</span><span>:</span> <span>continue</span>
            <span>if</span> <span>parts</span><span>[</span><span>7</span><span>]</span> <span>==</span> <span>'</span><span>વર્ષ</span><span>'</span><span>:</span> <span># date exists </span>                <span>idx</span> <span>=</span> <span>6</span>
                <span>obj</span><span>[</span><span>'</span><span>dob</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>]</span> <span>+</span> <span>'</span><span> વર્ષ</span><span>'</span>
                <span>idx</span> <span>+=</span> <span>1</span>
            <span>elif</span> <span>len</span><span>(</span><span>parts</span><span>[</span><span>6</span><span>])</span> <span>==</span> <span>8</span> <span>and</span> <span>parts</span><span>[</span><span>6</span><span>][</span><span>2</span><span>]</span> <span>==</span> <span>'</span><span>-</span><span>'</span><span>:</span>
                <span>idx</span> <span>=</span> <span>6</span>
                <span>obj</span><span>[</span><span>'</span><span>dob</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>]</span>
            <span>else</span><span>:</span>
                <span>print</span><span>(</span><span>"</span><span>warning: no date</span><span>"</span><span>)</span>
                <span>idx</span> <span>=</span> <span>5</span>
            <span>obj</span><span>[</span><span>'</span><span>marital_status</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>idx</span><span>+</span><span>1</span><span>]</span>
            <span>obj</span><span>[</span><span>'</span><span>extra_fields</span><span>'</span><span>]</span> <span>=</span> <span>'</span><span>::</span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>idx</span><span>+</span><span>2</span><span>:</span><span>-</span><span>2</span><span>])</span>
            <span>obj</span><span>[</span><span>'</span><span>blood_group</span><span>'</span><span>]</span> <span>=</span> <span>parts</span><span>[</span><span>-</span><span>1</span><span>]</span>
        <span>elif</span> <span>obj</span><span>:</span> <span># continuation lines </span>            <span>if </span><span>(</span><span>"</span><span>(</span><span>"</span> <span>in</span> <span>line</span> <span>and</span> <span>"</span><span>)</span><span>"</span> <span>in</span> <span>line</span><span>)</span> <span>or</span> <span>"</span><span>મો.ઃ</span><span>"</span> <span>in</span> <span>line</span><span>:</span>
                <span>obj</span><span>[</span><span>'</span><span>extra_fields</span><span>'</span><span>]</span> <span>+=</span> <span>'</span><span> </span><span>'</span> <span>+</span> <span>'</span><span>::</span><span>'</span><span>.</span><span>join</span><span>(</span><span>parts</span><span>[</span><span>0</span><span>:])</span>
    <span>if</span> <span>obj</span><span>:</span> <span>records</span><span>.</span><span>append</span><span>(</span><span>obj</span><span>)</span>        
    <span>jstr</span> <span>=</span> <span>json</span><span>.</span><span>dumps</span><span>(</span><span>records</span><span>,</span> <span>indent</span><span>=</span><span>4</span><span>)</span>
    <span>open</span><span>(</span><span>"</span><span>guj.json</span><span>"</span><span>,</span> <span>'</span><span>w</span><span>'</span><span>,</span> <span>encoding</span><span>=</span><span>'</span><span>utf-8</span><span>'</span><span>).</span><span>write</span><span>(</span><span>jstr</span><span>)</span>
    <span>print</span><span>(</span><span>f</span><span>"</span><span>written page </span><span>{</span><span>page_num</span><span>}</span><span> to json..</span><span>"</span><span>)</span>
nums = ['0', '૧', '૨', '૩', '૪', '૫', '૬', '૭', '૮', '૯']

def process_text(page_num, e_text):
    obj = None
    last_surname = None
    last_kramank = None
    print(f"processing page {page_num}..")
    for line in e_text.splitlines():
        line = line.replace('|', '').replace('[', '').replace(']', '')
        parts = [word for word in line.split(' ') if word]
        if len(parts) == 0: continue
        new_rec = True
        for char in parts[0]:
            if char not in nums:
                new_rec = False
                break
        if len(parts) < 2: continue

        if new_rec and len(parts[0]) >= 2: # numbered line             if len(parts) < 9: continue
            if obj: records.append(obj)
            obj = {}
            last_surname = parts[1]
            obj['kramank'] = parts[0]
            last_kramank = parts[0]
            obj['full_name'] = ' '.join(parts[1:4])
            obj['surname'] = parts[1]
            obj['pdf_page_num'] = page_num + 1
            obj['registered_by'] = parts[4]
            obj['village_vatan'] = parts[5]
            obj['village_mosal'] = parts[6]
            if parts[8] == 'વર્ષ':
                idx = 7
                obj['dob'] = parts[idx] + ' વર્ષ'
                idx += 1
            elif len(parts[7]) == 8 and parts[7][2] == '-':
                idx = 7
                obj['dob'] = parts[idx]
            else:
                print("warning: no date")
                idx = 6
            obj['marital_status'] = parts[idx+1]
            obj['extra_fields'] = '::'.join(parts[idx+2:-2])
            obj['blood_group'] = parts[-1]
        elif parts[0] == last_surname: # new member in existing family             if obj: records.append(obj)
            obj = {}
            obj['kramank'] = last_kramank
            obj['surname'] = last_surname
            obj['full_name'] = ' '.join(parts[0:3])
            obj['pdf_page_num'] = page_num + 1
            obj['registered_by'] = parts[3]
            obj['village_vatan'] = parts[4]
            obj['village_mosal'] = parts[5]
            if len(parts) <= 6: continue
            if parts[7] == 'વર્ષ': # date exists                 idx = 6
                obj['dob'] = parts[idx] + ' વર્ષ'
                idx += 1
            elif len(parts[6]) == 8 and parts[6][2] == '-':
                idx = 6
                obj['dob'] = parts[idx]
            else:
                print("warning: no date")
                idx = 5
            obj['marital_status'] = parts[idx+1]
            obj['extra_fields'] = '::'.join(parts[idx+2:-2])
            obj['blood_group'] = parts[-1]
        elif obj: # continuation lines             if ("(" in line and ")" in line) or "મો.ઃ" in line:
                obj['extra_fields'] += ' ' + '::'.join(parts[0:])
    if obj: records.append(obj)        
    jstr = json.dumps(records, indent=4)
    open("guj.json", 'w', encoding='utf-8').write(jstr)
    print(f"written page {page_num} to json..")

Enter fullscreen mode Exit fullscreen mode

Every PDF has its own nuances that must be accounted for. In this case, a new serial number (like 0૧ or 0૨) in the first field signaled a new group when the subsequent field (surname) changed.

pytesseract is a testament to the evolution and advancement in IT technology. About a decade ago, reading or parsing a PDF image using OCR in a non-English language on a modestly configured PC or laptop would have been nearly impossible. This is truly progress! Happy coding, and let me know how it goes in the comments below.

References

原文链接：Unlocking Text from Embedded-Font PDFs: A pytesseract OCR Tutorial

展开阅读全文

文章版权声明 1、本网站名称：拾光赋
2、本站永久网址：https://www.blogs.ink
3、本网站的文章部分内容可能来源于网络，仅供大家学习与参考，如有侵权，请联系站长QQ：805375623进行删除处理。
4、本站一切资源不代表本站立场，并不代表本站赞同其观点和对其真实性负责。
5、本站一律禁止以任何方式发布或转载任何违法的相关信息，访客发现请向站长举报
6、本站资源大多存储在云盘，如发现链接失效，请联系我们我们会第一时间更新。

THE END