Deep Dive into Microsoft MarkItDown

What is MarkItDown?

MarkItDown is a Python package developed by Microsoft, designed to convert a variety of file formats into Markdown.

Since its debut, the library has skyrocketed in popularity, gaining over 25k GitHub stars within just two weeks! 🤯

What Makes MarkItDown So Popular?

MarkItDown offers robust support for a wide array of file types, such as:

• Office formats: Word, PowerPoint, Excel
• Media files: Images (with EXIF data and descriptions), Audio (with transcription support)
• Web and data formats: HTML, JSON, XML, CSV
• Archives: ZIP files

Its ability to handle not just standard formats like Word but also multi-modal data makes it stand out. For example, it uses OCR and speech recognition to extract content from images and audio files.

The ability to convert anything into Markdown makes MarkItDown a powerful tool for LLM training. By processing domain-specific documents, it provides rich context for generating more accurate and relevant responses in LLM-powered applications.

Getting Started with MarkItDown

Using MarkItDown is incredibly straightforward - only 4 lines of code are needed:

from markitdown import MarkItDown

md = MarkItDown()
result = md.convert("test.xlsx")
print(result.text_content)

Enter fullscreen mode Exit fullscreen mode

Here's some use cases of MarkItDown.

Converting a Word document generates clean and accurate Markdown:

Even multi-tab Excel spreadsheets are handled with ease:

ZIP archives? No problem! The library parses all files inside them recursively:

Initially, image extraction might yield no results:

This is because MarkItDown relies on an LLM to generate image descriptions. By integrating an LLM client, you can enable this feature:

from openai import OpenAI

client = OpenAI(api_key="i-am-not-an-api-key")

md = MarkItDown(llm_client=client, llm_model="gpt-4o")

Enter fullscreen mode Exit fullscreen mode

With the configuration in place, image files can be successfully processed:

Note: LLM won't deal with image-based PDFs. PDFs need OCR preprocessing to extract content.

However, PDFs lose their formatting upon extraction, therefore headings and plain text are not distinguished:

Limitations

MarkItDown isn’t without its limitations:

• PDF files without OCR cannot be processed.
• Formatting is not available when extracting from PDF files.

Nonetheless, as an open-source project, it’s highly customizable. Developers can easily extend its functionality due to its clean codebase.

How MarkItDown Works

MarkItDown’s architecture is straightforward and modular. Its core logic resides entirely within a single file.

It has a DocumentConverter class, which defines a generic convert() method:

class DocumentConverter:
    """Base class for all document converters."""

    def convert(
        self, local_path: str, **kwargs: Any
    ) -> Union[None, DocumentConverterResult]:
        raise NotImplementedError()

Enter fullscreen mode Exit fullscreen mode

Individual converters inherit from this base class and are registered dynamically:

self.register_page_converter(PlainTextConverter())
self.register_page_converter(HtmlConverter())
self.register_page_converter(DocxConverter())
self.register_page_converter(XlsxConverter())
self.register_page_converter(Mp3Converter())
self.register_page_converter(ImageConverter())
# ...

Enter fullscreen mode Exit fullscreen mode

This modular approach makes it easy to add support for new file types.

File Conversion Workflows

Office Documents

Office files are transformed into HTML using libraries like mammoth, pandas, or pptx, and then converted to Markdown with BeautifulSoup.

Audio Files

Audio is transcribed with the speech_recognition library, which utilizes Google’s API.

(Microsoft, why not Azure here? 💔)

Images

Image processing involves generating a caption via an LLM prompt:
"Write a detailed description for this image."

PDFs

PDFs are handled by the pdfminer library but lack built-in OCR. You must preprocess PDFs for text extraction.

Deploying MarkItDown as an API

MarkItDown can run locally, but hosting it as an API unlocks additional flexibility, making it easy to integrate into workflows like Zapier and n8n.

Here’s a simple example of MarkItDown API using FastAPI:

import shutil
from markitdown import MarkItDown
from fastapi import FastAPI, UploadFile
from uuid import uuid4

md = MarkItDown()

app = FastAPI()

@app.post("/convert")
async def convert_markdown(file: UploadFile):
    unique_id = uuid4()
    temp_dir = f"./temp/{unique_id}"

    shutil.os.makedirs(temp_dir, exist_ok=True)

    file_path = f"{temp_dir}/{file.filename}"
    with open(file_path, "wb") as f:
        shutil.copyfileobj(file.file, f)
    result = md.convert(file_path)
    content = result.text_content

    shutil.rmtree(temp_dir)

    return {"result": content}

Enter fullscreen mode Exit fullscreen mode

To call the API:

const formData = new FormData();
formData.append('file', file);

const response = await fetch('http://localhost:8000/convert', {
  method: 'POST',
  body: formData,
});

Enter fullscreen mode Exit fullscreen mode.