Repository navigation
feat(converters): add OpenDocument (ODT/ODS/ODP) support - #2707
Open
buleboy (cyberspace-cs) wants to merge 1 commit into
Open
buleboy (cyberspace-cs) wants to merge 1 commit into
buleboy (cyberspace-cs) wants to merge 1 commit into
Conversation
Adds an `OdfConverter` for OpenDocument text, spreadsheet and presentation documents, plus the flat FODT/FODS/FODP variants. LibreOffice and OpenOffice write these by default and they are the one common office format MarkItDown could not read. No new dependency: OpenDocument is a ZIP of XML parts, so `content.xml` is read with `zipfile` from the standard library and parsed with `defusedxml`, which markitdown already requires. `defusedxml` is the right choice rather than a convenience here -- `content.xml` comes from an untrusted file, and the hardened parser blocks the entity expansion and external entity resolution that plain `xml.dom.minidom` allows. A 64 MiB cap on the uncompressed `content.xml` covers the case entity hardening does not. Format details the converter has to get right, and how it handles them: - **`table:number-columns-repeated` is compression, not data.** Every LibreOffice row ends with an empty cell repeated to the sheet's maximum width, so the table width is taken from the last column that holds a *value* rather than from the last cell. Without this a two-column file converts to a 16384-column table. Repeats of populated cells are expanded, and both `MAX_REPEATED_CELLS` and `MAX_COLUMNS` cap what a hostile file can ask for. - **Whitespace is markup.** `text:s` (with `text:c`) and `text:tab` are elements, so they survive the whitespace collapse that ordinary text nodes get; collapsing the assembled string instead would make `a<text:s/>b` and `ab` identical. - **A cell is a container.** Multi-paragraph cells join with `<br>` and pipes are escaped, so a cell never adds or removes a column. - `table:covered-table-cell` keeps its column, because Markdown has no merged cells and dropping it would misalign every other row. - The body kind is dispatched on which of `office:text` / `office:spreadsheet` / `office:presentation` is present, not on the extension. Sheet output follows `XlsxConverter` (a `##` heading per sheet, first row as the header) so the same data converted from `.ods` and `.xlsx` produces the same Markdown. Tests: 54 cases in `test_odf_converter.py`, with fixtures assembled in memory so the structural cases are visible in the test rather than hidden in a binary, and two committed fixtures (`test.odt`, `test.ods`) registered in the shared test vectors so the magika mime guess, the local/stream/URI paths and the CLI are all exercised against a real file. Stated limitations: images are not extracted (their hrefs are package-internal), footnotes are dropped rather than emitted as references with no definition, change tracking is not resolved, and ordered lists render as bullets because numberedness lives in the style cascade in `styles.xml`.
|
buleboy (@cyberspace-cs) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds OpenDocument format support:
.odt(text),.ods(spreadsheet),.odp(presentation), plus the flat XML variants.fodt/.fods/.fodp.Why
OpenDocument is a widely used office format (LibreOffice, Google Docs export), but it is currently unsupported. Per the project's contribution guidance ("be conservative about new formats because of dependencies"), this converter is implemented with zero new dependencies — ODF is just ZIP + XML, so it uses the standard library
zipfileand the project's existingdefusedxml.What changed
converters/_odf_converter.py(~600 lines): converter registration, ODF metadata extraction, text/spreadsheet/presentation rendering.tests/test_odf_converter.py: 54 test cases + 2 real fixtures.converters/_converters.py, README format table, and docs updated.Design notes
table:number-columns-repeated="16384"is a compression artifact, not data. Table width is taken from the last cell with actual content, not the last grid cell; repeated non-empty cells are expanded withMAX_REPEATED_CELLS/MAX_COLUMNScaps to prevent memory blowups.text:s/text:tabare elements that must survive whitespace collapsing — otherwisea<text:s/>bis indistinguishable fromab.<br>;table:covered-table-cellis kept to preserve column alignment; body type is determined byoffice:text/office:spreadsheet/office:presentationrather than the extension.Security
defusedxml(not the stdlib XML parser) becausecontent.xmlcomes from untrusted files — entity expansion and external entities are allowed underxml.dom.minidom.Tests
test_convert_http_uri[test_vector15/16], which fail because the new fixtures are not yet on GitHubmain; those tests are explicitly skipped viaskip_remotein CI.black --checkclean.Dependencies
No new dependencies (stdlib
zipfile+ existingdefusedxml).Checklist
black --checkclean