PR #17820 - onyx-dot-app/onyx

Description

This PR adds one shared file-type check and moves ten connectors to it.

is_indexable_file_type(file_name, mime_type, allow_images) in onyx/file_processing/file_types.py tells if Onyx can read a file from its name and mime type:

An image (a supported image mime type or image extension) is indexable only when allow_images is true.
Any other file needs a supported text or document extension, because text extraction selects its parser by extension. The mime type alone is not sufficient.

get_file_ext moves to file_types.py. extract_file_text.py imports it, so the old import path still works.

Connector rules stay in the connectors: size limits, path filters, and allow_images checks that run after the download. Stack A's Google Drive PR (#17550) will use the same function.

Per connector

Connector	Before	After
Azure Data Lake `has_indexable_name`	ext in `ALL_ALLOWED_EXTENSIONS`, image ext needs `allow_images`	`is_indexable_file_type(basename, None, allow_images)`
Box `_file_is_indexable`	image ext and `allow_images`, else ext in `TEXT_AND_DOCUMENT_EXTENSIONS`	`is_indexable_file_type(name, None, self.allow_images)`; size check stays
Confluence `validate_attachment_filetype`	image media type must be supported; else title ext in `ALL_ALLOWED_EXTENSIONS`	an unsupported `image/*` media type is refused (kept as a Confluence rule); then `is_indexable_file_type(title, media_type, True)`. Callers still apply `allow_images`
Drupal Wiki `_validate_attachment_filetype`	ext in `ALL_ALLOWED_EXTENSIONS`	`is_indexable_file_type(name, None, True)`. `allow_images` stays after the download, with no error
Egnyte `is_indexable`	ext in `TEXT_AND_DOCUMENT_EXTENSIONS`	`is_indexable_file_type(name, None, False)`. This one function is the filter for the folder walk, the slim listing (pruning), reindex and the sample-file capability check
File `_process_file`	ext in `ALL_ALLOWED_EXTENSIONS`	`is_indexable_file_type(name, None, True)` (uploaded images are always indexed)
Highspot `_get_item_content`	`content_name.split(".")[-1]` in `TEXT_AND_DOCUMENT_EXTENSIONS`	`is_indexable_file_type(content_name, None, False)`
OneDrive `_item_allowed`	ext in `ALL_ALLOWED_EXTENSIONS`	`is_indexable_file_type(name, None, True)`; `allow_images` stays in item extraction
Outlook `attachment_skip_reason`	ext in `TEXT_AND_DOCUMENT_EXTENSIONS`	`is_indexable_file_type(name, None, False)`
SharePoint connector and `_is_indexable_file` check	ext in `ALL_ALLOWED_EXTENSIONS`	`is_indexable_file_type(name, None, True)`

Not migrated

GitHub is_indexable_path: GitHub reads files as text with its own code. Its allow-list is narrower and has .markdown and .rst, which the shared function refuses. It also accepts names like README and LICENSE. A shared call would remove files from the index.
Teams files, Microsoft drive_items content: they refuse only some image mime types and accept any other type. A shared call would remove files.
Blob (S3 and others), Dropbox, Airtable attachments: no type filter. Text extraction reads unknown extensions as text when the bytes are text. A shared call would remove files such as .py.
storage_file_document, tabular_section_utils: they select a parser, they do not filter.
Google Drive: #17550 moves Drive to the shared function.

Behavior changes

No intended change. Two edge cases differ:

Highspot: a content_name that is only an extension (for example .pdf) was downloaded before. os.path.splitext gives no extension for it, so it now gets the default title and description content.
Confluence: the shared function makes the media type lowercase. An attachment with an uppercase media type such as IMAGE/PNG and a title with no supported extension is now accepted.

How Has This Been Tested?

New unit tests for is_indexable_file_type.
New unit tests for the file filters that had none: Azure Data Lake, Confluence, Drupal Wiki, Egnyte, Highspot. They pass on unmodified main and on this branch, so the result is the same for each tested file.
Box, OneDrive, Outlook and SharePoint filters already had unit test coverage.
Unit tests for all touched connectors, microsoft_utils, google_drive, capability_checks and file_processing: 2438 passed.

Docs

Customer-facing change: I updated documentation/ for the page's reader and task, at the level of detail of the whole page.
No docs change needed: the existing docs stay correct (internal, tests, CI, multi-tenant cloud only, or the UI explains it).

Additional Options

[Optional] Please cherry-pick this PR to the latest release version.

🤖 Generated with Claude Code

Summary by cubic

Adds a shared is_indexable_file_type check and moves ten connectors to it, so file-type decisions are consistent across connectors instead of being reimplemented per source.

is_indexable_file_type in onyx/file_processing/file_types.py accepts images only when allow_images is set, and any other file only when its extension is supported, since text extraction picks parsers by extension. get_file_ext moves into file_types.py; the old import path still works through extract_file_text.py.

Updates Azure Data Lake, Box, Confluence, Drupal Wiki, Egnyte, File, Highspot, OneDrive, Outlook, and SharePoint to use the shared function; connector rules like size limits, path filters, and post-download allow_images checks stay in the connectors. Connectors that read files with their own code (GitHub, Teams, Microsoft drive items) or index without a type filter (S3/Blob, Dropbox, Airtable) are intentionally not migrated.
No intended behavior changes. Highspot no longer downloads files whose names are only an extension (.pdf). Confluence matches the image/ prefix case-sensitively as before, so an uppercase IMAGE/PNG media type is refused unless the title has a supported extension.
Adds unit tests for the shared function and for connectors that previously had none (Azure Data Lake, Confluence, Drupal Wiki, Egnyte, Highspot). All full suite tests pass (2438).

Summary by CodeRabbit

Bug Fixes
- File eligibility is more consistent across cloud storage, collaboration, and wiki sources. Supported document and text files are recognized by extension, while image files are eligible only where the source allows them.
- Unsupported file types continue to be skipped, and existing size, path, and time-window restrictions remain in effect.
- Image attachment checks account for supported media types and filenames.
Tests
- Added coverage for supported and unsupported file types, image settings, case-insensitive extensions, and attachment media types.