Skip to content

cloud: clarify IMPORT INTO wildcard path matching #23488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alastori wants to merge 1 commit into
base: release-8.5
Choose a base branch
from
Open

cloud: clarify IMPORT INTO wildcard path matching #23488

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion sql-statements/sql-statement-import-into.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,21 @@ It specifies the storage location of the data file, which can be an Amazon S3 or
>
> If [SEM](/system-variables.md#tidb_enable_enhanced_security) is enabled in the target cluster, the `fileLocation` cannot be specified as a local file path.

In the `fileLocation` parameter, you can specify a single file, or use the `*` and `[]` wildcards to match multiple files for import. Note that the wildcard can only be used in the file name, because it does not match directories or recursively match files in subdirectories. Taking files stored on Amazon S3 as examples, you can configure the parameter as follows:
In the `fileLocation` parameter, you can specify a single file, or use the `*` and `[]` wildcards to match multiple files for import.

<CustomContent platform="tidb">

The wildcard can only be used in the file name, because it does not match directories or recursively match files in subdirectories.

</CustomContent>

<CustomContent platform="tidb-cloud">

Wildcards can appear in multiple path components, but each wildcard matches only within one `/`-separated component and never crosses `/`. Arbitrary-depth recursive matching is not supported.

</CustomContent>

Taking files stored on Amazon S3 as examples, you can configure the parameter as follows:

- Import a single file: `s3://<bucket-name>/path/to/data/foo.csv`
- Import all files in a specified path: `s3://<bucket-name>/path/to/data/*`
Expand All @@ -130,6 +144,12 @@ In the `fileLocation` parameter, you can specify a single file, or use the `*` a
- Import all files with the `foo` prefix and the `.csv` suffix in a specified path: `s3://<bucket-name>/path/to/data/foo*.csv`
- Import `1.csv` and `2.csv` in a specified path: `s3://<bucket-name>/path/to/data/[12].csv`

<CustomContent platform="tidb-cloud">

- Import all `.csv` files from matching subdirectories at one fixed directory depth: `s3://<bucket-name>/dir/subdir*/*.csv`

</CustomContent>

### Format

The `IMPORT INTO` statement supports three data file formats: `CSV`, `SQL`, and `PARQUET`. If not specified, the default format is `CSV`.
Expand Down