diff --git a/TOC.md b/TOC.md index 0bb0cbe9fa244..b77f98657347f 100644 --- a/TOC.md +++ b/TOC.md @@ -898,6 +898,7 @@ - [`SHOW FIELDS FROM`](/sql-statements/sql-statement-show-fields-from.md) - [`SHOW GRANTS`](/sql-statements/sql-statement-show-grants.md) - [`SHOW IMPORT JOB`](/sql-statements/sql-statement-show-import-job.md) + - [`SHOW IMPORT GROUP`](/sql-statements/sql-statement-show-import-group.md) - [`SHOW INDEXES`](/sql-statements/sql-statement-show-indexes.md) - [`SHOW MASTER STATUS`](/sql-statements/sql-statement-show-master-status.md) - [`SHOW PLACEMENT`](/sql-statements/sql-statement-show-placement.md) diff --git a/privilege-management.md b/privilege-management.md index c0b1bb3df426c..cd5154f021905 100644 --- a/privilege-management.md +++ b/privilege-management.md @@ -430,7 +430,7 @@ Requires the `INSERT` and `SELECT` privileges for the table. `SHOW PROCESSLIST` requires the `PROCESS` privilege to show connections belonging to other users. -`SHOW IMPORT JOB` requires the `SUPER` privilege to show connections belonging to other users. Otherwise, it only shows jobs created by the current user. +`SHOW IMPORT JOB` and `SHOW IMPORT GROUP` require the `SUPER` privilege to show import jobs and import groups belonging to other users. Otherwise, these statements only show jobs and groups created by the current user. `SHOW STATS_LOCKED` requires the `SELECT` privilege to the `mysql.stats_table_locked` table. diff --git a/sql-statements/sql-statement-cancel-import-job.md b/sql-statements/sql-statement-cancel-import-job.md index d4b911d846384..9f388bf5f352b 100644 --- a/sql-statements/sql-statement-cancel-import-job.md +++ b/sql-statements/sql-statement-cancel-import-job.md @@ -39,4 +39,5 @@ This statement is a TiDB extension to MySQL syntax. ## See also * [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md) -* [`SHOW IMPORT JOB`](/sql-statements/sql-statement-show-import-job.md) +* [`SHOW IMPORT JOB(s)`](/sql-statements/sql-statement-show-import-job.md) +* [`SHOW IMPORT GROUP(s)`](/sql-statements/sql-statement-show-import-group.md) diff --git a/sql-statements/sql-statement-import-into.md b/sql-statements/sql-statement-import-into.md index 7793534234871..56ef7d98b5fdf 100644 --- a/sql-statements/sql-statement-import-into.md +++ b/sql-statements/sql-statement-import-into.md @@ -141,6 +141,7 @@ The supported options are described as follows: | Option name | Supported data sources and formats | Description | |:---|:---|:---| +| `GROUP_KEY=''` | All file formats | Specifies an optional group key for the job. The group key does not affect the import itself; TiDB groups import jobs with the same group key so that you can monitor related jobs collectively using [`SHOW IMPORT GROUPS`](/sql-statements/sql-statement-show-import-group.md). The key can contain only alphanumeric characters, underscores (_), or hyphens (-), and can be up to 256 characters long. If no group key is set, the job does not belong to any group and does not appear in `SHOW IMPORT GROUPS` results. | | `CHARACTER_SET=''` | CSV | Specifies the character set of the data file. The default character set is `utf8mb4`. The supported character sets include `binary`, `utf8`, `utf8mb4`, `gb18030`, `gbk`, `latin1`, and `ascii`. | | `FIELDS_TERMINATED_BY=''` | CSV | Specifies the field separator. The default separator is `,`. | | `FIELDS_ENCLOSED_BY=''` | CSV | Specifies the field delimiter. The default delimiter is `"`. | @@ -240,7 +241,9 @@ IMPORT INTO t FROM '/path/to/small.csv' WITH DETACHED; ### View and manage import jobs -For an import job with the `DETACHED` mode enabled, you can use [`SHOW IMPORT`](/sql-statements/sql-statement-show-import-job.md) to view its current job progress. +For an import job with the `DETACHED` mode enabled, you can use [`SHOW IMPORT JOB`](/sql-statements/sql-statement-show-import-job.md) to view its current job progress. + +You can also use [`SHOW IMPORT GROUP(s)`](/sql-statements/sql-statement-show-import-group.md) to view the overall progress of jobs in the same group. After an import job is started, you can cancel it using [`CANCEL IMPORT JOB `](/sql-statements/sql-statement-cancel-import-job.md). @@ -258,6 +261,14 @@ IMPORT INTO t FROM '/path/to/file.csv' WITH skip_rows=1; IMPORT INTO t FROM '/path/to/file.csv' WITH DETACHED; ``` +#### Assign a group key to the import job + +To group related import jobs so that you can monitor them together using [`SHOW IMPORT GROUPS`](/sql-statements/sql-statement-show-import-group.md), specify the `GROUP_KEY` option: + +```sql +IMPORT INTO t FROM '/path/to/file.csv' WITH GROUP_KEY='user_group'; +``` + #### Skip importing a specific field in your data file Assume that your data file is in the CSV format and its content is as follows: @@ -363,4 +374,5 @@ This statement is a TiDB extension to MySQL syntax. * [`ADMIN CHECKSUM TABLE`](/sql-statements/sql-statement-admin-checksum-table.md) * [`CANCEL IMPORT JOB`](/sql-statements/sql-statement-cancel-import-job.md) * [`SHOW IMPORT JOB(s)`](/sql-statements/sql-statement-show-import-job.md) +* [`SHOW IMPORT GROUP`](/sql-statements/sql-statement-show-import-group.md) * [TiDB Distributed eXecution Framework (DXF)](/tidb-distributed-execution-framework.md) diff --git a/sql-statements/sql-statement-overview.md b/sql-statements/sql-statement-overview.md index c76059777c72d..1f6f0e4a6758a 100644 --- a/sql-statements/sql-statement-overview.md +++ b/sql-statements/sql-statement-overview.md @@ -131,6 +131,7 @@ TiDB uses SQL statements that aim to follow ISO/IEC SQL standards, with extensio | [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md) | Imports data into a table via the [Physical Import Mode](https://docs.pingcap.com/tidb/stable/tidb-lightning-physical-import-mode) of TiDB Lightning. | | [`LOAD DATA`](/sql-statements/sql-statement-load-data.md) | Loads data into a table from Amazon S3 or Google Cloud Storage. | | [`SHOW IMPORT JOB`](/sql-statements/sql-statement-show-import-job.md) | Shows the status of an import job. | +| [`SHOW IMPORT GROUP`](/sql-statements/sql-statement-show-import-group.md) | Shows the status of a group of import jobs. | ## Backup & restore diff --git a/sql-statements/sql-statement-show-import-group.md b/sql-statements/sql-statement-show-import-group.md new file mode 100644 index 0000000000000..f8d4834f6b853 --- /dev/null +++ b/sql-statements/sql-statement-show-import-group.md @@ -0,0 +1,77 @@ +--- +title: SHOW IMPORT GROUP +summary: An overview of the usage of SHOW IMPORT GROUP in TiDB. +--- + +# SHOW IMPORT GROUP + +The `SHOW IMPORT GROUP` statement is used to show the groups of `IMPORT` jobs created in TiDB. By default, `SHOW IMPORT GROUPS` shows groups created by the current user. To view all import groups, you need the `SUPER` privilege. + +## Required privileges + +- `SHOW IMPORT GROUPS`: if a user has the `SUPER` privilege, this statement shows all import groups in TiDB. Otherwise, this statement only shows groups created by the current user. +- `SHOW IMPORT GROUP `: only the creator of an import group or users with the `SUPER` privilege can use this statement to view a specific import group. + +## Synopsis + +```ebnf+diagram +ShowImportGroupsStmt ::= + 'SHOW' 'IMPORT' 'GROUPS' + +ShowImportGroupStmt ::= + 'SHOW' 'IMPORT' 'GROUP' GroupKey +``` + +The output fields of the `SHOW IMPORT GROUP` statement are described as follows: + +| Column | Description | +|------------------|-------------------------| +| Group_Key | The key of the group | +| Total_Jobs | Total number of jobs in this group | +| Pending | Number of pending jobs in this group | +| Running | Number of running jobs in this group | +| Completed | Number of completed jobs in this group | +| Failed | Number of failed jobs in this group | +| Cancelled | Number of cancelled jobs in this group | +| First_Job_Create_Time | The earliest creation time of the jobs in this group | +| Last_Job_Update_Time | The latest update time of the jobs in this group | + +## Example + +```sql +SHOW IMPORT GROUPS; +``` + +``` ++--------------+------------+---------+---------+-----------+--------+-----------+----------------------------+----------------------------+ +| Group_Key | Total_Jobs | Pending | Running | Completed | Failed | Cancelled | First_Job_Create_Time | Last_Job_Update_Time | ++--------------+------------+---------+---------+-----------+--------+-----------+----------------------------+----------------------------+ +| import_group_1 | 1 | 0 | 1 | 0 | 0 | 0 | 2025-08-07 01:36:18.479055 | 2025-08-07 01:36:18.479055 | ++--------------+------------+---------+---------+-----------+--------+-----------+----------------------------+----------------------------+ +| user_group | 1 | 1 | 0 | 0 | 0 | 0 | 2025-08-07 01:37:26.162268 | NULL | ++--------------+------------+---------+---------+-----------+--------+-----------+----------------------------+----------------------------+ +2 rows in set (0.01 sec) +``` + +```sql +SHOW IMPORT GROUP "import_group_1"; +``` + +``` ++--------------+------------+---------+---------+-----------+--------+-----------+----------------------------+----------------------------+ +| Group_Key | Total_Jobs | Pending | Running | Completed | Failed | Cancelled | First_Job_Create_Time | Last_Job_Update_Time | ++--------------+------------+---------+---------+-----------+--------+-----------+----------------------------+----------------------------+ +| import_group_1 | 1 | 0 | 1 | 0 | 0 | 0 | 2025-08-07 01:36:18.479055 | 2025-08-07 01:36:18.479055 | ++--------------+------------+---------+---------+-----------+--------+-----------+----------------------------+----------------------------+ +1 row in set (0.01 sec) +``` + +## MySQL compatibility + +This statement is a TiDB extension to MySQL syntax. + +## See also + +* [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md) +* [`SHOW IMPORT JOB`](/sql-statements/sql-statement-show-import-job.md) +* [`CANCEL IMPORT JOB`](/sql-statements/sql-statement-cancel-import-job.md) diff --git a/sql-statements/sql-statement-show-import-job.md b/sql-statements/sql-statement-show-import-job.md index 396f60558a137..60da0b01ec5dd 100644 --- a/sql-statements/sql-statement-show-import-job.md +++ b/sql-statements/sql-statement-show-import-job.md @@ -1,11 +1,11 @@ --- -title: SHOW IMPORT -summary: An overview of the usage of SHOW IMPORT in TiDB. +title: SHOW IMPORT JOB +summary: An overview of the usage of SHOW IMPORT JOB in TiDB. --- -# SHOW IMPORT +# SHOW IMPORT JOB -The `SHOW IMPORT` statement is used to show the IMPORT jobs created in TiDB. This statement can only show jobs created by the current user. +The `SHOW IMPORT JOB` statement is used to show `IMPORT` jobs created in TiDB. By default, `SHOW IMPORT JOBS` shows jobs created by the current user. To view all import jobs, you need the `SUPER` privilege. ## Required privileges @@ -16,19 +16,25 @@ The `SHOW IMPORT` statement is used to show the IMPORT jobs created in TiDB. Thi ```ebnf+diagram ShowImportJobsStmt ::= - 'SHOW' 'IMPORT' 'JOBS' + 'SHOW' 'IMPORT' 'JOBS' ShowLikeOrWhereOpt? ShowImportJobStmt ::= 'SHOW' 'IMPORT' 'JOB' JobID + +ShowLikeOrWhereOpt ::= + 'LIKE' SimpleExpr +| 'WHERE' Expression ``` -The output fields of the `SHOW IMPORT` statement are described as follows: +The output fields of the `SHOW IMPORT JOB` statement are described as follows: | Column | Description | |------------------|-------------------------| -| Job_ID | The ID of the task | +| Job_ID | The ID of the job | +| Group_Key | The group key of the job | | Data_Source | Information about the data source | | Target_Table | The name of the target table | +| Table_ID | The ID of the target table | | Phase | The current phase of the job, including `importing`, `validating`, and `add-index` | | Status | The current status of the job, including `pending` (means created but not started yet), `running`, `canceled`, `failed`, and `finished` | | Source_File_Size | The size of the source file | @@ -38,34 +44,46 @@ The output fields of the `SHOW IMPORT` statement are described as follows: | Start_Time | The time when the task is started | | End_Time | The time when the task is ended | | Created_By | The name of the database user who creates the task | +| Last_Update_Time | The time when the job was last updated | +| Cur_Step | The specific sequential processing step of this job | +| Cur_Step_Processed_Size | The amount of data that has been processed within the current step | +| Cur_Step_Total_Size | The total size of the data to be processed in the current step | +| Cur_Step_Progress_Pct | The estimated completion percentage of the current step | +| Cur_Step_Speed | The current data processing speed | +| Cur_Step_ETA | The estimated time remaining for the current step to complete | + +## Filtering import jobs + +Only `SHOW IMPORT JOBS` supports filtering import jobs with a `WHERE` or `LIKE` clause. `SHOW IMPORT JOB ` does not support these clauses. + +The `WHERE` clause can reference the output fields of `SHOW IMPORT JOBS`, including `Job_ID`, `Group_Key`, `Data_Source`, `Target_Table`, `Table_ID`, `Phase`, `Status`, `Source_File_Size`, `Imported_Rows`, `Result_Message`, `Create_Time`, `Start_Time`, `End_Time`, `Created_By`, `Last_Update_Time`, `Cur_Step`, `Cur_Step_Processed_Size`, `Cur_Step_Total_Size`, `Cur_Step_Progress_Pct`, `Cur_Step_Speed`, and `Cur_Step_ETA`. + +The `LIKE` clause matches its pattern only against the first column `Job_ID` (compared as a string). For example, `SHOW IMPORT JOBS LIKE '%1'` returns the jobs whose `Job_ID` ends with `1`. ## Example +Show all import jobs that the current user can view: + ```sql SHOW IMPORT JOBS; ``` -``` -+--------+-------------------+--------------+----------+-------+----------+------------------+---------------+----------------+----------------------------+----------------------------+----------------------------+------------+ -| Job_ID | Data_Source | Target_Table | Table_ID | Phase | Status | Source_File_Size | Imported_Rows | Result_Message | Create_Time | Start_Time | End_Time | Created_By | -+--------+-------------------+--------------+----------+-------+----------+------------------+---------------+----------------+----------------------------+----------------------------+----------------------------+------------+ -| 1 | /path/to/file.csv | `test`.`foo` | 116 | | finished | 11GB | 950000 | | 2023-06-26 11:23:59.281257 | 2023-06-26 11:23:59.484932 | 2023-06-26 13:04:30.622952 | root@% | -| 2 | /path/to/file.csv | `test`.`bar` | 130 | | finished | 1.194TB | 49995000 | | 2023-06-26 15:42:45.079237 | 2023-06-26 15:42:45.388108 | 2023-06-26 17:29:43.023568 | root@% | -+--------+-------------------+--------------+----------+-------+----------+------------------+---------------+----------------+----------------------------+----------------------------+----------------------------+------------+ -1 row in set (0.01 sec) -``` +Show a specific import job by job ID: ```sql -SHOW IMPORT JOB 60001; +SHOW IMPORT JOB 2; ``` -``` -+--------+--------------------+--------------+----------+-------+---------+------------------+---------------+----------------+----------------------------+------------+----------+------------+ -| Job_ID | Data_Source | Target_Table | Table_ID | Phase | Status | Source_File_Size | Imported_Rows | Result_Message | Create_Time | Start_Time | End_Time | Created_By | -+--------+--------------------+--------------+----------+-------+---------+------------------+---------------+----------------+----------------------------+------------+----------+------------+ -| 60001 | /path/to/small.csv | `test`.`t` | 361 | | pending | 16B | NULL | | 2023-06-08 15:59:37.047703 | NULL | NULL | root@% | -+--------+--------------------+--------------+----------+-------+---------+------------------+---------------+----------------+----------------------------+------------+----------+------------+ -1 row in set (0.01 sec) +The output is as follows: + +| Job_ID | Group_Key | Data_Source | Target_Table | Table_ID | Phase | Status | Source_File_Size | Imported_Rows | Result_Message | Create_Time | Start_Time | End_Time | Created_By | Last_Update_Time | Cur_Step | Cur_Step_Processed_Size | Cur_Step_Total_Size | Cur_Step_Progress_Pct | Cur_Step_Speed | Cur_Step_ETA | +|--------|-----------|-------------|--------------|----------|-------|--------|------------------|---------------|----------------|-------------|------------|----------|------------|------------------|----------|-------------------------|---------------------|-----------------------|----------------|--------------| +| 2 | import_group_1 | /path/to/file.csv | `test`.`bar` | 118 | global-sorting | running | 277.5GiB | 0 | | 2025-07-09 10:40:18.580706 | 2025-07-09 10:40:19.092528 | NULL | root@% | 2025-07-09 10:47:15 | encode | 4.55GB | 298GB | 1 | 10.96MB/s | 07:26:03 | + +Filter import jobs by output fields: + +```sql +SHOW IMPORT JOBS WHERE Group_Key = 'user_group'; ``` ## MySQL compatibility @@ -75,4 +93,5 @@ This statement is a TiDB extension to MySQL syntax. ## See also * [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md) +* [`SHOW IMPORT GROUP`](/sql-statements/sql-statement-show-import-group.md) * [`CANCEL IMPORT JOB`](/sql-statements/sql-statement-cancel-import-job.md)