cloud: add tiered storage docs - #23328
Conversation
ti-chi-bot
Bot
added
contribution
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds TiDB Cloud Essential Private Preview documentation for Tiered Storage, including concepts, architecture, configuration, operations, limitations, FAQ content, and table-of-contents navigation. It also ignores a local QodEr settings file. ChangesTiered Storage documentation
Local tooling ignore
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 14
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 8c969826-2f18-4a5e-9ea9-0868fa38a3a4
📒 Files selected for processing (5)
TOC-tidb-cloud.mdtidb-cloud/tieredstorage_concepts.mdtidb-cloud/tieredstorage_faq.mdtidb-cloud/tieredstorage_limitations.mdtidb-cloud/tieredstorage_operations.md
|
|
||
| ## 1 What is Tiered Storage | ||
|
|
||
| Tiered Storage is a **table-level and partition-level storage tiering capability** on TiDB Cloud Essential, designed for infrequently accessed data. Users can set a table or partition to the IA (Infrequent Access) storage class. The system automatically stores the full data in remote object storage (S3/OSS, etc.), keeping only metadata and on-demand cached hot data segments locally. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Replace etc. with explicit examples.
Vale rejects the current wording. Use a precise list without changing the technical meaning.
Suggested replacement
-Tiered Storage is a **table-level and partition-level storage tiering capability** on TiDB Cloud Essential, designed for infrequently accessed data. Users can set a table or partition to the IA (Infrequent Access) storage class. The system automatically stores the full data in remote object storage (S3/OSS, etc.), keeping only metadata and on-demand cached hot data segments locally.
+Tiered Storage is a **table-level and partition-level storage tiering capability** on TiDB Cloud Essential, designed for infrequently accessed data. Users can set a table or partition to the IA (Infrequent Access) storage class. The system automatically stores the full data in remote object storage, such as S3 or OSS, keeping only metadata and on-demand cached hot data segments locally.As per path instructions, this is an exact replacement for the contiguous changed line.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Tiered Storage is a **table-level and partition-level storage tiering capability** on TiDB Cloud Essential, designed for infrequently accessed data. Users can set a table or partition to the IA (Infrequent Access) storage class. The system automatically stores the full data in remote object storage (S3/OSS, etc.), keeping only metadata and on-demand cached hot data segments locally. | |
| Tiered Storage is a **table-level and partition-level storage tiering capability** on TiDB Cloud Essential, designed for infrequently accessed data. Users can set a table or partition to the IA (Infrequent Access) storage class. The system automatically stores the full data in remote object storage, such as S3 or OSS, keeping only metadata and on-demand cached hot data segments locally. |
🧰 Tools
🪛 GitHub Check: vale
[failure] 12-12:
[vale] reported by reviewdog 🐶
[PingCAP.Latin] Use 'such as' instead of 'etc.'.
Raw Output:
{"message": "[PingCAP.Latin] Use 'such as' instead of 'etc.'.", "location": {"path": "tidb-cloud/tieredstorage_concepts.md", "range": {"start": {"line": 12, "column": 310}}}, "severity": "ERROR"}
Sources: Path instructions, Linters/SAST tools
|
|
||
| Tiered Storage is a **table-level and partition-level storage tiering capability** on TiDB Cloud Essential, designed for infrequently accessed data. Users can set a table or partition to the IA (Infrequent Access) storage class. The system automatically stores the full data in remote object storage (S3/OSS, etc.), keeping only metadata and on-demand cached hot data segments locally. | ||
|
|
||
| **In a nutshell**: An IA table is still a regular table from the application layer — all query, transaction, backup, and recovery semantics remain unchanged. The difference lies in cost and performance: local disk usage drops significantly, but on a cold read (when the local cache is missing), data must be fetched from remote object storage, resulting in higher latency than Standard tables. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Remove spaces around the em dash.
Suggested replacement
-**In a nutshell**: An IA table is still a regular table from the application layer — all query, transaction, backup, and recovery semantics remain unchanged. The difference lies in cost and performance: local disk usage drops significantly, but on a cold read (when the local cache is missing), data must be fetched from remote object storage, resulting in higher latency than Standard tables.
+**In a nutshell**: An IA table is still a regular table from the application layer—all query, transaction, backup, and recovery semantics remain unchanged. The difference lies in cost and performance: local disk usage drops significantly, but on a cold read (when the local cache is missing), data must be fetched from remote object storage, resulting in higher latency than Standard tables.As per path instructions, this is an exact replacement for the contiguous changed line.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| **In a nutshell**: An IA table is still a regular table from the application layer — all query, transaction, backup, and recovery semantics remain unchanged. The difference lies in cost and performance: local disk usage drops significantly, but on a cold read (when the local cache is missing), data must be fetched from remote object storage, resulting in higher latency than Standard tables. | |
| **In a nutshell**: An IA table is still a regular table from the application layer—all query, transaction, backup, and recovery semantics remain unchanged. The difference lies in cost and performance: local disk usage drops significantly, but on a cold read (when the local cache is missing), data must be fetched from remote object storage, resulting in higher latency than Standard tables. |
🧰 Tools
🪛 GitHub Check: vale
[failure] 14-14:
[vale] reported by reviewdog 🐶
[PingCAP.EmDash] Don't put a space before or after a dash.
Raw Output:
{"message": "[PingCAP.EmDash] Don't put a space before or after a dash.", "location": {"path": "tidb-cloud/tieredstorage_concepts.md", "range": {"start": {"line": 14, "column": 83}}}, "severity": "ERROR"}
Sources: Path instructions, Linters/SAST tools
| - [ ] The data access frequency of the table/partition has been confirmed to be declining from a business perspective | ||
| - [ ] The table can be changed to a partitioned table, because cold/hot data separation is easier to manage with partitioned tables | ||
| - [ ] For regular table cold/hot separation, hot data accounts for less than 10% of the table | ||
| - [ ] Cold data access frequency is very low, e.g., query QPS does not exceed 10 concurrent (to avoid saturating object storage bandwidth) |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟠 Major | ⚡ Quick win
Separate QPS, concurrency, and throughput limits.
These lines currently treat “10 concurrent” as equivalent to a QPS or throughput limit. Define each dimension independently and explain the assumptions behind any conversion between them; otherwise users cannot determine whether their workload is within the documented safety limits.
tidb-cloud/tieredstorage_concepts.md#L60-L60: replace the QPS-based wording with an explicit concurrent-request limit.tidb-cloud/tieredstorage_limitations.md#L30-L30: document the1 GiB/sthroughput limit separately from the maximum concurrent request count.
📍 Affects 2 files
tidb-cloud/tieredstorage_concepts.md#L60-L60(this comment)tidb-cloud/tieredstorage_limitations.md#L30-L30
| - **Standard layer**: Three replicas are each stored on the local disks of three TiKV nodes | ||
| - **IA layer**: Three replicas are each uploaded to object storage independently in IA format | ||
|
|
||
| Each replica on each TiKV node runs its own independent LSM-Tree, performing independent flush and compaction operations. When a table switches to IA storage class, all subsequently generated SST files are written to S3 in IA type. The three replicas produce their own independent SST files — three separate objects in S3, not shared. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Restrict IA SSTable writes to the documented L1+ path.
The concepts document explicitly says memtable and L0 remain local; only eligible L1+ files are opened in IA mode after flush or compaction.
Suggested replacement
-Each replica on each TiKV node runs its own independent LSM-Tree, performing independent flush and compaction operations. When a table switches to IA storage class, all subsequently generated SST files are written to S3 in IA type. The three replicas produce their own independent SST files — three separate objects in S3, not shared.
+Each replica on each TiKV node runs its own independent LSM-tree, performing independent flush and compaction operations. Memtable and L0 writes remain on the local hot path; after flush or compaction, eligible L1+ SST files are written to S3 in IA format. The three replicas produce their own independent SST files—three separate objects in S3, not shared.As per path instructions, this is an exact replacement for the contiguous changed line.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Each replica on each TiKV node runs its own independent LSM-Tree, performing independent flush and compaction operations. When a table switches to IA storage class, all subsequently generated SST files are written to S3 in IA type. The three replicas produce their own independent SST files — three separate objects in S3, not shared. | |
| Each replica on each TiKV node runs its own independent LSM-tree, performing independent flush and compaction operations. Memtable and L0 writes remain on the local hot path; after flush or compaction, eligible L1+ SST files are written to S3 in IA format. The three replicas produce their own independent SST files—three separate objects in S3, not shared. |
🧰 Tools
🪛 GitHub Check: vale
[failure] 43-43:
[vale] reported by reviewdog 🐶
[PingCAP.EmDash] Don't put a space before or after a dash.
Raw Output:
{"message": "[PingCAP.EmDash] Don't put a space before or after a dash.", "location": {"path": "tidb-cloud/tieredstorage_faq.md", "range": {"start": {"line": 43, "column": 291}}}, "severity": "ERROR"}
Source: Path instructions
|
|
||
| Each replica on each TiKV node runs its own independent LSM-Tree, performing independent flush and compaction operations. When a table switches to IA storage class, all subsequently generated SST files are written to S3 in IA type. The three replicas produce their own independent SST files — three separate objects in S3, not shared. | ||
|
|
||
| IA is a **storage format/location optimization**, not a **replica reduction mechanism**. It changes "how each node stores its own copy," not "how many copies exist." The Raft write and replication flow is identical to the Standard layer. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Replace the ambiguous “how many copies exist.”
Suggested replacement
-IA is a **storage format/location optimization**, not a **replica reduction mechanism**. It changes "how each node stores its own copy," not "how many copies exist." The Raft write and replication flow is identical to the Standard layer.
+IA is a **storage format/location optimization**, not a **replica reduction mechanism**. It changes how each node stores its own copy, not the number of copies. The Raft write and replication flow is identical to the Standard layer.As per path instructions, this is an exact replacement for the contiguous changed line.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| IA is a **storage format/location optimization**, not a **replica reduction mechanism**. It changes "how each node stores its own copy," not "how many copies exist." The Raft write and replication flow is identical to the Standard layer. | |
| IA is a **storage format/location optimization**, not a **replica reduction mechanism**. It changes how each node stores its own copy, not the number of copies. The Raft write and replication flow is identical to the Standard layer. |
🧰 Tools
🪛 GitHub Check: vale
[warning] 45-45:
[vale] reported by reviewdog 🐶
[PingCAP.Ambiguous] Consider using a clearer word than 'many' because it may cause confusion.
Raw Output:
{"message": "[PingCAP.Ambiguous] Consider using a clearer word than 'many' because it may cause confusion.", "location": {"path": "tidb-cloud/tieredstorage_faq.md", "range": {"start": {"line": 45, "column": 147}}}, "severity": "INFO"}
Sources: Path instructions, Linters/SAST tools
ti-chi-bot
Bot
removed
the
missing-translation-status
lilin90
added
area/tidb-cloud
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Actionable comments posted: 7
♻️ Duplicate comments (1)
tidb-cloud/tiered-storage-concepts.md (1)
66-66:⚠️ Potential issue | 🟠 MajorKeep request concurrency and throughput as separate limits.
tidb-cloud/tiered-storage-concepts.md#L66-L66: replace the QPS wording with separate limits for no more than 10 concurrent cold-read requests and no more than 1 GiB/s aggregate throughput.tidb-cloud/tiered-storage-limitations.md#L36-L36: split the combined1 GiB/s (≤ 10 concurrent)value into separate throughput and concurrency rows.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: ac2090ff-f7bd-475c-a51f-7f1766f3ff3e
📒 Files selected for processing (6)
.gitignoreTOC-tidb-cloud-byoc.mdtidb-cloud/tiered-storage-concepts.mdtidb-cloud/tiered-storage-faq.mdtidb-cloud/tiered-storage-limitations.mdtidb-cloud/tiered-storage-operations.md
|
|
||
| ## What happens to IA tables when the object store (S3) experiences an outage? | ||
|
|
||
| IA tables will be affected and become unavailable — since all data resides remotely, read requests must fetch from S3. Additionally, if S3 bandwidth is saturated, IA read/write performance will also be impacted. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Do not state that every IA table becomes unavailable during an object-store outage.
The concepts document says hot data can remain in the local IA cache. During an outage, uncached reads and operations requiring object storage can fail, but cached data is not necessarily unavailable.
Suggested replacement
-IA tables will be affected and become unavailable — since all data resides remotely, read requests must fetch from S3. Additionally, if S3 bandwidth is saturated, IA read/write performance will also be impacted.
+IA tables will be affected: reads for uncached data and operations requiring object storage may fail, while cached data may remain readable. If object-storage bandwidth is saturated, IA read/write performance will also be impacted.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| IA tables will be affected and become unavailable — since all data resides remotely, read requests must fetch from S3. Additionally, if S3 bandwidth is saturated, IA read/write performance will also be impacted. | |
| IA tables will be affected: reads for uncached data and operations requiring object storage may fail, while cached data may remain readable. If object-storage bandwidth is saturated, IA read/write performance will also be impacted. |
🧰 Tools
🪛 GitHub Check: vale
[failure] 26-26:
[vale] reported by reviewdog 🐶
[PingCAP.EmDash] Don't put a space before or after a dash.
Raw Output:
{"message": "[PingCAP.EmDash] Don't put a space before or after a dash.", "location": {"path": "tidb-cloud/tiered-storage-faq.md", "range": {"start": {"line": 26, "column": 50}}}, "severity": "ERROR"}
Connor1996
left a comment
Connor1996
left a comment
There was a problem hiding this comment.
Why is the doc merged to release-8.5? Should be master
| | Range Columns partitioned table | Supported | Must use `ENGINE_ATTRIBUTE` | | ||
| | List partitioned table | Supported | Must use `ENGINE_ATTRIBUTE` | | ||
| | List Columns partitioned table | Supported | Must use `ENGINE_ATTRIBUTE` | | ||
| | Hash partitioned table | **Not supported** | — | |
There was a problem hiding this comment.
It has an ambiguity. Table-level IA is supported and applies to all partitions. TiDB rejects only partition-scoped storage-class definitions for HASH/KEY partitions.
| summary: Learn about Tiered Storage on TiDB Cloud BYOC/Premium/Essential, including its concepts, architecture, use cases, and read amplification. | ||
| --- | ||
|
|
||
| # Tiered Storage Concepts |
There was a problem hiding this comment.
| # Tiered Storage Concepts | |
| # Tiered Storage Overview |
Co-authored-by: Lilian Lee <lilin@pingcap.com>
3 participants
First-time contributors' checklist
What is changed, added or deleted? (Required)
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
AI agent involvement
Do your changes match any of the following descriptions?
Summary by CodeRabbit