lake: publish dbt integration docs - #23489
Conversation
ti-chi-bot
Bot
added
do-not-merge/work-in-progress
📝 WalkthroughWalkthroughThe PR replaces the generic dbt guide with TiDB Cloud Lake-specific setup instructions. It adds adapter support details, prerequisites, installation steps, profile configuration, connection options, and a table-of-contents link. ChangesTiDB Cloud Lake dbt documentation
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Skipping CI for Draft Pull Request. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
Bot
added
missing-translation-status
ti-chi-bot
Bot
removed
the
missing-translation-status
ti-chi-bot
Bot
removed
the
do-not-merge/work-in-progress
ti-chi-bot
Bot
added
the
do-not-merge/work-in-progress
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c871e6d5-f5d2-4e58-aac3-ac99e9130215
📒 Files selected for processing (2)
TOC-tidb-cloud-lake.mdtidb-cloud-lake/guides/load-with-dbt.md
| # Use dbt with TiDB Cloud Lake | ||
|
|
||
| [dbt](https://www.getdbt.com/) is a transformation workflow that helps you get more work done while producing higher quality results. You can use dbt to modularize and centralize your analytics code, while also providing your data team with guardrails typically found in software engineering workflows. Collaborate on data models, version them, and test and document your queries before safely deploying them to production, with monitoring and visibility. | ||
| [dbt](https://www.getdbt.com/) is an open-source data transformation tool that enables analytics engineers to build, test, and document SQL-based data models. The [TiDB Cloud Lake adapter for dbt](https://github.com/tidbcloud/lake-dbt) connects dbt to {{{ .lake }}} through the `tidbcloudlake-driver` Python package. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -u
printf '%s\n' '--- target document ---'
file=$(git ls-files | awk '$0=="tidb-cloud-lake/guides/load-with-dbt.md"{print; exit}')
if [ -n "${file:-}" ]; then
cat -n "$file" | sed -n '1,110p'
else
printf '%s\n' 'Target file not found'
fi
printf '%s\n' '--- repository references ---'
rg -n --glob '*.md' --glob '*.yml' --glob '*.yaml' \
'tidbcloud/lake-dbt|tidbcloudlake-dbt|warehouse|public repository|publication' . | sed -n '1,180p'
printf '%s\n' '--- GitHub repository and link status ---'
curl -L --max-time 20 -sS -o /tmp/lake_dbt_page -w 'page_http=%{http_code} final_url=%{url_effective}\n' \
'https://github.com/tidbcloud/lake-dbt'
curl --max-time 20 -sS -o /tmp/lake_dbt_api -w 'api_http=%{http_code}\n' \
'https://api.github.com/repos/tidbcloud/lake-dbt'
printf '%s\n' 'page_title_or_error:'
grep -o '<title>[^<]*' /tmp/lake_dbt_page | head -1 || true
printf '%s\n' 'api_response:'
cat /tmp/lake_dbt_apiRepository: pingcap/docs
Length of output: 41278
🌐 Web query:
Search for the public GitHub repository and published package documentation for "tidbcloud/lake-dbt", "tidbcloudlake-dbt", and "tidbcloudlake-driver".
💡 Result:
Based on current technical documentation and repository information as of August 11, 2026, the specific repositories and packages you inquired about are part of the TiDB ecosystem, though their naming conventions differ from your request. 1. tidbcloud/lake-dbt and tidbcloudlake-dbt There is no repository named tidbcloud/lake-dbt or tidbcloudlake-dbt. The official adapter for using dbt with TiDB is pingcap/dbt-tidb [1][2]. - GitHub Repository: https://github.com/pingcap/dbt-tidb [1][3] - Documentation: Available on the dbt Developer Hub [2] and the official TiDB documentation site [4]. - Package: The dbt adapter package is published as dbt-tidb [2]. 2. tidbcloudlake-driver There is no repository precisely named tidbcloudlake-driver. TiDB Cloud Lake provides various language-specific drivers instead of a single monolithic repository with that name [5]. - Core Driver Repository: The main repository for TiDB Cloud Lake native clients (including Rust, Python, and Node.js implementations) is tidbcloud/lakesql [6]. - Rust Driver: The Rust driver is available in the lakesql repository under the /driver directory [7]. It is published on crates.io as lake-driver [8][9]. - Other Languages: TiDB Cloud Lake provides specific drivers for Python, Go (lake-go), and Java (lake-jdbc) [10][5]. The official TiDB documentation maintains a dedicated page for connecting to TiDB Cloud Lake using these drivers [5]. Official documentation for all TiDB Cloud Lake connectivity can be found at the TiDB Docs website [11][5].
Citations:
- 1: https://github.com/pingcap/dbt-tidb
- 2: https://docs.getdbt.com/docs/local/connect-data-platform/tidb-setup
- 3: https://github.com/pingcap/dbt-tidb/blob/main/README.md
- 4: https://docs.pingcap.com/tidbcloud/integrate-tidbcloud-with-dbt/
- 5: https://docs.pingcap.com/tidbcloudlake/driver-overview/
- 6: https://github.com/tidbcloud/lakesql
- 7: https://github.com/tidbcloud/lakesql/tree/main/driver
- 8: https://crates.io/crates/lake-driver
- 9: https://docs.rs/crate/lake-driver/latest
- 10: https://github.com/tidbcloud/lake-go
- 11: https://docs.pingcap.com/tidbcloudlake/
Make the adapter repository accessible before publication. The https://github.com/tidbcloud/lake-dbt link returns 404, so readers cannot access the adapter. Publish the repository or update the link to its canonical public location.
Source: Path instructions
|
|
||
| - Python 3.8 or later | ||
| - A {{{ .lake }}} account and warehouse | ||
| - The host, database, username, password, and warehouse name for your {{{ .lake }}} connection |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- relevant document sections ---'
sed -n '1,110p' tidb-cloud-lake/guides/load-with-dbt.md
printf '%s\n' '--- warehouse references ---'
rg -n -C 3 '\bwarehouse\b|tidbcloudlake-dbt|lake-dbt' tidb-cloud-lake .github resources 2>/dev/null || true
printf '%s\n' '--- candidate adapter files ---'
git ls-files | rg -i 'dbt|adapter|profile|lake'Repository: pingcap/docs
Length of output: 50371
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import json
import urllib.request
def get(url):
req = urllib.request.Request(url, headers={"User-Agent": "review-inspection"})
with urllib.request.urlopen(req, timeout=20) as response:
return json.load(response)
pypi = get("https://pypi.org/pypi/tidbcloudlake-dbt/json")
info = pypi["info"]
print("--- PyPI metadata ---")
for key in ("name", "version", "summary", "home_page", "project_urls", "requires_python", "requires_dist"):
print(f"{key}: {info.get(key)}")
print("--- release files ---")
for item in pypi["releases"].get(info["version"], []):
print(item.get("filename"), item.get("url"))
repo = get("https://api.github.com/repos/tidbcloud/lake-dbt")
print("--- GitHub repository ---")
for key in ("html_url", "default_branch", "visibility", "archived"):
print(f"{key}: {repo.get(key)}")
for path in ("profiles.yml", "dbt/adapters/tidbcloudlake/connections.py",
"dbt/adapters/tidbcloudlake/impl.py",
"dbt/adapters/tidbcloudlake/__init__.py"):
url = "https://api.github.com/repos/tidbcloud/lake-dbt/contents/" + path
try:
item = get(url)
except Exception as exc:
print(f"--- {path}: unavailable ({exc}) ---")
continue
print(f"--- {path} ---")
print(item.get("download_url"))
PYRepository: pingcap/docs
Length of output: 2469
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- PyPI project metadata ---'
curl -k -fsSL https://pypi.org/pypi/tidbcloudlake-dbt/json |
jq '{name: .info.name, version: .info.version, project_urls: .info.project_urls,
requires_python: .info.requires_python, requires_dist: .info.requires_dist,
release_urls: [.releases[.info.version][] | .url]}'
printf '%s\n' '--- upstream repository metadata ---'
curl -k -fsSL https://api.github.com/repos/tidbcloud/lake-dbt |
jq '{html_url, default_branch, visibility, archived}'
printf '%s\n' '--- upstream tree paths containing profile/connection/adapter ---'
curl -k -fsSL 'https://api.github.com/repos/tidbcloud/lake-dbt/git/trees/HEAD?recursive=1' |
jq -r '.tree[] | select(.path | test("(?i)(profile|connection|adapter|config)")) | .path'Repository: pingcap/docs
Length of output: 1133
🏁 Script executed:
#!/bin/bash
set -eu
wheel=/tmp/tidbcloudlake_dbt-1.8.1-py3-none-any.whl
curl -k -fsSL \
'https://files.pythonhosted.org/packages/a7/48/34e912b921839e80f6d722136e9624ab89fe899aa3aef6c781ec56c99a2e/tidbcloudlake_dbt-1.8.1-py3-none-any.whl' \
-o "$wheel"
python3 - "$wheel" <<'PY'
import re
import sys
import zipfile
wheel = sys.argv[1]
with zipfile.ZipFile(wheel) as z:
names = z.namelist()
print("--- package files ---")
for name in names:
if name.endswith((".py", ".yml", ".yaml", ".json", ".toml")):
print(name)
print("--- relevant source lines ---")
for name in names:
if not name.endswith(".py"):
continue
text = z.read(name).decode("utf-8", "replace")
if re.search(r"warehouse|Profile|Connection|credentials|required", text, re.I):
print(f"### {name}")
for number, line in enumerate(text.splitlines(), 1):
if re.search(r"warehouse|Profile|Connection|credentials|required", line, re.I):
print(f"{number}: {line}")
PY
printf '%s\n' '--- upstream repository metadata ---'
curl -k -fsSL https://api.github.com/repos/tidbcloud/lake-dbt |
jq '{html_url, default_branch, visibility, archived}'
printf '%s\n' '--- upstream tree paths ---'
curl -k -fsSL 'https://api.github.com/repos/tidbcloud/lake-dbt/git/trees/HEAD?recursive=1' |
jq -r '.tree[] | select(.path | test("(?i)(profile|connection|adapter|config)")) | .path'Repository: pingcap/docs
Length of output: 4497
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import zipfile
wheel = "/tmp/tidbcloudlake_dbt-1.8.1-py3-none-any.whl"
with zipfile.ZipFile(wheel) as z:
for name in (
"dbt/include/tidbcloudlake/profile_template.yml",
"dbt/adapters/tidbcloudlake/connections.py",
):
print(f"--- {name} ---")
for number, line in enumerate(z.read(name).decode().splitlines(), 1):
if name.endswith("profile_template.yml") or 30 <= number <= 170:
print(f"{number}: {line}")
source = z.read("dbt/adapters/tidbcloudlake/connections.py").decode()
namespace = {}
# Extract only the deterministic DSN-building logic for a read-only probe.
start = source.index(" def build_dsn")
end = source.index(" `@classmethod`", start)
print("--- build_dsn source slice ---")
print(source[start:end])
PYRepository: pingcap/docs
Length of output: 6419
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import zipfile
wheel = "/tmp/tidbcloudlake_dbt-1.8.1-py3-none-any.whl"
with zipfile.ZipFile(wheel) as z:
for name in (
"dbt/include/tidbcloudlake/profile_template.yml",
"dbt/adapters/tidbcloudlake/connections.py",
):
print(f"--- {name} ---")
for number, line in enumerate(z.read(name).decode().splitlines(), 1):
if name.endswith("profile_template.yml") or 30 <= number <= 170:
print(f"{number}: {line}")
source = z.read("dbt/adapters/tidbcloudlake/connections.py").decode()
start = source.index(" def build_dsn")
end = source.index(" `@classmethod`", start)
print("--- build_dsn source slice ---")
print(source[start:end])
PYRepository: pingcap/docs
Length of output: 6419
Treat warehouse as an optional profile field.
Keep the warehouse resource in the prerequisites, but do not list the optional warehouse profile value as required.
Committable suggestion
| - The host, database, username, password, and warehouse name for your {{{ .lake }}} connection | |
| - The host, database, username, and password for your {{{ .lake }}} connection; optionally, the warehouse name if you want to select a warehouse explicitly |
Source: Path instructions
2 participants
What is changed, added or deleted? (Required)
tidbcloudlake-dbtadapter, including its supported features, installation, and connection profile.Validation completed: Markdown lint, manual line-break checks, Lake TOC verification, internal link checks, forbidden-term checks, and
git diff --check.Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
AI agent involvement
Do your changes match any of the following descriptions?
Summary by CodeRabbit