Skip to content

feat(bigquery): add bigquery_create_manifest_biglake_table sample to … - #14721

Merged
iennae merged 1 commit into
mainfrom
feat/bq-create-manifest-biglake-table
Nov 3, 2026
Merged

iennae merged 1 commit into
mainfrom
feat/bq-create-manifest-biglake-table

Conversation

@iennae

@iennae iennae commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

…create a BigLake table backed by a Symfoni/Hive manifest file

Problem:
The BigQuery documentation (https://cloud.google.com/bigquery/docs/query-open-table-format-using-manifest-files) needs a standalone, tested Python code sample demonstrating how to create a BigLake table backed by a Symfoni/Hive manifest file (region tag bigquery_create_manifest_biglake_table).

Solution:

  • Add create_manifest_biglake_table(project_id: str, dataset_id: str, connection_id: str, manifest_uri: str) -> None in bigquery/biglake/bigquery_create_manifest_biglake_table.py (Create a BigLake external table over open-table-format Parquet files using a manifest file URI and Cloud Resource connection.)
  • Add unit tests (bigquery/biglake/bigquery_create_manifest_biglake_table_test.py) covering both the success path and API error handling (GoogleAPICallError)

Checklist

Testing

  • I have tested this change on a live environment and verified it works as intended.

Compliance & Style


Post-Approval Actions

  • Please merge this PR for me once it is approved

@iennae
iennae requested review from a team and chalmerlowe as code owners November 1, 2026 20:24
@snippet-bot

snippet-bot Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Here is the summary of changes.

You are about to add 1 region tag.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@product-auto-label product-auto-label Bot added api: bigquery Issues related to the BigQuery API. samples Issues that are directly related to samples. labels Nov 1, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new Python sample and corresponding unit tests for creating a BigLake external table backed by a newline-delimited manifest file. The feedback highlights two main issues: first, the SQL query uses 'CREATE OR REPLACE EXTERNAL TABLE', which renders the 'AlreadyExists' exception handling block dead code and conflicts with the unit tests; second, the 'noxfile_config.py' incorrectly ignores Python 3.10, which is the oldest supported version, rather than ignoring intermediate versions like 3.11 and 3.12.


TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
"ignored_versions": ["2.7", "3.7", "3.8", "3.9", "3.10", "3.11"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository's general rules, we should only test the oldest and newest supported Python releases in test configurations, ignoring intermediate versions. Since requirements-test.txt specifies python_version >= "3.10", Python 3.10 is the oldest supported version and should not be ignored. Assuming Python 3.13 is the newest supported version, we should ignore intermediate versions like 3.11 and 3.12.

Suggested change
"ignored_versions": ["2.7", "3.7", "3.8", "3.9", "3.10", "3.11"],
"ignored_versions": ["2.7", "3.7", "3.8", "3.9", "3.11", "3.12"],
References
  1. Only test the oldest and newest supported Python releases in test configurations (such as noxfile_config.py), ignoring intermediate versions.

@XrossFox XrossFox left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dependency versions in requirements file are outdated, but code looks good.

Comment thread bigquery/biglake/requirements.txt Resolved
@iennae
iennae force-pushed the feat/bq-create-manifest-biglake-table branch 3 times, most recently from 7c8cc72 to bc84abe Compare November 2, 2026 23:33
…create a BigLake table backed by a Symfoni/Hive manifest file

Problem:
The BigQuery documentation (https://cloud.google.com/bigquery/docs/query-open-table-format-using-manifest-files)
needs a standalone, tested Python code sample demonstrating how to
create a BigLake table backed by a Symfoni/Hive manifest file (region tag `bigquery_create_manifest_biglake_table`).

Solution:
- Add `create_manifest_biglake_table(project_id: str, dataset_id: str, connection_id: str, manifest_uri: str) -> None` in `bigquery/biglake/bigquery_create_manifest_biglake_table.py`
  (Create a BigLake external table over open-table-format Parquet files using a manifest file URI and Cloud Resource connection.)
- Add unit tests (`bigquery/biglake/bigquery_create_manifest_biglake_table_test.py`) covering both the success
  path and API error handling (`GoogleAPICallError`)
@iennae
iennae force-pushed the feat/bq-create-manifest-biglake-table branch from bc84abe to e5278b2 Compare November 3, 2026 02:44
@iennae
iennae merged commit b908495 into main Nov 3, 2026
13 of 21 checks passed
@iennae
iennae deleted the feat/bq-create-manifest-biglake-table branch November 3, 2026 02:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery Issues related to the BigQuery API. samples Issues that are directly related to samples.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants