Skip to content

[Core] Explicitly Support gzip & deflate for Encodings in Aiohttp - #48388

Open
kashifkhan wants to merge 7 commits into
Azure:mainfrom
kashifkhan:azcore_comp_hdr
Open

[Core] Explicitly Support gzip & deflate for Encodings in Aiohttp#48388
kashifkhan wants to merge 7 commits into
Azure:mainfrom
kashifkhan:azcore_comp_hdr

Conversation

@kashifkhan

Copy link
Copy Markdown
Member

This PR addresses an issue with aiohttp and core. Today aiohttp automatically appends  br  (Brotli) and  zstd  to the Accept-Encoding  request header when it detects the corresponding optional library (e.g.  brotli / brotlicffi ) installed in the environment.

When the service honors that and returns a Brotli-encoded response, azure-core can't decode it: the aiohttp transport runs with auto_decompress=False  and decompresses manually, supporting only  gzip / deflate , so the raw bytes reach response.text() and raise a UnicodeDecodeError

Request does similar, but lets requests/urllib3 decompress the response itself, so it decodes whatever it advertised

We do the following:

  • Explicitly pass in gzip and deflate for aiohttp
  • Move the aiohttp body helper into the aiohttp section; its not generic and used only by aiohttp
  • Update tests

Copilot AI review requested due to automatic review settings July 31, 2026 14:56
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
7 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Restricts AioHttpTransport to supported response encodings, preventing decoding failures when optional aiohttp codecs are installed.

Changes:

  • Defaults Accept-Encoding to gzip, deflate while preserving caller overrides.
  • Relocates the aiohttp-specific body helper.
  • Adds tests and updates package metadata.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
azure/core/pipeline/transport/_aiohttp.py Sets supported encodings and hosts the body helper.
azure/core/rest/_aiohttp.py Imports the relocated helper.
azure/core/utils/_pipeline_transport_rest_shared.py Removes aiohttp-specific logic.
tests/async_tests/test_universal_http_async.py Tests defaults and caller overrides.
CHANGELOG.md Documents the fix.
azure/core/_version.py Bumps the version to 1.42.0.

Comment thread sdk/core/azure-core/CHANGELOG.md Outdated
Copilot AI review requested due to automatic review settings July 31, 2026 15:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comment thread sdk/core/azure-core/azure/core/pipeline/transport/_aiohttp.py

@JennyPng JennyPng left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@github-actions

This comment has been minimized.

Copilot AI review requested due to automatic review settings July 31, 2026 20:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

@github-actions

Copy link
Copy Markdown
Contributor
[Pilot] PR Pipeline Failure Analysis

A CI pipeline failed on this pull request. Here is an automated analysis of what went wrong and how to get the build green.

What failed

Two separate pipelines failed:

  1. azure-core mypy check (build 6646685) — validation failure. Mypy found one type error in azure/core/rest/_aiohttp.py:

    azure/core/rest/_aiohttp.py:170: error: Argument 1 to "_aiohttp_body_helper" has incompatible type
    "_RestAioHttpTransportResponseBackcompatMixin"; expected
    "AioHttpTransportResponse | RestAioHttpTransportResponse"  [arg-type]
    

    This is consistent with the PR's stated change of moving _aiohttp_body_helper into the aiohttp-specific section — the function's accepted type union likely needs to be updated to include the backcompat mixin.

  2. azure-ai-textanalytics test failures (macOS 3.11, Ubuntu 3.10, Windows 3.12 — build 6646684) — test failures across 14 test cases (mindependency, whl, sdist variants). The log also shows a pnpm ERR_PNPM_BAD_CONFIG_DEP error (Cannot resolve pnpm@9.5.0 as a package manager dependency because it has no integrity) which may be an infrastructure issue unrelated to this PR's changes. The azure-ai-textanalytics failures across all 3 OS/Python combinations are suspicious and could be infrastructure or pre-existing; this PR only touches azure-core.

Recommended next steps

  • Fix the mypy error: In azure/core/rest/_aiohttp.py (line 170), update the type annotation for _aiohttp_body_helper (or its call site) to accept _RestAioHttpTransportResponseBackcompatMixin in its union type. This is a direct consequence of the refactoring done in this PR.
  • Investigate azure-ai-textanalytics failures: Since this PR only modifies azure-core files, the 14 failing azure-ai-textanalytics tests may be pre-existing or infrastructure-related. Check the build logs to confirm whether these failures existed before this PR.
  • Re-run the pipeline for the azure-ai-textanalytics job to rule out the pnpm infrastructure issue (ERR_PNPM_BAD_CONFIG_DEP / no integrity for pnpm@9.5.0).
  • See the CI troubleshooting guide: https://aka.ms/ci-fix
  • Push new commits to address the mypy failure; this comment updates automatically on the next failing run.
Raw pipeline analysis (azsdk ci analyze)
Failed Tests (azure-ai-textanalytics, all platforms):
  test_abstract_summary, test_extract_key_phrases, test_extract_summary,
  test_health_care_lro, test_language_detection, test_multi_label_classify,
  test_recognize_entities, test_recognize_entities_ner,
  test_recognize_linked_entities, test_recognize_pii,
  test_recognize_pii_confidence_score, test_recognize_pii_redaction_policies,
  test_single_label_classify, test_text_sentiment
  (macos311, ubuntu2404_310, windows2022_312 — mindependency/whl/sdist)

Failed Task (build 6646684): ERR_PNPM_BAD_CONFIG_DEP
  Cannot resolve pnpm@9.5.0 as a package manager dependency because it has no integrity
  Pipeline: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6646684

Failed Task (build 6646685): azure-core mypy FAIL
  azure/core/rest/_aiohttp.py:170: error: Argument 1 to "_aiohttp_body_helper"
    has incompatible type "_RestAioHttpTransportResponseBackcompatMixin";
    expected "AioHttpTransportResponse | RestAioHttpTransportResponse"  [arg-type]
  Found 1 error in 1 file (checked 75 source files)
  Pipeline: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6646685

Copilot detected the failing pipeline and generated the analysis above. To have it attempt a fix automatically, reply with @copilot please fix the failing pipeline on this PR.

Generated by Pipeline Analysis - Next Steps · 31.5 AIC · ⌖ 6.32 AIC · ⊞ 6.6K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants