Skip to content

Conversation

@geropl
Copy link
Member

@geropl geropl commented Jan 14, 2026

Context

Some customers experience VSCode Desktop connection failures with HTTP/2 protocol errors, while browser-based IDEs work fine. These issues are often caused by corporate proxies, SSL inspection appliances, or firewalls that don't properly handle HTTP/2.

Diagnosing these issues is difficult because:

  • Customers can't access preview environments
  • Cycle times for testing fixes are long
  • We need visibility into the customer's network behavior

Solution

This PR adds a standalone diagnostic tool (dev/network-diag) that customers can run to identify network issues. The tool produces both human-readable and JSON output for easy sharing and comparison.

Features

Standard Tests

  1. DNS Resolution - Verifies hostname resolves correctly
  2. TCP Connection - Tests basic connectivity
  3. TLS Handshake - Checks TLS version, cipher suite, ALPN negotiation, key exchange, certificate chain
  4. HTTP/2 SETTINGS Exchange - Verifies HTTP/2 works and captures server settings
  5. HTTP/1.1 Test - Baseline HTTP/1.1 connectivity
  6. Connect-RPC Test - Makes a request similar to VSCode Desktop
  7. WebSocket Test - Tests WebSocket upgrade
  8. Reference Test - Compares against google.com to isolate Gitpod-specific issues

Extended Tests (--full flag)

TLS Compatibility:

  • TLS 1.2 fallback
  • P-256 curve only (simulates middlebox rejecting X25519)
  • Legacy cipher suite

HTTP/2 Robustness:

  • Large headers (8KB)
  • Many headers (50)
  • Concurrent streams (5)
  • HTTP/2 PING roundtrip

Protocol Variants:

  • gRPC request
  • Connect-RPC streaming

Middlebox Detection:

  • Response headers capture (Via, Server, X-Forwarded-*, proxy signatures)
  • Proxy signature detection (BlueCoat, Squid, CloudFlare, AWS CloudFront, etc.)
  • Full certificate chain details (subject, issuer, org, validity, DNS names, CA flag)
  • TLS session resumption test
  • Timing breakdown (DNS/TCP/TLS/HTTP2/first-byte separately)

Reliability Tests:

  • Retry test (3 attempts) to detect intermittent failures
  • Connection reuse test (second request on same connection)
  • Consistency check across attempts

Automatic Diagnosis

The tool automatically detects and reports:

  • Corporate SSL inspection (known vendor certificates)
  • HTTP/2 protocol errors
  • HTTP/2 downgrade to HTTP/1.1
  • Proxy signatures in response headers
  • Timing anomalies (slow TLS = possible inspection)
  • Intermittent failures
  • Connection reuse issues
  • Deep certificate chains (proxy injection)

Usage

# Basic diagnostics
./gitpod-network-diag api.gitpod.cloud

# Full test suite
./gitpod-network-diag --full api.gitpod.cloud

# JSON output for sharing/comparison
./gitpod-network-diag --full --json api.gitpod.cloud > diagnostic.json

Example Output

Gitpod Network Diagnostics
==========================
Target: api.gitpod.cloud:443
Time:   2026-01-15T11:22:53Z

Client: darwin/arm64
        HTTPS_PROXY=http://proxy.corp:8080

[1] DNS Resolution: OK (35.241.252.3, 12ms)
[2] TCP Connection: OK (23ms)
[3] TLS Handshake:  OK (45ms)
    Version:      TLS1.3
    Cipher:       TLS_AES_128_GCM_SHA256
    Key Exchange: ECDHE (TLS 1.3)
    ALPN:         h2
    Cert Chain:
      [0] *.gitpod.cloud (issuer: R12)
      [1] R12 (issuer: ISRG Root X1)
[4] HTTP/2 SETTINGS: OK (0ms)
    Server settings:
      MAX_FRAME_SIZE: 1048576
      MAX_CONCURRENT_STREAMS: 250
[5] HTTP/1.1 Test:  OK (HTTP 404, 12ms)
[6] Connect-RPC:    OK (HTTP 415, 13ms)
[7] WebSocket:      OK (HTTP 101, 12ms)
[8] Reference Test: OK (www.google.com:443)

Extended Tests
--------------
[TLS Compatibility]
  TLS 1.2 fallback: OK (HTTP 404)
  P-256 curve only: OK (HTTP 404)
  Legacy cipher: OK (HTTP 404)
[HTTP/2 Robustness]
  Large headers (8KB): OK (HTTP 404)
  Many headers (50): OK (HTTP 404)
  Concurrent streams: OK (5/5)
  HTTP/2 PING: OK (HTTP 404)
[Protocol Variants]
  gRPC request: OK (HTTP 200)
  Connect-RPC stream: OK (HTTP 415)
[Middlebox Detection]
  Cert chain depth: 2
  Proxy signatures: none detected
  Response headers:
    Via: 1.1 Caddy
  TLS session resumption: OK
[Timing Analysis]
  DNS:          0ms
  TCP connect:  10ms
  TLS handshake: 12ms
  HTTP/2 setup: 0ms
  First byte:   11ms
  Total:        35ms
[Reliability]
  Attempt 1: OK (HTTP 415, 35ms)
  Attempt 2: OK (HTTP 415, 12ms)
  Attempt 3: OK (HTTP 415, 11ms)
  Consistency: all attempts succeeded
  Connection reuse: OK
    Second request reused connection (faster)

Diagnosis
---------
No issues detected.

Compatibility Warnings:
  ⚠ TLS 1.3 in use - server may offer post-quantum key exchange (x25519mlkem768) which some older network equipment may reject

Cross-Platform Binaries

The Makefile builds binaries for:

make build-all

Testing

Tested against three preview environments with different Caddy versions:

  • Old Caddy (2.7.6)
  • Current Main (2.11.0-beta.2)
  • Fix branch (2.11.0-beta.2 + HTTP/1.1 backend)

All environments pass all tests identically, confirming the tool works correctly and can detect differences when they exist.

Related

This tool was created to help diagnose issues like those in #21258 where HTTP/2 behavior changes in Caddy 2.9+ may cause problems with certain corporate network configurations.

Adds a cross-platform diagnostic tool to help identify network issues
affecting Gitpod connectivity, particularly HTTP/2 and TLS problems
caused by corporate proxies and firewalls.

The tool checks:
- DNS resolution
- TCP connectivity
- TLS handshake (version, cipher, ALPN, cert chain)
- HTTP/2 SETTINGS exchange with server settings capture
- Connect-RPC test request (mimics VSCode Desktop)
- Reference test against google.com

It detects common issues like SSL inspection, HTTP/2 protocol errors,
and protocol downgrades, providing actionable recommendations.

Co-authored-by: Ona <[email protected]>
@socket-security
Copy link

socket-security bot commented Jan 14, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedgolang/​golang.org/​x/​net@​v0.34.07598100100100

View full report

Adds two additional tests to help differentiate between failure modes:

- HTTP/1.1 test: Forces HTTP/1.1 via ALPN to check if basic HTTPS works
- WebSocket test: Attempts WebSocket upgrade to /api/gitpod endpoint

This helps identify:
- HTTP/2 specific issues (HTTP/1.1 works, HTTP/2 fails)
- WebSocket blocking (HTTP works, WebSocket upgrade fails)

Co-authored-by: Ona <[email protected]>
- Add TLS key exchange detection (ECDHE, RSA, TLS 1.3)
- Add compatibility warnings section to diagnosis:
  - Warn if server advertises ENABLE_CONNECT_PROTOCOL
  - Warn about potential post-quantum key exchange with TLS 1.3
  - Warn if HTTP/2 works but Connect-RPC fails
- Display key exchange in TLS handshake output

These warnings help identify potential issues with corporate proxies
and older network equipment that may not support newer TLS/HTTP/2 features.

Co-authored-by: Ona <[email protected]>
Adds --full flag to run extended tests:

TLS Compatibility:
- TLS 1.2 fallback
- P-256 curve only (simulates middlebox rejecting X25519)
- Legacy cipher suite

HTTP/2 Robustness:
- Large headers (8KB)
- Many headers (50)
- Concurrent streams (5)

Protocol Variants:
- gRPC request
- Connect-RPC streaming

Bumps version to 0.2.0.

Co-authored-by: Ona <[email protected]>
New test categories:

Middlebox Detection:
- Response headers capture (Via, Server, X-Forwarded-*, proxy signatures)
- Proxy signature detection (BlueCoat, Squid, CloudFlare, etc.)
- Full certificate chain details (subject, issuer, org, validity, DNS names)
- TLS session resumption test
- Timing breakdown (DNS/TCP/TLS/HTTP2/first-byte)

Reliability Tests:
- Retry test (3 attempts) to detect intermittent failures
- Connection reuse test (second request on same connection)
- Consistency check across attempts

Also adds HTTP/2 PING roundtrip test to HTTP/2 robustness category.

Bumps version to 0.3.0.

Co-authored-by: Ona <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants