feat(py/samples): add web-endpoints-hello — REST + gRPC kitchen-sink sample by yesudeep · Pull Request #4498 · firebase/genkit

yesudeep · 2026-02-07T22:11:25Z

Summary

Adds web-endpoints-hello — a self-contained, template-ready sample that demonstrates all the ways to expose Genkit flows as network endpoints: REST (ASGI) and gRPC.

Designed to be copied out of the monorepo as a standalone project starter via ./scripts/eject.sh.

Highlights

REST via 3 ASGI frameworks: FastAPI (default), Litestar, Quart — selected via --framework
gRPC server: Async grpc.aio server with reflection, running in parallel on :50051
3 ASGI servers: granian (default), uvicorn, hypercorn — selected via --server
Both protocols share flows: REST and gRPC endpoints call the same Genkit flows
Server-side streaming: TellStory RPC streams chunks via gRPC
Secure by default: OWASP headers, CORS, rate limiting, body-size limits, request timeouts, secret masking — all production-ready with zero configuration
Production resilience: Response cache with stampede protection, async circuit breaker, connection tuning, multi-worker gunicorn support
OpenTelemetry built-in: Every request produces a trace; just dev auto-starts Jaeger
Multi-cloud deploy: Scripts for Cloud Run, App Engine, Firebase Hosting, Fly.io, AWS App Runner, Azure Container Apps
Self-contained dev workflow: ./setup.sh + just dev gets everything running
361 unit tests covering all modules; standalone just lint pipeline with ruff, ty, pyrefly, pyright, shellcheck, liccheck, pysentry

Changes outside the sample

CI (.github/workflows/python.yml): Added sample-tests job that installs and tests samples in isolation
License scripts (bin/check_license, bin/add_license): Ignore generated protobuf stubs (*_pb2.py, *_pb2_grpc.py)
Workspace type checking (py/pyproject.toml): Exclude web-endpoints-hello from workspace-level ty/pyrefly/pyright — the sample has its own lint pipeline with the correct deps. Added sentry-sdk to workspace lint deps for type resolution.
Samples naming (py/samples/README.md, all sample pyproject.tomls): Standardized naming taxonomy and added run.sh scripts with shared _common.sh utilities
Granian adapter fix (packages/genkit/src/genkit/web/manager/_adapters.py): Switched from sync granian.Granian to async granian.server.embed.Server API
Typos (py/typos.toml): Added caf to allowlist (substring of café in Latin-1 encoding tests)

Architecture

src/
├── app_init.py          # Genkit instance + cloud telemetry auto-detection
├── config.py            # Settings via pydantic-settings + CLI args
├── flows.py             # Genkit flow definitions (shared by REST + gRPC)
├── generated/           # Protobuf + gRPC stubs (auto-generated)
├── grpc_server.py       # gRPC service + interceptors
├── main.py              # Entry point: resilience → security → start servers
├── security.py          # ASGI security middleware stack
├── server.py            # ASGI server helpers (granian/uvicorn/hypercorn)
├── cache.py             # TTL + LRU cache with stampede protection
├── circuit_breaker.py   # Async circuit breaker for LLM APIs
├── rate_limit.py        # Token-bucket rate limiting (ASGI + gRPC)
├── telemetry.py         # OpenTelemetry setup
└── frameworks/
    ├── fastapi_app.py   # FastAPI adapter
    ├── litestar_app.py  # Litestar adapter
    └── quart_app.py     # Quart adapter
protos/
└── genkit_sample.proto  # gRPC service definition (9 RPCs)

Test plan

just test — 361 unit tests pass
just lint — all type checkers and linters pass
./run.sh starts both REST (:8080) and gRPC (:50051)
./test_endpoints.sh / ./test_grpc_endpoints.sh — integration tests
Workspace bin/lint and tests pass with the sample excluded from workspace-level type checking

gemini-code-assist · 2026-02-07T22:11:42Z

Summary of Changes

Hello @yesudeep, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new Python sample application that demonstrates the powerful integration of Genkit AI flows with the FastAPI web framework. The sample highlights FastAPI's native asynchronous capabilities, allowing Genkit flows to be called directly from API endpoints, and showcases features like real-time AI response streaming via SSE, context propagation from HTTP requests, and the reusability of Pydantic models across both frameworks. This addition provides a clear example for developers looking to build high-performance, AI-powered web services using Genkit and FastAPI.

Highlights

New FastAPI Sample: Introduces a new sample application, web-fastapi-hello, demonstrating how to integrate and serve Genkit AI flows using the FastAPI web framework.
Async Flow Invocation & Streaming: Showcases direct asynchronous invocation of Genkit flows from FastAPI handlers and implements Server-Sent Events (SSE) for real-time token streaming from AI models.
Context Injection: Illustrates how to inject HTTP request context, such as the Authorization header, into Genkit flows for personalized responses.
Pydantic Model Sharing: Highlights the seamless sharing of Pydantic models for both FastAPI request validation and Genkit flow input/output schemas.
Dual Server Setup: Provides a setup that concurrently runs the FastAPI application (via Uvicorn) and the Genkit DevUI for development and debugging.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

py/pyproject.toml
- Added the new web-fastapi-hello sample to the workspace sources and adjusted formatting.
py/samples/web-fastapi-hello/LICENSE
- Added the Apache License 2.0 for the new sample.
py/samples/web-fastapi-hello/README.md
- Created a comprehensive README detailing the sample's architecture, endpoints, setup, testing instructions, and an explanation of its inner workings.
py/samples/web-fastapi-hello/pyproject.toml
- Defined project metadata and dependencies, including fastapi, uvicorn, genkit, and genkit-plugin-google-genai.
py/samples/web-fastapi-hello/run.sh
- Implemented a shell script to simplify running the sample, which starts both the FastAPI server with auto-reload and the Genkit DevUI.
py/samples/web-fastapi-hello/src/main.py
- Developed the core FastAPI application, including Genkit flow definition, non-streaming and streaming chat endpoints, a health check, and server startup logic.

Activity

The pull request introduces a new feature, adding a FastAPI sample for Genkit.
The author, yesudeep, has provided a detailed description outlining the summary, rationale for FastAPI, endpoints, key features demonstrated, and affected files.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a new sample, web-fastapi-hello, which is a great addition to demonstrate Genkit's integration with FastAPI. The implementation correctly showcases non-streaming flow invocation, context passing, and dual server setup.

My review includes two main points of feedback:

A suggestion to refactor the streaming endpoint (/chat/stream) in src/main.py to use Genkit's native streaming API (flow.stream()). This simplifies the code, improves performance by removing polling, and avoids re-registering a flow on every request.
A minor correction in py/samples/web-fastapi-hello/pyproject.toml regarding an incorrect package path in the build configuration.

Overall, this is a valuable sample, and with these changes, it will be an even better example for users.

py/samples/web-fastapi-hello/src/main.py

py/samples/web-fastapi-hello/pyproject.toml

yesudeep · 2026-02-08T08:18:36Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces web-endpoints-hello, a comprehensive and well-structured sample demonstrating REST and gRPC endpoints for Genkit flows. The sample is of high quality, featuring production-ready patterns for security, resilience, and observability, along with extensive documentation and tests. The changes to the wider repository, such as updates to CI, linting, and dependency configurations, are well-integrated.

My review has identified a few issues that should be addressed:

A potential security risk in the root .gitignore file due to the removal of a global rule for .env files.
Portability issues in the shared shell script _common.sh for installing CLI tools on different Linux distributions.

Overall, this is an excellent addition to the repository. Addressing the identified issues will further improve the security and robustness of the codebase.

.gitignore

py/samples/_common.sh

@ai

Adds a new sample demonstrating FastAPI + Genkit integration, exercising several core Genkit capabilities: - Flows — @ai.flow() for tell_joke and translate_text - Tools — @ai.tool() for get_current_time (model-callable) - Structured output — Output(schema=TranslationResult) for typed JSON - Streaming — generate_stream() with SSE via StreamingResponse - Traced steps — ai.run() for auditable sub-spans in DevUI - Context injection — Authorization header forwarded to flows - uvloop — 2-4x faster async event loop Endpoints: POST /tell-joke non-streaming joke POST /tell-joke/stream SSE streaming joke POST /translate structured translation with tool use GET /health health check GET /docs auto-generated Swagger UI (auto-opened) FastAPI is ASGI-native, so Genkit flows work with plain await — no adapter layer needed unlike Flask which requires genkit_flask_handler. feat(py/samples): multi-cloud deploy, auto-telemetry, and endpoint tests Enhance the web-fastapi-hello sample with smooth multi-cloud deployment, auto-detection of cloud telemetry, and comprehensive endpoint test coverage. Rationale: Users deploying Genkit apps need zero-friction deployment to their preferred cloud with appropriate observability built in. This commit adds AWS and Azure as first-class deployment targets alongside GCP and Fly.io, with the app automatically detecting the cloud platform and enabling the right telemetry plugin (Cloud Trace, X-Ray, App Insights, or generic OTLP). Deployment: - Add deploy_aws.sh for AWS App Runner (ECR + auto-scaling) - Add deploy_azure.sh for Azure Container Apps (ACR Build + scale-to-zero) - Add CLI auto-detection/install helpers for gcloud, aws, az, flyctl in _common.sh with interactive platform-specific install prompts - Add auth check helpers (check_aws_auth, check_az_auth) alongside existing check_gcloud_auth - All deploy scripts now source _common.sh for consistent UX Telemetry: - Auto-detect GCP, AWS, Azure at startup via env vars (K_SERVICE, AWS_EXECUTION_ENV, CONTAINER_APP_NAME, etc.) - Enable platform-specific telemetry plugin automatically - Fall back to generic OTLP if OTEL_EXPORTER_OTLP_ENDPOINT is set - Add GENKIT_TELEMETRY_DISABLED env var to disable telemetry - Add --no-telemetry CLI flag - Add optional dependency groups: pip install .[gcp|aws|azure|observability] Tests: - Add pytest endpoint tests (tests/endpoints_test.py) covering all HTTP endpoints with mocked Genkit flows - Add test optional deps group in pyproject.toml Documentation: - Update README.md with AWS/Azure deploy sections, telemetry docs, and updated feasibility/secrets tables - Update justfile with deploy-aws, deploy-azure targets - Update py/samples/README.md with web-fastapi-hello entry fix(py): resolve CI lint/test failures and fix granian adapter Fix pre-existing type errors in GranianAdapter by switching from the sync granian.Granian API (whose serve() is not awaitable) to the async granian.server.embed.Server API designed for embedded use within an existing event loop. Changes: Granian adapter (packages/genkit/src/genkit/web/manager/_adapters.py): - Use granian.server.embed.Server instead of granian.Granian - Remove workers/loop params (embed API uses runtime_threads) - Import Interfaces and LogLevels correctly Type checker config (py/pyproject.toml): - Add granian, pydantic-settings to lint dependency group - Add web-fastapi-hello paths to ty environment root and pyrefly search - Add samples/*/tests/**/*.py to pyrefly project_includes - Ignore web-fastapi-hello tests in workspace pytest (run in isolation) CI (python.yml): - Add sample-tests job that installs and tests samples in isolation - Build job now depends on sample-tests Sample fixes (web-fastapi-hello): - Add conftest.py for sys.path setup (workspace + sample root) - Add pytest config to sample pyproject.toml for isolated test runs - Fix async fixture to use pytest_asyncio.fixture (strict mode) - Fix streaming test mock (use real coroutine, not AsyncMock) - Adopt .<name>.env convention (auto-gitignored by **/*.env) - Add local.env.example template for local development Env file convention: - Use .<name>.env instead of .env.<name> so all env files share the .env extension, are auto-gitignored by **/*.env, and get syntax highlighting in editors refactor(py/samples): rename web-fastapi-hello to web-asgi-hello Renames the sample to web-asgi-hello and adds multi-framework support (FastAPI + Litestar) via a framework adapter pattern. Changes: - Rename sample directory and package from web-fastapi-hello to web-asgi-hello - Add --framework=fastapi|litestar CLI flag for framework selection - Add src/frameworks/ adapter package with create_app() factory pattern - fastapi_app.py: FastAPI adapter with all routes - litestar_app.py: Litestar adapter with matching routes - Make telemetry.py framework-aware (auto-detects FastAPI vs Litestar) - Update config.py with framework setting and --framework CLI arg - Make server.py accept any ASGI app (not FastAPI-specific) - Update run.sh to pass all CLI args through ($@) - Update justfile with dev-litestar command - Update deploy scripts and Dockerfile with new service names - Update pyproject.toml with litestar and OTel ASGI dependencies - Update tests to use create_app() factory pattern - Update workspace pyproject.toml references - Update README with multi-framework docs and template usage guide - Remove old src/routes.py (replaced by framework adapters) fix(py/samples): resolve ty type checker errors in web-asgi-hello - Use Callable[..., Any] type alias for ASGI apps in server.py - Use fastapi.FastAPI concrete type and isinstance check in telemetry.py - Fix span.attributes narrowing in telemetry_test.py - Move gRPC import fallback to avoid unresolved-import error feat(py/samples): port ruff, ty, pyright, pyrefly configs to web-asgi-hello Adds standalone type checker and linter configurations to pyproject.toml so the sample works as a self-contained template when copied out of the monorepo. Includes ruff, ty, pyright, and pyrefly with the same strictness as the monorepo root. feat(py/samples): add gRPC support and rename to web-endpoints-hello Rename web-asgi-hello → web-endpoints-hello to reflect that it's now a kitchen-sink sample showing all endpoint types: REST (FastAPI, Litestar, Quart) and gRPC. gRPC additions: - src/grpc_server.py: async gRPC servicer that delegates every RPC to the same Genkit flows used by the REST endpoints - protos/genkit_sample.proto: service definition with 9 RPCs including server-side streaming (TellStory) - src/generated/: auto-generated protobuf + gRPC stubs - scripts/generate_proto.sh: regenerate stubs from proto definition - gRPC reflection enabled for grpcui/grpcurl discovery - Both servers run in parallel via asyncio.gather in ai.run_main() New CLI flags: --grpc-port (default 50051), --no-grpc New scripts: test_grpc_endpoints.sh (grpcurl-based integration tests) New justfile targets: proto, grpcui, grpc-list, test-grpc-endpoints, test-all Dependencies added: grpcio, grpcio-tools, grpcio-reflection fix(py/samples): make setup scripts cross-platform (macOS, Debian, Fedora) Rewrite setup.sh to auto-detect the OS and package manager: - macOS: Homebrew - Debian/Ubuntu: apt (with fallbacks for older versions) - Fedora: dnf - Arch: pacman Platform detection sources /etc/os-release on Linux and prints the detected OS/distro/pkg-manager at startup for debugging. Tool-specific install strategies: - uv: cross-platform curl installer (unchanged) - just: brew → apt/dnf (if available) → official install script fallback - podman: brew / apt / dnf - shellcheck: brew / apt / dnf - grpcurl: brew → go install → prebuilt binary from GitHub releases - grpcui: brew → go install (with helpful error message if Go missing) - genkit CLI: npm (cross-platform, unchanged) Also update test_grpc_endpoints.sh error messages with Linux install instructions, and add a Prerequisites section to the README with a per-platform install matrix. docs(py/samples): add instructions to pin genkit version when using as template Inside the monorepo, genkit and genkit-plugin-* dependencies resolve to local workspace packages via [tool.uv.sources]. When copying the sample out as a standalone project, users must pin these to a released PyPI version (e.g. genkit>=0.5.0) or uv sync / pip install will fail. Add a "Pin Genkit to a release version" section to README.md showing the before/after for both main dependencies and optional telemetry plugins, with a tip for finding the latest version. Update GEMINI.md with the same guidance.

The granian.Granian() constructor's 'loop' parameter no longer accepts the string literal 'auto' (now requires the Loops enum), and 'log_level' requires a LogLevels enum instead of a raw string. These type mismatches cause pyrefly and pyright errors. Migrate to granian.server.embed.Server, which provides a proper async serve() method designed for running granian inside an existing event loop (exactly our use case in the ServersManager). This also replaces the deprecated 'workers' parameter with 'runtime_threads'. Also removes the temporary pyrefly/pyright suppressions that were added in #4498 to work around these pre-existing type errors.

yesudeep requested review from MengqinShen, huangjeff5 and zarinn3pal February 7, 2026 22:11

github-project-automation bot added this to Genkit Backlog Feb 7, 2026

github-actions bot added docs Improvements or additions to documentation python Python config labels Feb 7, 2026

yesudeep marked this pull request as draft February 7, 2026 22:11

gemini-code-assist bot reviewed Feb 7, 2026

View reviewed changes

py/samples/web-fastapi-hello/src/main.py Outdated Show resolved Hide resolved

py/samples/web-fastapi-hello/pyproject.toml Outdated Show resolved Hide resolved

yesudeep force-pushed the yesudeep/samples/web-robyn-hello branch 14 times, most recently from 80fa459 to 9fa510a Compare February 7, 2026 23:19

github-actions bot added the root label Feb 7, 2026

yesudeep force-pushed the yesudeep/samples/web-robyn-hello branch 5 times, most recently from 3ea1c85 to d25b0b2 Compare February 7, 2026 23:36

yesudeep force-pushed the yesudeep/samples/web-robyn-hello branch 14 times, most recently from d566fb2 to 2c235db Compare February 8, 2026 08:18

yesudeep force-pushed the yesudeep/samples/web-robyn-hello branch from 2c235db to f8ba104 Compare February 8, 2026 08:22

gemini-code-assist bot reviewed Feb 8, 2026

View reviewed changes

.gitignore Show resolved Hide resolved

py/samples/_common.sh Show resolved Hide resolved

py/samples/_common.sh Show resolved Hide resolved

yesudeep force-pushed the yesudeep/samples/web-robyn-hello branch 3 times, most recently from e096bde to bc0f401 Compare February 8, 2026 08:38

yesudeep force-pushed the yesudeep/samples/web-robyn-hello branch from bc0f401 to b32db40 Compare February 8, 2026 08:49

yesudeep marked this pull request as ready for review February 8, 2026 08:50

yesudeep enabled auto-merge (squash) February 8, 2026 08:51

yesudeep disabled auto-merge February 8, 2026 08:51

yesudeep enabled auto-merge (squash) February 8, 2026 08:51

yesudeep merged commit 8614e5e into main Feb 8, 2026
18 checks passed

yesudeep deleted the yesudeep/samples/web-robyn-hello branch February 8, 2026 09:01

github-project-automation bot moved this to Done in Genkit Backlog Feb 8, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(py/samples): add web-endpoints-hello — REST + gRPC kitchen-sink sample#4498

feat(py/samples): add web-endpoints-hello — REST + gRPC kitchen-sink sample#4498
yesudeep merged 1 commit intomainfrom
yesudeep/samples/web-robyn-hello

yesudeep commented Feb 7, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot commented Feb 7, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

yesudeep commented Feb 8, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

yesudeep commented Feb 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Highlights

Changes outside the sample

Architecture

Test plan

Uh oh!

gemini-code-assist bot commented Feb 7, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

yesudeep commented Feb 8, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yesudeep commented Feb 7, 2026 •

edited

Loading