Skip to content

[FLINK-XXXXX][python] Add type hints and fix docstrings in datastream/functions.py#28216

Open
Vaibhav-S-Gowda wants to merge 1 commit into
apache:masterfrom
Vaibhav-S-Gowda:FLINK-XXXXX-add-type-hints-datastream-functions
Open

[FLINK-XXXXX][python] Add type hints and fix docstrings in datastream/functions.py#28216
Vaibhav-S-Gowda wants to merge 1 commit into
apache:masterfrom
Vaibhav-S-Gowda:FLINK-XXXXX-add-type-hints-datastream-functions

Conversation

@Vaibhav-S-Gowda
Copy link
Copy Markdown

@Vaibhav-S-Gowda Vaibhav-S-Gowda commented May 20, 2026

What is the purpose of the change

This pull request adds explicit generic type hints (IN, OUT, KEY)
to core abstract method signatures in pyflink/datastream/functions.py
and fixes several docstring issues. These methods form the foundational
public API for PyFlink DataStream users, and the absence of type
annotations was reducing IDE autocompletion quality and preventing
effective mypy static analysis on user code extending these classes.

Additionally, MapFunction, CoMapFunction and KeySelector are made
properly Generic (like AsyncFunction and WindowFunction already are),
enabling correct TypeVar binding across the class hierarchy.

Brief change log

  • Made MapFunction properly Generic[IN, OUT]
  • Made CoMapFunction properly Generic[IN, OUT]
  • Made KeySelector properly Generic[IN, KEY]
  • Added Iterator to typing imports
  • Added return type None to Function.open and Function.close
  • Added (IN) -> OUT to MapFunction.map
  • Added (IN) -> OUT to CoMapFunction.map1 and map2
  • Added (IN) -> Iterator[OUT] to FlatMapFunction.flat_map
  • Added (IN) -> Iterator[OUT] to CoFlatMapFunction.flat_map1 and flat_map2
  • Added (IN, IN) -> IN to ReduceFunction.reduce
  • Added (IN) -> bool to FilterFunction.filter
  • Added (IN) -> KEY to KeySelector.get_key
  • Added (IN, Context) -> Iterator[OUT] to ProcessFunction.process_element
  • Added (IN, Context) -> Iterator[OUT] to KeyedProcessFunction.process_element
  • Added (IN, Context) -> Iterator[OUT] to CoProcessFunction.process_element1 and process_element2
  • Fixed typo: "mthod" → "method" in FlatMapFunction.flat_map docstring
  • Fixed typo: "MapFUnction" → "MapFunction" in FlatMapFunction class docstring
  • Fixed doctest continuation lines: >>> → ... inside class body examples

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Verified with mypy:
mypy pyflink/datastream/functions.py --ignore-missing-imports --follow-imports=skip
Result: Success: no issues found in 1 source file

Does this pull request potentially affect one of the following parts:

  • Dependencies: no
  • The public API: yes
  • The serializers: no
  • The runtime per-record code paths: no
  • Anything that affects deployment or recovery: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable
Was generative AI tooling used to co-author this PR?
  • Yes

@flinkbot
Copy link
Copy Markdown
Collaborator

flinkbot commented May 20, 2026

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build


from py4j.java_gateway import JavaObject
from typing import Union, Any, Generic, TypeVar, Iterable, List, Callable, Optional
from typing import Union, Any, Generic, TypeVar, Iterable, Iterator, List, Callable, Optional
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • is there a migration consideration with these changes?
  • I am curious whether that can be tested maybe by having tests implementing the generic types.
  • in terms of the PR title - the title should be either a square bracketed JIRA number or start with [hotfix].

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @davidradl, thanks for the feedback!

  • These changes are backward compatible since Python generics aren't enforced at runtime, so existing subclasses won't be affected.
  • I'll add tests that implement the generic types as you suggested.
  • Will also create a JIRA ticket and fix the PR title.

Updates coming soon!

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've requested a JIRA account and will update the ticket number once it's approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants