python-stdlib-idioms

Stdlib idioms — iteration and tree composition

Two common-but-distinct Python data idioms that don't fit the typing-idioms or value-objects skills, bundled here because each on its own is too thin to be a standalone skill. Use the section that matches your task.

• Iteration — generators, itertools, lazy pipelines, async iteration
• Tree composition — modeling leaf/composite structures with Protocol + dataclass, recursive traversal

When to use this skill

• Writing any for ... in ... loop
• Producing a sequence of values lazily
• Composing transformations on a stream
• Iterating async results from a DB query, HTTP stream, message queue
• Modeling a filesystem, expression tree, AST, UI tree
• Building a query DSL where conditions combine (A AND (B OR C))
• Implementing the visitor pattern on a tree

---

Part 1 — Iteration

The GoF Iterator pattern exists for languages without first-class iteration. Python has had the iterator protocol (__iter__ / __next__), generators, and itertools since 2.2. There is almost never a reason to write a class with __next__ by hand.

1. Generators over hand-rolled iterators

# Good — three lines, all the iterator machinery is implicit
def line_numbers(lines: Iterable[str]) -> Iterator[tuple[int, str]]:
    for i, line in enumerate(lines, start=1):
        yield i, line

# Bad — implementing the protocol when generators would do it
class LineNumberIterator:
    def __init__(self, lines: Iterable[str]) -> None:
        self._lines = iter(lines)
        self._i = 0
    def __iter__(self) -> "LineNumberIterator":
        return self
    def __next__(self) -> tuple[int, str]:
        self._i += 1
        return self._i, next(self._lines)

Generators are the iterator protocol with the right defaults. yield makes a function pause and resume; the generator object handles __iter__, __next__, StopIteration, and send/throw for free.

2. itertools for stream composition

from itertools import chain, groupby, islice, takewhile, pairwise

combined = chain(orders_today, orders_yesterday)     # concatenate lazily
first_100 = islice(big_iter, 100)                    # first 100 only
for status, group in groupby(orders, key=lambda o: o.status):
    print(status, list(group))                       # consecutive runs by key