Generators

Generators are a special type of function in Python that enable the creation of iterable sequences of values, generated one at a time as needed.

Unlike lists or tuples, which store all elements in memory, generators produce values on-the-fly, making them particularly useful for working with large datasets or sequences where memory efficiency is crucial.

How Generators Work

Generators use the yield statement instead of return. When a generator function is called, it does not execute its body immediately. Instead, it returns a generator object. Each call to the generator retrieves the next value and resumes execution from where it last yielded.

Creating a Generator

A generator function is defined just like a regular function, but it uses yield to return values one by one.

Example:

def my_generator():
    for i in range(5):
        yield i  # Yield one value at a time
 
gen = my_generator()  # Create the generator object
print(next(gen))  # Output: 0
print(next(gen))  # Output: 1
print(next(gen))  # Output: 2

Here,

• The my_generator function generates numbers from 0 to 4.
• Each call to next(gen) resumes the generator’s execution, returning the next value and pausing at yield.

If you call next(gen) after all values are exhausted, it raises a StopIteration exception.

Using Generators in Loops

Generators integrate seamlessly with for loops, which handle StopIteration automatically.

Example:

def my_generator():
    for i in range(5):
        yield i
 
for value in my_generator():
    print(value)
 
# Output:
# 0
# 1
# 2
# 3
# 4

Key Point: The generator does not store all values in memory, it generates each value only when needed.

Generator Expressions

You can create a generator object using a compact generator expression, similar to a list comprehension but with parentheses instead of square brackets.

Example:

gen = (x**2 for x in range(5))
 
for value in gen:
    print(value)
 
# Output:
# 0
# 1
# 4
# 9
# 16

Key Difference from List Comprehension:

• List comprehension [x**2 for x in range(5)] creates and stores the entire list in memory.
• Generator expression (x**2 for x in range(5)) creates values on demand, saving memory.

When to Use Generators

• When working with large datasets (e.g., processing log files or CSVs line by line).
• When streaming data from an API or database.
• When creating infinite sequences (e.g., Fibonacci numbers, counters).
• When memory efficiency matters more than random access to elements.

👉 Next tutorial: Python Walrus Operator