Python Iterators and Generators Exercises

To master iteration, you must understand these two distinct roles:

1. The Iterable (The Container)

An Iterable is anything you can loop over (a list, a string, a dictionary). It has an __iter__ method that, when called, returns a brand new "cursor" or "bookmarker."

2. The Iterator (The Bookmarker)

The Iterator is that bookmarker. It keeps track of where you are in the sequence. It has a __next__ method that gives you the next item every time you call it.

In Python, the Iteration Protocol is defined by two specific methods:

• __iter__: This method should return the iterator object itself. it is required so that an iterator can also be used in for loops.
• __next__: This method must return the next value in the sequence. If there are no more items, it must raise a StopIteration exception.

# A simple Iterator structure:
class MyIterator:
    def __iter__(self):
        return self
    def __next__(self):
        # logic to return next value...
        pass

Unlike some other languages that return a "null" value, Python uses Exceptions to signal the end of a sequence.

How the Loop Sees It:

When you use a for loop, Python is actually calling next() behind the scenes. When StopIteration is raised, the for loop catches it and stops the loop gracefully.

it = iter([1, 2])
print(next(it)) # 1
print(next(it)) # 2
print(next(it)) # Raises StopIteration

Key Takeaway: StopIteration is not an "error" in the bad sense; it is a signal that the iterator is exhausted.

This is the "Superpower" of Iterators and Generators: Memory Efficiency.

The Memory Difference:

• List: To store 10 million integers, Python must allocate enough RAM to hold every single number simultaneously. This could take hundreds of Megabytes.
• Generator: It only stores the formula to create the next number and the current state. It uses the same tiny amount of memory whether it's generating 10 numbers or 10 billion numbers.

This is known as Lazy Evaluation: don't do the work until you absolutely need the result!

To solve this, you must track the internal state (the cursor) of the iterator.

Step-by-Step Execution:

1. it = iter(nums): The iterator starts before the first element.
2. next(it): The cursor moves to the first element (10) and returns it, but we don't print it.
3. next(it): The cursor moves to the second element (20).
4. print(...): The value 20 is printed.

Note: Once an iterator moves forward, it cannot move backward. It is a one-way street!

This is the "magic" that separates a Generator from a Regular Function.

Suspension vs. Termination

• Return: When a function hits return, it is finished. Its local scope is destroyed.
• Yield: When a generator hits yield, it "pauses." Python takes a snapshot of the current local variables and where the code stopped. When you call next() again, it "wakes up" and continues from that exact spot.

To answer this correctly, you must follow the execution flow of a generator step-by-step.

Execution Trace:

1. counter = count_up(): This does not run any code inside the function. It only creates the generator object.
2. next(counter): Execution starts. It prints "Start".
3. It hits yield 1. The value 1 is sent back to the caller, and the function pauses.
4. print(val): The caller prints the value received (1).

The code yield 2 and print("End") are never reached because next() was only called once.

Generators are the only way to handle Infinite Sequences in Python without crashing your computer.

Why Option 2 works:

• The while True loop would normally freeze a program.
• However, because of yield, the loop pauses every time it produces a number.
• It only calculates the next power of two when you specifically ask for it with next().

Option 1 would only return 2 and stop. Option 4 would eventually cause a RecursionError and doesn't actually yield values correctly.

This is a major point of confusion for new developers.

The "Wrapper" Effect

When Python sees the yield keyword inside a function definition, it marks that function as a "generator factory."

• Calling my_func() does not execute the code inside.
• Instead, it creates and returns a Generator Object.
• This object "wraps" your function logic and provides the __next__ and __iter__ methods required to run the code later.

def test():
    yield 5

res = test()
print(type(res)) # <class 'generator'>

This exercise highlights the difference between Eager Evaluation and Lazy Evaluation.

Memory Footprint Analysis:

• Line A (List Comprehension): Square brackets [] tell Python to execute the loop 100 million times right now and store every result in a list. This will likely cause a MemoryError on most personal computers.
• Line B (Generator Expression): Parentheses () create a Generator Object. It doesn't calculate any squares yet. It only stores the "rule" for squaring numbers. It uses the same tiny amount of RAM regardless of the range size.

A critical rule of Iterators and Generators is that they are One-Time Use.

The Concept of Exhaustion:

Think of an iterator as a "stream" of water. Once the water has flowed past a certain point, it is gone. You cannot "reset" a generator to the beginning.

• During the First Pass, the generator runs until it raises StopIteration.
• During the Second Pass, the generator is already "exhausted." When the for loop asks for the next value, the generator immediately says "I'm done."

Solution: If you need to iterate over the data twice, you must either convert it to a list (data = list(get_data())) or create a fresh generator instance for the second loop.

The yield from expression is a "cleaner" way to handle nested iterators.

Before yield from:

You would have to write a loop inside your generator:

for item in sub_gen():
    yield item

With yield from:

Python handles the link between the main_gen and the sub_gen automatically. It is more efficient and allows for advanced features like passing data directly into the sub-generator (used in coroutines).

Output: [0, 1, 2, 3]. Notice how it feels like one single sequence!

In Python, every Iterator should also be an Iterable.

The Logic:

• A for loop first calls iter() on whatever object you give it.
• By having __iter__ return self, the iterator tells the loop: "I am already my own bookmark/cursor. You can start calling __next__ on me immediately."

Without this method, your class would strictly be an iterator, but it would fail when used in a standard for loop, which is counter-intuitive for most Python developers.

Generators are designed to be "robust," even when they aren't fully exhausted.

Resource Safety:

If you break a loop or an exception occurs, the generator object eventually gets destroyed by the Garbage Collector. When this happens, Python is smart enough to "throw" a GeneratorExit exception into the generator.

• This allows the generator to run any finally blocks or with statements it was currently inside.
• This is why generators are excellent for managing file handles or network connections.

When a generator function is called, it returns a generator object but no code is executed yet.

The Concept of "Priming":

The .send() method sends a value to the result of the current yield expression. If the generator hasn't reached a yield yet, there is nowhere for the value to go!

• The first next(gen) starts the function and runs it until it hits the first yield.
• Once it is "suspended" at the yield, it is ready to receive data via .send().

Note: Calling gen.send(None) is equivalent to next(gen) and is often used to prime coroutines.

This is a subtle but vital feature used heavily in asyncio and complex generator pipelines.

Return vs Yield:

• yield: Produces a value for the iteration sequence.
• return: Signals the final result of the generator's computation.

When return "Done!" is executed, Python raises StopIteration as usual to stop any loops, but it attaches the string "Done!" to the exception instance. This allows the caller to retrieve a final status message or calculation result after the stream of data has finished.

This is how Python ensures Resource Integrity.

Generator Life-Cycle:

If a generator is closed (either explicitly with .close() or via Garbage Collection), Python injects a GeneratorExit exception at the point where the generator was paused.

def my_gen():
    try:
        yield 1
    finally:
        print("Cleanup!") # This runs even if we 'del' the generator

This mechanism is why it is safe to use with open(...) statements inside generators; the file will be closed properly even if you stop iterating early!

This exercise tests your ability to track state across multiple suspensions.

The Sequence:

1. next(acc): Runs until yield total. Returns 0. Suspends.
2. acc.send(10): 10 becomes the result of yield total. total becomes 10. Loops back to yield total. Returns 10. Suspends.
3. acc.send(5): 5 becomes the result of yield total. total becomes 15 (10 + 5). Loops back to yield total. Returns 15.

This is the "elegant" way Python handles sub-routine results in 3.3+.

The yield from Expression:

The yield from syntax is not just a loop; it's an expression that evaluates to the return value of the sub-generator.

def sub():
    yield 1
    return "Finished"

def main():
    res = yield from sub()
    print(f"Sub said: {res}")

# list(main()) will print "Sub said: Finished"

This allows generators to behave like lightweight threads or coroutines that can produce a stream of data and then "hand back" a final status report to their parent.