How to use Python's itertools for time-series feature engineering

Python's itertools is a built-in module for working with iterators, and it becomes an indispensable tool for time-series feature engineering. If you build features with loops and copy datasets along the way, itertools can do the same job several times faster and with far less memory.

What is itertools and why do you need it

The itertools module provides functions for building fast, memory-efficient iterators. That matters for time series: instead of holding lists in memory, we generate features on the fly. On datasets with millions or billions of points, this goes beyond convenience: it's what makes the work possible at all.

The key itertools functions for feature engineering: combinations() — all pairs and combinations of elements (lag features, variable interactions) permutations() — all permutations (testing feature orderings and their impact) islice() — extracting subsequences and sliding windows without copying zip_longest() — merging series of different lengths while handling gaps * chain() — joining iterators into a single stream without copying data ## Practical examples: from a naive approach to a scalable one A classic task: you have a time series of prices and need to build lag features with lags of 1, 2, and 3. The naive (slow) way: python import pandas as pd data = pd.read_csv('prices.

csv') for lag in range(1, 4): data[f'price_lag_{lag}'] = data['price'].shift(lag) The problem: every shift() call creates a copy of the dataset in memory. On 10 million rows, that's ~300 million copy operations plus 3x the memory usage.

With itertools (fast and memory-efficient): python from itertools import islice prices = [100, 102, 105, 103, 107, 110] for lag in range(1, 4): lagged = list(islice(prices, lag, None)) print(f'lag {lag}: {lagged}') islice doesn't copy the list — it simply moves a pointer to the right position. On large datasets, that saves hours of compute. The second example is sliding windows: python from itertools import islice, tee def sliding_window(iterable, n): iterables = tee(iterable, n) for i, it in enumerate(iterables): for _ in range(i): next(it, None) return zip(*iterables) prices = [100, 102, 105, 103, 107, 110] windows = list(sliding_window(prices, 3)) This builds sliding windows of size 3 using lazy iterators instead of copying the array.

Performance: where you save memory and time

On datasets with billions of points: Lazy evaluation — elements are generated only on demand rather than loaded all at once Chained operations — chain() combines sources without intermediate copies * Composition — by combining itertools functions, you can build a complex pipeline with minimal memory The result: memory usage can drop by 10-100x. In the cloud, that means a smaller bill. On a local machine, workloads that previously didn't fit in RAM now run.

What this means for you For data scientists: work with datasets that previously just didn't fit in memory.

For ML engineers: production models need fewer resources and run more reliably. itertools isn't magic — it's a built-in way to work with data more intelligently. Start with lag features and sliding windows, and you'll find other places to apply lazy evaluation later.