A Swiggy-scale delivery platform logs everything about every order: the restaurant's coordinates, the customer's coordinates, the exact second the order was placed, the cuisine, the order value, how many times this customer has ordered before, and whether the delivery arrived late. Someone on the data science team is asked to build a model that predicts, the moment an order is placed, whether it is likely to be late, so the app can warn the customer honestly instead of showing an optimistic ETA that quietly slips.
The columns are all sitting right there in the database. So the obvious first move is to hand them straight to a model: restaurant_lat, restaurant_lon, customer_lat, customer_lon, order_placed_ts, cuisine, order_value_rupees, prior_orders_count. A logistic regression trained on exactly these eight raw columns performs barely better than guessing the majority class. Nothing is wrong with the data. What is wrong is the assumption that a model can find a pattern in whatever shape the data happens to arrive in.
Why raw columns defeat the model, not the pattern
A logistic regression (like linear regression, and like the "linear part" hiding inside many other models) can only combine its inputs as a weighted sum: w1·x1 + w2·x2 + ... + b. Whether an order is late almost certainly depends on how far the rider has to travel. But "how far apart two GPS points are" is not a weighted sum of their four raw coordinates. It is a trigonometric function of differences between coordinates. No setting of weights w1 through w4 on restaurant_lat, restaurant_lon, customer_lat, customer_lon can reconstruct that relationship, because the true relationship is nonlinear in exactly the columns the model was given. The pattern is real. The representation hides it.
The same problem shows up in different disguises for every raw column here. order_placed_ts is a timestamp string: a model cannot learn "lunch rush is risky" from a string that looks like 2026-08-27T13:04:00, because nothing about that string's numeric encoding is close to the string for 13:06:00 the way "these two are both lunchtime" should be close. cuisine is text, and most models require numbers. order_value_rupees ranges from about ₹150 to ₹1000 while prior_orders_count ranges from 0 to several hundred. On wildly different scales, several algorithms (anything using distance or gradient magnitude) will let the large-scale column dominate for reasons that have nothing to do with which one actually predicts lateness.
Feature engineering is the deliberate construction of new input columns, called features, from raw data, so that a pattern which already exists in the world becomes something the chosen model's mathematics can actually represent. It does not invent signal that was not there. It translates signal from a shape the model is blind to into a shape the model can use.
Four transformations that make data ML-ready
Extraction. Pull a new, more useful number out of a raw field instead of feeding the raw field itself. From a timestamp, extract hour_placed and a derived flag like is_rush_hour. From two GPS coordinate pairs, extract the actual distance between them using the haversine formula, which accounts for the Earth's curvature:
a = sin²(Δφ/2) + cos(φ1)·cos(φ2)·sin²(Δλ/2)
c = 2 · atan2(√a, √(1−a))
distance_km = R · c (R = 6371 km, Earth's radius)where φ is latitude and λ is longitude, both converted to radians, and Δφ, Δλ are the differences between the two points. Trace it for a Koramangala restaurant (12.9352° N, 77.6245° E) delivering to a customer in Indiranagar (12.9784° N, 77.6408° E): φ1 = 0.22576 rad, φ2 = 0.22652 rad, so Δφ = 0.00075 rad and Δλ = 0.00028 rad. Then a = sin²(0.000377) + cos(0.22576)·cos(0.22652)·sin²(0.000142) ≈ 1.613×10⁻⁷, c = 2·atan2(√a, √(1−a)) ≈ 8.033×10⁻⁴ rad, and distance_km = 6371 × 8.033×10⁻⁴ ≈ 5.12 km. That single derived number carries more predictive weight for "will this be late" than all four raw coordinates combined, because it is the actual physical quantity the outcome depends on.
Encoding. Categorical fields need to become numbers, and there are real tradeoffs in how. One-hot encoding turns a category with k possible values into k binary columns: safe and interpretable, but a restaurant_id field with 40,000 distinct restaurants would explode into 40,000 columns, most of them almost always zero. Label encoding maps each category to an arbitrary integer (0, 1, 2…): compact, but it invents a false ordering: a model may learn that "cuisine 2 is between cuisine 1 and cuisine 3" when cuisines have no such order. Target (mean) encoding replaces a category with the average outcome observed for that category in the training data: compact and often predictive, but it directly bakes label information into a feature, which is exactly the mechanism the misconception below abuses.
Scaling. Once every feature is numeric, features on different scales still distort many algorithms. Standardization (z-scoring) recenters a feature to mean 0 and standard deviation 1: z = (x − mean) / std. Min-max scaling instead squeezes a feature into a fixed range, usually 0 to 1: x_scaled = (x − min) / (max − min). Neither changes the relative ordering of the values. Both just change the ruler.
Taming skew. Some numeric features are heavily right-skewed: five customers with 1–8 prior orders and one power user with 150. A log transform (usually log(1 + x), written log1p, so that zero stays defined) compresses the extreme end without discarding it, so the model does not treat every prediction as if it hinges on outrunning a single outlier.
Worked example: engineering six delivery orders end to end
Here are six raw orders exactly as they would sit in the database, alongside the label the model is being trained to predict (whether the delivery was late):
order_id distance_km* hour_placed cuisine order_value_rupees late
1 5.12 13 North Indian 450 1
2 2.30 20 Chinese 320 0
3 8.70 11 South Indian 180 0
4 1.50 21 North Indian 600 1
5 6.40 14 Chinese 250 0
6 3.90 13 South Indian 390 1
*already derived from raw lat/lon via haversine, as shown aboveThe following code builds the engineered features (a rush-hour flag from the hour, a target-encoded cuisine rate, a standardized distance, and a min-max-scaled order value) and prints the resulting table:
import pandas as pd
orders = pd.DataFrame({
"order_id": [1, 2, 3, 4, 5, 6],
"distance_km": [5.12, 2.30, 8.70, 1.50, 6.40, 3.90],
"hour_placed": [13, 20, 11, 21, 14, 13],
"cuisine": ["North Indian", "Chinese", "South Indian",
"North Indian", "Chinese", "South Indian"],
"order_value_rupees": [450, 320, 180, 600, 250, 390],
"late": [1, 0, 0, 1, 0, 1],
})
# EXTRACT: rush-hour flag (12-14 lunch, 19-21 dinner)
orders["is_rush_hour"] = orders["hour_placed"].apply(
lambda h: 1 if (12 <= h <= 14 or 19 <= h <= 21) else 0
)
# SCALE: standardize distance
mean_d = orders["distance_km"].mean()
std_d = orders["distance_km"].std(ddof=0)
orders["distance_z"] = ((orders["distance_km"] - mean_d) / std_d).round(3)
# ENCODE: target-encode cuisine using the training rows themselves
cuisine_means = orders.groupby("cuisine")["late"].mean()
orders["cuisine_late_rate"] = orders["cuisine"].map(cuisine_means)
print(orders[["order_id", "distance_km", "distance_z", "hour_placed",
"is_rush_hour", "cuisine", "cuisine_late_rate",
"order_value_rupees"]].to_string(index=False))Running this prints exactly:
order_id distance_km distance_z hour_placed is_rush_hour cuisine cuisine_late_rate order_value_rupees
1 5.12 0.191 13 1 North Indian 1.0 450
2 2.30 -0.965 20 1 Chinese 0.0 320
3 8.70 1.659 11 0 South Indian 0.5 180
4 1.50 -1.293 21 1 North Indian 1.0 600
5 6.40 0.716 14 1 Chinese 0.0 250
6 3.90 -0.309 13 1 South Indian 0.5 390Walk through what each engineered column did. distance_z centers the six distances (mean 4.653 km, standard deviation 2.439 km) around zero, so order 3's 8.70 km (the farthest) becomes +1.659, nearly a std deviation and a half above average, while order 4's 1.50 km becomes −1.293, well below. is_rush_hour turns a timestamp into the single binary fact that actually matters for lateness risk; order 3 at 11:00 is the only one placed outside lunch or dinner rush, and it is also the only far-distance order that was not late; that is a hint that distance alone does not decide the outcome, timing interacts with it. cuisine_late_rate replaces three text categories with three numbers computed straight from this data: North Indian orders were late 100% of the time in this sample, Chinese 0%, South Indian 50%. A tree-based model can now split directly on that number; a text column would have forced it to search over three separate branches instead of one ordered threshold.
How the pipeline fits together
The bottom half of the diagram plots the same six orders twice, on two number lines drawn to their own scales. On the raw axis, order 3 (8.70 km) sits far to the right and order 4 (1.50 km) sits far to the left, a spread of 7.2 km. On the standardized axis, the same two orders sit at 1.659 and −1.293, a spread of about 3 standard-deviation units. Nothing about which order is "farthest" changed. What changed is the ruler, and that ruler swap is precisely why a model that compares feature magnitudes (nearest-neighbour distance, gradient step size, regularization penalty) needs scaling to treat distance_km and order_value_rupees fairly, instead of letting whichever column happens to have bigger raw numbers dominate.
The misconception: "fit the scaler on all the data, then split"
The single most common mistake in feature engineering (made by students and by production pipelines alike) is computing scaling statistics, or target-encoding means, using the entire dataset before splitting into train and test. It feels harmless: "the mean and standard deviation are just descriptive numbers about the data, why would order matter?" It matters because those numbers are supposed to describe only what the model is allowed to have seen (the training set), and once a single test-set value contaminates the mean or standard deviation, every training row's engineered feature silently absorbs a trace of information the model was never supposed to have during training. That is data leakage, and it inflates validation accuracy in a way that will not survive contact with genuinely new orders.
Here is the effect in exact numbers. Suppose a seventh order (a 20 km corporate catering delivery) belongs only to the held-out test set. Computed correctly, the standardization statistics come only from the six training orders: mean 4.653 km, std 2.439 km, giving order 1 (5.12 km) a z-score of +0.191. Compute the statistics the leaky way, over all seven orders including the test-only 20 km outlier, and the mean jumps to 6.846 km with std 5.826 km, giving that same order 1 a z-score of −0.296. The sign flips. A feature that correctly says "slightly farther than average" now says "slightly closer than average," purely because a value the model will never see during training was allowed to shift the ruler. Any weight the model learns against that feature is now calibrated to a ruler that will not exist at prediction time.
The fix is a strict ordering: split the data into train and test first; compute every mean, standard deviation, min, max, and target-encoding rate using only the training rows; then apply those frozen numbers to transform both the training rows and the test rows. The test set is never allowed to influence the numbers used to transform it.
Active recall
Attempt each question before reading its answer.
1. A new restaurant-customer pair sits at (12.9352° N, 77.6245° E) and (12.9698° N, 77.7500° E), Koramangala to Whitefield. Using the haversine steps shown earlier, is the resulting distance closer to 5 km, 14 km, or 25 km?
2. Order 3's distance was mis-recorded. The corrected value is 4.20 km, not 8.70 km. Recompute the standardized distance_z for all six orders, not just order 3.
3. The training data has three cuisines, so one-hot encoding cuisine produces 3 binary columns. At deployment, an order arrives with cuisine = "Continental", a category the encoder never saw during training. What happens, and what does that reveal about a risk one-hot encoding shares with target encoding?
4. Target encoding a cuisine that has exactly one order in the training set produces a rate of either 0.0 or 1.0 for that category. Explain why this is a more severe problem than ordinary data leakage.
5. prior_orders_count for six customers is [1, 2, 3, 5, 8, 150]. After a log1p transform it becomes [0.693, 1.099, 1.386, 1.792, 2.197, 5.017]. Compare the ratio of the largest to smallest value before and after the transform, and explain what that ratio change means for a model.
6. A new order has order_value_rupees = 700, above the training range of ₹180–₹600. Using the training statistics (min=180, max=600, mean=365, std=136.96), compute this order's min-max scaled value and its standardized z-score. Which one signals a problem more clearly?
Answers
1. φ1=12.9352°, φ2=12.9698°, λ1=77.6245°, λ2=77.7500°, so Δφ≈0.0346° and Δλ≈0.1255°. Δλ≈0.1255° is about 7.7× larger than the Koramangala–Indiranagar gap's Δλ (0.0163°), while Δφ≈0.0346° is actually slightly smaller than that gap's Δφ (0.0432°). The much greater distance comes almost entirely from the larger Δλ term, since the haversine formula weights it by cos(φ1)·cos(φ2) (≈0.95 at this latitude) — not from both angular gaps growing by the same factor. Carrying the same steps through gives a≈1.230×10⁻⁶, c≈2.218×10⁻³ rad, and distance = 6371 × 2.218×10⁻³ ≈ 14.13 km. It is closer to 14 km: Koramangala to Whitefield is a genuinely long cross-city trip.
2. The new distances are [5.12, 2.30, 4.20, 1.50, 6.40, 3.90]. The mean is no longer 4.653: it drops to 3.903 km (the sum fell by 4.5 km across 6 rows), and the standard deviation shrinks from 2.439 to 1.641 km, because the single most extreme value in the set just moved toward the middle. Every order's z-score changes, not only order 3's, because mean and standard deviation are shared statistics computed across all six rows: order 1 → 0.742 (up from 0.191), order 2 → −0.977 (down from −0.965), order 3 → 0.181 (down from 1.659), order 4 → −1.465 (down from −1.293), order 5 → 1.522 (up from 0.716), order 6 → −0.002 (up from −0.309). This is the general lesson: fixing one row's raw value in a standardized feature requires recomputing the feature for every other row too, because they all share the same ruler.
3. A "Continental" order becomes a row of three zeros ([0,0,0]), indistinguishable to the model from "no cuisine information at all," which is not the same claim as "this is a real fourth category the model has opinions about." Some encoders instead raise an error on an unseen category. Either way, the shared risk with target encoding is that both techniques are frozen at training time: one-hot encoding fixes the set of known categories, and target encoding fixes the rate assigned to each. Neither can represent a category it never met in training, so both require an explicit fallback plan (an "other" bucket, a global average rate) for categories that appear only later.
4. Ordinary leakage (like the scaling example above) lets test-set values distort a shared summary statistic: a diffuse, indirect contamination. A single-order category's target-encoded rate is not diffuse: it is a direct, deterministic copy of that one row's own label. The "feature" and the "answer" become the same number for that row, so the model does not learn a generalizable relationship between cuisine and lateness for that category: it memorizes one training example's outcome and will confidently misapply it to every future order from that same rarely-seen cuisine, whatever the real cause of lateness was that day.
5. Before the transform, the ratio 150/1 = 150: the busiest customer's raw count is 150 times any of several ordinary customers'. After log1p, the ratio is 5.017/0.693 ≈ 7.24. A model sensitive to feature magnitude (nearest-neighbour distance, a gradient step, an L2 penalty) would otherwise let that single power user's row dominate any distance-based or magnitude-based computation nearly two orders of magnitude more than an ordinary customer's row does. The log transform keeps the ordering (more prior orders is still "more") while cutting the outlier's leverage roughly twenty-fold.
6. Min-max: (700−180)/(600−180) = 520/420 ≈ 1.238, outside the [0,1] range every training row was scaled into, because min-max scaling has no defined behavior beyond the training extremes; it silently extrapolates. Standardized: (700−365)/136.96 ≈ 2.446, still a perfectly ordinary z-score, readable as "about 2.4 standard deviations above the mean order value," a number the model has almost certainly encountered training rows near even if not exactly at 700. The z-score degrades gracefully for out-of-range inputs; the min-max score does not, which is one practical reason standardization is often preferred for features expected to see values beyond the training set's observed range.
Think About It
Think about this: How would you explain feature engineering: the art of making data ml-ready to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Practice Exercises
Now it is time to practice! Complete these challenges to solidify your understanding:
- Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
- Exercise 2: Find a real-world example where feature engineering: the art of making data ml-ready is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
- Exercise 3: Create a mind-map connecting feature engineering: the art of making data ml-ready to at least 3 other topics you have studied.