{
 "id": "customer_retention",
 "kind": "analysis R4",
 "title": "When do customers come back, and when do they stop?",
 "question": "Of the customers whose first order was in a given month, what share bought again in each later month, how soon do the ones who return come back, and after how long without an order is a customer unlikely to return?",
 "source": {
  "name": "UCI Online Retail II",
  "publisher": "Chen, D. (2012), UCI Machine Learning Repository",
  "url": "https://archive.ics.uci.edu/dataset/502/online+retail+ii",
  "license": "CC BY 4.0",
  "as_of": "December 2011",
  "rows_raw": 1067371,
  "rows_used": 1021254,
  "caveat": "A UK online gift retailer selling mostly to wholesale customers, so repeat buying runs higher than at a typical consumer shop. Customers without a Customer ID are left out. The data starts on Dec 1, 2009, so the December 2009 cohort includes customers who first bought earlier."
 },
 "kpis": [
  {
   "label": "Placed a second order",
   "value": "72.4%",
   "note": "of 5,852 identified customers; 27.6% bought once"
  },
  {
   "label": "New customers who buy again the next month",
   "value": "21.0%",
   "note": "cohorts Jan 2010 to Oct 2011, weighted; Dec 2009 cohort 35.0%"
  },
  {
   "label": "New customers back within 90 days",
   "value": "40.6%",
   "note": "first purchase Jan 1 to Dec 9, 2010 (3,334 customers)"
  },
  {
   "label": "Median days to come back",
   "value": "71 days",
   "note": "2010's new customers who came back within a year"
  },
  {
   "label": "Order again within 6 months, after 180 quiet days",
   "value": "40.3%",
   "note": "against 83.9% just after an order"
  }
 ],
 "finding": "The biggest single drop comes after the first order: 27.6% of 5,852 identified customers had not ordered again by Dec 9, 2011, while 77.7% of those with a second order placed a third. Customers new in 2010 or 2011 bought again in the month after their first 21.0% of the time, against 35.0% for the December 2009 cohort, which includes customers already buying before the data starts. Returns come early: half of 2010's new customers who came back within a year did so within 71 days. The longer a customer goes quiet, the less likely an order: 83.9% order within six months of a purchase, 59.2% after 90 days without one, 40.3% after 180.",
 "recommendation": "Treat the second order as the target: a follow-up inside the first 71 days, when half of the returns within a year happen, and a win-back step at about four months without an order, when the odds of another order in the next six months fall to about even (51.5%). Track it by first-purchase month, not as one overall repeat rate.",
 "cohort_heatmap": {
  "months": [
   0,
   1,
   2,
   3,
   4,
   5,
   6,
   7,
   8,
   9,
   10,
   11,
   12,
   13,
   14,
   15,
   16,
   17,
   18,
   19,
   20,
   21,
   22,
   23
  ],
  "rows": [
   [
    "2009-12",
    951,
    100.0,
    35.0,
    33.3,
    42.5,
    37.9,
    36.0,
    37.6,
    34.4,
    33.8,
    36.2,
    42.2,
    49.6,
    37.6,
    28.2,
    24.4,
    30.3,
    26.3,
    30.3,
    28.4,
    25.9,
    25.4,
    31.3,
    30.4,
    40.6
   ],
   [
    "2010-01",
    368,
    100.0,
    21.5,
    32.1,
    31.5,
    27.2,
    31.0,
    26.9,
    23.4,
    28.5,
    32.6,
    31.0,
    17.9,
    22.8,
    17.9,
    19.6,
    15.2,
    24.5,
    20.4,
    19.3,
    20.1,
    25.3,
    19.8,
    25.3,
    null
   ],
   [
    "2010-02",
    375,
    100.0,
    23.5,
    22.7,
    29.3,
    24.5,
    19.7,
    19.2,
    28.8,
    25.6,
    27.7,
    11.5,
    12.5,
    15.2,
    17.6,
    12.3,
    20.0,
    16.0,
    16.3,
    14.4,
    22.9,
    22.9,
    16.5,
    null,
    null
   ],
   [
    "2010-03",
    441,
    100.0,
    19.0,
    23.1,
    24.3,
    23.1,
    20.4,
    24.7,
    30.6,
    27.7,
    10.9,
    11.6,
    14.5,
    20.0,
    16.3,
    20.0,
    17.0,
    17.5,
    15.6,
    17.7,
    20.4,
    21.3,
    null,
    null,
    null
   ],
   [
    "2010-04",
    294,
    100.0,
    19.0,
    19.0,
    16.0,
    18.4,
    22.1,
    27.6,
    26.5,
    10.5,
    10.9,
    7.5,
    13.9,
    14.3,
    15.6,
    15.6,
    15.6,
    13.9,
    15.0,
    18.0,
    22.8,
    null,
    null,
    null,
    null
   ],
   [
    "2010-05",
    255,
    100.0,
    15.7,
    16.9,
    17.6,
    17.6,
    25.5,
    21.2,
    12.5,
    5.9,
    8.2,
    11.4,
    13.3,
    15.3,
    15.3,
    9.8,
    12.5,
    13.7,
    16.5,
    15.3,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2010-06",
    267,
    100.0,
    17.6,
    18.7,
    20.6,
    23.2,
    28.5,
    12.7,
    9.0,
    8.2,
    11.2,
    10.5,
    13.9,
    15.0,
    12.4,
    11.2,
    12.4,
    13.1,
    20.2,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2010-07",
    185,
    100.0,
    15.7,
    18.4,
    29.7,
    29.2,
    14.1,
    11.4,
    14.6,
    14.6,
    11.4,
    13.5,
    14.6,
    13.0,
    13.5,
    19.5,
    17.3,
    23.8,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2010-08",
    163,
    100.0,
    19.6,
    28.8,
    32.5,
    16.6,
    11.7,
    9.8,
    12.9,
    13.5,
    12.9,
    12.9,
    11.7,
    14.7,
    17.2,
    17.2,
    19.6,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2010-09",
    239,
    100.0,
    23.0,
    23.4,
    13.0,
    8.8,
    10.5,
    13.8,
    9.6,
    13.0,
    13.0,
    12.1,
    10.0,
    22.6,
    16.7,
    16.7,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2010-10",
    375,
    100.0,
    25.9,
    14.7,
    12.5,
    8.8,
    8.3,
    13.1,
    13.9,
    10.7,
    9.3,
    10.7,
    12.8,
    19.2,
    23.2,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2010-11",
    326,
    100.0,
    17.5,
    9.5,
    9.5,
    7.7,
    8.9,
    12.9,
    10.1,
    8.6,
    9.2,
    11.0,
    14.7,
    25.2,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2010-12",
    76,
    100.0,
    9.2,
    5.3,
    9.2,
    11.8,
    6.6,
    5.3,
    9.2,
    5.3,
    2.6,
    9.2,
    19.7,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-01",
    72,
    100.0,
    16.7,
    22.2,
    19.4,
    22.2,
    15.3,
    15.3,
    12.5,
    11.1,
    20.8,
    25.0,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-02",
    125,
    100.0,
    16.0,
    16.0,
    18.4,
    21.6,
    16.0,
    16.0,
    15.2,
    15.2,
    18.4,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-03",
    179,
    100.0,
    17.9,
    21.8,
    20.1,
    22.3,
    15.1,
    20.7,
    20.1,
    24.0,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-04",
    106,
    100.0,
    24.5,
    19.8,
    19.8,
    17.9,
    23.6,
    17.9,
    25.5,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-05",
    111,
    100.0,
    23.4,
    24.3,
    16.2,
    21.6,
    20.7,
    26.1,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-06",
    108,
    100.0,
    23.1,
    21.3,
    26.9,
    20.4,
    27.8,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-07",
    101,
    100.0,
    20.8,
    29.7,
    27.7,
    34.7,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-08",
    107,
    100.0,
    26.2,
    29.9,
    26.2,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-09",
    188,
    100.0,
    27.1,
    37.8,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ],
   [
    "2011-10",
    221,
    100.0,
    31.7,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null,
    null
   ]
  ],
  "last_observed_month": "2011-11"
 },
 "curve": [
  [
   1,
   35.0,
   21.0,
   22,
   4682
  ],
  [
   2,
   33.3,
   21.5,
   21,
   4461
  ],
  [
   3,
   42.5,
   21.1,
   20,
   4273
  ],
  [
   4,
   37.9,
   19.4,
   19,
   4166
  ],
  [
   5,
   36.0,
   18.6,
   18,
   4065
  ],
  [
   6,
   37.6,
   18.4,
   17,
   3957
  ],
  [
   7,
   34.4,
   18.6,
   16,
   3846
  ],
  [
   8,
   33.8,
   16.4,
   15,
   3740
  ],
  [
   9,
   36.2,
   15.0,
   14,
   3561
  ],
  [
   10,
   42.2,
   13.5,
   13,
   3436
  ],
  [
   11,
   49.6,
   14.0,
   12,
   3364
  ],
  [
   12,
   37.6,
   18.4,
   11,
   3288
  ],
  [
   13,
   28.2,
   16.9,
   10,
   2962
  ],
  [
   14,
   24.4,
   15.9,
   9,
   2587
  ],
  [
   15,
   30.3,
   16.2,
   8,
   2348
  ],
  [
   16,
   26.3,
   17.5,
   7,
   2185
  ],
  [
   17,
   30.3,
   17.3,
   6,
   2000
  ],
  [
   18,
   28.4,
   17.0,
   5,
   1733
  ],
  [
   19,
   25.9,
   21.4,
   4,
   1478
  ],
  [
   20,
   25.4,
   23.1,
   3,
   1184
  ],
  [
   21,
   31.3,
   18.2,
   2,
   743
  ],
  [
   22,
   30.4,
   25.3,
   1,
   368
  ],
  [
   23,
   40.6,
   null,
   0,
   0
  ]
 ],
 "ladder": [
  [
   "1st to 2nd",
   72.4,
   5852,
   4234
  ],
  [
   "2nd to 3rd",
   77.7,
   4234,
   3289
  ],
  [
   "3rd to 4th",
   80.0,
   3289,
   2630
  ],
  [
   "4th to 5th",
   81.4,
   2630,
   2141
  ],
  [
   "5th to 6th",
   83.4,
   2141,
   1786
  ],
  [
   "6th to 7th",
   84.4,
   1786,
   1508
  ],
  [
   "7th to 8th",
   85.5,
   1508,
   1290
  ],
  [
   "8th to 9th",
   86.5,
   1290,
   1116
  ],
  [
   "9th to 10th",
   86.1,
   1116,
   961
  ],
  [
   "10th to 11th",
   89.9,
   961,
   864
  ]
 ],
 "return_days": [
  [
   "7 days",
   12.9,
   3.2
  ],
  [
   "14 days",
   23.6,
   7.0
  ],
  [
   "30 days",
   29.1,
   16.0
  ],
  [
   "60 days",
   49.6,
   30.8
  ],
  [
   "90 days",
   59.3,
   40.6
  ],
  [
   "120 days",
   68.5,
   47.3
  ],
  [
   "180 days",
   77.5,
   57.2
  ],
  [
   "270 days",
   83.5,
   64.7
  ],
  [
   "365 days",
   89.4,
   69.7
  ]
 ],
 "lapse": [
  [
   "Just ordered",
   83.9,
   10621,
   8914
  ],
  [
   "30 days without an order",
   76.6,
   6426,
   4922
  ],
  [
   "60 days without an order",
   67.8,
   4217,
   2858
  ],
  [
   "90 days without an order",
   59.2,
   3102,
   1837
  ],
  [
   "120 days without an order",
   51.5,
   2431,
   1251
  ],
  [
   "180 days without an order",
   40.3,
   1707,
   688
  ],
  [
   "270 days without an order",
   28.3,
   1265,
   358
  ]
 ],
 "numbers": {
  "customers": 5852,
  "dec2009_size": 951,
  "new_cohort_customers": 4682,
  "new_cohorts": 22,
  "ladder_counts": {
   "1": 5852,
   "2": 4234,
   "3": 3289,
   "4": 2630,
   "5": 2141,
   "6": 1786,
   "7": 1508,
   "8": 1290,
   "9": 1116,
   "10": 961,
   "11": 864
  },
  "return_groups": {
   "d09": {
    "customers": 951,
    "back_within": {
     "7": 123,
     "14": 224,
     "30": 277,
     "60": 472,
     "90": 564,
     "120": 651,
     "180": 737,
     "270": 794,
     "365": 850
    },
    "never_back": 85,
    "median_days_if_back_within_365": 53.0
   },
   "new": {
    "customers": 3334,
    "back_within": {
     "7": 107,
     "14": 235,
     "30": 535,
     "60": 1026,
     "90": 1354,
     "120": 1576,
     "180": 1908,
     "270": 2156,
     "365": 2325
    },
    "never_back": 836,
    "median_days_if_back_within_365": 71.0
   }
  },
  "lapse_purchase_days": 10621,
  "lapse_window_days": 180,
  "lapse_last_purchase_day": "2010-08-31",
  "live_simple_avg_m3": 23.3,
  "live_simple_avg_m12": 19.6
 },
 "sample_queries": [
  {
   "label": "Cohort retention (DuckDB, the full cleaned history)",
   "sql": "-- Cohort retention: share of each first-purchase month still buying k months later\nWITH active AS (            -- one row per customer and month with a purchase\n    SELECT DISTINCT customer_id, date_trunc('month', invoice_date) AS month\n    FROM retail_lines\n    WHERE customer_id IS NOT NULL AND NOT is_cancelled\n      AND invoice_date < DATE '2011-12-01'        -- December 2011 is partial (to Dec 9)\n),\ncohort AS (\n    SELECT customer_id, min(month) AS cohort_month FROM active GROUP BY 1\n)\nSELECT c.cohort_month,\n       date_diff('month', c.cohort_month, a.month) AS months_since,\n       count(*) AS customers,\n       round(100.0 * count(*) / max(s.size), 1) AS retention_pct\nFROM active a\nJOIN cohort c USING (customer_id)\nJOIN (SELECT cohort_month, count(*) AS size FROM cohort GROUP BY 1) s USING (cohort_month)\nGROUP BY 1, 2\nORDER BY 1, 2"
  },
  {
   "label": "Time to come back (DuckDB)",
   "sql": "-- Time to come back: days from a customer's first purchase day to the next one\nWITH days AS (              -- several invoices on one day count as one purchase day\n    SELECT DISTINCT customer_id, invoice_date::DATE AS d\n    FROM retail_lines\n    WHERE customer_id IS NOT NULL AND NOT is_cancelled\n),\nnexts AS (\n    SELECT f.customer_id, f.first_day, min(d.d) AS next_day\n    FROM (SELECT customer_id, min(d) AS first_day FROM days GROUP BY 1) f\n    LEFT JOIN days d ON d.customer_id = f.customer_id AND d.d > f.first_day\n    GROUP BY 1, 2\n)\nSELECT CASE WHEN first_day < DATE '2010-01-01' THEN 'Dec 2009 cohort'\n            ELSE 'First purchase Jan 1 to Dec 9, 2010' END AS grp,\n       count(*) AS customers,\n       round(100.0 * count(*) FILTER (WHERE next_day - first_day <= 90) / count(*), 1) AS back_within_90_days_pct,\n       median(next_day - first_day) FILTER (WHERE next_day - first_day <= 365) AS median_days_if_back_within_a_year\nFROM nexts\nWHERE first_day <= DATE '2010-12-09'          -- every customer here has at least 365 days to come back\nGROUP BY 1\nORDER BY 1"
  }
 ],
 "cleaning": [
  "Dropped non-product stock codes (POST, DOT, M, m, D, BANK CHARGES, AMAZONFEE, C2, PADS, CRUK, ADJUST, ADJUST2, S, B, TEST001/002, gift card vouchers) that represent postage, fees, manual adjustments, and test rows rather than real merchandise.",
  "Dropped rows with price <= 0 (6,207 rows); nearly all of these have no Customer ID and are internal stock corrections, not customer sales.",
  "Dropped negative-quantity rows on invoices that do not start with 'C' (3,457 rows); a true return in this dataset is a cancellation invoice (prefix 'C'), and these unlabeled negative-quantity rows all have null Customer ID and mostly null description, so they read as inventory write-offs rather than customer returns.",
  "Removed 34,035 exact duplicate line items (after the rules above).",
  "Kept rows with missing Customer ID (22.2% of clean rows) for revenue and product analysis since they are real sales; excluded them from the cohort retention analysis, which requires a known customer.",
  "Customers: rows with a Customer ID, on invoices that are not cancellations. A month or a day counts once per customer however many invoices fall in it.",
  "December 2011 (Dec 1 to 9) is left out as an observation month because it is partial; November 2011 is the last month observed."
 ],
 "checks": [
  "The cleaning reproduces the live Online retail page: 1,067,371 source rows, 1,021,254 after cleaning, 5,852 identified customers, the 27.6% one-time share and all 156 cells of the published cohort chart.",
  "Every number on this page was recomputed by separate code (verify_r4.py: plain Python reading the original UCI zip, with no DuckDB and no intermediate files) and matches.",
  "The cohort and time-to-come-back queries below were run in DuckDB 1.5.5 and return the figures on this page."
 ]
}