Before computing anything, you need to know what type of data you are handling. It determines which calculations are allowed.
Individuals, variables, categories
In a data table, each row is an individual (a customer, an order, a day) and each column is a variable. The possible values of a variable are called categories (or levels).
Four families of variables
| Family | Subtype | Examples |
|---|---|---|
| Qualitative | nominal (no order) | acquisition channel, country, color |
| Qualitative | ordinal (natural order) | satisfaction “low / medium / high”, size S/M/L |
| Quantitative | discrete (countable values) | number of items, number of children |
| Quantitative | continuous (any value in an interval) | amount, duration, temperature |
Classic trap: a postal code or a customer ID is written with digits, but averaging them makes no sense. They are qualitative variables.
Counting categories
The first summary of a qualitative (or discrete) variable is its count table: how many times each category appears.
from collections import Counter
canaux = ["web", "magasin", "web", "appli", "web", "magasin"]
effectifs = Counter(canaux) # counts each category
print(effectifs) # Counter({'web': 3, 'magasin': 2, 'appli': 1})
total = sum(effectifs.values())
frequences = {k: v / total for k, v in effectifs.items()}
print(frequences["web"]) # 0.5, i.e. 50%The frequency of a category is its count divided by the total count. The frequencies always sum to 1 (or 100%).
With pandas
On a real dataset, you use pandas:
import pandas as pd
s = pd.Series(["web", "magasin", "web", "appli", "web", "magasin"])
print(s.value_counts()) # counts, sorted from largest to smallest
print(s.value_counts(normalize=True)) # frequenciesOrdinal variables and cumulative frequencies
For an ordinal or discrete quantitative variable, you can accumulate frequencies in the order of the categories: “70% of customers bought 2 items or fewer”. You must then sort by category, not by count.
nb_articles = pd.Series([1, 2, 2, 3, 1, 2, 4, 1, 2, 3])
freq = nb_articles.value_counts(normalize=True).sort_index() # sort by category
print(freq.cumsum()) # 1 -> 0.3, 2 -> 0.7, 3 -> 0.9, 4 -> 1.0For a continuous variable (amounts), you first group the values into bins (intervals), for example with pd.cut, before counting.
Key takeaways
- Identify the type of each variable before any calculation.
- Count = tally; frequency = count / total.
- Cumulative frequencies only make sense if the categories are ordered.
Exercises
Solutions and tests stay on the server. In the student space, your code is run and checked, and the tutor gives hints before the solution.
Channel frequencies
You have the list
canaux. Create a dictionaryfrequencesthat maps each channel to its frequency (between 0 and 1), usingcollections.Counter.from collections import Counter canaux = ['web', 'magasin', 'web', 'appli', 'web', 'magasin', 'web', 'appli'] # frequences = ...Cumulative frequencies with pandas
The series
nb_articlescontains the number of items per order. Computefreq_cum, the series of cumulative frequencies sorted by increasing number of items. Then store inpart_2_ou_moinsthe share of orders with 2 items or fewer.import pandas as pd nb_articles = pd.Series([1, 2, 2, 3, 1, 2, 4, 1, 2, 3]) # freq_cum = ... # part_2_ou_moins = ...Identify a variable's type
What is the type of the variable “customer postal code”?
- Quantitative discrete
- Quantitative continuous
- Qualitative nominal
- Qualitative ordinal
End-of-lesson quiz
Clothing size (S, M, L, XL) is a variable that is:
- Qualitative nominal
- Qualitative ordinal
- Quantitative continuous
Out of 200 orders, 50 come from the app. What is the frequency of the app?
- 50
- 0.25
- 0.5
- 4
What does s.value_counts(normalize=True) return?
- The raw counts
- The frequency of each category
- The mean of the series
The rest of the course
- Central Tendency: Mean, Median and Mode15 min
- Spread: Range, Variance and Standard Deviation18 min
- Quantiles, Box Plots and Outliers18 min
- Relationship Between Two Variables: Cross-Tabulations and Correlation20 min