Where does marketing data come from?
A marketing department works with three main families of data:
- sessions (or visits): one row per visit to the site, with its origin (source, medium, campaign), the device, and whether it led to a conversion;
- touchpoints: one row per interaction of a person with a campaign before converting (click on an ad, opening of an email, visit from a social network);
- conversions: one row per goal reached (purchase, quote request, sign-up), with its value.
This data is produced by an audience measurement tool (such as Google Analytics 4), by advertising platforms, and by your own orders database. They never tell exactly the same story: each tool has its own counting rules. Your first job as an analyst is therefore to know the grain of the table you are handling: does a row correspond to a session, a person, a click, an order?
UTM parameters
To know where a visit comes from, you add UTM parameters to campaign links:
https://boutique.example/soldes?utm_source=newsletter&utm_medium=email&utm_campaign=soldes_hiver
utm_source: the precise sender (newsletter, google, facebook);utm_medium: the type of medium (email, cpc for paid click, social, referral);utm_campaign: the campaign name;utm_contentandutm_term(optional): the ad variant and the keyword.
Python reads these parameters with the standard urllib.parse module:
from urllib.parse import urlparse, parse_qs
url = "https://boutique.example/?utm_source=Newsletter&utm_medium=Email&utm_campaign=Soldes_Hiver"
params = parse_qs(urlparse(url).query)
print(params) # {'utm_source': ['Newsletter'], 'utm_medium': ['Email'], ...}
parse_qs returns a list of values per parameter, because a parameter can appear several times. You generally take the first one.
Normalize before counting
Measurement tools distinguish Facebook, facebook, and facebook: without normalization, the same source ends up split across three rows and each looks small. Two simple rules:
- put all values in lowercase and remove extra spaces;
- publish a naming convention (for example: lowercase, words separated by
_, a closed list of allowed mediums) and have everyone who creates links follow it.
When a visit has no UTM and no identifiable referring site, it is classified as direct access: source (direct), medium (none). Be careful, “direct” does not mean “the user typed the address”: an untagged link in a PDF, a messaging app, or a badly tagged email also arrives as direct.
Group into channels
A marketing manager thinks in channels (paid search, email, social networks…) rather than in source / medium pairs. Grouping is done with ordered rules: the first rule that applies wins.
def channel_group(source, medium):
if source == "(direct)":
return "Direct"
if medium == "email":
return "E-mail"
if medium in ("cpc", "ppc"):
return "Recherche payante"
if medium == "paid_social":
return "Social payant"
if medium == "organic":
return "Recherche naturelle"
if medium in ("social", "organic_social"):
return "Social naturel"
if medium == "referral":
return "Sites référents"
return "Autres"
Market tools offer their own default groupings, with richer rules. Writing your own has two advantages: you know exactly what each channel contains, and you can apply them to data that does not come from that tool (CRM, advertising export).
A first table by channel
sessions["channel"] = [channel_group(s, m) for s, m in zip(sessions["source"], sessions["medium"])]
by_channel = sessions.groupby("channel").agg(
sessions=("converted", "size"),
conversions=("converted", "sum"),
)
by_channel["conversion_rate"] = by_channel["conversions"] / by_channel["sessions"]
This rate is a conversion rate per session. If you compare it with a rate computed per user in another report, you will be comparing two different things. Always note the grain next to the figure.
Key takeaways
- Identify the grain of each table before computing.
- Normalize UTM values and enforce a naming convention.
- Channel grouping is a documented, shared set of ordered rules.
Exercises
Solutions and tests stay on the server. In the student space, your code is run and checked, and the tutor gives hints before the solution.
Read a URL's UTMs
Write
parse_utm(url)that returns a dictionary{"source": ..., "medium": ..., "campaign": ...}:- values in lowercase and without surrounding spaces;
- if
utm_sourceis missing or empty, return{"source": "(direct)", "medium": "(none)", "campaign": "(not set)"}; - if only
utm_mediumorutm_campaignis missing, put"(not set)"for the missing value.
Use
urlparseandparse_qs.from urllib.parse import urlparse, parse_qs def parse_utm(url): ...Table by channel
Write
channel_group(source, medium)applying, in this order, the rules from the lesson (Direct, E-mail, Recherche payante, Social payant, Recherche naturelle, Social naturel, Sites référents, otherwise Autres).Then add a
channelcolumn tosessions, and buildby_channel(index = channel) with the columnssessions(number of rows),conversions(sum ofconverted), andconversion_rate, sorted bysessionsdescending.import pandas as pd sessions = pd.DataFrame({ "source": ["google", "google", "(direct)", "newsletter", "facebook", "google", "facebook", "blog-partenaire", "(direct)", "google"], "medium": ["cpc", "organic", "(none)", "email", "paid_social", "cpc", "social", "referral", "(none)", "organic"], "converted": [1, 0, 1, 0, 0, 1, 0, 0, 0, 1], }) def channel_group(source, medium): ... # by_channel = ...A split source
In a report, the Facebook source appears on three rows:
Facebook,facebook, andfb. What is the best lasting answer?- Add up the three rows by hand every month
- Normalize the values (lowercase, spaces), fix the known aliases, and enforce a naming convention for future links
- Delete the smallest rows
- Move all Facebook traffic to direct access
End-of-lesson quiz
What does
utm_medium=cpcmean?- A visit from an email
- A visit from a paid click
- A direct visit
A visit classified as
(direct)/(none)comes from:- Always from manually typing the address
- An unidentified origin: typed address, bookmark, or untagged link
- A search engine
Why does the order of channel rules matter?
- Because the first rule that applies decides the channel
- Because pandas sorts the rules
- It does not matter
The rest of the course
- Campaign Attribution: Sharing the Credit for a Conversion18 min
- Conversion Funnel: Measuring Drop-Off Step by Step18 min
- Advertising ROI: Breakeven, Incrementality, and Marginal Return18 min
- Campaign A/B Tests and the Marketing Dashboard18 min