When I look at a competitor's ads, the first line matters most. The hook. "Stop living paycheck to paycheck." "No gym? No problem." After copying fifty of them into a note, you see a few angles coming back, but it's hard to say which ones and how often. So I wanted a script that groups hooks by the words they share.

This post builds that script in Go, using only the standard library. It tokenizes a CSV of hooks, builds TF-IDF vectors, compares them with cosine similarity, and merges them with average-linkage agglomerative clustering. For each cluster it prints the top terms, a representative hook and the members. Each hook has a hand label (the app category), so the script can check how often a cluster mixes categories.

I ran everything below with Go 1.27.3 on Linux on 16 November 2026. The output excerpts are copied from those runs.

The data is made up, on purpose

The hooks.csv in this post is a made-up sample dataset. I wrote all 50 lines myself, in the style of habit, fitness and budgeting app ads. They are not real ads, and numbers like "1 million people" are not claims about any real app. I added a few Spanish and German lines and some emoji, because real hook lists look like that.

For real data, open the public ad libraries you can access, read the ads, and paste the first lines into the same CSV by hand. Respect each library's terms of service and don't scrape them. Fifty hooks from your own niche is a good start.

The format is three columns:

id,label,hook 5,habit,Stop scrolling. Start a streak 🔥 18,fitness,Drop 10 pounds before summer with 15-minute workouts 35,budget,Stop living paycheck to paycheck 

The label column is only for checking. The clustering never sees it.

Setup

mkdir adhooks && cd adhooks go mod init adhooks # save main.go and hooks.csv here, then: go run . 

The code

The whole program is about 200 lines. Save it as main.go:

// adhooks: cluster ad hooks (first lines of ad copy) with TF-IDF and cosine // similarity, using only the Go standard library. package main import ( "encoding/csv" "flag" "fmt" "math" "os" "sort" "strings" "unicode" ) type Hook struct{ ID, Label, Text string } var stop = map[string]bool{} func init() { for _, w := range strings.Fields(`a an the and or but to of in on for with at by from is are was be it its this that these those you your i my me we our do does did don't doesn't isn't won't not no can will just here's it's what how why who when where than then so if as up out more most all one`) { stop[w] = true } } // tokenize lowercases, splits on anything that isn't a letter, digit or // apostrophe, drops stopwords and 1-letter tokens, and strips a plural "s". func tokenize(s string) []string { f := strings.FieldsFunc(strings.ToLower(s), func(r rune) bool { return !unicode.IsLetter(r) && !unicode.IsDigit(r) && r != '\'' && r != '’' }) var out []string for _, w := range f { w = strings.Trim(strings.ReplaceAll(w, "’", "'"), "'") if len([]rune(w)) < 2 || stop[w] { continue } if len(w) > 3 && strings.HasSuffix(w, "s") && !strings.HasSuffix(w, "ss") { w = strings.TrimSuffix(w, "s") } out = append(out, w) } return out } type Vec map[string]float64 // tfidf builds one L2-normalised vector per document. // idf = ln((1+N)/(1+df)) + 1, the "smooth" variant, so no term gets weight 0. func tfidf(docs [][]string) ([]Vec, map[string]float64) { df := map[string]int{} for _, d := range docs { seen := map[string]bool{} for _, t := range d { if !seen[t] { df[t], seen[t] = df[t]+1, true } } } n := float64(len(docs)) idf := map[string]float64{} for t, c := range df { idf[t] = math.Log((1+n)/(1+float64(c))) + 1 } vecs := make([]Vec, len(docs)) for i, d := range docs { v := Vec{} for _, t := range d { v[t] += idf[t] } norm := math.Sqrt(cosine(v, v)) for t := range v { v[t] /= norm } vecs[i] = v } return vecs, idf } // cosine of two unit vectors is just their dot product. func cosine(a, b Vec) float64 { var s float64 for t, x := range a { s += x * b[t] } return s } // cluster runs average-linkage agglomerative clustering: keep merging the two // clusters with the highest mean pairwise similarity until it drops below // minSim, or until only k clusters are left (if k > 0). func cluster(sim [][]float64, minSim float64, k int) [][]int { cs := make([][]int, len(sim)) for i := range sim { cs[i] = []int{i} } avg := func(a, b []int) float64 { var s float64 for _, i := range a { for _, j := range b { s += sim[i][j] } } return s / float64(len(a)*len(b)) } for len(cs) > 1 && (k <= 0 || len(cs) > k) { bi, bj, best := -1, -1, -1.0 for i := range cs { for j := i + 1; j < len(cs); j++ { if s := avg(cs[i], cs[j]); s > best { bi, bj, best = i, j, s } } } if k <= 0 && best < minSim { break } cs[bi] = append(cs[bi], cs[bj]...) cs = append(cs[:bj], cs[bj+1:]...) } sort.SliceStable(cs, func(i, j int) bool { return len(cs[i]) > len(cs[j]) }) return cs } func main() { path := flag.String("f", "hooks.csv", "CSV with id,label,hook") minSim := flag.Float64("sim", 0.10, "stop merging below this average similarity") k := flag.Int("k", 0, "stop at k clusters instead (0 = use -sim)") flag.Parse() fh, err := os.Open(*path) rows, err2 := csv.NewReader(fh).ReadAll() if err != nil || err2 != nil || len(rows) < 2 { fmt.Fprintln(os.Stderr, "cannot read", *path, err, err2) os.Exit(1) } var hooks []Hook var docs [][]string for _, r := range rows[1:] { hooks = append(hooks, Hook{r[0], r[1], r[2]}) docs = append(docs, tokenize(r[2])) } vecs, idf := tfidf(docs) sim := make([][]float64, len(vecs)) for i := range vecs { sim[i] = make([]float64, len(vecs)) for j := range vecs { sim[i][j] = cosine(vecs[i], vecs[j]) } } cs := cluster(sim, *minSim, *k) fmt.Printf("%d hooks, %d terms, %d clusters\n", len(hooks), len(idf), len(cs)) agree, singles := 0, 0 for n, c := range cs { if len(c) == 1 { singles++ continue } weight, labels := map[string]float64{}, map[string]int{} rep, repScore := c[0], -1.0 for _, i := range c { labels[hooks[i].Label]++ for t, x := range vecs[i] { weight[t] += x } var s float64 // medoid: highest mean similarity to the others for _, j := range c { if j != i { s += sim[i][j] } } if s > repScore { rep, repScore = i, s } } terms := make([]string, 0, len(weight)) for t := range weight { terms = append(terms, t) } sort.Strings(terms) // alphabetical first, so ties stay stable between runs sort.SliceStable(terms, func(a, b int) bool { return weight[terms[a]] > weight[terms[b]] }) top, most := "", 0 for l, cnt := range labels { if cnt > most || (cnt == most && l < top) { top, most = l, cnt } } agree += most fmt.Printf("\n#%d size=%d labels=%v\n", n+1, len(c), labels) fmt.Printf(" top terms: %s\n", strings.Join(terms[:min(4, len(terms))], ", ")) fmt.Printf(" representative: %q\n", hooks[rep].Text) for _, i := range c { if i != rep { fmt.Printf(" - %s\n", hooks[i].Text) } } } fmt.Printf("\nsingletons: %d\n", singles) fmt.Printf("purity of clustered hooks: %d/%d\n", agree, len(hooks)-singles) } 

A few choices worth explaining:

  • Tokenizing. Split on anything that isn't a letter, digit or apostrophe, so emoji and punctuation disappear. Accented letters survive, and curly apostrophes become straight ones. The only stemming is cutting a final "s".
  • TF-IDF. Common words get a lower weight, using the smooth formula ln((1+N)/(1+df)) + 1. Each vector is normalised to length 1, so cosine similarity is a dot product.
  • Clustering. Every hook starts alone. The loop merges the two clusters with the highest average similarity until it drops below -sim (default 0.10) or -k clusters are left. With 50 hooks the O(n³) loop takes milliseconds.
  • Representative hook. The medoid: the member most similar to the others. It's always a real hook, which reads better than centroid weights.

main_test.go has three small tests (tokenizing, cosine, the two stop rules). go vet and go test pass.

What it found

go run . on the sample prints:

50 hooks, 192 terms, 23 clusters #1 size=6 labels=map[budget:1 fitness:1 habit:4] top terms: day, free, try, changed representative: "Tiny habits. Big results. Try it free for 7 days" - I tracked one tiny habit a day for a year, here's what changed - Your morning routine is broken. Fix it in 5 minutes a day - Journaling 2 minutes a day changed my mornings - Try a free 7-day workout plan built for beginners - Save money on autopilot. Try it free for 7 days #2 size=4 labels=map[fitness:4] top terms: workout, minute, 15, 10 representative: "Drop 10 pounds before summer with 15-minute workouts" - No gym? No problem. Home workouts in 15 minutes - Burn belly fat with a 10-minute daily workout - The workout plan that adapts to your body #3 size=3 labels=map[habit:3] top terms: habit, build, 21, willpower representative: "Build a habit in 21 days without willpower" - Don't break the chain: build daily habits that stick - The habit tracker that actually makes you keep your habits #4 size=3 labels=map[fitness:2 habit:1] top terms: start, stop, calorie, counting representative: "Stop counting calories. Start eating smarter" - Stop scrolling. Start a streak 🔥 - Calorie counting made simple: snap a photo of your meal #5 size=3 labels=map[budget:1 habit:2] top terms: people, join, million, habit representative: "Join 1 million people building better habits" - Why do most people quit new habits? (It's not motivation) - Join 1 million people saving more every month #6 size=3 labels=map[budget:1 fitness:1 habit:1] top terms: finally, pov, checked, full representative: "POV: you finally checked your subscriptions 😬" - POV: you finally kept a streak for 30 days 🔥🔥 - POV: you finally did a full push-up 💪 #7 size=3 labels=map[budget:2 fitness:1] top terms: without, spend, feeling, less representative: "Spend less without feeling broke" - Lose weight without giving up the food you love - Track every dollar you spend without spreadsheets #8 size=3 labels=map[budget:3] top terms: budget, rule, 30, 20th representative: "I saved $5000 in 6 months with one simple budget rule" - The 50/30/20 budget rule explained in 30 seconds - Broke by the 20th? Here's a budget that works #9 size=2 labels=map[fitness:1 habit:1] top terms: ohne, aufbauen, gewohnheiten, willenskraft representative: "Gewohnheiten aufbauen ohne Willenskraft" - Abnehmen ohne Fitnessstudio: 15 Minuten am Tag ... singletons: 8 purity of clustered hooks: 33/42 

Cluster #2 is the "short home workout" angle, #3 is "build a habit without willpower", and #8 is the budget rule. Each has one label.

The mixed clusters are more interesting. #6 is three "POV: you finally..." hooks, one per category. #5 joins the two "Join 1 million people" lines, and #1 collects "Try it free for 7 days" hooks from all three apps. As category checks they are mistakes. For ad research they show a format used across niches. 33 of 42 clustered hooks share their cluster's main label, and 8 hooks stayed alone. I treat that as a number to watch, not a goal.

The walls you'll hit

1. Hooks are too short for TF-IDF. The average hook in my sample has 5.7 tokens after stopword removal. 142 of the 192 distinct terms appear in only one hook, and 1,075 of the 1,225 hook pairs have a similarity of exactly 0. So the clustering decides on one or two shared words. One "without" put "Lose weight without giving up the food you love" and "Spend less without feeling broke" together (#7). A longer stopword list helps a little. Adding each ad's second line helps more.

2. Synonyms don't match. "Lose weight", "Drop 10 pounds" and "Shed those extra kilos" mean the same thing to a person. To TF-IDF, "Lose weight without giving up the food you love" and "Drop 10 pounds before summer with 15-minute workouts" have a similarity of 0.000. "I lost 20 pounds" doesn't match "lose" either, because my stemmer only removes an "s". Embeddings fix this, but then you need a model. A small synonym map you maintain yourself (pounds, kilos, weight → weight) is a decent middle ground for one niche.

3. Tokenizing decides more than you'd think. Emoji are dropped, so "🔥" never links the two streak hooks. Fine for topics, bad if you study emoji use. "50/30/20" becomes three number tokens. Copied ad text often has curly apostrophes, and without the normalising line "don’t" would slip past the stopword list. "Habits" becomes "habit", but "spending" and "spend" stay apart. Print the tokens for ten hooks before you trust any cluster.

4. Choosing the threshold or k is the real work. Same data, different -sim values:

-sim clusters singletons purity
0.05 16 4 31/46
0.10 23 8 33/42
0.15 28 12 30/38
0.20 34 22 21/28
0.30 41 34 10/16

Lower thresholds cluster more hooks but mix more categories. Higher ones leave most hooks alone. -k 3 was worse: one cluster held 46 of the 50 hooks, with 15 or 16 of each label. Once it's forced to keep merging, average linkage on sparse vectors chains everything together. I pick a threshold, read the clusters and adjust by hand. Your data will need a different number.

5. Languages cluster by language, not by topic. The stopword list is English only. Cluster #9 joined a German habit hook and a German fitness hook because both contain "ohne" (without). Cluster #12 joined a Spanish fitness hook and a Spanish budget hook through "sin" and "de". "Crea hábitos que duran" never matched an English habit hook. Split hooks by language first and give each group its own stopwords.

6. Small, hand-made data is easy to overfit. I wrote these hooks, so they repeat my own phrasing. I wrote the stopword list and picked the 0.10 threshold while looking at the same 50 lines I'm reporting on, so the 33/42 is optimistic. Real hooks will be messier. Keep some hooks aside while tuning and check the setting on them. And collect them the slow way: copying lines by hand is fine for research, a scraper usually breaks the library's terms.

Where this stops

This script tells me which hooks in my list share words. It doesn't know when an ad started, how long it ran, or what the video showed, and it only sees the hooks I had time to copy.

That's the part I built PeekPanda* for. It's an iOS competitor catalogue, and each app's case file has a paid ads library: dated public paid-ad observations with the creative media, the copy and the destination, across millions of ads. You can filter by format, start date and run length, and see ad transcripts and creative analysis. I still run this Go script on the hooks I pick out, because seeing my own clusters is the fastest way to see which angles a niche leans on.

Wrap-up

  • About 200 lines of standard-library Go can tokenize, weight, compare and cluster ad hooks.
  • Read clusters as angles and formats, not ground truth. Mixed clusters are often the useful ones.
  • Short texts, synonyms, emoji, thresholds and languages are where it goes wrong. Check tokens and sweep the threshold.
  • Collect hooks by hand, respect each library's terms, and don't scrape.

Paste in 50 hooks from your own niche and run it with -sim 0.05, 0.10 and 0.15. The clusters that show up at every setting are probably the angles worth testing.

*I build it.