Your Marketplace's Most Valuable Asset Is Hiding in Its Conversational Data

If your product is a contact-heavy platform (classifieds, rentals, home services, jobs, dating), the richest product signal you have is conversational data. The marketplace, in the broad sense of the word, is the whole family of platforms where strangers transact, and the messenger is where most of those contacts start and end. That makes the messenger the place where AI/ML earns the most.

Two forces, one dial

Growth needs more contacts: more supply, more traffic, a lower-friction "write" button. Quality needs contacts that turn into deals: live listings, responsive counterparts, real intent. The two pull against each other, and usually sit with two owners.

A platform's flywheel spins when buyers and sellers keep buying and selling successfully without paid acquisition pushing them; Every turn goes through a contact. Quantity spins the wheel; quality is what makes the turn count.

The theory behind it: Alvin Roth's market design says a matching market (one where both sides have to choose each other) works only when it is thick (enough participants), uncongested (people can get through their options in time) and safe (no reason to lie, game the timing or take the deal elsewhere). Congestion is what you get when you go thick too fast. A messenger flooded with "Is it still available?" is a congested market, not a busy one.

The two axes are not equally visible. Volume is free to observe: chats, contacts, listings. Quality is not: the deal usually closes off-platform. ML's first job is to make quality observable; its second is to move it.

Three buckets, not one number

A contact becomes one of three things, and each has a different owner:

The label is the product. The hard part is turning a fuzzy human idea ("did they agree?") into reproducible classes: one labelling instruction shared by data science, analysts and the business; a seed set labelled together to find the grey zones, then labelling at volume with a second pass on the disagreements. Then check the classes against the deals you can see: a "sold here" on takedown, a review after a service, a both-sides confirmation in the chat, an in-app payment. That is the ground truth the off-platform majority is calibrated against.

An example: one week, three channels

Every chat gets a class and a reason once a week: rules mark the unanswered, a model classifies the rest. The three buckets are the invariant; the reasons under them are whatever your product needs to act on. "Already sold" and "no deal: price" are supply and pricing problems, "details asked, no decision" is a conversation problem, "no answer" is an onboarding problem.

What to score a chat with

Start cheap and add cost only where the cheap layer is unsure.

Heuristics first. Deterministic rules on metadata, not text: no reply within a window, a call under a few seconds, a first message identical across fifty chats, a listing already closed. Cheap, auditable, and they cover a large share of trash. They also become the first features of the next step.

Then one binary model. One question: did this chat lead anywhere, yes or no. Features: messages per side, reply latency, who spoke last, whether a structured offer was made, the buyer's prior activity on the listing, the seller's reply history, message text only as embeddings. Two classes keep the labels clean; typed labels explode into overlapping categories nobody applies consistently. Set the threshold on precision for the target class, because a false "target" inflates the north star.

Then an LLM, sparingly. It classifies the outcome and the reason the conversation ended: sold, price, terms, no next step, pulled off-platform. Run it on the unsure middle of the score distribution and on a sample for diagnostics. Cost per inference is the constraint, not accuracy.

The trade-off, in one table:

	Heuristics	Binary model	LLM	LLM as labeller → small model
Cost per chat	~0	~0	high	high once, ~0 after
Labels needed	none	thousands, clean	none to start	a few hundred gold labels
Works on calls (no text)	yes	yes	no	partly
Explains itself	fully	partly	in words, not reliably	no
Best at	obvious trash	the yes/no at volume	the why, new patterns, cold start	bootstrapping labels fast

The binary model wins production because half the signal is not in the text: reply latency, who spoke last, offer made, listing history. An LLM works on text alone, and a call has none. The LLM wins where there are no labels yet (a new market, a new category, a new scam) and where you need the reason, not the score. The combination is the answer: the LLM labels a sample, a few hundred gold labels check it, the cheap model trains on those labels, and the LLM stays on for the unsure middle and a monthly diagnostic sample.

Two disciplines around the model. Recalibrate on a schedule, because behaviour drifts and the thresholds drift with it; a monthly labelling pass on a random sample is the regression test the behavioural metrics cannot be. And log the reply before you trust the metric: if calls are not logged or a channel drops replies, a whole segment scores as trash, and the model learns it.

Seven levers, four places

Once every contact has a class, the score does two jobs. As a metric, it is the number to run A/B tests against: a feature wins when it moves target contacts, not chats. As a signal, it feeds the features themselves: which chat to answer first, which silent chat to nudge, which sender to throttle. The seven levers below use it one way or the other, grouped by where they sit in a chat’s life.

Before the chat starts: shape what comes in

1. Openers that carry intent. Openers tuned to the listing instead of a blank box (“Can I pick it up today?”, “Would you take X?”): ranked templates, no generation. Easier chat start → more chats → more target contacts if the rate stays. Hinge does the same with a like on a specific photo or prompt.
2. Throttle low-odds senders. Re-distribute liquidity in search and recommendations towards the users that are responsive, throttle channels that give low-response rates.

At the first message: take the volume off the seller

3. Help the seller answer the opener. "Is it still available?" is the dominant first message everywhere, and the item usually is. An intent classifier answers it from the listing status; "last price?" and "can you post it?" get a price surface and a shipping quote. Mind precision: a system reply is not a seller reply.

Inside the conversation: get to the agreement

4. The next step as a chat action. "Make an offer", "Book a viewing", "Buy with delivery" inside the chat. A structured offer is a labelled event; a free-text haggle is invisible. An on-platform transaction plus the chat is the strongest signal.
5. Close the reply gap. The median seller reply lands over an hour after the buyer writes; most buyers are gone by then. Templates first (most of the value), then ranked templates, then answers retrieved from the listing, never free-form generation.
6. Make staying safe the easy path. The scam playbook barely varies: pull the buyer off the chat to a "payment" link or a "courier" app. Signals combine into a confidence score; the reaction is graded: allow, warn, hold, block. Warn on a shared phone number; block only high-confidence fraud. And ship the features people leave for (photos, voice, location, in-chat payment).

After the conversation: follow up the ones that can convert

7. Follow up the conversations that can convert. The deal-probability score says which silent chats deserve a nudge: a follow-up on the seller's behalf when the buyer went quiet on a chat scored "could agree", visible to the seller, switchable off. The same score orders the inbox. Without the score, a nudge to every silent seller moved reply rate a little, deals not at all, and faded within weeks.

---

---