In the realm of user communication and customer engagement, it's not just about the words you say but also when you say them

We built Aampe to allow apps to communicate with their customers in a way that doesn’t make them hate everything and uninstall the app (pro tip: constantly nudging all your users with mindless messages about discounts is a great way to make them hate everything). Our system allows you to figure out what each of your users wants to hear, but that’s only half the battle - the right message sent at the wrong time is still the wrong message.

Let's break down a pivotal component of our system: the personalization score

When we message users, we deploy an AI model that estimates the probability they will respond. We use this model to generate two different estimates: their baseline probability to respond (which only looks at their historical behavior) and their target probability to respond (which considers each possible messaging choice one by one).

We compare the probabilities to calculate a personalization score, which represents the degree to which the target probability outpaces the baseline probability.

So if you write a message that emphasizes how fast your delivery is, each user will get a personalization score that tells you the probability that they’ll respond if you send them a message about fast delivery.

For every messaging choice, every user gets a personalization score of 0.5 to start off - that means they have a 50/50 chance of responding to it. As we get more information by experimenting with different messages, the score could go as high as 1.0 (meaning the user is very likely to respond) or as low as 0.0 (meaning the user is very unlikely to respond).

(If you want a more in-depth but still pretty easy introduction to how personalization scores work, go here. If you want a technical deep-dive, go here.)

We calculate personalization scores for each copy choice - for each value proposition, each product offering, even for particular greetings of emoji choices - but we also calculate personalization scores for timing choices, showing how likely each user is to respond to a message sent at, say, Monday at noon versus Friday and 9:00 in the morning.

‍

‍Note: In addition to calculating personalization scores for message content and timing, we also calculate them for each user's preferred messaging channel. Some people don't mind getting SMS or WhatsApp messages. For others, they're fine receiving push notifications but find these other communication channels to be obtrusive. Sending users the right message on the right channel (and, mind you, these preferences can change over time) is just as important as sending the user the right message at the right time.

‍

Here’s a results breakdown of personalization scores, looking at two different countries in which we have been managing user communication for a large app with over three million users. Each box represents the click-through rate for messages sent on the basis of different personalization scores: so the box at the intersection of a 0.3 copy score and 0.8 timing score represents the click-through rate for notifications where were Aampe’s algorithms were pretty sure about the timing but not sure at all about the copy. The biggest, darkest boxes represent the highest click-through rates:

Notice a few things:

  • Messages sent with low copy scores and low timing scores don’t perform very well at all. Yes, that’s what we’d expect, and we don’t send messages with generally low scores unless we have no better choice or we decide to try a new combination to see if a user’s preferences have changed.
  • Messages with high copy scores and low timing scores also perform poorly - and there are even fewer of these instances than there are of generally low scores. That means (1) it’s hard to get the copy right unless you get the timing right, and (2) even when you do get the copy right, if you get the timing wrong users still don’t respond.
  • Messages with low copy scores and high timing scores, however, performed decently - not as good as getting both copy and timing right at the same time, of course, but just sending a moderately interesting message at exactly the right time does yield a fair amount of success - far more than sending an awesome message at a time when users won’t respond to it.

It’s important to optimize copy - users aren’t going to sit by patiently as you message them about irrelevant offerings and try to tap into motivations that don’t speak to them. However, you won’t win on copy alone. You need to get the “what” right, but you also need the “when”. Get both of those right (plus the right messaging channel), and you’re in a position to delight your customers, which puts them in a position to reward you for speaking to them the way they want to be spoken to.