Analysis to impact: Building a compelling narrative
Updated: Oct 2, 2020
The purpose of this article is to help people improve “building a narrative”. What I landed on is a framework to help think through whether the ingredients are there for a compelling narrative. The benefits of building compelling narratives should be clear: The work will likely have more impact. While this note focuses on analysis and seeks to primarily benefit data scientists, I think the framework is flexible enough to be applied to any presentation looking to have impact.
The framework I came up with is a simple hierarchy of necessary conditions to meet, each increasing the likelihood we reach a higher apex, which is impact. Here it is:
Each level of this hierarchy will be described in detail in the later sections of the note. From bottom to top.
Here’s how to interpret this diagram. Starting from the bottom, each condition is necessary to meet before the next. That is, it is pointless, or even harmful, to have a an analysis that is not sound (either logically flawed, or premises incorrect --> perhaps used the wrong data) but is inspiring and actionable. You can have impact with simply a sound, clear analysis with some new information (imagine we remove the top layers of inspiring & actionable). What this diagram asserts however, is that you are more likely to have more impact by also making it inspiring & actionable (higher apex). More on each of the buckets and the various scenarios later.
How I imagine someone could use this framework is by asking the questions in sequence (Though I recommend reading through their dedicated sections later on in this note):
Is my argument valid and sound? Does it logically make sense?
Is there new information in what I’m presenting? Does it lead to new conclusions?
Is the argument clear? Is the new information and conclusions clear to the readers?
Does the new information I present inspire people to come up with new product ideas or to think of a problem differently? Do I make it easy for people to reach this understanding? Will people walk away with sufficient depth of understanding?
Am I clear in what I want people to do with this new information/understanding? Are there clear asks and next steps? Do people know what is being asked of them?
There are more nuances to these 4 buckets than just the above questions that I’ll spend more time going into more detail in this note. In particular, there are a couple of themes to consider that cut across these buckets:
Quite simply, the more content there is (could be measured by number of slides, as an example), the harder it is to follow. People have limited attention, the more stuff you have to share, the higher likelihood they will be lost or miss the point you’re trying to get across.
Who you are presenting to will influence what and how you present. What may be novel to one audience may not be to another. The information needed for a particular audience to determine if an argument is sound or not will also differ. Audiences can vary significantly from peer Data Scientists, to XFN, to product leads, to senior leads, etc.
I will be referencing these themes in each of the buckets as I go through them.
This logical concept can be broken down into two important components:
Is the argument valid?
Are the premises true?
The first part, validity, is basically the question of, “if what you say is true (premises), does your conclusion logically follow?”. The second question is then a confirmation of whether the premises are actually true or not. Together, they determine if your argument is sound. This first step, soundness, is crucial because all the other layers of the hierarchy are built on top of it. Can you imagine convincing and inspiring people to take action based off of faulty premises or invalid arguments? This would almost certainly fall in the category of negative impact.
Let's consider each component individually.
Validity. Constructing a valid argument is not as trivial as it may seem, especially in a complex environment with incomplete information. The skill commonly referenced here is framing. Can you structure a problem or solution well enough that others can agree that if the component pieces were solved, it would lead to some greater conclusion? Often times structuring a problem/solution and getting alignment on that is just as important (if not more so) than the work needed to validate the pieces.
Truth. Determining if the premises of an argument are true or not is a little more straightforward. The considerations here are more around due diligence, technical errors, and sense checking the data. The less straightforward pieces here are when a premise cannot be validated, in which case assumptions must be made. Being comfortable with and knowing how to make reasonable assumptions is what can make this component difficult as well.
Knowing what the arguments you’re trying to make (validity) are will provide the foundation of a good narrative. The act of identifying this explicitly for yourself also empowers you to pare down the content volume. Specifically, the information that supports non-critical premises can be moved into the appendix (in case the question does come up) or cut out entirely which strengthens the focus on the core argument and frees it from needless distraction. Having the argument finely tuned (while remaining valid) will also enable your audience to more easily accept the conclusions you are asserting.
Audience can play a big role in how you construct a sound argument and what content is needed. For example, when presenting to peer data scientists, spending more time on strong methods of validating premises and details around data accuracy is probably a good idea. However, when presenting to more senior leads, these details become less important as more faith is given to the due diligence around determining the truth of the premises. Framing here, becomes key, and an unclear and poorly communicated argument, even if valid, will have a greater chance of being misinterpreted or ignored. These considerations move us to the much related second tier of the hierarchy, Clarity.
While a sound argument is the foundation of a narrative, it is all for nothing if the intended audience cannot perceive it accurately. This is where clarity comes in. Clarity is on top of Soundness because what would be the point of being very clearly wrong... Anyways, when I think of clarity, I think of two key dimensions: Form & Focus.
When I say form, I am referring to the type of medium one is trying to use to communicate the information. This can happen at many levels. It could be a choice between powerpoint/keynote (slide deck), a note, a post, or something else. It could also be a choice between a bar chart, line graph, table, plain text, pie chart, etc. The key here is that the chosen form has a large impact on level of achievable clarity. I’ll give a few examples for how this can come into play.
The first choice one usually needs to make is on the form of the presentation container. This can be influenced by if the intended audience is meant to consume the contents of it asynchronously or via a live presentation. Where notes & posts & emails may sometimes be preferred for the former and a slide deck/photo album of slides may be preferred for the latter. From my experience, if there is flexibility, go for the slide deck as a default. The reasoning here is that working with a slide deck forces the presenter to break their thoughts down into slides which lends itself better argument construction. Further, the form of a slide will tend to err on the side of visuals, since blocks of text look worse on slides than on emails/notes/posts. This isn’t to say that it has to be slides all the time, but that there are some benefits in using slides that will bias the presentation construction in a good way.
The form chosen to present and determine the truth of premises is also crucial in impacting the level of clarity. I will list a few points that I’ve found to be true and effective:
Avoid more than 2 sentences in a row on slides. Keep it in bullets. Keep it visual. This not only forces you to be succinct, it also helps to prevent the dreaded “reading from slides” behavior which comes across poorly in presentations.
No Pie Charts. People do not have good area perception and inferring relative sizes from them. Generally, anything that can be communicated via the areas in a pie chart is better communicated by using varying lengths on a bar chart.
Is a comparison necessary? Keep it on the same slide, side by side. This seems obvious but it happens all too often that a conclusion is drawn from a comparison that necessitates flipping or scrolling between two slides repeatedly.
Vary the visuals. Even if line charts are the most effective way of communicating or validating each individual premise, a long sequence of 10 line charts is makes it incredibly hard to sit through. In these cases, varying the visuals and form can help regain attention.
If it is between text and visual, pick visual, even if it takes up more space. People take a lot from visuals, whether it’s relationships, flows, relative sizes, etc. It helps people organize thoughts and is often well worth the effort. Even bullet points can be considered a visual when compared to the equivalent statement in sentences.
Moving on to Focus. When I talk about Focus, I’m referring to all the things we can do to free the reader from distraction. It is inevitable that when different people look at the same chart, or even read the same sentence, different interpretations can arise which all can derail an argument due to lack of clarity. Again, there are many things we can do in this space so I’ll give a shortlist of some techniques that can be done:
Consider the point you are trying to get across. Do you need to display the full years worth of data? Or do you simply need to zoom in on a particular set of events. Discard unnecessary information and help the reader focus their attention.
Formatting. Make the spacing, font, colors, templates etc. for all your slides as consistent as possible. Make them look good. Why? Because after the design theme is established with the first few slides, people can then focus on the content. Poor formatting and design is absolutely a distraction at best and at worst, can negatively affect interpretation of the content.
Annotate. Arrows, dotted lines, etc. are all super helpful to guide a readers eyes to the parts of the chart/visual that you want them to focus on. Help them understand where to start on a long time series, where to pause, where to look next. The sequence and highlights can all be key to communicating an important premise.
The application of Clarity towards the reduction of the volume of content should be obvious. Zoom in, remove unnecessary information, maintain focus. As hard as it was to compile all the necessary information to build a compelling narrative, it is even harder to identify the critical components and remove the rest to further strengthen that narrative.
For audience, clear sometimes means “don’t get into the details”, especially when presenting to XFN & leads. The more the overarching argument and premises are brought to the surface, the more clear the narrative. Details and further supporting documentation have a space to go --> in the appendix.
Ok, you now have a sound argument and it is clearly presented. People accept your conclusions. What next?
Novelty is a fairly simple concept, make sure there is new information or conclusions. Clear articulations of information that we already knew in the past is unlikely to have impact because most likely, we have already acted on them (this is not always the case as we’ll see later). It should also be fairly obvious how novel information is pointless if it cannot be perceived (unclear) or even wrong (unsound). The concept here is simple, but can grow more complex when we think about audience.
The key to understanding how novelty can differ by audience is to realize that not everyone has perfect shared information. Everyone has incomplete information and they are all incomplete in different ways. Knowing the context of what peoples’ current understanding of the world is will allow you to tailor and further focus your presentation draw their attention and show them why some action must be taken. This is why context gathering is so important, the more you know what your audience’s mental model of the world is, the more effective you can be at influencing them or even identifying opportunities to influence them. It is not uncommon, I think, that we believe people to have more context than they actually do.
The identification of what information is novel also gives us the inverse, information that is not novel. This gives further opportunity to pare down the volume of content spent on repeating things people already know and move that content into the appendix.
Ultimately, paying attention to Novelty allows presentations to draw attention to the deltas between the audience’s current understanding of the world vs. the presenter’s (hopefully more accurate and insightful) understanding of it. This allows the viewer to be able to see the incremental value of the work and probably brings them closer to being inspired to take action.
Novel information can often be inspiring by itself, but extra work is often required to get there. When I talk about inspiration, I’m talking about the ability that the narrative has to help the viewers generate new ideas or think about old problems in new and different ways. For example, the early days of the sharing team were focused significantly in solving problems around sharing mechanics, audience reduction, and inspirational prompts. Finding out that feedback was a significant component of the sharing experience inspired a whole new space of work around distribution, getting feedback from the right people, getting sufficient feedback, exploring different types of feedback, etc.
There are many things we can do to inspire and help people think through what impact an insight might have on their product or strategy. Here are some techniques:
Show the insight in the context of the greater problem/solution. To take the above example, the conclusion could have simply been that we had found another sharing lever, feedback. However, what really inspired a change in the way we thought about the sharing problem was to propose an alternate framework of understanding sharing behavior as a part of a feedback loop that reinforces future sharing. Showing how an individual finding fits into our broader understanding of a problem can make a huge difference in the impact of the finding.
Help build the intuition. Some findings can be very abstract and difficult for some people, especially non-technical people, to fully appreciate. Identify where this might be the case and find different ways of describing the insight. It could be spinning up a user story, or giving some sample numbers in a hypothetical scenario. Building on peoples’ intuition will help elevate them from a state of vague understanding to truly internalizing the insight enabling them to be inspired.
Give examples of ideas that may come from the finding. Sometimes giving an list of ideas that follow from the insight can inspire people to think of others and understand the insight in a different way.
The key to inspiring someone is to simultaneously enable deeper understanding of the insight as well as elevating the understanding in showing how it fits into the context of other higher level problems & solutions. The requirements for getting people to this level of understanding will obviously differ based on the audience and spending more time to make sure this lands well is often worth it.
If we can get people to be inspired by our narratives, that’s great... but it is still quite a distance away from achieving impact. Taking the next step and actually taking the cognitive burden of figuring out the full list of ideas and actions that need to be taken from the insight and proposing a clear plan of action brings us another huge step forward. This doesn’t necessarily have to be product ideas, it could be process recommendations, follow up analysis or experimentation, etc. They key is that we need to help answer the question of, “What next?”
I want to spend extra time on this because I think that sometimes, as a data scientist, we are more excited by the finding than anything else and we end up slapping together some next steps without putting too much thought into it. What I’ve found over time that this step is actually very important to get right in closing the loop for impact, especially when presenting to leads. Here’s why:
When you are presenting, especially to leads, you don’t have a lot of time. But during the allocated time you should have their full attention. If you stop at the inspiration stage and leave the immediate next steps as a question mark, what will you have gotten from that meeting? Sure the attendees will be more aware, and maybe they’ll do something differently, or if you’re lucky someone might reach out to help you figure out next steps. The meeting will end with mentions of good work and some half thought ideas will be thrown around but action is unlikely to be taken until the next time the topic is brought up. Depending on the amount of time in between, topics and discussion may need to be repeated resulting in a far longer process than you would like.
Instead, present the well thought out next steps. Even better, frame it in such a way that someone can simply say, “yes, that makes sense, go do it”. That way, the way is opened to enable the best case scenario where people agree that nothing crucial was missed and you have the go ahead to move forward. And worst case, you get some valuable feedback for how to modify your next steps. By presenting your next steps, a reference point is created for the attendees, and rather than spending the remainder of the meeting (probably 5 minutes or less) trying to work on building those out from scratch, the attendees can instead focus on the deltas between the current plan vs. the more optimal one based on their individual expertise.
This last step is so critical to reducing long timelines of back and forth discussions and ensuring good insights land impact in a timely manner. If you take one thing from this section, take this: “Make it easy for your stakeholders to simply say yes and clear the path for your impact”. Not only does this help your work land more impact quickly, it also gives the other stakeholders confidence that you’ve thought through the details and will likely execute well.
At this point, you’ve done just about all you can do to ensuring your narrative will be strong and successful. The rest really just comes down to execution.
Here are some final tips that apply to all sections:
Get feedback. Get it from a variety of people. Get it from stakeholders 1:1. This not only helps to get a signal on how well each of the pieces of the hierarchy are addressed thus strengthening the narrative, it also has the advantage of building consensus ahead of time. Having people answer questions for you is incredibly powerful.
I’ve said it before, but it cannot be emphasized enough: Keep it as short as possible. It’s so easy to end up with a bloated presentation, it takes a lot of conscious effort to shorten it to the important bits. Throw everything else into appendix. Still, not all insights need to be polished to the max so judge accordingly based on expected impact and audience.
Different versions for different audiences. If the insight is important enough to be super polished, it almost certainly has to look different if you are presenting to different audiences. Keep this in mind, always.
Focus on impact. Do not fall into the trap of falling in love with the insight alone. Make sure the insight gets the treatment it deserves and matures to a huge amount of impact. Do not neglect the top two to three tiers of the hierarchy.
Not all narratives need to be super polished. As mentioned above, a lot of the effort depends on how important it is to be polished. Doing all of these things for all presentations will likely slow things down. Though with practice, some things might make the process go faster. Exercise good judgement.
That’s it. It’s a long note but I hope it will be helpful in bringing landing more impact from peoples’ analyses (or any other types of presentations).