Using Social Listening to Detect and Respond to Software Outages

Using social listening data to watch software and respond to outages

Customers dislike it when their software stops working. These outages alienate your clients and damage your brand. While it’s true for relatively minor outages, like when customers can't order a burger on the McDonald's app, it becomes even more infuriating when it's where their finances are held, like banking, brokerage, or investing software.

Customers of Fidelity Investments, the second largest brokerage in the United States, experienced such an outage on August 15, 2023. For a few hours, customers could log in but could not buy or sell stocks or even see any account balances. Clients took to social media to express their fear as to when they could access their money again.

We used a customized social listening dataset using Infegy Atlas to identify the outage and quantify its brand consequences. We'll identify the key indicators we used to show evidence of the issue and its damage to your brand so your brand can do the same with your social listening tool.

Traditional ways of detecting outages

When software gets buggy, developers can use two types of data sources to determine what's wrong. First, developers put error-handling signals such as logging in their code to catch errors. Logs help developers know if their app is working as it should and help them troubleshoot problems after identification. Since all this data is private and internal data doesn't harm a brand's reputation.

Second, there's public data from websites like www.downdetector.com. These sites rely on crowdsourced outage data that users self-report when they can't access a web resource.

Social listening data can also be a public outage indicator showing your application isn't working. Next, we'll discuss what social listening indicators you can use to identify software outages and how to respond.

Surge in post volume

Post volume is the first social listening indicator that something might be up with your product or service. Traditionally, r/FidelityInvestments is a sleepy place. It usually garners between 200 and 300 comments per day in August. This lack of activity makes sense: August is a summer month and traditionally quiet for the investing community. However, on the outage day, the subreddit rose 187% from its two-week mean of 348 posts. Here is your first indicator that something was afoot within the Fidelity community.

Figure 1: Post Volume surges August 15, 2023 on Fidelity's subreddit (August 1, 2023 through August 15, 2023); r/FidelityInvestments data processed by Infegy Atlas.

Rising negative sentiment

While post volume is a great first indicator to detect anomalous behavior about your software product, it's not sufficient to suggest that something terrible is happening.

To detect the brand concern, we next turned to aggregated sentiment. Traditionally, r/FidelityInvestments is a neutral place, meaning that positive sentiment roughly equals negative sentiment. However, on the outage day, we saw a two-week high surge in negative sentiment. Negative sentiment reached 72%, approximately three out of every four posts.

Figure 2: Negative sentiment surges August 15, 2023 on Fidelity's subreddit (August 1, 2023 through August 15, 2023); r/FidelityInvestments data processed by Infegy Atlas.

Identifying negative keywords

We've detected anomalous post volume and negative sentiment around your brand at this stage. It's time to dig deeper into what the actual problem is.

To do this, we'll use Infegy Atlas's Topics, which aggregates nouns and verbs that are particularly important within chunks of text. On August 15, we see apparent keywords like "access," "service," "outages," "logging," "problem," and "website" that rise to the top of aggregation. These all have primarily negative sentiments, another clue that something's wrong. If you set automated alerts, you can use these aggregated indicators to show that something is wrong with your application and that the issue is widespread, suggesting many users are experiencing the same problem.

Figure 3: Negative topics on Fidelity's subreddit (August 1, 2023 through August 15, 2023); r/FidelityInvestments data processed by Infegy Atlas.

Indicator emotions drop

Now that we've looked at aggregated sentiment and topics let's look at how those meld together to create waves and both positive and negative emotions about your brand. Infegy Atlas achieves this with Infegy IQ, our natural language processing engine.

Figure 4: Emotional documents distribution Fidelity's subreddit (August 1, 2023 through August 15, 2023); r/FidelityInvestments data processed by Infegy Atlas.

Declining customer loyalty

Customers are sticky within online brokerages. Once brokerages have attracted customers, they tend to stick around: moving money is a pain, and transferring funds between brokerages can take weeks. That fact makes Figure 5 particularly concerning for Fidelity's brand strategists. Upon the August 15 outage, we saw Intent->Loyalty drop to a 2-week low. Intent->Loyalty aggregates loyalty-related keywords that signal customers are happy with your service and are willing to stick around. Falling customer loyalty signals that your clients are leaving your platform, and your brand needs to respond to keep them happy.

Figure 5: Dropping customer loyalty via Infegy Atlas Themes (August 1, 2023 through August 15, 2023); r/FidelityInvestments data processed by Infegy Atlas.

Social listening data to inform outages

The Fidelity Investments outage on August 15, 2023, underscored how you can use social listening data as a non-traditional signal for software outages and the potential brand damage they can cause. After collecting and analyzing data from r/FidelityInvestments, we identified key indicators such as a surge in negative sentiment, anomalous post volume, and declining customer loyalty, shedding light that the outage occurred and the significant brand consequences of such disruptions. This experience highlights the importance of swiftly addressing software issues to mitigate brand damage and maintain customer trust in today's competitive digital investing landscape.

Key Takeaways from Fidelity's Outage

Using social listening tools can quickly identify software outages, as seen during the Fidelity Investments service disruption on August 15, 2023.

Anomalies in social media activity, such as increased post volume and negative sentiment, are early indicators of potential software issues.

The outage led to a significant drop in customer loyalty, emphasizing the importance of timely outage responses to protect brand reputation.

Implementing social listening as part of outage detection strategies can help brands address problems proactively, maintaining user trust.