All posts by Linda Tsao

Using SysTrack to Resolve User Issues with SaaS

Cloud services have fundamentally shifted the landscape for the average consumer of IT resources, and too often this leaves the support organization stuck as the discoverer of the root cause of issues that span the endpoint, the network, and the supporting service providers. Our own organization is in a similar situation, having adopted Office 365 services for all users. However, because we’re consumers of our own dog food (figuratively—I only have a guinea pig), we’ve got the ability to pinpoint the real source of user-impacting problems. As an example, we recently had a user experience trouble with making and receiving calls with Skype for Business.

Our investigation starts at the point of interaction for the user. In Resolve, we’ve got the ability to look up the user in question and immediately start reviewing the context surrounding their performance issue.

The RAG analysis provided in the overview indicates a handful of issues, but nothing directly related to Skype performance on the client itself. Further, there’s nothing impacting the network for the user or their NIC. That means we need to dig in a little deeper. This leads us to the Black Box area, which allows us to focus on the particulars of what was happening when the user had their issue.

Selecting the time interval where they first started having an issue we notice something interesting: the Skype for Business app (lync.exe) shows essentially no network traffic. Kind of a curious turn of events, and it’s something we wanted to investigate further. Luckily, we’ve been doing some internal auditing work with service status reporting from Office 365.

Here we find the cause: A/V components of Skype for Business were degraded for that user in the time interval where he was having the problem. A little bit of patience and some conversations with the Skype Online team had the service back up and running shortly. In fact, you can see the normal call traffic resume in the Black Box record above, and there you have it.

With SysTrack, we could find that the underlying cause for this specific user’s problem was not related to a client problem, a local network problem, or any service centrally provided by IT. Instead, there was a temporary service degradation that was resolved in short order. As essential software moves to the cloud, it is increasingly important to have the ability to gain detailed insights at the endpoint in order to identify whether the root cause of an issue can be traced back to the service provider.

How does Office 365 perform across Windows operating systems?

Modern users have the choice between a variety of Windows OS and Office versions. In relation to this mix, a common question we have come across in the past is “How does Windows 10 performance compare with Windows 7?” While we have addressed the situation in the past, it remains a popular question to this day. However, users are now becoming curious about the performance implications of Office versions against the operation systems. Through analysis of SysTrack Community data, we were able to reevaluate Windows 7 and Windows 10 performance implications against Office 365.

A feature that Windows 10 has is its integration with various components of Microsoft’s cloud portfolio. With this new component, we felt compelled to look at how Office 365 ran against past operating systems and how past versions of Office, specifically Office 2013, ran against current operating systems. Office 365 may look very similar to older versions, there are quite a few notable differences. While Office 2013 required a product key, Office 365 handles licensing more efficiently for users, potentially allowing each job role to be given a best fit licensing level. This is just an example of how Office 365 is now closely reliant on the cloud. The cloud allows Office 365 applications to be available from any device and encourages collaboration among users while Office 2013 requires a local installation. Office 2013 did not allow for as smooth of collaboration, requiring the user to share files that have been saved locally or manually stored in a place that can be reached by others. Finally, with Office 365 being software-as-a-service, it has improved security and user experience by seamlessly providing small, frequent patches.

With all these updates to Office 365, how does it affect the overall performance characteristics? We ran a comparison of Office 365 against Office 2013 with different operating systems to see how their load times compared (displayed below in Figure 1).

Figure 1

While it is interesting that Office 365 seems to take a slightly longer time to load, it is mostly due to external connectivity and tying the user account context for Office 365 to the application itself. However, looking at application stability (displayed below in Figure 2), Office 365 has significantly fewer faults than Office 2013.

Figure 2

As displayed above, Office 365 faults significantly less in Windows 10 than Office 2013. This can be a result of both being “as-a-service” products ultimately resulting in less downtime users (thus a higher end user experience) and less maintenance for IT administrators. It can be concluded that while Office 365 takes a little more time to load, it is more stable than Office 2013 among the various operating systems.

So what does all this analysis mean? Ideally, Windows 10 and Office 365 should be used together to achieve high end user experience. Office 365, overall, is more stable providing less application faults. However, other operating systems are also compatible with Office 365 despite the slightly longer load time. To evaluate readiness for a Windows 10 migration, or performance monitoring, check out our free Windows 10 assessment, with the addition of SysTrack to provide the transparency of end user experience monitoring.

Key Considerations for Office 365 Planning

This blog post is a continuation of our Office 365 series on license provisioning. In this post, we’ll cover the metrics relevant to this discussion. By focusing on specific characteristics of user behavior, you can make better informed decisions when assigning Office 365 licenses. Choosing the right metrics to examine is crucial, as we all know that user application activity is complex and that not all data tells the full story.

In the previous post, “Office 365 Planning Kit: The Method Overview,” we covered  how we organized each license based on application category. The table below displays a clear breakdown of categories for each license. In order to fit users with licenses, you have to find out how users are interacting with their applications.

To determine what each user requires to fulfill their job role, it is important to understand a user’s current application needs. The categories make it easier to visualize what kind of applications each user uses and which applications can potentially be replaced based on the frequency of usage. While Office 365 applications work best together, other applications in the category can be just as compatible and ultimately save the company money. We have sorted usage based on 3 tiers: None, Low, and High (displayed below) with each tier determined by the amount of focus time and active time a user has with an application.


SysTrack collects a unique metric, “focus time”, that offers the most insight into the quality of a user’s engagement with an application. Compared to active time, which is simply the amount of time that a user has an application open, SysTrack collects focus time only when a user is actually interacting with an application. For example, it’s common for users to open applications, such as Skype, and leave it open while they work on their report in Word resulting in Word have a greater focus time than Skype, but with Skype having a greater active time than Word.  This metric provides further optimization for suggesting a license to each user, because it not only takes into account which applications they use, but also considers how they may use the application.

Perhaps a user fits the criteria for an E3 license, but the frequency of usage for the Presentation, Document Sharing, and Meeting applications are categorized as low. While they will use all the applications provided with an E3 license, the best fit for them would be an E1 license with the other applications being replaced. For a more specific user, perhaps one suggested an E3 or E5 license, IT administrators have the ability to look into the specific users and observe the users active time, focus time, and the ratio of active to focus time (displayed below). This will allow the administrators to determine if they are viewing (shorter focus time) vs editing (longer focus time).

The table also provides more data such as the application names that users are already using. This can help determine if all users are using the same compatible applications or if they are all choosing their own versions. This knowledge allows IT administrators to continue to suggest the same compatible application to users to further promote a clean environment.

Having to keep all these considerations in mind may seem like a headache; luckily, our Office 365 Planning Kit simplifies the process. In a future post, we will further discuss the various ways companies can continue to benefit from optimal provisioned licensing in detail along with real world use cases.

The SysTrack Office 365 Planning Kit

Office 365 is Microsoft’s latest cloud-based group of software and service subscriptions. Due to its features and monthly/yearly licensing plan, many businesses choose a single license for their entire company. After all, selecting the same license for everyone seems like the easiest choice–why go through the hassle of fitting users to their optimal licenses?

Generically assigning Office 365 licenses may be convenient, but doing so is ultimately a poor business decision. A poorly fit license can lead to clutter in the environment as some applications go unused while other various applications are installed to make up for the license fit. Different job roles within a company likely have different application needs, and it is worthwhile for a business to investigate the unique needs of users to avoid overspending on licenses. SysTrack’s Office 365 Planning Kit makes it easy to identify the best license for each user in an environment while providing a clear picture of potential savings (displayed below).

The table above shows how moving users to reduced licenses would result in costs savings. If a user with an E3 or E5 license is found to only require E1-level functionality, dropping the user down to the more basic license would result in a savings of at least $12/month. Over an entire company, this kind of movement could result in significant savings on licensing costs alone. You may have noticed that there are suggested licenses that are not actual license options for Office 365. We built the “E3 to E1” and “E3 or E5” options into the SysTrack Kit to allow administrators to make the call on determining the best licenses are for edge users that fall into these categories. If a user is suggested an E3 to E1 license it simply means they use some applications (word processor, spreadsheet, or email) that are included in the E3 licensing. However, the applications that they are using are also included in the E1 license as  web-based applications instead of locally installed applications. Switching from E3 to E1 can save money at the infrastructure level since web-based applications and light bandwidth use for the applications can be cheaper and easier to maintain than locally installed instances. The locally installed applications are also more difficult to update and deploy and can introduce versioning issues and a whole slew of other problems into the environment. If a user is suggested an E3 or E5 license, this means the user uses applications that only the E5 license can provide. However, based on how they use the applications (editing versus viewing the documents), the user may still fit within the E3 license.

Since each job role will only have the applications that they need with their newly fitted license, their environment will be less cluttered. This leads to a less impacted and overall faster environment allowing IT to work on more valuable projects. Needing less IT help lets the company potentially save even more money. Having a condensed application portfolio also allows IT to learn and troubleshoot faster because they will be able to get a deeper understanding of the applications.

Provisioning optimal licenses to each user can appear as an unnecessary, tedious task. However, due to potential savings, a more organized environment and less burden on IT, the company will benefit greatly. Still don’t feel like undertaking the task? SysTrack’s Office 365 Planning Kit greatly simplifies the process by providing automated suggestions of license assignments based on real user needs. In future posts, we will discuss our new kit in further detail to show how easy it can be to harness the benefits of fitting users to their ideal Office 365 subscription license.

SysTrack Use Case: What’s Impacting My System?

One of SysTrack’s many uses includes providing insight on negative sources of impact in an end user’s environment. Specifically, it can be used by an IT administrator to find out what the source of issues are that cause user experience problems. This helps the IT organization find underlying causes for common user complaints like “my system is running slow” or “I just can’t open my email!”

Because there are numerous reasons a system could be having issues, it can be difficult for an IT administrator to know where to start. Let’s take a concrete example with a user that calls into the help desk with a generic complaint.

The IT administrator receives a complaint from a user who says his system is running too slow. The administrator begins by targeting the user’s system in SysTrack Resolve. They start searching for the user in the General: Change Focus tab and selecting the appropriate system. The user can potentially have multiple systems, however, the online system is most relevant because it’s the system they are currently using. The administrator continues on to Analytics Overview, where they notice some applications and events in critical state. However, due to the immense amount of time that has passed since they were stated as critical, they can be reasonably ignored for this situation. The administrator decides to check the health of the system under the Health tab of Resolve because it will provide a summary on any source of impact on the system. They notice the Total Impact graph (displayed below) in the summary section highlighting how many minutes of impact each category has on this system.

The administrator takes note that the network category has the highest total impact on the system but they decide to continue to look at the Quality Trend graph. It displays the quality of the environment per day compared with the average quality. This allows them to be able to see how the system was performing for the end user compared to how it performs on most days. They notice that it begins to go towards a downward trend on the day that the user started to notice the poor performance.

The administrator notices that the network seems to have the highest total impact overall, but decides to check out the daily impact chart to see the day that the user started noticing a slower performance. As displayed below, the administrator notices that the day the user started noticing a poor performance shows a high network impact and continues to show a high network impact.

The administrator now knows the source of what is causing the largest impact in the user’s system and can use that knowledge to fix the decreasing quality of the user’s system and the environment overall. They continue further into Resolve’s Black Box and looks closer at the network system data. They choose to switch the filter in the System Data panel from System to Network since this is the category they are most concerned with (displayed below). This reveals further details on the network interface cards such as a high retransmit rate resulting in the large network impact.

The IT administrator discovered the source of the impact for that specific user, but is wondering if other users are also having this same problem. They dive into Visualizer Enterprise and look at the health for the overall environment. The health appears to have good user experience across the board and the administrator concludes that this situation was only applicable to that one user. Finding the largest source of impact for an environment is just one of SysTrack’s many uses to promote a successful environment.

Software Asset Optimization with SysTrack

Workplace analytics encompasses a vast amount of end user computing related information collected from a variety of sources, and a vital component of the topic is the observation of software assets. Obviously, a broad topic, we’ve chosen to break that further into three key categories: performance, usage and dependencies. Software performance monitoring is driven by the need to understand how well applications are working in the environment. Software usage is predicated on the idea of optimizing licensing and delivery to provide necessary applications. The last category, Dependencies, is vital to understand what pieces are necessary for software to function.

Software performance is itself a complex topic, but broadly the idea is to identify the answer to key questions like “why does my application keep crashing?” and “what applications take the longest time to load?” This incorporates key metrics like resource consumption details (CPU, memory, IOPS, network bandwidth) as well as number and frequency of faults or hangs. In many ways, this is one of the first items thought of in the context of software asset analytics, and it’s often one of the first things an end user notices about the environment. Diagnosing performance issues and understanding the resource consumption for the average user can help steer hardware requisition and delivery methods. Clearly, though, a preliminary question in many cases is exactly what packages belong in the environment.

Accurately observing software usage can be invaluable to a company. The ability to know which applications are used versus installed directly relates to the distribution of licenses, and that’s a direct cost driver. Another consideration is support cost savings made possible by making images less complicated. Intrinsic to rationalization is a host of potential ways to make sure that the delivery of applications to end users is as closely tailored to their needs as possible. There are some technical considerations to this as well, not the least of which is exploring the components or backend connections required for software in the environment.

Gaining insight into the required components a given package needs to function can be very important to choosing appropriate delivery mechanisms and options. Application compatibility concerns driven by incompatible components, fundamentally unsupportable system components, and complex networking requirements are all key to understand. Identifying what applications call on to function on a day to day basis dictates many of the decisions IT need to make to modernize and continually innovate with their delivery options.

We’ll be going into more depth on each of these categories as we release our upcoming Software Asset Analytics Kit. With each area, we’ll expand on some real use cases and provide some real-world examples of how each provides essential information for an environment.

Digital Experience Management and Event Correlation with SysTrack

SysTrack provides the ability to score an environment’s end user experience using digital experience management metrics. The resulting end user experience score provides a clear view of the end user’s experience in that environment and is composed by a series of Key Performance Indicators (KPIs). These KPIs are structured to provide direction to any problems in the environment that may affect the end user. The key to the philosophical approach with SysTrack, though, is the joining of this scoring to event correlation and analysis through the use of proactive alerts. These proactive alerts tie that overarching score to triggered, targeted events to provide a fuller and easier to understand portrait of the IT environment.

This starts with our end user experience score, and it’s best thought of as a simple grade. Basically, the score comes in a range of 0 to 100, with a score of a 100 implying the environment is providing the best end user experience. The score is composed of 13 different KPIs that represent aggregate data points on potential sources of impact. These  roughly fall into the categories of resource issues, networking problems,  system configuration issues, and infrastructure problems. This results in a great, normalized score to understand across a broad set of different systems what kind of performance issues are occurring. Even more importantly, it provides a platform for long-term analysis for trending to see the trajectory and evolution of that experience over time. The image below displays an overall view of the end user’s experience of the environment and the ability to monitor the evolution of those impacts over time. 

For more operational items that require an immediate response the alerting mechanism comes into play. Alerts are an active component that are triggered by events generally correlated with negative impact. Alert items roughly correlate with the end user experience score KPIs to help further IT administrators’ direction towards resolving problems. The image below demonstrates an environment with active alerts.
The key piece is correlating these items to that impact in a meaningful way. So, the big question is this: how do they work with one another?

One of the most common ways alerts and user experience scores are used in conjunction is through a feedback loop.  An administrator determines which KPI is causing the largest source of impact and continues to drill down providing a clear view of placed and potentially triggered correlating alerts. The alerts will direct the administrator towards the underlying causes of the problem and finally to the potential source of impact. After the resolution the administrator can track the increase in user experience as a result of their improvements to see how successful their changes have been.

End user experience scores provide an overall indicator of the quality end users are experiencing, while alerts provide immediate information on the source of impact. The integration of both tools provides an easy and clear way for IT administrators to discover the source of a system’s impact. To learn more on this topic check out our white paper, Utilizing KPIs and Alarm Methods!