Brief: Take Application Performance To The Next Level With - Soasta

Brief: Take Application Performance To The Next Level With - Soasta

For Infrastructure & Operations Professionals Brief: Take Application Performance To The Next Level With Digital Performance Management Performance M...

125KB Sizes 0 Downloads 10 Views

For Infrastructure & Operations Professionals

Brief: Take Application Performance To The Next Level With Digital Performance Management Performance Management: The Modern Service Delivery Playbook by Milan Hanson and James McCormick February 19, 2016

Why Read This Brief

Key Takeaways

Today’s digital businesses and their customers need I&O professionals to go beyond problem solving to deliver an excellent customer experience. Operations teams are seeking something more than application performance management (APM) as we know it — and some vendors are starting to provide it. Managing today’s customer experience requires tighter alignment with business requirements and everchanging customer needs and expectations. This report tells I&O pros how to go beyond APM toward digital performance management.

Digital Performance Management Is An Evolutionary Step Beyond APM Digital performance management is APM with a companywide perspective. It goes beyond APM by managing performance in a customer and business context. Understand The Customer To Understand Performance Good or bad performance depends on the customer’s expectations, which can be influenced by a range of factors such as geography or even by the customer’s stage in a single transaction. Adopt A DPM Approach For That Competitive Edge DPM will subsume APM, providing better insights that lead to action that is more finely tuned to business improvement.

forrester.com

For Infrastructure & Operations Professionals

Brief: Take Application Performance To The Next Level With Digital Performance Management Performance Management: The Modern Service Delivery Playbook by Milan Hanson and James McCormick with Laura Koetzle, Eveline Oehrlich, Michael Facemire, Julie A. Ask, and Megan Doerr February 19, 2016

Current APM Approaches Fail The Business Today’s APM solutions fall short of business needs because they don’t adequately equip I&O to work with or quantify its importance to the business. Even with APM, I&O pros focus on technology performance and availability while developers monitor for efficient transaction navigation; marketers monitor page visits; and market researchers gauge customer sentiment. No one sees the big picture. APM still fails to consistently support business needs because today’s practices: ›› Operate in data silos. Performance is only one variable impacting the experience and behavior of customers using your site or applications. Other factors such as delivering the right content, delivering the right offer, or presenting a call to action also impact the customer experience. Yet performance metrics are kept separate from these other factors.1 Thus, the business is blind to the impact of performance on KPIs such as click-throughs, revenue generation, and life-time value. For example, the marketers for Star Wars decided to update the web page for The Force Awakens film just days before its premier.2 While the update may have increased visitor appeal, the site slowed down because the update increased the number of page objects. ›› Don’t understand the impact of performance on business. Businesses build applications to generate business value — to engage customers with content, to make a sale, etc. — not to deliver performance. Performance does impact these objectives, but quantifying the exact impact on the business is not an APM priority. A Forrester analytics survey found that 43% of firms believe that correlating multiple digital data dimensions is the most beneficial to their business.3 And yet firms do not consider performance metrics data for analysis alongside other digital data, such as visitor path and conversion data. ›› Fail to align performance resources with business needs. Modern digital application architectures are a complex mix of technologies that support many customer roles and use cases. Focusing resources on the right projects depends on knowing what customers want and need from each step in the application or website. For instance, customers’ sensitivity to a page’s response Forrester Research, Inc., 60 Acorn Park Drive, Cambridge, MA 02140 USA +1 617-613-6000 | Fax: +1 617-613-5000 | forrester.com © 2016 Forrester Research, Inc. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. Unauthorized copying or distributing is a violation of copyright law. [email protected] or +1 866-367-7378

For Infrastructure & Operations Professionals

February 19, 2016

Brief: Take Application Performance To The Next Level With Digital Performance Management Performance Management: The Modern Service Delivery Playbook

time differs based on how far they have moved through a transaction.4 Yet I&O professionals tend to treat all parts of the application equally and fail to focus issue resolution and performance improvement efforts where they are needed most.

Digital Performance Management Drives Business Outcomes Customer obsession unmistakably boosts revenue — Forrester estimates that a one-point improvement in a US wireless provider’s Customer Experience Index (CX Index™) score results in an additional $175 million in revenue.5 Tight coupling with customer experience, marketing, and eCommerce pros lets I&O pros react faster — and even be proactive — in meeting customer needs and delighting customers. I&O pros are shifting from monitoring infrastructure SLAs to including a basic understanding of technology’s impact on end user experiences. Thus, modern performance management approaches must likewise evolve into digital performance management (DPM) practices (see Figure 1). Forrester defines the practice of DPM as: Optimizing customer experience and business KPIs through comprehensive performance monitoring and analysis of technology, application, and business metrics.

© 2016 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. [email protected] or +1 866-367-7378

2

For Infrastructure & Operations Professionals

February 19, 2016

Brief: Take Application Performance To The Next Level With Digital Performance Management Performance Management: The Modern Service Delivery Playbook

FIGURE 1 The Evolution Of Digital Performance Management

1993-1999

The golden age of APM 2006-2014 2000-2006

2015-present

System performance management

1st generation application 2nd generation application performance management performance management

Digital performance management

• Consolidate network • Monitor and manage and system mgmt., user SLAs manage technical SLAs • Third-party technologies • Client-server software grow, such as CDNs, model entirely on-prem marketing, and media advertising • Firms pursue single pane of glass, single • Mainstream acceptance source of truth for IT of distributed digital applications, such as • ITIL-based ITSM suites websites and email are common • Event correlation pursued aggressively • Firms try to correlate events to user services: reduce volume, improve prioritization • Vendors build performance monitoring and management that measures impact on end users’ performance: fat clients and desktop browsers

• Dev and ops • Customer experience is cooperation: dev goes app performance plus agile, ops needs faster business factors repairs analyzed together; ultimate measure is • Mainstream adoption of smart devices and social business benefits or revenue, customer media, virtualization experience goes big, cloud gains momentum • Mobility still accelerating; complexity increasing; • Big advances in big data, microservices, customer experience and Internet of Things monitoring coming strong • Firms try to manage performance of mixed • Firms want predictive analytics that bind on-premises and thirdtogether performance party components and business data to • Vendors assemble produce actionable comprehensive APM insights for business suites via M&A or organic dev: broad back decisions; fine-tune performance per precise end monitoring, thirdcustomer segmentation party components, • Vendors near parity on customer BYOD APM, add machine monitoring, big data learning, predictive analytics analytics, automation, and integrated tools for DevOps; break down data barriers between DevOps and marketing, sales, and customer insight

DPM Practices Link Performance Directly To Business And Customer Outcomes For DPM to be successful, I&O professionals must work with developers, customer intelligence (CI) pros, CX experts, marketers, and eCommerce professionals to link application performance with KPIs that matter to the business. Take the lead from firms that blend business, performance, and operations insights with a DPM approach. For example:

© 2016 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. [email protected] or +1 866-367-7378

3

For Infrastructure & Operations Professionals

February 19, 2016

Brief: Take Application Performance To The Next Level With Digital Performance Management Performance Management: The Modern Service Delivery Playbook

›› Align performance management efforts with business needs. Today, modern application delivery chains impact performance across an array of devices, third-party infrastructure services, and complex in-house multi-tier infrastructures. APM detects and isolates performance problems in this technology jungle, but DPM also blends business metrics into the analysis. Splunk’s CTO Snehal Antani described a banking I&O team monitoring deposits in real time to see the business impact of performance issues. When deposits at mid-day were at $2 million instead of the typical $5 million, I&O launched an immediate investigation. Finding no performance issue, the I&O team notified the CEO and CMO, and they together discovered a competitor’s offer pulling away depositors. The bank launched a counter-offer and the teams watched deposits recover in real time. The bank’s CEO said that without this monitoring, they would not have noticed the problem until 30 days later.6 ›› Make customer obsession a companywide imperative. When you combine customer satisfaction survey results with classic I&O metrics, the whole firm gains insight. For example, right before its anniversary sale — its biggest event of the year — retailer Nordstrom saw declining customer satisfaction for online performance. Nordstrom used SOASTA mPulse to extend APM toward DPM, combining customer perception feedback with synthetic and real-user performance monitoring. Nordstrom broadcasted SOASTA mPulse data on televisions throughout its headquarters so employees could see key business metrics — especially conversions — as a function of response time.7 ›› Achieve customer-obsessed performance management. Customer sensitivity to performance varies depending on the customer’s commitment to the process. For example, T-Mobile used Dynatrace DPM capabilities to reveal that slow pages early in the transaction funnel (e.g., product pages) were abandoned more often than slow pages later in the process (e.g., checkout pages).8 Dynatrace’s DPM capabilities provide the data and analysis to substantiate such observations. Analyzing and differentiating the performance needs of disparate parts of the customer funnel allows I&O professionals to focus their efforts on what matters to the business. ›› Dissolve cross-team barriers. Shifting from APM to DPM solutions can improve cooperation among siloed teams. For example, DirecTV’s traditional APM solution was not being used by teams other than I&O, which complicated troubleshooting. After switching to AppDynamics’ Application Intelligence Platform, which provides a broader DPM view and advanced collaboration and automation capabilities, DirecTV application development teams began using AppDynamics on a regular basis, resulting in streamlined processes and improved DevOps collaboration.9 DPM Makes Performance Action-Optimized APM is primarily about reporting and reacting to performance problems, whereas DPM adds decision support for precise action — spending only the necessary resources to improve performance in the areas with the highest business benefit. To do this, DPM analytics takes a page from general-purpose analytics practices: It generates actionable insights that inform decisions.10 DPM practices deliver optimized actions and decision support by: © 2016 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. [email protected] or +1 866-367-7378

4

For Infrastructure & Operations Professionals

February 19, 2016

Brief: Take Application Performance To The Next Level With Digital Performance Management Performance Management: The Modern Service Delivery Playbook

›› Understanding the different performance needs of diverse customers. Performance may appear acceptable when averaged over all customers in aggregate, but if you analyze performance by different customer segments, I&O pros will find unpleasant surprises. For example, you may find that high-value customers experience terrible page-download speeds because of defective content targeted specifically at them. But page-download speed averaged over all customers — most of whom do not receive the defective content — will conceal this problem. DPM enables that kind of segmentation (and more) by including customer data not previously available to I&O professionals. ›› Delivering the right performance to the right audiences. Customer behavior and geographic location as well as factors like device type, operating system, and browser version impact the customer’s performance expectations and experience. Using the DPM approach, I&O professionals must expand their technology toolkits to consider offerings by vendors such as Maxymiser and SiteSpect. These tools use the above-mentioned customer and device attributes to associate different system components with different customer segments and thus optimize performance for each customer. What It Means

DPM Will Disrupt Both Your Analytics Norms And Vendors’ Businesses I&O professionals have been collecting application performance data, but it’s not a mature and stagnant practice. Thus, I&O pros and APM vendors should be ready to disrupt or be disrupted. Disturbances to “business as usual” will occur as: 1. DPM feeds digital analytics practices. Marketing and eCommerce teams that look to customer data and analytics to drive better marketing, customer experiences, and sale completion rates know that along with content, offers, and design, app performance also drives customer success. Thus, I&O teams that are ready with a DPM approach will become valuable business partners. 2. Technology vendors transition to DPM platforms. Existing APM vendors such as AppDynamics, Dynatrace, New Relic, and SolarWinds will grow toward DPM-style data integration and analytics capabilities to ingest business and customer data and merge it with native performance data for business-relevant insights. Or APM vendors may make performance data available to other digital analytics systems to do the same, feeding it to digital behavioraltargeting vendors such as Adobe, Maxymiser, and SiteSpect.11 Those firms will continue to invest in capabilities to ingest, integrate, and analyze performance data with customer and behavioral data to deliver a superior customer experience. 3. Performance insights become democratized. Marketing, eCommerce, product teams, and services teams will realize the value of performance in optimizing customer interactions, which will push I&O teams to deliver a DPM approach. Monitoring and analysis practices will transition from silos by data type (e.g., performance, business, customer) into a cooperative practice that

© 2016 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. [email protected] or +1 866-367-7378

5

For Infrastructure & Operations Professionals

February 19, 2016

Brief: Take Application Performance To The Next Level With Digital Performance Management Performance Management: The Modern Service Delivery Playbook

derives greater business insight. Thus, I&O teams will increasingly work with many different lineof-business teams to ensure that systems and applications can support performance within various marketing campaigns, CX projects, eCommerce changes, and other initiatives.

Engage With An Analyst Gain greater confidence in your decisions by working with Forrester thought leaders to apply our research to your specific business and technology initiatives. Analyst Inquiry

Analyst Advisory

Ask a question related to our research; a Forrester analyst will help you put it into practice and take the next step. Schedule a 30-minute phone session with the analyst or opt for a response via email.

Put research into practice with in-depth analysis of your specific business and technology challenges. Engagements include custom advisory calls, strategy days, workshops, speeches, and webinars.

Learn more about inquiry, including tips for getting the most out of your discussion.

Learn about interactive advisory sessions and how we can support your initiatives.

Supplemental Material Abbreviations Used In This Document APM: application performance management

I&O: infrastructure and operations

DPM: digital performance management

KPI: key performance indicator

CX: customer experience

SLA: service-level agreement

CI: customer insight

Endnotes “At many of the companies we’ve surveyed, nobody sees the big picture. Different people watch different parts of their web presence: IT tests a site’s uptime; marketers count visitors; UI designers worry about how test subjects navigate pages; and market researchers fret over what makes customers tick.” Source: Alistair Croll and Sean Power, Complete Web Monitoring, O’Reilly Media, 2009.

1

© 2016 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. [email protected] or +1 866-367-7378

6

For Infrastructure & Operations Professionals

February 19, 2016

Brief: Take Application Performance To The Next Level With Digital Performance Management Performance Management: The Modern Service Delivery Playbook

Source: David Jones, “Update: Countdown to Premiere of Star Wars, The Force Awakens: Digital Performance Is Essential for Related Sites,” Dynatrace, December 14, 2015 (http://apmblog.dynatrace.com/2015/12/14/only-4-daysto-premiere-of-star-wars-the-force-awakens-digital-performance-is-essential-for-related-sites/).

2

In parallel with its research into the Q2 2015 Web Analytics Forrester Wave™, Forrester surveyed 53 users of top web analytics platforms about their digital analytics practices. For more information on the current state of digital analytics practices, see the “Gauging Web Analytics Practices In The Age Of The Customer” Forrester report and Forrester’s Q1 2014 Global Web Analytics Forrester Wave™ Customer Online Survey.

3

“It may be acceptable to make purchasers wait 2 to 4 seconds after typing in their address and credit card number, but not at an earlier stage when they may be comparing various product features.” Source: Ian Molyneaux, The Art of Application Performance Testing: From Strategy to Tools, O’Reilly Media, 2014.

4

What’s a potential improvement in a company’s customer experience worth in terms of dollars and cents? To help answer this question, Forrester used data from Forrester’s Customer Experience Index (CX Index) to model the impact of CX on revenue potential by industry. See the “The Revenue Impact Of Customer Experience, 2015” Forrester report.

5

Source: “Keynotes from .conf2015,” Splunk (https://conf.splunk.com/speakers/keynotes/2015.html).

6

Source: SOASTA (http://www.soasta.com/customers/nordstrom/).

7

T-Mobile recently linked performance data from 370,000 web visitor sessions with customer order and customer service data. Source: “Going beyond page load speed to optimize customer experience,” Dynatrace (https://info. dynatrace.com/apm_all_cs_tmobile_going_beyond_en_fulfillment.html).

8

Source: AppDynamics DirecTV Case Study (http:// www.appdynamics.com/case-study/directv/).

9

Forrester calls the modern practice of digital analytics “digital intelligence,” and defines it as the capture, management, and analysis of customer data to deliver a holistic view of the digital customer experience that drives the measurement, optimization, and execution of digital customer interactions. See the “Supercharge Analytics With Digital Intelligence” Forrester report.

10

Behavioral targeting tools allow users to deliver tailored content, promotions, and functionality to visitors using rules or predictive algorithms based on visitor behavior, characteristics, and historical interactions. See the “Decipher The Digital Intelligence Technology Code” Forrester report.

11

© 2016 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. [email protected] or +1 866-367-7378

7

We work with business and technology leaders to develop customer-obsessed strategies that drive growth. Products and Services ›› ›› ›› ›› ›› ››

Core research and tools Data and analytics Peer collaboration Analyst engagement Consulting Events

Forrester’s research and insights are tailored to your role and critical business initiatives. Roles We Serve Marketing & Strategy Professionals CMO B2B Marketing B2C Marketing Customer Experience Customer Insights eBusiness & Channel Strategy

Technology Management Professionals CIO Application Development & Delivery Enterprise Architecture ›› Infrastructure & Operations Security & Risk Sourcing & Vendor Management

Technology Industry Professionals Analyst Relations

Client support For information on hard-copy or electronic reprints, please contact Client Support at +1 866-367-7378, +1 617-613-5730, or [email protected] We offer quantity discounts and special pricing for academic and nonprofit institutions.

Forrester Research (Nasdaq: FORR) is one of the most influential research and advisory firms in the world. We work with business and technology leaders to develop customer-obsessed strategies that drive growth. Through proprietary research, data, custom consulting, exclusive executive peer groups, and events, the Forrester experience is about a singular and powerful purpose: to challenge the thinking of our clients to help them lead change in their organizations. 128141 For more information, visit forrester.com.