Welcome!

Mobile IoT Authors: Mauro Carniel, Rostyslav Demush, Pat Romanski, Elizabeth White, Liz McMillan

Related Topics: @DevOpsSummit, Mobile IoT

@DevOpsSummit: Article

Testing Tips For Today | @DevOpsSummit #DevOps #IoT #Microservices

HTML5 and CSS3 together were one of of the biggest transformations to happen to web development

Testing Tips For Today's Tech: HTML5, WebSockets, RTMP, Adaptive Bitrate Streaming

One of the most dangerous things that can happen to a development team is complacency. The modern world moves so fast, and new technologies are coming out all the time. If you stop innovating and stop adapting, you'll be sunk. It's critical for web developers to be continuously responding to the changes around them.

The web community is constantly churning out new technologies that make it easier to develop the applications that savvy users are demanding. It may be obvious, but new technologies often require new ways of thinking about testing - sometimes new tools, other times entirely new testing methods. You just can't rely on the same old same old when things change as much as the web does.

This means that you, as a performance tester, must constantly be preparing for the latest the web has to offer. We thought we'd provide some tips on how to conduct performance testing on newer technologies, so you can stay one step ahead of the curve.

HTML5 + CSS3
HTML5 and CSS3 together were one of of the biggest transformations to happen to web development in the past decade. With the introduction of these W3C specifications, finally a common standard for rich media and web interactivity was put in place that applied across all browsers on desktops, laptops, set-tops, and mobile devices.

Since most of what happens in an HTML5 application takes place on the client, much of the performance testing job comes in the form of end-user experience testing. You want to know that your client devices can handle all the processing involved in a high fidelity HTML5 app. For example, here's a processor-heavy client-side HTML5 app that you can use for benchmarking the client-side experience across different devices.

There are, however, some implications of HTML5 that do matter when viewed through the lens of server-side scalability and load testing. These are:

  • The complexity of the app. HTML5 apps are often very dynamic, so they don't necessarily all load at once. Make sure your test scenarios fetch all the necessary sub-components, are running correlated scripts, and are exercising AJAX calls appropriately. These will matter under load.
  • HTML5 also supports server-sent events, so you'll want an infrastructure that can support these events with a normalized method for receiving various types of push notifications and server-sent messages.
  • Finally, you'll want a good way to record traffic back and forth from the server and then replay those communications as a scalable series of simulated users, with the ability to insert randomized dummy data into each app instance. The richness of an HTML5 app can make these traffic patterns more complicated than a legacy app would typically be. You'll want the flexibility in your testing infrastructure and scenario design to fully exercise HTML5 as a realistic simulation of load.

Ultimately, HTML5 has new communication features that you need to accommodate and a higher degree of potential complexity in the apps that are built on the platform. But you'll still want to see all the performance metrics and insights that you do with classic HTML application performance tests.

WebSockets
WebSockets is actually part of the HTML5 specification, but nonetheless it deserves its own callout. WebSocket technology creates a persistent connection between the client and the server. That way a server can push info to a client without waiting to piggy-back on an incoming HTTP request.

As the web grew more and more interactive, companies started using browsers as a platform for low-latency and real-time apps, like running a game, or real-time communications, or notifications. However, since HTTP required every communication to start with a client request, this forced developers and toolkits to create workarounds that simulated server-pushed events, while still fitting in the request-response framework. These solutions worked well, but were inefficient.

With HTML5 and WebSockets, that process became highly efficient. So WebSockets became a great way to push information from the server to the client. However, because they are so speedy and don't require a new connection to be established any time information needs to be communicated in either direction, web developers often use WebSockets as a way of maintaining a fast channel between client and server, even for traditional request-response purposes.

When it comes to performance testing, the key to testing WebSockets is simulating the bi-directional nature of your app. Test scenarios aren't always one-way. They aren't always request-response. They aren't always server-pushed. Applications that employ WebSockets often contain a mix of communication patterns. To build your load test scenarios you'll want to record and playback WebSocket communications with your app to create realistic testing scenarios. You'll also need to handle messages pushed over WebSockets just like you would handle messages pushed using a traditional request-response, piggy-back architecture. Load test variables should include the time it takes to establish a WebSocket connection, as well as the time it takes to send a request over that connection. Finally don't forget to include tests for both text and binary data.

One more thing to keep in mind - a lot of tools designed to test WebSockets are only built to handle push notifications, but the way modern apps are developed, a lot of request-response apps leverage WebSockets anyway. Make sure your load testing tool can deal with both communication protocols.

If your application doesn't take advantage of WebSockets, and if you care about performance, consider checking the technology out. WebSockets are a great way to boost your interactive applications.

Real-Time Messaging Protocol (RTMP)
RTMP was originally developed for streaming audio, video, and data over the Internet, between a server and a Flash player. It was created by Macromedia (now part of Adobe) and was later released as an open specification that's commonly used for Flash and Flex/Air applications. The protocol supports AMF, SWF, FLV, and F4V file formats.

Today, most people agree that the video support included within HTML5 will reduce the need for these file formats. However, because there is so much video out there, and much of it is not HTML5-compliant yet, these traditional formats still carry a lot of weight.

Like WebSockets, RTMP creates persistent connections between a server and a client application written in Flash or Flex/Air. The technology is used to reduce the overhead involved in establishing and tearing down connections for low-latency or highly-interactive apps. Also like WebSockets, performance testing for RTMP is typically focused around the optimizing data that's pushed from the server to the client.

What's unique when testing RTMP apps is that you need testing tools that have the RTMP standard built-in. If your app employs this protocol, you'll want to develop test scenarios that really exercise it. However, RTMP isn't part of the native HTML5 package the way WebSockets is. Your load testing solution should incorporate RTMP directly, and provide a lightweight way of creating virtual users, recording RTMP traffic, processing it appropriately, and playing it back in a realistic way. Your test scenarios will also need to be able to understand and process events that come from the server in order to simulate a diverse and active user population.

Adaptive Bitrate Streaming
All the technologies listed above focus primarily on interactive apps and bi-directional communications between servers and clients. Adaptive bitrate streaming is altogether different. Built directly on HTTP, adaptive streaming detects a user's bandwidth and CPU capacity in real-time, and adjusts the quality of the video stream accordingly. That means you get a different data stream if you are watching a movie on your 4" phone than you would get on a 27" monitor at your desk.

Bitrate streaming has become important because it's a way to reduce buffering and load times for videos. It's all about being able to serve video content immediately, the moment the user asks for it. That's the reason you no longer get those annoying "waiting for video to buffer" messages any more.

The thing about adaptive bitrate streaming is that the interaction between the client and server is very complex, so that makes it a difficult thing to simulate in load tests. Plus there are a lot of different kinds of streaming - that adds a whole other layer of complexity. So you need a load testing tool that first can handle a wide variety of streaming technologies like MPEG/DASH, Adobe Dynamic Streaming for Flash, Apple HTTP Live Streaming, and Microsoft Smooth Streaming.

Second, your testing software should be able to handle a large library of videos. Usually streaming apps involve users accessing a broad set of content. To accurately simulate their behavior, you want to avoid having the same videos get served over and over again. You need to be able to fully exercise the behavior of the population as a whole.

Finally, video streaming is very demanding for a server, but it's not the only thing going on. Apps continue to function in other ways, concurrently while video is streaming. For example, ads get served, related content is browsed, or transactions get executed - all while that video is playing. You'll need to build this into your test scenarios as part of a realistic media load test.

Conclusion
We all know how important it is to stay on top of the latest trends in technology. At Neotys this is something we take very seriously. That's why our team is always looking to see what technologies web developers are using and building those capabilities directly into our products. Have questions about testing a particular technology? We'd love to hear from you.

Photo Credit: Susanne Nilsson

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

@ThingsExpo Stories
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Conference Guru has been named “Media Sponsor” of the 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. A valuable conference experience generates new contacts, sales leads, potential strategic partners and potential investors; helps gather competitive intelligence and even provides inspiration for new products and services. Conference Guru works with conference organizers to pass great deals to gre...
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, demonstrated how to move beyond today's coding paradigm and shared the must-have mindsets for removing complexity from the develop...
In his Opening Keynote at 21st Cloud Expo, John Considine, General Manager of IBM Cloud Infrastructure, led attendees through the exciting evolution of the cloud. He looked at this major disruption from the perspective of technology, business models, and what this means for enterprises of all sizes. John Considine is General Manager of Cloud Infrastructure Services at IBM. In that role he is responsible for leading IBM’s public cloud infrastructure including strategy, development, and offering m...
"Evatronix provides design services to companies that need to integrate the IoT technology in their products but they don't necessarily have the expertise, knowledge and design team to do so," explained Adam Morawiec, VP of Business Development at Evatronix, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
To get the most out of their data, successful companies are not focusing on queries and data lakes, they are actively integrating analytics into their operations with a data-first application development approach. Real-time adjustments to improve revenues, reduce costs, or mitigate risk rely on applications that minimize latency on a variety of data sources. In his session at @BigDataExpo, Jack Norris, Senior Vice President, Data and Applications at MapR Technologies, reviewed best practices to ...
Widespread fragmentation is stalling the growth of the IIoT and making it difficult for partners to work together. The number of software platforms, apps, hardware and connectivity standards is creating paralysis among businesses that are afraid of being locked into a solution. EdgeX Foundry is unifying the community around a common IoT edge framework and an ecosystem of interoperable components.
Large industrial manufacturing organizations are adopting the agile principles of cloud software companies. The industrial manufacturing development process has not scaled over time. Now that design CAD teams are geographically distributed, centralizing their work is key. With large multi-gigabyte projects, outdated tools have stifled industrial team agility, time-to-market milestones, and impacted P&L stakeholders.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
"IBM is really all in on blockchain. We take a look at sort of the history of blockchain ledger technologies. It started out with bitcoin, Ethereum, and IBM evaluated these particular blockchain technologies and found they were anonymous and permissionless and that many companies were looking for permissioned blockchain," stated René Bostic, Technical VP of the IBM Cloud Unit in North America, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Conventi...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
"Cloud Academy is an enterprise training platform for the cloud, specifically public clouds. We offer guided learning experiences on AWS, Azure, Google Cloud and all the surrounding methodologies and technologies that you need to know and your teams need to know in order to leverage the full benefits of the cloud," explained Alex Brower, VP of Marketing at Cloud Academy, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clar...
Gemini is Yahoo’s native and search advertising platform. To ensure the quality of a complex distributed system that spans multiple products and components and across various desktop websites and mobile app and web experiences – both Yahoo owned and operated and third-party syndication (supply), with complex interaction with more than a billion users and numerous advertisers globally (demand) – it becomes imperative to automate a set of end-to-end tests 24x7 to detect bugs and regression. In th...
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...
"There's plenty of bandwidth out there but it's never in the right place. So what Cedexis does is uses data to work out the best pathways to get data from the origin to the person who wants to get it," explained Simon Jones, Evangelist and Head of Marketing at Cedexis, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
SYS-CON Events announced today that Telecom Reseller has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5-7, 2018, at the Javits Center in New York, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...