Month: October 2018

Part 2: Applying new testing methods to the latest technologies

In the first part of this blog post series, I outlined How the process of testing is accelerating. In this second part, I am excited to share some ways we are applying these testing methods to various digital technologies and use cases.

Microservices and APIs

Traditionally, the primary focus of any test automation approach was around front-end user interface testing. However, with large monolithic applications now being broken down into microservices in pursuit of efficiency, each (micro) service must be functionally tested as a whole to ensure that the hand-shaking between two services is maintained and validated. That means API and service test automation are in high demand and are areas in which we do a lot of work. It’s always a unique challenge as you help a company modernize, and we often have to use service virtualization to create mock services to simulate functionality that hasn’t been developed yet or to replicate components or functionality that are hard to access in a live environment. All of this increases confidence in the quality and helps us test – and then lock down – functionality. (We will then look to automate the test in a subsequent phase).

We are also continually giving APIs a good testing workout – whether we have inherited them, are integrating them, or are designing them ourselves – using our version of globally-accepted Best Practice standards. API maturity is now expected from all clients, and we help new clients adopt those standards quickly.

Device Testing

In the era of digital transformation, new technologies and competitive pressures have forced companies to innovate to give their customers the best experience possible. To deliver those experiences, we need to test products across all the devices and channels which will consume them. In days gone by, we had drawers full of mobile devices from all over the world, but now we make use of advances in Device Farms and Emulators.

The emulators that our developers and testers use locally are far better than those I remember from the early days, which were more testing in hope than expectation. But now, when running SMOKE and Regression tests, we use a cloud-based Device Farm like SauceLab (the world’s largest) to give us results back on the tests we have designed for a long list of variables (new and old devices, form factors and operating system configurations).

Open Source

Open source tools have always been the choice for non-production testing and are more easily embraced by our clients than for testing legacy production (i.e. live and still working) platforms. At Ness we are product neutral, so we are enthusiastic supporters of all the evolving platforms that make free community tools available to those like us building next-generation digital propositions. We practice what we preach, and we won’t adopt paid-for licensed testing tools unless it’s an imperative within an engagement. We use open source tools whenever possible for non-production testing.

For live platforms of a certain vintage, there is an understandable reticence to modernize something that might be slow and cumbersome but is still working; in that scenario moving to open source tools occurs alongside some more significant and urgent improvement.

Big Data and IoT engagements need specialized testing

As organizations adopt Big Data and Data Analytics, they are setting up data lakes and processing huge volumes of data that have been collected from various sources, including Internet of Things (IoT) sensors. Unlike traditional database testing (where all the data is structured), the data in a Big Data lake is usually structured and unstructured. In order to accommodate testing in this new world of enormous data volumes, the provisioning of suitable test environments is both essential and challenging.

Infrastructure testing and testing data before it even gets to the storage environment is yet more of the “shift-left” mentality, which means we are planning the test strategy early in the process so that everything that follows is more efficient and manageable. Knowing what you will be testing, what you are looking to find out from the test, and what success looks like are all factors in consolidating a test strategy before you fill a data lake. We don’t want to test all of the data. We want to test the useful data, the outliers, the changes. We want to find patterns that reveal insights. We want to test less but test it smarter and more intelligently.

Key questions we need to answer are: How do I get the data into my development or test environment in the first place, and do I have enough visibility to be able to maintain quality with such a huge volume of data involved?

The Internet of Things is one of the fastest growing technology areas and has created another area in which testing has become a key factor. IoT’s mesh of various inter-connected “things” over the Internet creates complexity. With this complexity, it becomes imperative that each of these distinct entities is thoroughly tested for functionality, quality, performance, security, and accessibility, and that now inextricably covers the hardware and the software components of the IoT. We need to do that with the same testing maturity and coverage that we have in normal everyday product development.

Together, Big Data and IoT challenge us with a new dynamic in the variety of forms and structures. Everything in this field needs organizing differently and more astutely than in less sophisticated times to generate value. This is a change in technology and business mentality.

Security

Of course, data security is a business-critical issue. With companies facing the changing nature of internal and external threats, Ness partners with pure specialists in this field where security is of paramount importance. The time, energy, effort, and cost of staying ahead of this field are intense, and we always want to pull in the most knowledgeable resources.

Service Design and Business Process Assurance

This modern approach is to truly step into the shoes of a modern user and try and accomplish something they would do. This is where we are using Service Design techniques to modernize what was previously known as Business Process Assurance. Traditionally this has been done by business analysts on the client side to map out the workflows they use within the company.

However, incumbent (some might say legacy) businesses are tackling digital transformation and having to change the way their processes work. To be more customer-centric and much less attached to their own internal, silo-based workflows (to which customers understandably react badly), companies are increasingly open to new workflow design and tests from partners like Ness.

Certain steps within a single flow might well be used by many other workflows, so it’s not easy, and we need to demonstrate expertise with coolness under pressure, sharp thinking and a steady hand. We need to keep testing ALL the related flows to keep the speed and quality assurance in place. If one small change (improvement) is made, it needs to be tested across the flows with a view to all business function users being able to understand the change and enjoy the improvement.

This is a new wave of sophistication (more intelligent, agile “shift-left”). Ness is part of the wave and is helping clients with their internal transformations in pursuit of digital excellence.

In summary

Across all of our digital transformation engagements, Ness helps customers adopt new methodologies, including transitioning from traditional to Agile development methodologies for those who haven’t completed that part of the journey. We set about implementing continuous testing with modern, Quality Engineering methods and frequently embed the DevOps tool chain to improve predictability indicators of product quality and increase the occurrence of hassle-free product releases. Testing is a crucial factor in a company’s digital success story.

In our pursuit of simplicity in a technology world of increasing complexity, we have consolidated all the knowledge, experience and learnings set out in this overview into NessQ, which is a framework for building automated test cases. It is an accelerator to jump start test automation capabilities and is an integration of world-class open source tools for web, mobile, API, accessibility and security testing.

We frequently win new business because we can demonstrate such a solid approach to building and testing digital propositions. Proactively suggesting ways to fix failed tests is something we are renowned for, and I get plenty of calls from non-client CIOs asking me how quickly Ness could put in place a better test strategy than one causing stress somewhere in the world. Sometimes, fixing a testing strategy can be a high-impact, lower risk entry point for companies that are pushing digital transformation into other parts of their business.

Either way, I feel pride that our team has been through this whole journey many times. We are always looking for ways to further the modern test strategy. Whether it’s been global leaders battling to efficiently integrate data from numerous, organically acquired Learning Management Systems, or digital pure play companies determined to keep improving test efficient by 1% every month to reach zero defects, we have learned the lessons – and continue to learn new ones every day. We enjoy sharing them in new conversations springing up from unique and unexpected sources all the time.

Part 1-How the process of testing is accelerating

It is self-evident that technology is advancing quickly. Every company wants to gain as much commercial value as possible from these advancements, and the expectations of both employees and customers are that they should have access to “the latest” tools. Every company has to decide which new digital technologies to adopt to satisfy this demand. The technology products companies develop and launch are integral to their futures, but they must also be inextricably linked to a rigorous approach to testing and Quality Assurance (QA). Confidence and capability in one without the other is potentially disastrous, so testing processes are now as high profile and critical as the engineering processes.

In the era of Microservices, APIs, Big Data, Mobility, IoT, Blockchain and AI, let’s take a look at how and why these new movements affect testing and how Ness is addressing them.

In Part 1 of this two-part blog post series, I’ll share some observations about how the testing process itself is changing to support faster release of digital platforms and products to market:

QA is changing

In the testing domain, I see the industry becoming more focused on Quality Engineering (QE) versus just Quality Assurance.

Back in the formative years of Agile, we got used to two-week sprints for development followed by two weeks of testing the code in the next sprint. There was a real split between rudimentary “what works/doesn’t work” testing of the code and then, much later on, testing for performance, usability, security. It was quite linear and waterfall-like and not at all in the spirit of the Agile manifesto which focused on “early and continuous delivery of valuable software… with a preference for the shortest timescale.”

Quality Assurance (QA) is really a remnant of the waterfall development lifecycle era and geared toward long delivery cycles with a natural inclination to “test when done.”

With the modern urgency to release products and applications faster, traditional QA has become an impediment to product companies because the process is too slow and QA frequently becomes the main bottleneck in the process. Working out where to gain efficiency led to the emergence of Quality Engineering (QE).

QE is taking QA’s place

Where Quality Assurance is the overall process (the quality system) of ensuring a final product meets specified requirements, QE is the approach used to define, maintain and improve that system. It is a more ambitious, prominent, and muscular version of what has gone before, and it signifies a monumental “shift-left” where the subject of testing and quality arises much earlier in the timeline of software creation.

QE’s emergence has everything to do with the pursuit of speed in getting new features to market with confidence. Quality Engineering is all about managing testing much earlier: test often, test fast, fail fast, fix fast and automate as much as possible to speed up testing even more.

QE’s role in testing today

By testing earlier, things have become more agile and much smarter. We now test smaller packets of completed code and are therefore able to make adjustments more economically than in the old days when it felt much more like we were testing everything all the time.

With modern approaches like Test Driven Development (TDD), Acceptance Test Driven Development (ATDD), and Behavioral Driven Development (BDD), there is an increasing benefit to surface QA much earlier in the development life cycle. QE delivers that benefit with a fixed eye on increasing efficiency through the automation of tests–and isolating what has been tested, so we don’t need to test it again.

Now that we are testing as we are developing, our automation test suite is growing all the time. This means that at the end of every sprint, we can now run an entire automation program to see if anything has broken: we have automated regression testing. Automation points towards increased certainty and quality predictability that everything still works. It removes the frailties of manual testing, which sometimes caused confusion when changing things that already passed testing.

A New Type of Engineer Emerges

With the new emphasis on testing (and test automation) much earlier in the cycle, there is a growing demand for a new kind of engineer: a Software Development Engineer in Test (SDET).

SDETs are developers themselves. They understand code and development practices and have the technical mind set to write, run and automate test cases. Crucially, they also have the skills to find reasons for failed tests AND adjust the code to fix the failure. I can tell this new role is helping Ness scrum teams move with even greater speed and efficiency.

This is a new career path for test engineers to follow. In training and building out our SDET capabilities, we are cross-training test automation engineers, so they are able to automate processes for thorough testing and develop code on their own. They are playing a contributory role in the creation of all our new software. We know we are in esteemed company, because SDET was originally a Microsoft term and is now popular with Facebook, Amazon and Google as they strive to release at speed with high quality.

As was the case with Agile and then DevOps, Ness has always been on the leading edge of implementing practices that facilitate faster, quality software releases. It’s a priority for us because we know speed and reliability create an advantage for our clients when they release new products. Over the years, we have also gotten very good at training and scaling new capabilities across our engineering teams, so we can deploy new practices quickly. In Part 2 of this blog post, I’ll share perspective on how we’re using QE approaches to support various development opportunities in IoT, open source, and service design, to name a few.

Click here for part 2