What would it mean to your retail business if you could test changes to your global IT systems overnight? What sort of competitive advantage would that give you?
This isn’t a theoretical question for some retailers: they’ve already benefited from testing automation which has significantly reduced the time it takes to fix bugs, enhanced their systems, accelerated development projects and decreased cost.
In this article, we talk to Antony Kaplan, test director at REPL, about the advances that are being made in the test automation arena.
Faster Development Methods Create Pressure on Testing
As product development adopts more continuous, streamlined methodologies, there has been a fly in the ointment – slow and laborious testing phases. This has meant significant proportions of time have been taken up by testing when the rest of the process is slimming down and speeding up.
This entire process is influenced by the type of technology platform involved. For example, a custom-built ecommerce platform with self-contained boundaries and a micro-serviced architecture lends itself to a testing driven development underpinned by extensive automation. This scenario leads to a very fast release cycle, measured in hours or days.
However, on other platforms, such as monolithic POS application via a third party package software company, a set of restraints will inhibit this speed. While not making automation and new development methodologies impossible, it does mean more innovative ideas are required to make it work.
Of course, high quality testing is an essential part of any product release. Without it issues slip through the gaps, running the risk of creating problems for organisations and their customers, and more work in the long run. As automated testing advances at pace, we decided to catch up with Antony to see how these new tactics are helping retailers and their customers, and where the drive for change is coming from.
Making the Move From Manual to Behaviour Driven Testing
“We realised we needed to implement automation because customers needed to get products out quickly but still with excellent quality,” says Antony. “This has led us down the path of reducing testing timescales and time to market while maintaining resource and reducing costs to our customers.”
The key to this transformation has been a shift from manual testing scripts to behaviour driven development (BDD). Instead of writing out test scripts step by step, a BDD approach tests the behaviour of a solution which can then be easily automated.
For example, a manual script would say: sign on, enter operator or user number, then steps detailing how to run the script with an expected result at the end. In comparison, BDD works by stating: when I’m signed on and I complete a receiving function I expect a certain behaviour to happen. Instead of writing down every aspect of a scenario you’re telling the developer in test that in x situation, your expected result would be y.
By shifting from highly detailed scripts that need to be manually completed step by painful step, BDD scenarios should be a maximum of five lines that can be automated easily. This might sound like a simple shift, however, the results are spectacular, as Antony explains: “By automating 90% of the testing for one global retailer’s JDA back office solution, we expect their testing effort to reduce by 90+%.”
Enormous improvements like this enable retailers to test overnight which means they can move from bi-monthly or quarterly to monthly releases. And because automated scripts are so quick to run – typically around 30-60 seconds – this opens up the opportunity to complete the testing phase even faster.
“By the time you’re automating thousands of scripts,” says Antony, “It’s possible to run them overnight across multiple countries in parallel. This means customers will get the test results the very next day enabling development project timescales to be significantly reduced.”
Faster Testing + Quality Results = Competitive Advantage
The other major benefit of automated testing is that customers can be confident they’re getting quality results. “Because the testing is automated,” says Antony, “The system will always run the same tasks, it won’t make mistakes and it doesn’t get tired. And when there are defects, they can be raised the very next day instead of over a two week period for example. This allows them to be dealt with very quickly, accelerating the product development cycle further.”
For retailers, this means enhancements can be rolled out far more quickly. Whether that’s defect fixes, the introduction of a new promotion engine or any other system enhancement; the faster retailers move, the more competitive they are. And it’s testing automation that’s driving this improved experience for REPL’s retail clients and their customers.
Combining Robotics With Test Automation
Another interesting area of development is the introduction of robotic card testing with automated testing. Antony believes REPL is the only technology company that’s cracked this area: “No-one’s pushing testing automation like REPL. We’re the first to automate payment, fuel and loyalty card testing and this has been integrated with POS, back office and online systems-integrated regression testing. This means REPL can provide the complete, fully automated, end-to-end piece.”
What happens if retailers don’t use end-to-end testing automation? What are the risks? The main problem is that the gaps between the different stages require manual testing to be carried out. This defeats the purpose of moving to automated testing and adds time and a greater risk of error back into the development cycle.
Robotic card testing enhances automated retail testing by removing the need to manually test cards which increases the speed and accuracy of the testing process. Let’s unpack this a little. When testing JDA back office systems, a point of sale transaction is needed to create a back office extract. In the past this meant someone would have to make a payment manually. However, now automated robotic card testing has been introduced bringing automation to this stage.
This hasn’t been an easy achievement. For the robotics to know when to introduce the card, the system has to be programmed to recognise the terminal status using the terminal’s logs. We are now able to hook into the terminal logs to dynamically extract every status as it happens. Within 1 millisecond of a status appearing on the terminal screen, (insert card, insert pin, remove card etc.) we can send that command to our automated card robot to action that command immediately.
To achieve these results requires close collaboration with the main software vendors who have to support achieving these goals with their end customer.
In the future, REPL is considering additional technology – optical character recognition (OCR) – to save even more time and to validate where text and pictures are on the terminal screen.
To automate the POS terminal steps, knowing what the terminal is doing at all points of the transaction is key. The OCR kit uses a camera above the card terminal to read the status on the screen. It takes lots of pictures and as soon as it captures and understands the status it communicates with the robotic testing equipment. This enables the relevant action to be taken, for example checking that a specific logo is correct or entering the PIN.
Testers who aren’t using this technology must assume how long it will take for each status to change. This results in time being built into the testing script. However, this could mean the robot is waiting around adding unnecessary time to the testing script.
With this technology, the robot can react in real time to the commands on the card terminal. “From our terminal log or OCR implementation, we’ve seen that we can reduce each test by about ten seconds,” says Antony. “When you’re running 4,000 test scripts across a project this is a huge time saving – over 11 hours.
How Testing Automation is Enhancing Supply Chain
Another area that testing automation is about to significantly speed up is supply chain systems development.
“To automatically test changes to the web platform, we’re introducing an emulator – an automated version of warehouse operatives’ handheld devices – that picks stock on the handheld terminal,” says Antony. “The menus can be reviewed automatically, stock picked and BDD scenarios used to check the stock has been instantly reduced on the web platform.”
Although this has passed proof of concept stage, this technology hasn’t yet been tested in a real-life environment. However, early indications show there will be a massive time saving for retailers’ testing.
What’s Driving Testing Automation Forward?
As technology continues to move at pace, speed is of the essence. Not just in terms of reducing the time it takes to test new products or enhancements but in terms of moving the technology forward that enables this to happen.
Key to this acceleration in testing automation methodology is a willingness to partner with other experts. As Antony recognises: “We’re continuously working with our partners to find ways to get around any blockers that emerge. There will always be a way to automate, it’s about finding the right tooling for each solution and being very partner oriented. This helps us move test automation forward in ways we wouldn’t be able to otherwise.”
It’s this kind of approach that’s accelerated test automation, making it an area to watch for the future.
Take your testing to new levels. Get in touch with REPL’s expert testing consultants on +44 (0) 808 200 7375 or at firstname.lastname@example.org today.