Test Automation: Becoming Inclusive

Test automation practices frequently appear to reside in the hands of a small, specialized group of individuals within most testing organizations. Although automation tooling (e.g. macro engines, native code scripting, automation products) have been around for decades, many quality assurance teams continue to rely (sometimes exclusively) on manual efforts. While some organizations express that they wouldn’t be able to function without automation, the vast majority appear to be still bound by the efforts available through manual testers.

Test automation practices frequently appear to reside in the hands of a small, specialized group of individuals within most testing organizations. Although automation tooling (e.g. macro engines, native code scripting, automation products) have been around for decades, many quality assurance teams continue to rely (sometimes exclusively) on manual efforts. While some organizations express that they wouldn’t be able to function without automation, the vast majority appear to be still bound by the efforts available through manual testers.

In the past, I would have understood such constraints. Some automation options in the market are focused on specialized individuals that have a need for testing solutions and the skills of a developer. Newer tools (or newer releases of older products) have evolved to support automation coding in more visual and user-friendly IDE’s to combat this problem. Regardless of the tooling, there will of course be some individuals that are more suited to the demands of logic coding than others.

The challenge I pose to the test automation community is expressed simply… how do we adapt automation practices so that more people per organization may contribute to exercising automated test? As an inverse of this question, we should also ask how the efforts of a single automation engineer can be better leveraged to serve a larger number of testing needs.

The answer resides in our implementation of test automation. Implementation of automation is expressed by how an automation framework is developed and used in an organization. No matter the type of test automation tool or scripting employed, a test automation framework defines how that technology is put to use in the organization.

Traditional approaches (or initial approaches) to test automation started as linear coding whereby all of the actions for a test case or transaction were captured in a single script. Over time, this adapted to practices that were modular in design so that a driver (or parent) script called a subordinate (or child) script. Libraries could be formed from these modules of automation such that parent scripts could call child scripts that called grandchild scripts, and so on. These approaches grew to encompass logic that responded to input data (or data from applications) to determine which subordinate functions to exercise (e.g. if password-length=0, test for an error message to be displayed, else log into an application).  These approaches are employed across many organizations currently, and these approaches have served the software testing industry well over decades. These practices have also limited organizations such that only key individuals who have a development skill set can often comprehend the automation that has been build. This is an exclusive approach to automation that fosters specialists, rather than encouraging inclusion of automation in the service of all testers. It is now time for a new paradigm.

Newer approaches to test automation are developing across the quality assurance market.  These approaches include keyword-driven, behavior-driven, and state-driven frameworks. These framework styles each present various merits. However, they share some commonality that shift the automation perspective from an exclusive practice to an inclusive approach.
These frameworks implement automation by identifying that test automation may be comprised of multiple process areas:

  1. Automation Design
  2. Automation Implementation
  3. Test Case Design

In automation design, the elements or behaviors in an application are identified where automation is to be applied. Each transactional element (or atomic function) is identified. In more tangible terms, automation design may determine that elements are comprised of discrete elements on a screen (e.g. a text field, a radio button, a frame with a collection of a few controls). Alternatively, these elements may be seen as common (yet small and discrete) actions in the application. This may be actions such as “enter a form value”, “navigate a header menu list”, or even “login”. These elements may be considered analogous to individual steps in a manual script. The analysis required for automation design may be performed by automation specialists, or non-technical resources. This allows for inclusion of more individuals with less specialized skill sets to participate in automation implementation. These resources are focused on “what actions do we need to be able to perform to execute transactions?”. The focus is shifted from worries on how to technically achieve the automation to an abstraction based on real world needs.

In automation implementation, an automation engineer (or someone capable of building an automation script) builds the automation code to strictly meet each element from automation design. While this component may be more technically challenging and require deeper knowledge of automation tools or scripting languages, this process is still simplified when compared to traditional approaches. The automation engineer has less logic to develop as functions are bounded by the need to create small, discrete modules for specific needs. This leads to rapid development of many small functions rather than laborious creation of comprehensive transactional scripts. Furthermore, this promotes increase maintainability over time. If a small function or element of a transaction is changed in the application, maintenance is quickly applied to the smaller function. For example, if an application adds a “secret question” during login, the automation engineer knows that the login function is the only location in the automation libraries which needs to be modified.

Test case design is the process area where testing is actually assembled. This may also be a more inclusive process. Test case design is a process whereby the smaller building blocks of elements or functions are assembled to represent a test case. From the elements defined by automation design, elements are triggered sequentially to determine pass/fail status. This test design may be written in the automation tooling language, or in many cases, driven by rows in a spreadsheet. When implemented through spreadsheets, this enables less-technical resources to define the tests to be executed in an automated fashion. The automation framework may be designed to open a file with the defined steps, and execute the implementation code.

By developing frameworks that separate test case development from implementation details, automation coding becomes faster, reduces maintenance overhead, and improves overall efficiency. More importantly, this separation of automation functions offers the possibility of including more people in the automation practices. More individuals in the organization may realize the benefits of any implementation efforts applied. More testing may be achieved in less time as constraints imposed by available automation engineering personnel are reduced.

Please contribute to this discussion. How does your organization apply automation to testing processes? What type of framework approaches do you use? What problems and constraints do you face?

Chris Meranda
Senior Solutions Engineer
Avatar

 

Test Automation: Uncovering the Myths

Why is it that so often I visit customers where test automation is in the hands of a few technical individuals, while the greater majority of the testing teams are bound in manual processes?

Why is it that so often I visit customers where test automation is in the hands of a few technical individuals, while the greater majority of the testing teams are bound in manual processes? While the greater majority of the users are pounding away at processing functional and regression tests, a small minority (or even an individual) is assigned to running an automated replay engine. On the surface, this seems to be counter-productive. The potential is that the individual with the automation tooling is actually exercising the vast majority of the tests using scalable solutions and running operations 24 x7, and the remaining individuals are addressing the outlying test cases. However, this is not what I have witnessed visiting companies in different markets, and with different levels of testing sophistication. I repeatedly see that the automation tools are used by specific individuals in a specialized, or niche, application within the organization.

If we, the testing team, have so many tests required (a nearly infinite number of potential test cases), why is the implementation of automation practices limited to the minority? Shouldn’t the majority of the testing team be building and maintaining automation scripts so that the efforts of each individual may be multiplied by the number of scripts that can be executed by hardware? Instead, we bind our processes so that each individuals efforts are additive to covering all test cases in a cycle manually, rather than multiplicative with automation tooling.

What is holding organizations back from more effective implementation of test automation practices? From what I witness across many organizations, the primary limitation to automation reside in vision and perspective. If we perceive a problem to be hard, we try to avoid it. But, what if this is a perception and not reality? What if we have notions that something is hard or daunting, but it turns out that our conceptualization is wrong?  Test automation is fraught with these perspectives.

Here are some myths that I encounter when I visit organizations:

  1. Test automation practices must require a highly advanced skill set.
  2. A very specialized type of person must be needed to implement test automation.
  3. Test automation must consume an excessive amount of time to build and maintain, so only a few people can be afforded to be removed from manual testing efforts to get the automation built.
  4. Test automation must need highly advanced hardware.
  5. Test automation must be expensive.
  6. Applications must be so dynamic that they can’t be automated in any practical way.

In upcoming articles, I will be addressing these myths and perceptions. I would also like to add to this list. Please let me know what challenges you face when trying to apply automation to your efforts.  What is holding your organization back from adopting automation strategies or approaches? Why aren’t more of your teams supplementing manual efforts with tooling?

Chris Meranda

Senior Solutions Engineer
Avatar

 

Highlighting the complexities of mobile testing at Appsworld!

Appsworld has grown to be a regular on the events circuit taking on Earls Court 2 for a couple of days each October. Wooing the crowds with a mix of tracks catering to a varied audience of technical professionals and developers all looking to find out the latest and greatest, this year was set to be the biggest and best show yet!

Appsworld has grown to be a regular on the events circuit taking on Earls Court 2 for a couple of days each October. Wooing the crowds with a mix of tracks catering to a varied audience of technical professionals and developers all looking to find out the latest and greatest, this year was set to be the biggest and best show yet!

Well the show didn’t disappoint. The audience really was quite varied, covering everyone from students through to CTO’s, ensuring some really engaging discussion and keen interest in demos on our Borland stand throughout the two days.

Early on day 2 the crowds were entertained by none other than Steve Wozniak, aka “Woz” which was a real treat! This could have been a tough act to follow but Becky Wetherill, Product Director, pulled in a large crowd (see below) with her session covering the maturity of mobile testing and its integration into the Continuous Integration Testing (CIT).  If you’d like to learn more about how you can improve your testing and raise your app quality bar then take a look at the resources we have here


So another year of Appsworld comes to a close, thanks again to everyone who got a chance to drop by our stand! And don’t forget you can find all the latest information about our products that optimize your software delivery on MicroFocus.com.

Mark Plant

Director Marketing Operations

Avatar

Lights…camera…it’s showtime! Garter Symposium / IT Expo Orlando

So. My big moment. A whole week of them.

Not a great day – there were no marching bands, parades or military flypasts to welcome me when I got here. Orlando? Did you not get the memo?

At this stage, I’m ruled by my to-do list. It’s a biggie and it’s getting bigger so I’ve had to prioritize it. The stuff written in RED BOLD UPPER CASE lettering comes first.

1) Check that fingers are crossed (This should guarantee all the boxes and materials have made it onsite).

2) Re-cross them to make sure all the staff are on their way. (Yes, Frank – I understand. I really do.)

3) Check hotel reservations. Then re-check them. Then do it again. (Note to self: have you checked the hotel reservations?)

4) Start sorting shirts for the staff who are on their way. They are, right? Yes?

5) Double-check the video showreel (thanks John Wyer & Purple!) and ensure the right file goes on the memory stick. A Gartner show is no place for my holiday snaps.

6) Meet with the presenters for our speaking session. Check they can all speak.

So in short – my life is an endless loop of checking, double and triple-checking. This is where the rubber hits road – and where any miscommunication will start hurting, big time.

I heard from my taxi driver last night – and they never mislead you, right? – that the Gartner Symposium/ITxpo will bring in more than 13,000 people. So no pressure there, then … (Note to self: We’re gonna need a bigger stand)

The movers and shakers are all here. And I mean literally – the movers are carrying in all the vendor materials and the shakers (ie all of us vendors) are ‘shaking’ their boxes to make sure their giveaways and collateral made all in one piece.  AV is being set up here, Internet folks are draping cabling around there. If this is the calm before the storm, it isn’t very calm…

If you are anyone in the IT world, you are here. Or thinking that you should be. Who’s not here, you ask?  Frank Borland is off saving another company from another disastrous testing failure, so unfortunately he’s just here in spirit. And video.

Several of our twitterific staff have made it – Kevin Brearley, Archie Roboostoff, Andrew Wickett, Emerson Sklar – tweet them to say hello or better yet, stop by booth #500 to say hi in person.  We’ll be there ready to shake hands with new and old friends.

Must run, the delivery guy is here with MORE boxes. Nearly time for curtain up….seeya soon

COBOL: the language for future generations

After many busy years of getting to grips with the latest programming language, it’s time to venture out into the daunting world of jobs. Learning older programming languages such as COBOL as well as more modern languages is something few, within today’s generation, consider – but in many cases it’s this very skill that employers in the business industry are looking for.

This blog explains why so many organizations still value COBOL and how broadening your existing skill set by learning the language can give you competitive edge when it comes to landing a great job.

COBOL makes the world go round
Grace Hopper brought COBOL into the world in 1959. Over fifty years later, it powers 70 percent of all business transactions. COBOL is everywhere – from ATMs, to point of sales systems and healthcare prescriptions. “The language is present within 85 percent of the world’s business applications[1], and it’s place in behind the scenes business software is as prominent as ever. COBOL is woven far too deeply into the business world to simply tear out and throw away.  The world would  be a very different place.  Without it in many situations, communications around the world would collapse.

COBOL: the language of longevity
Despite these facts, newer languages appear to be the popular choice as they frequently catch the eye of the younger generation developer.  The world’s application do need different languages.  But certain languages are better at certain tasks than others. For instance, COBOL’s strengths lie in processing financial-style data and number crunching. Java and C# are used more effectively in the front-end user experience. Languages must be for fit for purpose: “The problem to solve should determine the language to use.”[2]

So why do organizations choose to keep COBOL instead of rewriting applications using the latest one? After all, keeping up-to-date with the latest technology is important in IT. But when other factors are taken into account – the length of downtime during transition from old to new, the fall in return on investment it triggers, the amount of resources it uses, the training involved – the thought of a ‘rip and replace’ loses its appeal.

As many as 75%[1] of all rewrite projects have resulted in failure. Businesses which already have COBOL established in their systems are unlikely to wake up one day and replace it with Java. Newer languages have not had the chance to stand the test of time, so no-one knows how robust they will be several years down the line. That risk could cost an organization, the business.  So for many, COBOL is here to stay.

The pending problem
The significant concern is that those skilled COBOL programmers are disappearing without being replaced, forming a widening skills gap. Development teams also often work in siloed environments, broken down by programming language or the tools they use, which can inhibit application development. But there are many positive reasons to learn COBOL. The next generation of developers must pick up COBOL skills and carry them forward into the future.

COBOL is robust
It’s been adapting to business change for decades, so it really knows the ropes when it comes to keeping things running. Throughout its life so far, it has come across many different obstacles – such as new platforms and devices – and has (with the sponsorship of Micro Focus) met each of these challenges. It even integrates with the next generation of modern languages, such as C#, Java, and VB.NET.

COBOL is intuitive. It’s easy to learn because of its English-like structural components and it has been ported to virtually every hardware platform. It runs within modern IDEs – Visual Studio and Eclipse – so there’s no need to worry about learning a new toolset. COBOL is the perfect language to broaden the coding experience – and it’s a language skills desired by employers. Why wouldn’t you learn it?

It could make you much more marketable in the jobs market: “the more languages learned by developers the better, as a range of abilities will increase their chances of employment.[3]

Visual COBOL: a modernized language
As the COBOL developer cooking pot boils dry due to retiring veterans, there is likely to be an increase in demand for COBOL developers. Learning COBOL could therefore make you highly desirable.There is a real need for greater community collaboration between academic universities and business organizations.

Micro Focus Visual COBOL enables older and newer languages to unite in a familiar environment. It enables COBOL teams to work in the same development environment as the Java or C# developers, so they can collaborate effectively. It also provides an easy-to-use environment which assists modern language developers in picking up new COBOL skills. Forget green screens – its modern development IDE is elegant, efficient and a powerful tool for the modern application developer.

Academic program
Micro Focus’ Academic Program supports this initiative, providing the latest tools and technologies to help build the next generation of COBOL developers. Students, educators and businesses are joining forces to bridge the skills gap. Why not join them today?

Micro Focus Visual COBOL Personal Edition helps fine-tune your skills and gain experience. Working with the IDE of their choice, students and aspiring developers can take full advantage of the new capabilities available within this innovative development environment. Take this opportunity to learn the enterprise language behind 70% of today’s business transactions.

To learn more about how Visual COBOL bridges the skills gap, watch this inspiring animation and check out www.microfocus.com/bridgethegap to see how you can get involved.


[1] http://edtechdigest.wordpress.com/2013/05/17/cobol-anyone/

[2] http://msdn.microsoft.com/en-us/library/ms836794.aspx

[3] http://www.siliconrepublic.com/careers/item/31776-cobol-and-legacy-programmi