What you need to know about software testing automation
A comprehensive collection of articles, videos and more, hand-picked by our editors
A QA tester gives the lowdown on what developers, IT operations and management should know about QA testing, automation, tools and training, and DevOps.
Automated QA testing is an afterthought or poorly understood step in the application lifecycle for developers, IT operations and management. Some might say it slows down the agile process, when speed is critical to continuous delivery.
But quality assurance (QA) testers such as Nadia Makhnovskiy-Linares beg to differ. With a few changes and communication, integrating automated tests into the development and operations processes would actually increase deployment speeds -- and quality.
SearchITOperations sat down with Makhnovskiy-Linares to find out what devs, IT ops and management need to know about integrating automated QA testing into their DevOps processes.
What should developers know about automated QA testing?
By creating unit testing before creating new code, developers save precious time and finding bugs is cheaper for testers, Makhnovskiy-Linares says. But just creating those unit tests isn't enough.
"I would love them to run those unit tests on their machines before they check in the code … sometimes it happens that developers check in something and they didn't run unit tests, they did not get [the] latest version from repository and then a lot of time is wasted," she says, in putting together the software build, running tests and finding out something doesn't work because of outdated versions.
Automation is also something to think about when anyone, be it dev or ops, creates a test or chooses tools, she says.
What should IT ops know about automated QA testing?
When talking about IT ops and infrastructure administrators, Makhnovskiy-Linares focused on efficiency.
Real data is important for better testing, she says. Problems arise when the testers get a bunch of data straight from the ops team that hasn't been prepared. Since ops teams may have a lot of regulations and security protocols when dealing with this data, she asks that IT ops groups help automate data cleanup to remove sensitive information.
Second, make the QA testers part of infrastructure planning decisions.
"If we know how the system is set up, its integration points, we can test more efficiently," she says. "When infrastructure changes, we need to know that it changed, because it might change our testing."
Makhnovskiy-Linares' advice on IT ops tools also might come as a surprise. The number of tools IT ops pros use can create silos and redundancies in the IT department -- so what does it do to testers?
What should management know about automated QA testing?
The leaders at companies need to invest in training, big time, Makhnovskiy-Linares says.
"Today's QAs [are] not the same as QAs 20, 30 years ago when you just had to kind of be the surrogate for users," she says. When QAs have to be highly technical, training is anything but a waste of money. Better training means that QAs know what to look for when something is going wrong, letting developers, system administrators and others spend their time doing more productive tasks.
One metric Makhnovskiy-Linares wants managers to drop from their key performance indicators? Bug count. Usually, Makhnovskiy-Linares says, automation will find a bunch of bugs since it is used to test regressions. Instead, use other metrics like test coverage.
Learn more about Nadia Makhnovskiy-Linares.
Transcript - How dev, ops and execs should treat automated QA testing
Austin: Welcome to the SearchITOperations podcast about DevOps and testing and the overlap between the two. We're gonna be talking about things that testing QA engineers need to know. What is a QA engineer's role during the testing process, and how code interacts with the infrastructure. With me, today, I have Nadia Makhnovskiy-Linares. Is that, did I get that right?
Austin: I did?
Nadia: Yes, it's very rare.
Austin: Yes, awesome. Where exactly do you work, and what do you do?
Nadia: I work at SmashFly Technologies, I really love my job. I've been doing automation for about 12 years, QA automation and testing. And, I guess I have some experience to share with you today.
Austin: What would you like developers to know about QA testing that would make things go easier for you?
Nadia: I'd really like them to invest time in creating unit tests, because unit tests and yes, there are some cases when the problem will pass, but unit test is the first line of defense for our code. And, it's also useful for developers because, for example, task-driven development. It means that you create unit test first, and then you create the code. So, instead of like creating new code and then manually testing it, as a developer, you can just create a test and just run it. Run it when you change the code. So, I would love them to really, really invest time if possible, in unit testing. Also, it's much more cheaper to find bugs and much cheaper to fix. So, that's first thing.
Also, I would love them to run those unit tests on their machines before they check in the code. We have sometimes those...no, not we but you know in general sometimes it happens that developers check in something and they didn't run unit tests, they did not get latest version from repository, and then a lot of time is wasted to make a build, then testing and then something is not working just because of that. So, would be nice if they just get latest from repository. And, also when they write code, I would like them to think that it should be automated. So, if they can kind of think about automation folks, that'll be great and one of the examples I can give is it'll be helpful to create unique IDs for elements on UI [user interface]. Because this way, we can automate much more easy and faster and at the end of the day, the whole project benefits.
Austin: So, we just covered developers, so what would you like IT operations people, specifically infrastructure people, to know about testing and test that would make things go easier?
Nadia: I would love them to have QAs part of the infrastructure decisions in plan, because it's really critical for us to know what infrastructure people know, because if we know how the system is set up, its integration points, we can test more efficiently. When infrastructure changes, we need to know that it changed, because it might change our testing. And we don't want to run around and bother people for things which are actually known and take their time. We want to be as efficient as we can.
Austin: Yeah, it's DevOps, you wanna be as efficient as possible.
Nadia: Yeah, yeah, efficient. It's about efficiency, about making everyone happy, even the people who work with you, not just customers, that's the big part of it. So, real data is also important to us to test better. So, if you guys can assist us in preparing that data, because it's not a very simple task, sometimes you really have a lot of regulations and security things you have to go for. So, if you can maybe help us to automate data clean up, removing all the sensitive information, that will be really helpful. That will be really helpful. And please be careful with the number of tools you are choosing, because yes, one, tool is probably not gonna solve every problem but please don't get new tool for everything. Just learn it...
Austin: Don't pile on the tools?
Nadia: Yes, now don't pile and if you know how the tool works, if you know the capabilities of the tool, you can save a lot of time too.
Austin: All right. So, last part of this, I guess, three-part question. So, what would you like management to know, about testing and tests that would make things go easier?
Nadia: First of all, know your goals for testing and automation, like you have to really know what exactly and why you have to test. Because, then it really helps us to pick up priorities, and concentrate on them instead of wasting time. Second of all, I'd really want them to know that they should invest in training. Yeah, I know, I know, but it's not a waste of money. Today's QAs, it's not the same as QAs 20, 30 years ago when you just had to kind of be the surrogate for users. Now, these QAs have to be highly technical. They have to really know what's going on behind the scenes, and we have to also, not they, we, have to know what's going on behind the scenes and really sometimes are very helpful because we can sometimes find what's going wrong. And developers and our people don't have to go and look for that, so we can deliver the solution in the box, if we are trained. So, please invest in training; that will be nice.
And, I now want to switch to automation. We already discussed it, but please treat automation as a separate software project, not a side activity, right? And, please don't measure automation success by how much bugs automation finds because usually automation is used to test regressions and unless you had a horrible product, and you get like regressions all over the place, that's actually not a good thing [to measure automation success by the amount of bugs found]. You just cannot measure success by the amount of bugs which automation finds. You can use other metrics, and they exist. For example, one of them is, you can just measure the test coverage. So, you can see how much automation is coded to, for example, to cover risk areas or to cover our important areas for product, yes?
So, that's my 10 cents. My personal, please don't take it as a company endorsement or a company opinion.
Austin: Before we end, do you have anything else that you'd like to say?
Nadia: Anything that I'd like to say?
Austin: Yes, let the people know.
Nadia: Yes, I'd like really people to, learn more. I mean, I think it's important, and so fast paced technology world we are in. You really have to learn. You really have to try to educate yourself about what's going on. And again, stay positive, meditate. Stress is counterproductive. So, I hope you guys all have great testing time and not just testing, developing and whatever you do, and I enjoyed this.
Austin: Thank you, I enjoyed myself too. Thank you, Nadia. Again this is SearchITOperations podcast for TechTarget. Thank you, Nadia, for being here.
Nadia: Thank you, Austin. That was really cool and interesting experience. I will remember that for all my life.