This past year we launched the Breeze Team, a customer-driven initiative to help other customers and potential customers gain useful insights about our Zephyr products. Along with my co-worker Fraser Colmer, we interviewed a selection of enthusiastic Zephyr customers and handpicked Super Zephyr Users to join the Breeze Team program.

The benefit of the Breeze Team for Zephyr program is to help educate a wider community of users about Zephyr. On a personal level, I enjoy the interaction with the customers and hearing about how they are using our tool. Many report back about how much easier it makes their jobs. When we were in Barcelona for the Atlassian Summit I got to speak to many of our Breeze Team members in person and was humbled by how truly thankful they were for Zephyr.

Here are some of our Breeze Team members' favorite benefits of Zephyr for Jira:

Zephyr’s Integration with Jira

“The best thing is that it smoothly integrates into JIRA that you don’t even notice that you’re working in another tool.”

-Taras M., QA Lead in the Insurance Industry

Efficient Execution of Tests

“As the test teams execute the tests, you can raise deflects with a flag. It’s all linked, it’s all within one system and that’s the biggest value that we get with Zephyr.”

-Vanna C., Senior Test Consultant in the Finance Industry

Speedier Process of Development to Deployment

“Everyone eventually learns to start using it in a way that’s definitely benefitted our company. I think that it’s made things a lot more streamlined, a lot more organized, and it’s made the time to deployment a lot quicker.”

-Cheryl L., QA Lead in the Education Industry

The Traceability of the Test Cycle

“The traceability report to see the flow of the test case from the requirement through the sprint cycle and where they fail, as part of regression is a great plus. As well as the ability to filter them and then add to the regression suite has been a huge help to us.”

-Chitra A., QA Lead Engineer in the Software Industry

Cloning For Regression Test Activities

“I like the ability to clone, especially for the regression test activities. The regression folder, sprint over sprint, I will clone the previous and then we add to it in the new sprint so that we continue to expand our regression test portfolio.”

-Sherri D., QA Manager in the Real Estate Industry

Simple, Easy Test Management

“It’s really the best tool for managing tests, especially the automation and continuous integration feature, ZQL. It’s the best because it’s easy to use and you can filter out tests, especially the execution status, which is really helpful if you have big numbers of tests.”

-Web Developer, Automation Solutions

Now It's Your Turn, Share Your Story

Is there anything we might have missed? We’d love to hear from you! Email me at matthew.king@getzephyr.com to let me know what you and your team think about using Zephyr for JIRA for your industry.

Insights from Zephyr Capture for Jira Survey

If you’ve ever struggled to share concise feedback with a co-worker or make your review process more efficient, you are not alone. As a visual, collaborative tool for annotating screenshots with detailed, visual feedback, Capture for Jira was created to solve that problem. A picture might be worth a thousand words, but Capture for Jira enables you to combine pictures and words together, without anything getting lost in translation.

Capture for Jira can be used on a wide variety of projects, from user interface and web design to feedback on sales and marketing content. We recently sent out a survey to our customers to find out how their teams were using Capture for Jira and the ways in which visual feedback is beneficial to their businesses. Overall, respondents agreed that visual feedback enhances collaboration and communication, but here were some of the insights from our survey:

Who is Using Capture For Jira?

Out of the 120 people who responded to our product survey, 67% are small businesses, 12% are medium businesses, and 22% are enterprise-level businesses. Nearly 37% of them have been using Capture for Jira as a tool to provide visual feedback for one to three years and 21% have been using the product for three to five years.

A wide variety of teams are regularly using Capture for Jira, from Quality Assurance, to Software Development, to Project or Program Management and beyond.

How Are They're Using It?

The majority of respondents (42.5%) found the biggest benefit of Capture for Jira to be the time they save by having clear and concise visual feedback. Many others (32.5%) like that it allows for increased collaboration on testing, and a small group (18.3%) thought the biggest benefit was that it created a faster and more efficient review process. Our biggest take away from this survey? People learn and communicate in a variety of ways, but visual communication tools can improve collaboration among teams.

Here’s what some of our users are saying:

“Love it, templating defects for different environments and defaults helps us to obtain consistent bug reports from developers, support and the test team.”

– Michael H.

“Very useful plugin. It improves our productivity and saves time. A great way to communicate UI issues and bugs, efficiently.”

- John Z.

“Effective tool for every team! Connected with our Jira platform, Capture fits for all needs of every team, and in particular for our Marketing team. Capture allows our team to easily provide annotated visual feedback of the website directly to Jira. Another plus is the Zephyr support. If we have any questions, the Support Team is always committed to resolve any issues fast and effectively. For us, Zephyr's Capture for Jira is definitely a 'thumbs up'. 

- Julia Z.

Top Capabilities For Users

The top three attributes or capabilities respondents liked about Capture for Jira were the:

• High quality of the product
• Ease of use
• Ability for collaborative testing

In fact, almost 39% of respondents said there are not any features that could be added to improve the tool or experience with the tool and they like it as is! Other highly valued features included:

• Saving time with clear and concise visual feedback
• Faster and more efficient review process
• Efficient prioritization and implementation of feedback

Making Improvements With Video

The other 60% of respondents who believed that Capture for Jira had room for improvement mostly wanted to see a video recording feature implemented. We took the feedback to heart and version 2.9.17, an update to the tool that launched in July 2018, has exactly that. The same update also gave users the ability to add audio, which was the second most popular suggestion about ways the tool could be improved.

The entire team here at Zephyr takes our customer feedback seriously, as we always strive to provide all of our users with the best possible experience. In fact, if you have anything, you’d like to tell us about your experience with Capture for Jira, email us: marketing@getzephyr.com

Gartner, the world’s leading information technology research and advisory company, recently chose SmartBear as  a Leader in its 2018 Gartner Magic Quadrant report for Software Test Automation,[1] designed to help enterprises select Software Test Automation tool vendors.  This is the fourth consecutive year Gartner has recognized SmartBear, and its first year as a Leader.

“Test automation tools are essential elements of a DevOps toolchain and enablers for achieving the continuous quality approach required for successful DevOps. Continuous quality is a systematic approach toward process improvement to achieve the quality goals of business and development. It synchronizes quality assistance and continuous testing with DevOps processes to help mitigate risks before progressing to subsequent stages of the software development life cycle.”

Here’s SmartBear’s product portfolio, in brief:

Plan and Track

• HipTest Agile & Behavior Driven Development (BDD) Test Management
• Zephyr Native Jira & Enterprise Test Management
• Capture for Jira Visual Feedback and Annotation
• QAComplete Manual & Automated Test Management

Design and Build

• SwaggerHub OAS API Design & Documentation
• ServiceV Pro UI & Service Virtualization
• Collaborator Code & Document Review
• AQTime Pro Memory & Performance Profiling
• Swagger Open Source OAS API Tooling

Test and Validate

• SoapUI Pro API Automated Functional & Security Testing
• TestComplete UI Automated Functional Testing
• CrossBrowserTesting Cloud-based Browser & Device Testing
• TestLeft UI Functional Testing Framework for Developers
• SoapUI Open Source API Testing

Load and Monitor

• LoadUI Pro API Load & Performance Testing
• LoadNinja Web Load & Performance Testing
• AlertSite Web, Mobile & API Performance Monitoring

Report Details

Gartner examined 11 software test automation vendors in the report based on two criteria: Completeness of Vision and Ability to Execute.  The Gartner Magic Quadrant Report classifies each vendor into four different quadrants:  Leaders, Challengers, Visionaries and Niche Players.

Magic Quadrant for Software Test Automation

To access a complimentary copy of the 2018 Gartner Magic Quadrant for Software Test Automation, visit https://smartbear.com/resources/white-papers/gartner-magic-quadrant-2018/

¹Gartner Magic Quadrant for Software Test Automation, by Joachim Herschmann, Thomas E. Murphy, Jim Scheibmeir, November 27, 2018. 

About SmartBear Software

Supporting more than six million software professionals and over 22,000 companies in 194 countries, SmartBear is the leader in software quality tools for teams. The company’s products help deliver the highest quality and best performing software possible while helping teams ship code at nearly impossible velocities. With products for API testing, UI testing, code review and performance monitoring across mobile, web and desktop applications, SmartBear equips every development, testing and operations team member with the tools to ensure quality at every stage of the software cycle. For more information, visit: http://smartbear.com, or for the SmartBear community, go to: LinkedIn, Twitter or Facebook.

Gartner, Magic Quadrant for Software Test Automation, Joachim Herschmann, Thomas Murphy, Jim Scheibmeir, 27 November 2018

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from [insert client name or reprint URL].

What is your level of Ninjility? Zephyr’s Ninjility quiz helps testing organizations better understand where they stand when it comes to software development agility. Achieving Ninjility, which is a heightened state of software development, is a highly coveted task that requires a variety of moving pieces all working in sync with one another.  

Zephyr took the Ninjility quiz on the road this year in an effort to get teams to start a conversation with one another about their testing journey. We visited over 1000 people at about a dozen events around the globe to see how far along organizations were on their journey towards reaching software delivery enlightenment. The biggest surprise? Many team members at the same organization found that their Ninjility scores differed. This served as a starting point for important discussions between team members and allowed them to better understand each other’s perspectives.

“Our goal was to create a fun interactive exercise to encourage discussions between team members,” said Tom Alexander, Vice President of Zephyr Marketing. “Ultimately, Ninjility is a self-assessment tool for organizations seeking to improve their process and skill sets.”

From Eurostar in Hague, Netherlands, to Developer Week in Brooklyn, to QUEST in San Antonio and beyond, we spoke with QA Managers, QA Engineers, QA Analysts, Developers, Testers, Scrum Masters, CEO’s and CPO’s at organizations both big and small and discovered that although the road to Ninjility can be a little bumpy along the way, it’s well worth the effort in the long run.

We’re Halfway There: Ninjility Is A Process

Ninjility is ranked on a scale of 1-5, with the highest state of Ninjility being 5. Of the 1000+ participants that were surveyed, the average Ninjility score was a 3. This means that the majority of organizations who are working towards achieving agility in a DevOps ecosystem are making headway. It also shows, however, that there’s still some way to go until total Ninjility has been reached.

Our 2018 annual report on “How The World Tests,” revealed that 52 percent of Zephyr customers are working towards the maturation of their DevOps team and continuous testing as part of their agile journey. And 20 percent want to invest in new or better tools for test management. Organizations continue to make progress in their push to become more agile.

Culture Shock: Adopting A DevOps Ecosystem

For organizations looking to compress the software development lifecycle, they have to adopt a DevOps ecosystem, which can be a real shock to their office culture. In fact, 30 percent of our survey participants found the shift over to DevOps to be challenging for their work culture. This is because DevOps fundamentally changes how teams interact with one another, confusing those who are used to legacy systems and a silo’d approach.

A DevOps ecosystem on the other hand, starts at the team level, streamlinescommunication andcreatesmore frequent collaboration between departments. If implemented properly, a DevOps ecosystem can create more trust and efficiency, as well as faster software releases and the ability to solve critical issues more quickly. “How The World Tests” found that a quarter of organizations find the biggest benefit of a DevOps ecosystem to be faster speed to market and the fast-paced delivery of features.

Size Doesn’t Matter: Organizations of All Sizes Can Obtain Ninjility

In addition to exploring how agile organizations have become, our Ninjility survey also discovered that larger organizations are further along in the process of adopting a DevOps culture than smaller organizations. In fact, larger teams reported a 20 percent higher use of testing automation tools, and were 20 percent better at using analytics when testing.

Digging deeper, “How The World Tests” found that small and medium-sized businesses are not that far behind when it comes to adopting a DevOps culture. Both small and medium-sized businesses reported being about 40 percent of the way there, while enterprise-level organizations are 50 percent there. Our report also found that over 50 percent of organizations are currently using some type of Descriptive Analytics tools, regardless of team size.

Looking To The Future

Both our Ninjility survey and our “How The World Tests” report seem to come to the same conclusion: The testing industry is clearly making a global shift towards DevOps and Continuous Testing Agility.

Where are you and your team on the journey to software development enlightenment? Take our Ninjility quiz to find out: http://www.getninjility.com/

Dr. Milan Verma, Zephyr’s Lead Client Services Engineer and Greg Hanson, Senior Director of Global Sales Engineering for SmartBear recently gave a presentation and answered some essential questions on automated testing and test management with SmartBear and Zephyr.  “The focus of our talk today is test automation and the power it brings to your testing journey,” Greg said, by way of introduction.

“Once we help you figure test automation, we're going to talk about how to combine it with your test management platform to ensure that you get the highest level of insight into your testing practices.  Then I’ll be pass the mic to Milan and he will give you an in-tool demonstration of Zephyr and TestComplete, combining those two worlds of test management and test automation.”

SmartBear Overview

Greg noted that SmartBear was founded in 2009 and currently supports more than six and a half million software professionals and over 22,000 companies in 194 countries, on almost every continent.  “We do have some deals with penguins in Antarctica that we're working on, which may take a little more time,” he joked.

Beginning with a high-level overview of SmartBear product offerings, Greg explained that SmartBear has products that stretch the entire software development life cycle, from development through testing and operations--for both front-end and back-end applications.

“From the development side, we have collaboration tools for traditional and API development to allow developers, testers and product teams to really communicate as the code is being developed to help find those defects earlier in the process.”

“Moving into the testing phase, we have test automation tools for developers, for people brand new to testing, and for sophisticated automation engineers--again for both the front-end and APIs in the back end.”  

“All the way through into Ops and monitoring where we have our brand new LoadNinja tool for SaaS and web application load testing, as well as AlertSite for monitoring all of your applications and web assets after they've been deployed--because nothing is quite as satisfying as finding a defect before your customers do and fixing it before anyone knows that there's an issue.”

He called attention to two tools in the middle of the flowchart, Zephyr and Hiptest, which are both recent SmartBear acquisitions.  “These tools are vital for test management, which you need to have full insight into your testing practices and how well your manual testing efforts are integrated with any or all of your test automation tools.”

Test Automation

Time, cost and quality are three widely-recognized constraints of software project management, Greg said. “There are always going to be tradeoffs. You're either going to spend more money to increase quality and save time, or you're going to increase quality and cost yourself more time.”  Test automation helps conquer the constraints of time, cost and quality in this iron triangle of tradeoffs. It is a cost-effective solution to accelerate software development while ensuring quality through earlier bug identification and quicker fixes.

“We don't want to go all the way to the point of automating 100 percent of the tests like they did with Windows Vista because I think we all know how that turned out, but we want to make sure that we are automating the right things in an intelligent way.” Greg said.

After a real-time poll found that the majority of audience members were just getting started with automated testing, Greg emphasized the need to concentrate test automation efforts at the unit testing level.  “This is really important because that's the level where you can actually test the true functionality, whether it works, yes or no. If programmers are doing this testing as they're doing development, they're going to be able to find the defects and fix them much more rapidly.”

“The UI testing at the top is the icing on the cake that uncovers user experience hiccups, design layout flaws and workflow issues.  This robust framework, where you have pillars of unit testing underlying API and UI tests, supports your manual testing efforts. It’s all designed to give the human tester as much flexibility and time as possible to do what we do best, which is the intelligent part of testing. “

The Automated Testing Pyramid is just one of many things that should be part of your organization’s test automation strategy.  He cautioned that there are numerous challenges in getting test automation right, citing results from a recent Zephyr How the World Tests, 2018 survey.

“There are several different ideas that we can take from this but what it boils down to is people don't have the knowledge necessary to really implement automation…  Everyone wants to automate, everyone wants to save time, everyone wants to release on a much more rapid development cycle, but they just don't know how to tackle that,” he said.

In spite of these problems, test automation continues to grow year over year, Greg said, citing the results of  SmartBear’s State of Software Testing, 2018 Industry Report .  

 

 

Benefits of Test Automation and Test Management

Greg stated that one notable fact from this year’s survey is that a third of the people are still using spreadsheets and Word documents for test automation instead of a proprietary tool or integration with JIRA.

Perhaps the most surprising result of the survey is that respondents using a standalone test management tool are more likely to automate their testing across a majority of testing types, including integration testing, unit testing, UI testing, API testing, and performance testing, as compared to those who use spreadsheets and Word documents.  Greg added emphasis to this point by using a comic book voice— “the kind I use when I’m reading to my kids”—to read the following: “Those who invested in a test management tool automated more for every type of testing than those using spreadsheets only!”

 

 

 

Zephyr Overview  

Milan began the next part of the presentation with an overview of the two products that make up the Zephyr test management suite:  Zephyr for Jira and Zephyr Enterprise. Both products provide scalable test environments, he said, via the data center inside Zephyr for Jira and also via Zephyr Enterprise's scalable SOA- and microservices-based architecture.  Zephyr has an architecture that is continuously “bashed” or tested, using APIs, virtual users, physical users, and various configurations to ensure that that the application holds up from a load testing, security and safety perspective and can perform at scale.

He said there are quite a few approaches to reporting in Zephyr, including widgets within the Zephyr dashboard as well as live charts that are clickable to allow you to drill down to view data points in more detail.  In addition to a range of traceability and custom reports, he stated that it’s also possible to link up automation frameworks within Zephyr Enterprise so that you can have a consolidated view of semi-automated, fully automated and manual testing in one place.

Since most of the webinar’s audience considered themselves heavy users of JIRA, Milan said Zephyr offers multi-JIRA, multi-project support:  meaning you can simultaneously connect multiple instances of JIRA and multiple projects, which can be set up within both Zephyr Enterprise and Zephyr for JIRA.  It’s also possible to report against all of these instances and projects by leveraging widgets available in the dashboard.

Using Zephyr and TestComplete to do Test Automation

Milan said Zephyr offers a couple of different ways to do test automation.  Zephyr Enterprise is a live HTML5 environment that connects with JIRA and it has a whole host of applications inside it.  For example, there is an artificial intelligence (AI) engine add-on that looks at all of your requirements and testing data and helps figure out whether you're going to finish your sprint on time and what steps you can take if you're running short on time like adding additional users.  It also uses a probabilistic model to ascertain whether a test case is likely to fail and flags it as a predicted fail. There's also a DevOps add-on and an add-on that allows you to work with automation frameworks using a full set of REST-based APIs to design your desired automation scripts.  “Since we use these APIs to develop the UI for Zephyr in the first place, you can look at network traffic and see which APIs are being used,” Milan said. You can also use the Vortex application to trigger automation jobs directly from Zephyr Enterprise.

TestComplete + Zephyr

Using Vortex, it’s relatively straightforward to configure one-click automation in TestComplete and Zephyr.  Vortex features include control of test executions, auto-creation of test cases, the ability to track and manage automation runs, centralized location of results for easy reporting, and analytics on testing activities.  “You can trigger TestComplete suites directly from Zephyr,” Milan said. “You can also automatically create test cases that have been executed using a build automation solution, a CI engine, or a CI/CD framework like Jenkins, Bamboo or TeamCity.  The scripts will be auto created inside of Zephyr Enterprise for you. For instance, if you have a job or a Java class that's being executed that runs a Selenium script, Vortex will auto create a test case and place that into the test repository. It also places the executed version of that test case into the test case execution folder.”

Milan explained that Vortex is an automation platform that uses automation agents called Zephyr Bots or ZBots, which can be installed on laptops, machines, servers, or any consolidated results location on either Linux- or Windows-based machines.  Once you've installed an agent on a given machine, you have the option of setting up a job that triggers different automation frameworks directly from the Zephyr server or setting up a folder watcher that listens in on a given results location.

“In this case,” he said, “the ZBot is looking at a particular location where TestComplete results are pumped in from a Jenkins build.  As the Jenkins build happens, it pushes the results into the folder and the ZBot checks every 60 seconds to see whether there are new results in this folder. As soon as it finds new results, it then populates the Zephyr test planning application with new details.”

“If we look at the execution results updated by Vortex, we see all the passes and indeed a failure (red circle above) as well.   The results of any automation run can be viewed in live reports (blue circle), but also in a dashboards area (green circle).  You can also have a sprint-oriented view, which shows all of your manual, semi-automated and fully automated test results in one single place.”

There are a number of dashboard widgets for showing automation results including one that shows the automated status of a particular release and another one that displays what percent of tests are automated with respect to every phase or by tags in a release.   This information can be published by the Daily Pulse widget that offers a running view on testing activity and tracks the progress of tests created, tests executed, and defects found.

 

“When you observe a test case execution failure,” Milan said, “you can immediately click on a button and file a defect against any failure you've observed.  These defects will go straight into JIRA for you. And then you can come back into Zephyr and build a comprehensive traceability matrix for the defect, as well.”

Questions and Answers

In response to a question about Vortex in the Q&A section of the presentation, Milan clarified that Vortex is an application within Zephyr Enterprise.  “It’s there on the bottom left hand side (orange circle, above) within Zephyr Enterprise.  Vortex doesn’t work directly with Zephyr for JIRA.  In order to bring TestComplete results over into Zephyr for JIRA, you need to leverage ZAPI (or Zephyr API).”  (ZAPI is an add-on to Zephyr for JIRA that allows access to its testing data including the ability to view and upload information programmatically.)  Zephyr for JIRA users can integrate with test automation tools using ZAPI capabilities, he explained. “You can also leverage the Zephyr for Jira, Jenkins or Bamboo add-ons. In this case the Zephyr for Jira Jenkins add-on is the better one to use since it’s open source, so you'll be able to configure it and tweak the data sent over.”

Greg answered in the affirmative when asked whether the Zephyr/TestComplete integration would work “headless” on a device without a graphical user interface.  He referred the questioner to SmartBear’s TestExecute product, which runs TestComplete tests on computers where TestComplete is not installed. TestExecute is a lightweight version of TestComplete that supports all the testing functionality provided by the Desktop, Web, and Mobile modules, but uses fewer resources than TestComplete.  “This is the most common use case in a CI/CD pipeline where people will have TestExecute set up on all of their test runner machines and Jenkins on their other CI/CD platforms, which will call out to TestExecute to run the tests,” he said. “All of that backend information can then be pushed into JIRA (where you can create JIRA issues for failed tests.)”

 

Asked whether Vortex works with Selenium, Milan said Vortex has automation triggering capabilities that let you can run Selenium scripts directly from Vortex within Zephyr Enterprise.  “You can also have Jenkins trigger your Selenium scripts and put the output in a folder you’ve set up to listen for Selenium results.”

 

In response to another question about Zephyr and TestComplete, Greg said that on the SmartBear product roadmap going into 2019, the company was committed to integrating Zephyr with SmartBear’s TestComplete and SoapUI automation tools.  “We're really taking a very thoughtful approach to how we want to integrate all of our tools into Zephyr to make sure that it’s the best for all of our users,” he said.

 

Asked about TestComplete’s ease-of-use, Greg recommended that the questioner download a full feature trial of TestComplete at SmartBear.com or TestComplete.com and take it for a spin. “It’s designed to be easy to use:  you just have to click, record and navigate through your application.  Once you're done with that, you click play and it will run the script back so you’ll know how well your app runs on different browsers and different machines.” SmartBear also has a plethora of video tutorials, trainings and documentation available freely online without any login necessary.

 

Milan responded positively to a question about using Vortex to run Selenium scripts against code developed with AngularJS, a JavaScript-based open-source front-end web application framework.  “Absolutely. It doesn't matter which application or type of application you're testing using Selenium. You can use it to test mobile platforms, AngularJS Javascript applications or HTML5 applications. It really doesn't matter.”

 

He concluded: “If your question is whether X, Y or Z tool can be integrated with Vortex and Zephyr, the answer is generally ‘Yes.’”

 

About Zephyr

 

Zephyr, a SmartBear company, provides the world’s most widely used software test management solutions, powering more than 18,000 customers and 5 million users across 100 countries. Zephyr is leading the global transformation toward DevOps and Continuous Testing Agility through widely adopted advanced quality management, automation and analytics tools. Leading product and IT teams in finance, healthcare, media, mobile, IT services and enterprise leverage the Zephyr family of products to keep pace with accelerating software delivery lifecycles. Dedicated to helping organizations spend less time testing and more time building, Zephyr launched the first testing solution natively inside Jira nearly a decade ago. Today, companies and teams of all sizes rely on Zephyr’s end-to-end solutions, unmatched scalability and support to move from ideas to impact with increasing velocity and ease. Zephyr is headquartered in San Jose, Calif., with regional offices in Philadelphia, Europe and India.

 

 

In an in-depth interview with Alan Shimel of DevOps.com, Ryan Lloyd, VP of Products at SmartBear, explained how the recent acquisition of Zephyr’s Test Management Suite fits into SmartBear’s portfolio of products.

“If you don’t recognize the [SmartBear] company brand, it’s very likely that you’ll recognize one or more of our product brands, since we’re the company behind some key product brands like Swagger, SoapUI, CrossBrowserTesting and TestComplete. We have a diverse portfolio of products that are all focused on helping our customers build, test and monitor their software applications. Zephyr’s collection of tools (will help them) manage the testing process.”

Zephyr’s presence within the Atlassian ecosystem, where SmartBear is already a big player, also played a role in the acquisition decision, Lloyd said.

“The reality is 60 to 70 percent of our customers are using Atlassian’s tools to manage some part of their software development process. So, it’s a very strategic and important ecosystem for SmartBear as a company. Zephyr improves our ability to reach that ecosystem of users and make sure that they’re aware of all of the tools and capabilities that SmartBear can offer.”

“What Zephyr brings to the table is a solution that is natively built around Jira and delivered through the Atlassian marketplace. It’s a solution that Jira administrators and customers can install directly from within their Jira instance. It provides a seamless extension of their workflows within Jira to handle test-management scenarios. This is really important to teams that are trying to take a unified approach to consolidating their processes.”

SmartBear and Open Source Software

Asked about the differences between open-source and commercial versions of SmartBear products, Lloyd said there two philosophies core to SmartBear’s business:

“One is that we want to be open and interoperable with all the tools that are part of the ecosystems that software teams are using. In some cases, that means (taking advantage of) commercial opportunities to partner with companies like Atlassian; in other cases, it means we want to be a part of an existing open-source ecosystem or help build up and nurture an open-source ecosystem, which we did when we brought SoapUI and Swagger into our product portfolio.”

“There’s a brand and a culture that exists around open-source products and it’s our responsibility, as stewards of those brands, to continue to nurture that community. That means we’ve have to make balanced decisions about where we choose to add features to the open-source products versus where we commercialize some of those features, so there’s a constant balance in evaluating and making sure we’re engaging with the community.”

Biggest Challenges in Software Tools Space

“More broadly, beyond just our open-source products, one of the biggest challenges in the software tools space and market is the interoperability that’s required between tools these days,” Lloyd said. "Developers and testers are demanding practitioners. They want their tools to work together and there’s a huge diversity of tools out there in the market. As a product manager, what keeps me up at night is making sure all these tools are compatible and work (together) to support these complex workflows.”

Security testing is merging with other kinds of testing as part of the new Agile workflows, Lloyd observed. “As teams increase the pace of trying to deliver and release software, they’re being forced to evaluate quality and security earlier in the life cycle and more from the context of the development team, rather than having a separate, isolated team that deals with (security issues) downstream or after the fact. There’s definitely a lot of demand for Agile workflows to support testing by design and security by design. From our perspective, that opens up some unique opportunities for adjacencies and areas where SmartBear can continue to expand in the future.”

“The acquisition of Swagger is a great example of us providing tools that help developers earlier in the life cycle. Where we had SoapUI to help with API testing from a QA perspective, Swagger aims to solve some of that further upstream by enabling more collaboration around the definition of those APIs to begin with.” 

“Continuing down this path of modernizing our portfolio, we acquired CrossBrowserTesting about two years ago to help companies test modern types of applications that are integrated more deeply with Selenium, which has become hugely popular with developers to drive automated Web and mobile testing. Hiptest and Zephyr are furthering our relevance for those software teams that are on that journey.”

“In addition to integrating our products in the right way, we’re building up the SmartBear Academy, our SmartBear community and our big user conference, to help elevate the SmartBear brand and create more awareness of the portfolio of products we have – how they work together, how they fit into the ecosystem.  We’re just at the start of that journey right now. You’ll see a lot more, as we launch into next year, of what we’re doing from a brand perspective to galvanize the company and all of the great products we have,” he concluded.

Watch the DevOps.com Video Chat here: https://devops.com/devops-video-chat-devops-testing-with-ryan-lloyd-of-smartbear/

Over the past few years, agile methodologies have been adopted by organizations big and small in order to streamline the ways in which they develop and test their software. In fact, adopting agile in an organization has become so popular we were pleasantly surprised to discover in our 2018 annual testing report, “How The World Tests”, that 52 percent of the people who took part in the survey are currently using agile or agile-like methodologies.

Adopting agile in an organization can be challenging, however, because it forces organizational structures to take a new approach to processes, tools and departmental communication. Without the right infrastructure in place, organizations that are used to legacy approaches could require more time and effort to get their agile testing practices up to speed.

What Is Agile?

Agile, which is considered by many to be the gold standard when it comes to testing methodologies, is rooted in adaptive planning, early delivery and continuous improvement. The premise behind agile methodologies is that it gives organizations the ability to respond quickly and easily to change in order to bring new products to market and keep up with the competition.

Unlike standard methodologies, agile moves organizations away from a documentation-heavy process to a more lightweight story card approach. There are different agile frameworks that can be used, and it’s up to each organization to figure out which one works best for their employees and their goals.

The Benefits of Adopting Agile In An Organization

If you’re on the fence about adopting agile in your organization, consider the benefits of ultimately making the shift:

• Greater Team Engagement. When you adopt agile as an organization, you provide more opportunity for collaboration among teams. Everyone has a better understanding of the other’s goals and when issues arise, and teams can work together to problem solve.
• More Productivity. Agile methodologies enable projects to be completed in shorter sprints, allowing them to be more manageable, rolled out quickly and changed easily at any point in the process.
• Allows For Flexibility and Change. Testing is continuous with agile, allowing for constant updates and tweaks. Project teams can process customer feedback and make necessary updates to improve a product.
• Improves Quality. When you use agile methodologies, the overall quality of projects improve because you have the ability to identify and solve problems quickly and efficiently.

We recently spoke with Zephyr client, AJ, who works in the Administrative Service industry, about why his team decided to make the jump to agile. 

“We wanted more communication across the organization when completing projects and efforts,” said AJ. “Previously we had siloed teams completing their own efforts and only involving other groups when needed or when they are needed to complete a task. In many cases, assumptions were being made about others work.  This created confusion at times and caused projects to come to a halt because of these assumptions. In order to circumvent this, we are now including all parties in the planning of the projects and then planning tasks based on the combined discovery sessions. By employing agile methods all parties are included throughout the development of the project which is more efficient and helpful.”

Additionally, another Zephyr client, Taras, in the Software Industry, found that speed to market was most important when it came to switching over to agile for his organization.

“We wanted to be able to make changes really quickly and deploy to production and receive feedback straight away,” Taras told us. “Our business needed to grow and in order to do that we needed to adapt quickly to the market. We couldn’t achieve that with any models aside from agile methodologies.”

Challenges Of Agile Change Management Process & How To Work Through It

Our report on “How The World Tests” revealed that the biggest challenge facing organizations when it comes to adopting agile testing is actually cultural. Agile is a more collaborative, faster paced, and more dynamic process than non-agile methodologies, so organizations who do things “the old way” can have trouble adjusting to a new way of working.

Taras has found his organization’s greatest challenge to be the shifting the mindset of all the people involved. To overcome this, all of Taras’ team members attended standard Scrum training.

“Newbies had mentors assigned who explained all the details and answered questions,” said Taras. “There were training plans developed for everyone, complete with self-learning courses and self-evaluations. We also were practicing in estimation and writing user stories.  The most interesting was when the host was simulating different scenarios for us.”

Taras recalls two different role play examples that were presented to the team during training: one in which a team member gets sick and is out for 2 days and they had to decide what to do as a team in order to deliver successful sprint, and one in which a story was underestimated and 4 additional days were required to finish it and the team had to figure out how to proceed and how to estimate requirements that are unclear.

For AJ, they found success in overcoming challenges by starting small and building a network of support.

“We first educated the teams on Agile and Scrum methodologies with lunch and learn sessions, and other meetings,” he says. “Then we started employing scrum with the smaller teams to prove success. There was some resistance but after explaining the value of the change, it was an easier sell. It was also very helpful to have the support of individuals at an executive level help push for the change.”

Taras mirrors that sentiment, saying that a strong leader and “knowledge keeper who drives the change” is the key to successfully adopting agile in an organization. 

How JIRA Can Help Your Agile Adoption Process

For AJ, JIRA was a key part in moving to Agile, in part because of the access to better metrics.

“We did have to modify how we use it for some teams, because they were using Kanban boards and simply moving tasks through a simple workflow, which worked for them,” says AJ. “However, we had to switch them over to scrum boards with story point estimations and timeboxes in order to keep better metrics for forecasting work and metrics on the work being completed.” 

Taras’ team turned to JIRA to gain more feedback. “JIRA was one of the main reasons we were interested in adopting agile for our organization,” he says. “It’s the best tool for agile projects. We can get feedback right away and move to continuous delivery.”

What A Successful Agile Adoption Looks Like

Depending on what an organization’s goals are, success will be defined differently based on those needs. 

At AJ’s organization, streamlined communication and greater transparency were top priorities of shifting to agile methodologies.

“We are more communicative now,” says AJ. “Decisions on infrastructure and thFe best tech use are decided by all parties involved and not just pushed from one group to the next. We are able to solve complex issues more efficiently when we are all working as one team and not passing deliverables in a Waterfall format. And we’ve increased transparency regarding the workload and the work involved in completing certain tasks.” 

For Taras, speed to market was an organizational achievement of adopting agile testing methods. “There’s definitely less stress,” he says. “And we get quicker feedback.”

Choosing The Right Tools For Agile Adoption

Adopting agile in an organization is only the first step in achieving testing automation success. You also need to choose the right tools that provide you with the best testing solutions during the process. Zephyr provides the best testing solutions for agile methodologies, empowering your team to continuously strive towards efficiency, speed and excellence.

If your organization wants to build, maintain or sell quality software, you need to seriously consider using a quality management system (QMS) to document your processes, procedures and responsibilities for achieving quality objectives and policies.  Implementing and maintaining a QMS is a crucial part of regulatory compliance if you’re doing development in industries like medtech, biotech and pharma where you often have to show traceability (tying requirements to pieces of code to specific tests to bugs, and so forth), as well as robust change and version management capable of incorporating things like electronic signatures for sign-offs and approvals. 

A QMS is a great way to demonstrate that your company’s development, testing and deployment practices are reliable, repeatable and traceable.  Fortunately, the Jira ecosystem has numerous apps, add-ons, plugins and integrations available to help you build a straight-forward and cost-effective QMS solution you can use to coordinate your software operations to meet both regulatory and customer requirements.

Jira Quality Management

According to Atlassian, Jira is used for issue, project and workflow tracking by over 75,000 customers in 122 countries around the globe; not just for software projects but also for business uses like tracking manufacturing supply chain processes, patients in pharmaceutical trials or job candidates through the HR interview process.

A Jira 'issue' refers to a single work item of any type or size that is tracked from creation to completion.   For example, an issue could be a feature being developed by a software team, a to-do item for a marketing team, or a legal contract being developed with a customer.  Jira projects are flexible on-line working spaces that allow you to group similar issues by team, business unit, product, or stream of work.  Examples of projects are a software development project, a marketing campaign, a helpdesk system, or a product enhancement request system.   A Jira workflow represents the sequential path an issue takes from creation to completion. Here’s an example of a basic Jira workflow:

In this case, Open, In Progress, Under Review, Final Approval and Done represent the status an issue can take, while the arrows represent potential transitions from one status to another. Workflows can be both simple and complex, and can include collaboration features like notifications, comments and assignments that will help get everyone engaged in the Jira Quality Management process.

Jira Quality Management on Agile Software Projects

Making sure the code you write complies with quality standards is not just a management responsibility but also a team level responsibility, especially if you’re doing agile development and testing.  One of the principles of agile development is "working software over comprehensive documentation," which means agile projects shouldn't get bogged down by checklists and documentation when work is handed off from one group to another.  Agile project management focuses on continuous improvement, scope flexibility, team input, and delivering high-quality working software rather than documentation.  Managing quality and compliance on agile projects requires tight collaboration between all team members.

The de facto standard for the agile software development and testing, Jira Software has powerful collaborative functionality that makes planning sprints, tracking daily work, and reporting on project progress across even globally distributed teams easy.  Jira also has major feature sets designed for Scrum and Kanban, the two main types of agile process frameworks used by agile software teams, which are detailed in the following tutorials:

• “Learn Scrum with Jira Software Tutorial”:  https://www.atlassian.com/agile/tutorials/how-to-do-scrum-with-jira-software (Step-by-step instructions on how to drive a Scrum project)
• “Learn Kanban with Jira Software Tutorial”:  https://www.atlassian.com/agile/tutorials/how-to-do-kanban-with-jira-software (Step-by-step instructions on how to drive a Kanban project with Jira Software)

Regulatory Compliance

In certain markets and industries such as the Medical Devices and Pharmaceutical industries in the United States, a formal Corrective Action Preventive Action (CAPA) process may be required as part of a QMS.  CAPA is a process which investigates and solves problems (identified by customer complaints, internal audits, etc.), finds causes, takes corrective action and prevents recurrence of the root causes.  While it’s possible to comply with CAPA standards with a manual paper-based QMS or one built in a program like Microsoft Excel, the reality is that these kinds of solutions don’t scale and are outdated if your team is doing agile development and testing. 

As described above, Jira workflows have forms, rules and automatic escalations that can be built and/or customized to follow almost any regulation or process (like CAPA).  Jira also has dashboards and custom reports that help your entire agile team see the bigger picture and highlight opportunities for improvement, an important part of any QMS.

Jira and Testing 

Although Jira Software is designed for issue, project and workflow tracking on IT projects, many agile teams are also using it for test case management so that development and testing teams can work together in one system.

There are two types of integrations you can use to add testing functionality to Jira:

1. Add-ons like Zephyr for Jira, which are internal integrations that live inside the Jira application and add testing functionality.  Test issues can be created, executed, tracked and reported on just like any other Jira issue in Zephyr for Jira, which has a look n’ feel that is exactly like Jira’s.
2. An external integration with a dedicated test case management tool like Zephyr Standalone, a test management solution managed independently outside the Atlassian ecosystem that offers real-time Jira integration.

Because Jira currently offers no direct integration with automated testing frameworks like Selenium or JUnit, the preferred approach is to handle automated testing via a continuous integration (CI) server and to use Jira primarily for manual testing.  This means setting up acceptance, integration and functional tests inside Jira, which can involve considerable workflow customization.  

You also have to solve the problem of storing test results.  One way is with Atlassian’s Confluence product, which is a knowledge base, or wiki, that stores and organizes all of your information assets around the projects you’re doing in Jira (such as product requirements).  Confluence is integrated with Jira, so these assets become an integrated piece that allows teams to create, collaborate and update these assets together.

Automated Testing with Zephyr for Jira  

You also need to make sure that test cases are written and executed to ensure that each of your Agile Stories are successfully implemented.  Zephyr for Jira allows test cases to be documented and linked to Stories and linked back to requirements in Confluence.  Test cases can then be organized into test cycles, or groups of tests to be executed together.  The results of these test cycles can also be stored as test executions, giving a clear view of the current status of the project in terms of how many of the test cases are passing successfully.

Conclusion

Agile software projects thrive in a culture of collaboration.  Thanks to its ability to integrate with powerful tools like Confluence, Zephyr for Jira and Zephyr Standalone, Jira provides a solid way for agile development and testing teams to collaborate in real-time on requirements, user stories and test cases.  This approach adds to the transparency needed to make sure your company has a successful Quality Management System in place and your agile development, testing and deployment practices are reliable, repeatable and traceable.

About SmartBear Software

SmartBear is behind the software that empowers developers, testers, and operations engineers at over 20,000 of the world’s most innovative organizations including Adobe, JetBlue, MasterCard, and Microsoft. More than 6 million people use our tools to build, test, and monitor great software, faster. Our high-impact tools are easy to try, easy to buy, and easy to use. These tools are backed by a team of people passionate about helping you create software that transforms our world. Those tools are SmartBear tools. That team is SmartBear. For more information, visit: http://smartbear.com, or follow us on LinkedIn, Twitter, or Facebook.

Change is never easy. But in business, as in life, change can be necessary for moving forward. Putting processes in place that can help aid that change can make the experience a lot easier and set your organization up for success in the long run.

We spoke with Jim Beckner III, who is one of our clients that works in the telecommunications industry, about how Zephyr for Jira has benefited his team, allowing them to add a Quality Assurance process to their testing cycle, improving their product quality and creating greater speed-to-market for their clients.

Putting A Process In Place

Jim and his company needed a way to test all of their software before sending it out to customers. For the past six months, they’ve used Zephyr for Jira to manage all their testing and development.

“Without Zephyr for Jira we weren’t able to fully test everything,” he says. “But with it, we can really put the focus on making sure we have a quality product for our customers.”

Prior to investing in Zephyr for Jira, Jim and his team did not have a QA process even though developers were regularly releasing new versions of their software for clients.

“Now that we have Zephyr for Jira we’re able to integrate our development processes with our QA processes,” he says. “We’re able to make sure that we’re testing what we need to test, that we are regression testing to ensure that previous features are still working correctly, and we’re able to tie all of this together and document it so that we can have some visibility of how the different parts of the process are going.”

Integrating To Agile With Ease

Their organization has been in the process of becoming agile over the past year or two, and the development team was the first to adopt an agile methodology, using Jira to help with the transition.

“When the QA team became more of a formalized function here, we were able to take that methodology and use it for our testing as well,” he says. “We work with the schedule of the developers who put together new software every couple of weeks and use Zephyr for Jira to test everything—not only for new features and the bug fixes, but also the previous versions and making sure that everything is still lining up the way our customers expect.”

He reports that there have been zero problems with Zephyr for Jira since integrating it into their testing process over the last several months. Not only has it been easy to set up, but it’s also been a huge time saver since it allows them to tie together all of their testing plans and test a variety of versions of software that is going to be released—all within one platform.

“We use Jira internally for our development team and our support team. Being able to integrate our testing into the same platform has been invaluable,” he says. “It’s hard to find a product that just works right out of the box and that provides you with everything you need without causing a lot of problems.”

Creating More Confidence In Product Releases

Jim’s QA team uses Zephyr for Jira on a day-to-day basis. Zephyr for Jira gives them the ability to look at different versions that have been released by the developers, or that will be released soon, and develop plans based on the tests that have already been developed, as well as write new tests for any new pieces of software.

“Just today we released about 12 different items of software and thanks to Zephyr for Jira we were able to run all of our tests and feel comfortable with what we were releasing.”

He reports that Zephyr for Jira has provided them with the ability to write test plans with all different sorts of test steps and use them for multiple tests, which has been a major asset to their testing plans.

“We can test common features with every release to make sure that we are still functioning the way that we need to,” he says.

A Proud Member Of The Breeze Team

Jim is a member of our Breeze Team, which is a group of Zephyr for Jira user advocates. He loves being a part of the Breeze Team because he believes in supporting great products, which in turn support him.

“The QA team is new here and we are always looking for ways to get some quick results. Zephyr has been able to consistently do that for us,” he says. “It’s great to be part of the Breeze Team and help promote the product, not only for others here at my company, but also for those in other industries as well.”

If you and your team are looking for a tool to help you succeed in your testing capabilities or in making the transition over to agile, Zephyr for Jira will get you there. Check out Zephyr for Jira on the Atlassian Marketplace to sign up for a FREE trial and experience the benefits yourself.

With the ever increasing demand to deploy high quality and reliable software faster, Jira is quickly emerging as a go-to software for agile teams for the end-to-end planning and tracking of their product development cycle.

Recognizing this increasing adoption of Jira and other Atlassian products into the daily workflows of QA and development teams, SmartBear is further extending its support for the test automation efforts of Atlassian users by introducing four new product integrations between Zephyr, the world’s most widely used test management solution, and its leading test automation tools, including SoapUI Pro, TestComplete, CrossBrowserTesting, and LoadNinja.   

We are also excited to announce that ZAPI, our powerful add-on for Zephyr for Jira, is now available for free. This is big news for Zephyr for Jira users who can now utilize advanced automation and continuous testing capabilities, without incurring any additional add-on costs.

By leveraging the advanced test management capabilities in Zephyr and seamlessly integrating the user’s functional and performance test automation efforts, these new product integrations will empower software teams to unify test management and test automation, all within a native Jira experience.

Enhance Test Automation and Continuous Testing 

Zephyr for Jira enables development and testing teams of all sizes to utilize full-featured test management capabilities by seamlessly integrating testing into the product life cycle with Jira.

Linking requirements all the way through to test cases helps to bring traceability into the development process and offers the flexibility needed to build high-quality, competitive software.

Seamless integrations play a critical role in achieving continuous testing for teams on their path towards Agile and DevOps adoption. ZAPI (Zephyr API) creates limitless testing possibilities for Zephyr for Jira users by enabling them to integrate with their favorite test automation and continuous integration tools, and leverage well documented REST APIs to consolidate testing data across multiple tools. These APIs offer key capabilities to project teams, enabling them to easily create new tests and test execution cycles, update test execution status, retrieve search results via zephyr query language, and more.

With ZAPI now available for free, it’s a win-win for QA and development teams!

Experience the Power of Test Management and Test Automation

The integration of Zephyr for Jira with leading test automation tools from SmartBear enables Jira users to streamline their development activities and extend their workflows with test automation capabilities.

Integrate with TestComplete

Create and execute all your automated tests with a native Jira experience! TestComplete is a popular functional and automated testing tool SmartBear provides to enable teams to scale their automated UI testing efforts and maximize automated test coverage.

Zephyr for Jira’s integration with TestComplete will allow you to take advantage of a centralized system to define and plan your test strategy right inside Jira, create and execute automated tests with TestComplete, and seamlessly extend the existing Jira workflows to view, track, and share test results. The result is better insights into the testing outcomes, faster defect resolution, enhanced traceability and better collaboration between teams. Learn more.

Integrate with SoapUI Pro

Manage all of your functional and security API tests right inside Jira! SoapUI is the world's most widely-used open source automated testing tool, enabling teams to effortlessly automate advanced API tests.

With the Zephyr for Jira and SoapUI integration, you can now manage and execute all of your functional and security API tests from a single interface, and gain end-to-end visibility into the testing progress by capturing and reporting relevant quality metrics across QA and testing activities without having to leave Jira. Learn more.

Integrate with CrossBrowserTesting

Get centralized insights into your continuous testing activities! CrossBrowserTesting is the only all-in-one testing platform that enables teams to run automated, visual, and manual tests on thousands of real desktop and mobile browsers.

With this integration, you can bring Zephyr and CrossBrowserTesting into your CI/CD pipeline to get centralized insights across testing activities, discover defects faster, and get continuous feedback on the executed test cases via Jenkins. The ability to view comprehensive insights into the parallel test executions running across real browsers and devices will empower your teams with the real-time visibility needed to make informed release decisions. Learn more.

Integrate with LoadNinja

Load test continuously within your CI/CD pipeline! Zephyr for Jira’s integration with LoadNinja, the leading cloud-based platform to load test web applications, will now enable you to easily automate and manage your UI performance tests in the CI/CD pipeline.

Teams can record and create new test cases with LoadNinja, launch load tests with Jenkins, and gain complete visibility into the load testing activity by capturing and analyzing test performance with Zephyr for Jira, resulting in faster identification and resolution of performance issues. Learn more.

With these new product integrations SmartBear demonstrates its commitment to the Atlassian ecosystem and a focus on bringing new test automation capabilities to the Atlassian users.

Maximize testing efficiency and simplify user management with our latest Zephyr 6.3 release.

In an agile DevOps environment, development, testing and operations all must work together to meet frequent iterations, releases and delivery goals. But often software teams face a number of challenges: globally distributed teams, slow release cycles, business decisions based on stale data, and an ever-changing collection of tools, from a variety of vendors. These problems are only compounded by error-prone manual processes such as handoffs from a development group to a QA group.

The solution, of course, is a collaborative environment where development, testing and operations teams all work together to meet frequent iterations and releases.  Zephyr Standalone is a robust test management solution offering best-in-class Jira integration, along with seamless integration with an array of automation and CI/CD tools. Zephyr provides end-to-end traceability across both manual and automated test cases, which helps your teams gain complete visibility over the entire development and QA cycle, and collaborate better.

The latest enhancements in Zephyr 6.3 make it that much easier for agile teams to be on track to achieve their software quality and release goals by realizing the benefit of Zephyr’s tight integration with Jira, advanced user management, and a series of enhancements in test case design, custom fields, global repository and reporting.

User Group Sync and Role Assignments

Zephyr 6.3 now allows auto creation of users and assignment of user roles in Zephyr, based on their Crowd and LDAP group assignments. Within user groups a list of users can be allocated to specific projects rather than individually assigning a user to a project. By simplifying user management and significantly reducing the user administration time for admins, this capability will help boost administration efficiencies, especially for organizations with a large user base.

6.3 also brings added support for different authentication methods when connecting to multi-Jira instances and provides seamless login experience for Zephyr SSO.

Enhanced User Roles & Permissions

By default, Zephyr Standalone users are assigned one of 5 roles: Test Manager, Test Lead, Tester, Defect and Dashboard user, although custom roles can be defined as needed.  The new release further advances role permissions with the ability to assign project specific roles to users. For instance, you can now assign more granular permissions by customizing a testers role at the project level and also grant permissions for certain actions at the project level.

This enables effective user management by empowering QA managers and admins to better manage the users/QA resources via well-defined access permissions.

Advance Test Case Reusability 

Zephyr enables centralized test case management and facilitates test case re-use by allowing users to share and copy test cases stored in the Global Test Case Repository across Projects and Releases. With this release teams can further avert the need to re-write test cases and make better use of their testing time by easily moving the ‘ownership’ of a test case from the 'project' to the 'global' repository at the individual or folder level, making it available to other users and projects.

Enhanced Tracking and Traceability

In keeping with the interoperability theme of the latest Zephyr Standalone release, users also get better traceability and tracking whenever a requirement in Jira is mapped to a test case in Zephyr via the Zephyr test case issue link that reflects both test execution status and cycle details.

6.3 allows synchronization at the individual test execution level, so users can easily see the status of the latest version of a test case when it’s updated in the test case repository.

Custom Field Enhancements

Teams will benefit with greater customizability with custom fields now added for test steps, test cases and requirements. These custom fields can be managed at a global level across all projects, and on a project level to configure fields exclusive to specific projects.

Rich Text custom fields have also been introduced for test steps, test repository and requirements, which makes it easy for customers migrating to Zephyr to retain the original format of all their test case data.  An additional comment text field has been added to test steps to provide further context for a test step, together with the ability to bulk append text to the test case comments field. The addition of Cascading custom fields will help users with faster filtering and searching of relevant data.

These are just some key feature additions we've made to Zephyr 6.3! The new release offers a host of new features and functionality that will make the lives of your QA teams, admins and Test managers easier while providing valuable information about how your entire release is performing.  

Join our live Webinar on May 15th: Maximize Your Testing Efficiency and Simplify User Management with Zephyr 6.3, to learn more about Zephyr 6.3 enhancements and see the new features in action.

An old business adage says you can't manage what you can't measure, which is why test metrics and reports are essential for testers, agile teams and QA managers who want to boost the effectiveness and efficiency of their QA and testing efforts.

A recent SmartBear webinar, “Advance Your Testing Strategy with Robust Test Metrics & Reporting,” discussed how to use advanced reporting capabilities in Zephyr for Jira to measure, analyze and report on testing data, as well as how to use specific software test metrics to track the effectiveness of your quality assurance process.

During the webinar, we asked our audience which tools they were currently using for test management and reporting, and almost 80% responded that they were using Jira, with smaller percentages using spreadsheets and proprietary testing tools.  A solid majority (69%) of respondents also said they preferred continuing to work with a test management solution native to Jira rather than one outside of it.

Here are some of the main takeaways from the webinar to help teams measure what’s important and identify bottlenecks in their QA process that can prevent them from releasing high-quality software.

Why Test Metrics are Critical to Your Testing Strategy

Software test metrics are quantitative measures that help estimate the process quality and health of your software testing efforts.  Test metrics are generally classified into three categories:

• Process metrics.These metrics are used to gauge and improve the process efficiency of your software development lifecycle. Process metrics provide insights into things like your test execution coverage, test execution productivity, and so on.
• Product metrics. These help you to monitor and continually improve the quality of your software products by telling you things like your defect density, defect fix rate, and so forth.
• Project metrics. Some of the most important testing metrics can be gathered at the software project level itself, across a specific release cycle or over the course of multiple releases and projects. QA members can use this information to gain a better idea of how effective their efforts have been on any specific project. This can include metrics like requirements coverage or defect distribution by cycle or phase.

Building software without metrics in place is like running a time trial race without a stopwatch.  If software testing and development is a marathon, then a metric like requirements coverage determines what the finish line and ideal finishing time is, and the times needed at all stages to meet the final goal.  Without a requirements coverage metric, developers and testers may miss certain key requirements that are absolutely required in the final product.

Test metrics allow you to assess if your project progress is in line with your release goals. This is a big deal since it allows teams to clearly identify areas that need improvement and make better informed decisions about process or budget changes that are backed by real time data.

Similarly, metrics allow quality and development teams to track software defects, prioritize them based on insights such as which defects have maximum impact on your requirements or which ones present risks for future bugs.  Testing metrics will also help management teams streamline your overall testing process, which can improve the cost efficiency of future releases.

Why Reporting is Critical to Your Testing Strategy

Reporting is one of the most important phases in your software testing cycle because it makes it easy to evaluate the efficiency and timeline of your development process so you can make continual improvements to it.  Reports and dashboards allow you to visualize testing data in a form that’s easy to understand by developers, testers and managers.

You can track defects and software quality with test reports such as test execution charts that show which defects are impacting the most test cases.  If you have the right test management tool in place, you can also easily share these findings with other people—which will ultimately help your teams make better informed quality release decisions.

Test metrics and reporting are critical to your testing strategy for three main reasons.  One is to facilitate communication and increased collaboration between quality assurance and development teams.  Another is to provide project teams information on the status of their test executions, top defects and other key testing elements.  Finally, they also provide management with deeper insights into the efficiency and productivity of all your software development activities.

How Zephyr for Jira Helps Advance Your Testing Strategy

Enterprises looking for test management software that provides more than just basic code coverage will be excited by the advanced test planning, dashboard and automation features in the latest Zephyr 6.4 Standalone Edition. Especially useful for Agile DevOps project teams that need to release products in shorter and more frequent release cycles, the new version takes advantage of Zephyr's brand-new approach to customer driven product development where customers vote on features they desire, via a public customer feedback Trello board. Highlights of this release include a user-friendly automation interface and easier dashboard creation and navigation. You now have the ability to bulk update test steps across multiple test cases and customize the names of test case priority fields (i.e. "urgent", "high", "medium", "low").

Enhanced Test Automation

Automation testing makes software testing easier, faster and more reliable and is essential in today's fast-moving software delivery environment. Usually seen as an alternative to time-consuming and labor-intensive manual testing, automation testing uses software tools to run a large number of tests repeatedly to make sure an application doesn't break whenever new changes are introduced.

Agile teams can execute one-touch control of test automation from within Zephyr with Vortex, which allows you to integrate with a large and growing suite of automated testing frameworks (including EggPlant, Cucumber, Selenium, UFT, Tricentis, and more) with minimal configuration. In addition to being able to create and reuse manual tests on agile projects, Vortex makes it easy to bring in and work with automation information from across your development stack, including from systems external to your organization. Vortex allows users--wherever they are in your organization-- to integrate, execute, and report on test automation activities. By providing an intuitive screen that lets users access both manual and automated test cases at the same time, Vortex helps agile teams better monitor their overall automation effort (that is, the number of manual versus automated tests from one release to another.

Integration with TestComplete and SoapUI Pro

Zephyr Standalone 6.4 now integrates with leading SmartBear test automation tools, TestComplete and SoapUI Pro, which are included in Vortex’s list of supported automation frameworks. Users can now select TestComplete and SoapUI Pro from the Vortex setup from either ‘Automation’ or ‘Folder Watcher’ setup and retrieve execution results to be visible within Zephyr. TestComplete tool has a powerful and comprehensive set of features for web, mobile, and desktop application testing. In addition to having an easy-to-use record and playback feature, TestComplete allows testers to use JavaScript, VBScript, Python, or C++Script to write test scripts. The tool also has an object recognition engine that can accurately detect dynamic user interface elements, which makes it especially useful in applications that have dynamic and frequently changing user interfaces. SoapUI is a test automation tool for functional testing, web services testing, security testing, and load testing. Specifically designed for API testing, SoapUI supports both REST and SOAP services. It provides ‘drag and drop’ options for creating test suites, test steps and test requests to build complex test scenarios without having to write scripts.

Dashboard Improvements

The capability of live, automated dashboards, which can keep your whole company updated on every aspect of testing and product quality, has been further expanded in the new release. Dashboards in Zephyr Standalone have always provided users with the ability to filter data based on testing activities in a variety of areas (such as Requirements, Test Repository, Test Planning, and Test Execution) and to compare trends over different times. The new release allows multiple dashboard owners--Users, Leads, Managers, and Administrators--to manage individual dashboards in parallel, giving multiple owners the ability to manage a particular dashboard and also solves the problems caused when a dashboard manager is unavailable or has left the company. The new improvements also gives users the ability to clone all or specific gadgets on a dashboard, or easily clone an already existing complex dashboard for their next release, making dashboard creation much faster and saving users a significant amount of time.

Improved Jira Integration

Zephyr Standalone is a test management solution that offers real-time, bi-directional integration with Jira, which means your requirements, user stories, epics (collections of stories), sprints, new features, improvements, tasks and sub-tasks can be connected to your test cases, test case assignments, and execution information, as well as your defects. The new release improves the way Zephyr automatically handles real-time changes in requirements in Epics when Zephyr Standalone is sync’d with Jira.

Security and Privacy Updates

Maintaining Zephyr’s relentless focus on fixing any bugs that impact its customers, this upgrade addresses a security vulnerability in the Zephyr Jenkins plugin, which also makes it easier to find the subset of test cases impacted by this or other vulnerabilities. Jenkins runs tests automatically every time a developer pushes new code into the source repository. Because CI (Continuous integration) detects bugs early on in development, bugs are typically smaller, less complex and easier to resolve.

The General Data Protection Regulation (GDPR) is a regulation that requires businesses to protect the personal data and privacy of European Union (EU) citizens for transactions that occur within EU member states. To help customers maintain GDPR compliance, the latest Zephyr Standalone release also now provide a method (available upon request) to completely remove or anonymize a user's PII (Personally Identifiable Information) from the database, while still keeping historical execution and audit data.