UI automation from scratch: Part 1 – your test in testNG

I like so far that we are doing all this from scratch, following only the guidelines provided by the originators. For TestNG class generation, I’d like to follow this document: http://testng.org/doc/documentation-main.html

TestNG comes with many benefits but if I were to pick one, that would be the use of DataProvider.

But before starting with TestNG classes, we need to take care of the dependencies on TestNG first. For any maven dependency, it’s a good habit to search on this page: https://mvnrepository.com

Searching on ‘testng’ results below:

<!-- https://mvnrepository.com/artifact/org.testng/testng -->
<dependency>
 <groupId>org.testng</groupId>
 <artifactId>testng</artifactId>
 <version>6.10</version>
</dependency>

Let’s add it in pom.xml (and don’t forget to Source > Format from Eclipse menu). This section should replace JUnit section under dependencies.

We also need to install TestNG plugin for Eclipse by following this: http://testng.org/doc/eclipse.html (Eclipse > Help > Install New Software > work with: http://beust.com/eclipse > hit Add, give name TestNG > check on everything under TestNG > hit Next > Accept Agreement > Say yes when it’s asking to restart Eclipse)

Now we can create our AppTest in TestNG

package com.stonecress;
import org.testng.annotations.Test;

public class AppTest {
  @Test(groups = { "functest", "checkintest" })
  public void testMethod1() {
  }
 
  @Test(groups = {"functest", "checkintest"} )
  public void testMethod2() {
  }
 
  @Test(groups = { "functest" })
  public void testMethod3() {
  }
}

And you can try running it in Eclipse by right click > Run As > TestNG test

Screen Shot 2016-12-23 at 11.48.21 PM

Also try on command line

mvn test

To find

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.stonecress.AppTest
Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNG652Configurator@484b61fc
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.458 sec

Results :

Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

[INFO] ----------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ----------------------------------------------------------

Try these to find the meaning of test groups and features of surefire plugin working with maven

mvn test -Dgroups=functest
mvn test -Dgroups=checkintest
mvn test -Dtest=AppTest
mvn test -Dtest=AppTest#testMethod3
mvn test -Dtest=AppTest#testMethod3+testMethod2

If the last line does not work, probably your Eclipse came with surefire plugin older than 2.19. Let’s add below to the pom.xml

<project>
...
 <dependencies>
...
  <dependency>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19.1</version>
  </dependency>
 </dependencies>
...
 <build>
  <pluginManagement>
   <plugins>
    <plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-surefire-plugin</artifactId>
     <version>2.19.1</version>
    </plugin>
   </plugins>
  </pluginManagement>
 </build>
</project>

Converting non standard test output to unit test xml

Let’s say a build generates a json file that contains test names and the test results in this form.

{
  "Integration Test1": "Passed",
  "Integration Test2": "Failed",
  "Integration Test3": "Passed"
}

This type of summary won’t be so useful for Jenkins to track and report as a post build process. A simple python based test can convert that to a standard JUnit format xml file.

test/test.py

from nose_parameterized import parameterized, param
import unittest, json

class TestPSRResults(unittest.TestCase):
	def load_results():
		with open('TestResults.json') as f:
			summary_dict = json.load(f)
		print summary_dict
		summary = []
		for name in summary_dict.keys():
			summary.append((name, summary_dict[name]))
		return summary

	@parameterized.expand(load_results)
	def test_result(self, name, result):
		assert result == 'Passed' , "Actual Result : {}".format(result)

unittest.cfg

[unittest]
plugins=nose2.plugins.junitxml

[junit-xml]
always-on=True
test_fullname=True

Configure Jenkins to run this as a build step after the actual test that generates TestResults.json

nose2 -v

Then Jenkins now can publish JUnit test result report for nose2-junit.xml.
JUnitJenkinsChart

UI automation from scratch: Part 0 – create an empty project

To start a UI test automation project, You can start everything from scratch but it’s easier if you have some skeleton project to work with. I am going to explain the steps I had to go through to create this skeleton project. So if you want to start building your test automation right away, you can jump to a much later part. This ‘Part 0’ will go through creating an empty git repo, make a maven project, then push it back to git.

  1. I created a github repo named ‘ui-automation’
  2. Before cloning it, I created my ssh key pair from my macbook
    ssh-keygen -C <my email address>
  3. And copied the ~/.ssh/id_rsa.pub content into Github – settings – SSH key section
  4. Now I can clone and do all the git transactions without typing in my github credentials
    cd ~/workspace
    git clone https://github.com/changgull/ui-automation.git
  5. This is an empty project. Now we need to create a maven project so we can benefit from its build facilitation. Creating a maven project is just one command liner. Reference here. From your workspace folder and run this command:
    mvn archetype:generate -DgroupId=com.mycompany.ui -DartifactId=ui-automation-2
  6. That will create a directory named ui-automation-2. Now modify the pom.xml to change the artifactId from ui-automation-2 to ui-automation then move all contents to ui-automation. Now I see the build pass from ui-automation directory.
    mvn test
  7. Now you can push the change made to this project back to the git repo
    mvn clean
    echo *.class >.gitignore
    git add * .gitignore
    git commit -am "Converted to Maven"
    git push origin master
  8. If Eclipse is your choice of IDE, now you may import this project at this step.
    Screen Shot 2016-12-14 at 11.35.45 PM
    Screen Shot 2016-12-14 at 11.37.42 PM