TestNG testing framework

Overview

Build Status Maven Central Bintray License Sonarqube tech debt Sonarqube Quality Gate Status

Documentation available at TestNG's main web site.

Release Notes

Need help?

Before opening a new issue, did you ask your question on

If you posted on both sites, please provide the link to the other question to avoid duplicating the answer.

Are you sure it is a TestNG bug?

Before posting the issue, try to reproduce the issue in a shell window.

If the problem does not exist with the shell, first check if the issue exists on the bugtracker of the runner, and open an issue there first:

Which version are you using?

Always make sure your issue is happening on the latest TestNG version. Bug reports occurring on older versions will not be looked at quickly.

Have you considered sending a pull request instead of filing an issue?

The best way to report a bug is to provide the TestNG team with a full test case reproducing the issue. Maybe you can write a runnable test case (check the src/test/ folder for examples) and propose it in a pull request Don't worry if the CI fails because it is the expected behavior. This pull request will be a perfect start to find the fix :)

How to create a pull request?

Refer our Contributing section for detailed set of steps.

We encourage pull requests that:

  • Add new features to TestNG (or)
  • Fix bugs in TestNG

If your pull request involves fixing SonarQube issues then we would suggest that you please discuss this with the TestNG-dev before you spend time working on it.

Comments
  • Release TestNg more frequently

    Release TestNg more frequently

    Hi,

    I see a need of release TestNg more frequently, Today it seems to be 6-9 month between the releases. And the number of features included in TestNg is kind of big, In our project we do releases for every commit, which allow fast feed back.
    And in our project we have restriction on using forks.

    The risk I see now is if we find a bug in a release we need to wait very long time until next release, and then there is a risk that something else have changed.

    Maybe a release every quarter wouldn't create to much extra job for your team.

    What do you think?

    opened by martinaldrin 75
  • Execution of test methods in different classes is interleaved when dependsOnMethod is used

    Execution of test methods in different classes is interleaved when dependsOnMethod is used

    I am seeing an odd order of test executions in TestNG.

    I created a simple project, with this testng.xml:

    <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
    <suite name="Simple suite">
        <test name="Simple tests">
            <packages>
                <package name="testngtests"></package>
            </packages>
        </test>
    </suite>
    

    The package testngtests contains classes MyTest1, MyTest2 with several test methods, all of which are empty:

    @Test
    public void testEins(){
    
    }
    

    etc.

    Two of the tests use "dependsOnMethods":

    @Test(dependsOnMethods ="testEins")
    public void testZwei(){}
    
    @Test(dependsOnMethods ="testZwei")
    public void testDrei(){}
    

    If I run the suite using TestNG V 5.10-jdk15 or V 6.0.1, they always run in the same order:

    testngtests.MyTest1.testEins()
    testngtests.MyTest1.testZwei()
    testngtests.MyTest1.testDrei() 
    testngtests.MyTest2.testEinsA2()
    testngtests.MyTest2.testDeins2()
    testngtests.MyTest2.testDreiB2() 
    testngtests.MyTest2.testZweiZ2()
    

    as expected.

    However, with TestNG 6.2.1, the order is:

    MyTest1.testEins()
    MyTest2.testDeins2()
    MyTest2.testDreiB2()
    MyTest2.testEinsA2()
    MyTest2.testZweiZ2()
    MyTest1.testZwei()
    MyTest1.testDrei()
    

    Note that now the tests from different classes are interleaved.

    I expected tests from the same class to always run together (though not always in the same order). From the TestNG manual: "TestNG will try to group test methods by class."

    So did some change to TestNG break ordering in 6.2.1?

    Or does TestNG simply not guarantee that test will be grouped by class? In that case the docs should be clarified to explain when TestNG will group by class - or not.

    Also, in that case it would be nice to have a setting that forces grouping tests by class. Should I file a feature request for that?

    opened by sleske 67
  • There was an error in the forked process

    There was an error in the forked process

    Hello, I've tried to migrate my project from jdk 1.6 to jdk 1.7, but faced with following issue:

    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on project test: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test failed: There was an error in the forked process
    [ERROR] java.lang.NullPointerException
    [ERROR] at java.lang.String.length(String.java:623)
    [ERROR] at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:414)
    [ERROR] at java.lang.StringBuffer.append(StringBuffer.java:237)
    [ERROR] at org.testng.reporters.TestHTMLReporter.generateTable(TestHTMLReporter.java:114)
    [ERROR] at org.testng.reporters.TestHTMLReporter.generateLog(TestHTMLReporter.java:305)
    [ERROR] at org.testng.reporters.TestHTMLReporter.onFinish(TestHTMLReporter.java:40)
    [ERROR] at org.testng.TestRunner.fireEvent(TestRunner.java:1241)
    [ERROR] at org.testng.TestRunner.afterRun(TestRunner.java:1032)
    [ERROR] at org.testng.TestRunner.run(TestRunner.java:621)
    [ERROR] at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
    [ERROR] at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
    [ERROR] at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
    [ERROR] at org.testng.SuiteRunner.run(SuiteRunner.java:240)
    [ERROR] at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    [ERROR] at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    [ERROR] at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
    [ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
    [ERROR] at org.testng.TestNG.run(TestNG.java:1057)
    [ERROR] at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:91)
    [ERROR] at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:204)
    [ERROR] at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:107)
    [ERROR] at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:113)
    [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
    [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
    [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
    [ERROR] -> [Help 1]
    

    I'm not sure is this a testng problem. Please close issue, if not. P.S. jdk: 1.7.0_25 maven: 3.0.5 maven-surfire-plugin: 2.16

    Feature: report 
    opened by maxymshg 60
  • Getting Unreachable browser exception after a while when I am running the TestNG tests. OS : MAC

    Getting Unreachable browser exception after a while when I am running the TestNG tests. OS : MAC

    I am running my tests in MAC as well as in Windows. In windows, the code works like a charm. In MAC OSX, I run the same code and it will be working fine BUT after a while I get unreachable browser exception. I am running my test through maven surefire plugin. maven version : 3.3.9 surefire plugin : 2.19.1 testng version : 6.9.10 java.version: 1.8.0_77

    Here is my stacktrace

    UnreachableBrowserException: Error communicating with the remote browser. It may have died.
    Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
    System info: host: 'Hemanths-MacBook-Pro.local', ip: '192.168.0.8', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.4', java.version: '1.8.0_77'
    Driver info: driver.version: RemoteWebDriver
    Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEnabled=false, chrome={userDataDir=/var/folders/cd/vbgjmfqj67s2_7lxl03xm5wc0000gn/T/.org.chromium.Chromium.uH5fCE}, takesHeapSnapshot=true, databaseEnabled=false, handlesAlerts=true, hasTouchScreen=false, version=50.0.2661.94, platform=MAC, browserConnectionEnabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=true, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true}]
    Session ID: 900637934c566c1fb4ddc890c4eaaa10
    org.openqa.selenium.remote.UnreachableBrowserException: Error communicating with the remote browser. It may have died.
    Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
    System info: host: 'Hemanths-MacBook-Pro.local', ip: '192.168.0.8', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.4', java.version: '1.8.0_77'
    Driver info: driver.version: RemoteWebDriver
    Capabilities [{applicationCacheEnabled=false, rotatable=false, mobileEmulationEnabled=false, chrome={userDataDir=/var/folders/cd/vbgjmfqj67s2_7lxl03xm5wc0000gn/T/.org.chromium.Chromium.uH5fCE}, takesHeapSnapshot=true, databaseEnabled=false, handlesAlerts=true, hasTouchScreen=false, version=50.0.2661.94, platform=MAC, browserConnectionEnabled=false, nativeEvents=true, acceptSslCerts=true, locationContextEnabled=true, webStorageEnabled=true, browserName=chrome, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true}]
    Session ID: 900637934c566c1fb4ddc890c4eaaa10
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:665)
        at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:363)
        at org.openqa.selenium.remote.RemoteWebDriver.findElementByXPath(RemoteWebDriver.java:500)
        at org.openqa.selenium.By$ByXPath.findElement(By.java:361)
        at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:355)
        at org.openqa.selenium.support.pagefactory.DefaultElementLocator.findElement(DefaultElementLocator.java:69)
        at org.openqa.selenium.support.pagefactory.internal.LocatingElementHandler.invoke(LocatingElementHandler.java:38)
        at com.sun.proxy.$Proxy16.isDisplayed(Unknown Source)
        at org.openqa.selenium.support.ui.ExpectedConditions.elementIfVisible(ExpectedConditions.java:302)
        at org.openqa.selenium.support.ui.ExpectedConditions.access$100(ExpectedConditions.java:41)
        at org.openqa.selenium.support.ui.ExpectedConditions$10.apply(ExpectedConditions.java:288)
        at org.openqa.selenium.support.ui.ExpectedConditions$10.apply(ExpectedConditions.java:285)
        at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:238)
        at org.etna.utils.Waiting.explicitWaitVisibilityOfElement(Waiting.java:36)
        at org.etna.customer.pageobjects.HomePageObjects.waitForProfileDropdownLink(HomePageObjects.java:508)
        at org.etna.modules.LoginModuleTest.TC_Login_009_010_RememberPassword_UserNameAndPasswordRefill(LoginModuleTest.java:115)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:86)
        at org.testng.internal.MethodInvocationHelper$1.runTestMethod(MethodInvocationHelper.java:197)
        at org.etna.maincontroller.MainController.run(MainController.java:208)
        at org.testng.internal.MethodInvocationHelper.invokeHookable(MethodInvocationHelper.java:209)
        at org.testng.internal.Invoker.invokeMethod(Invoker.java:639)
        at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:820)
        at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1128)
        at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
        at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
        at org.testng.TestRunner.privateRun(TestRunner.java:782)
        at org.testng.TestRunner.run(TestRunner.java:632)
        at org.testng.SuiteRunner.runTest(SuiteRunner.java:366)
        at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:361)
        at org.testng.SuiteRunner.privateRun(SuiteRunner.java:319)
        at org.testng.SuiteRunner.run(SuiteRunner.java:268)
        at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
        at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
        at org.testng.TestNG.runSuitesSequentially(TestNG.java:1244)
        at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
        at org.testng.TestNG.run(TestNG.java:1064)
        at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:281)
        at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75)
        at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:121)
        at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
    Caused by: org.openqa.selenium.WebDriverException: java.net.SocketException: Bad file descriptor
    Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'
    System info: host: 'Hemanths-MacBook-Pro.local', ip: '192.168.0.8', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.11.4', java.version: '1.8.0_77'
    Driver info: driver.version: RemoteWebDriver
        at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:91)
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:644)
        ... 45 more
    Caused by: java.net.SocketException: Bad file descriptor
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
        at org.apache.http.impl.conn.LoggingOutputStream.write(LoggingOutputStream.java:77)
        at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:126)
        at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:138)
        at org.apache.http.impl.io.SessionOutputBufferImpl.flush(SessionOutputBufferImpl.java:146)
        at org.apache.http.impl.io.ContentLengthOutputStream.flush(ContentLengthOutputStream.java:101)
        at org.apache.http.entity.ByteArrayEntity.writeTo(ByteArrayEntity.java:117)
        at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:158)
        at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)
        at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:237)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:122)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.openqa.selenium.remote.internal.ApacheHttpClient.fallBackExecute(ApacheHttpClient.java:144)
        at org.openqa.selenium.remote.internal.ApacheHttpClient.execute(ApacheHttpClient.java:90)
        at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:142)
        at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:82)
    

    PFB my TestNG.xml file

    <suite name="Etna Automation Suite">
        <listeners>
                <listener class-name="org.etna.maincontroller.MainController"/>
                <listener class-name="org.etna.utils.ReportGenerator"/>
        </listeners>
      <test name="End to End Testing">
        <groups>
        <run>
        <include name="regression"/>
        </run>
        </groups>
    
        <classes>
          <class name="org.etna.modules.ShopByBrandsModuleTest"/>
          <class name="org.etna.modules.GeneralSearchModuleTest"/>
          <class name="org.etna.modules.PDPModuleTest"/>
          <class name="org.etna.modules.ShopByManufacturersModuleTest"/>
          <class name="org.etna.modules.LoginModuleTest"/>
          <class name="org.etna.modules.MyCartModuleTest"/>
          <class name="org.etna.modules.CategoryModuleTest"/>
          <class name="org.etna.modules.HomePageModuleTest"/>
          <class name="org.etna.modules.APAModuleTest"/>
        </classes>
      </test>
    </suite> 
    

    *Each class contains atleast 20 test cases each. *

    Here is my pom.xml too.

       <modelVersion>4.0.0</modelVersion>
        <groupId>ETNA</groupId>
        <artifactId>inhouse.NewStandardEcommerceTemplate</artifactId>
        <version>1.0-SNAPSHOT</version>
        <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
            <compiler.version>1.7</compiler.version>
            <allure.version>1.5.0.RC2</allure.version>
            <aspectj.version>1.8.9</aspectj.version>
            <version.selenium>2.53.0</version.selenium>
             <version.slf4j>1.7.7</version.slf4j>
        </properties>
        <name>ETNA</name>
        <description>New standard ecommerce template</description>
        <dependencies>
         <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.7.0</version>
      </dependency>
        <dependency>
        <groupId>com.pojosontheweb</groupId>
        <artifactId>monte-repack</artifactId>
        <version>1.0</version>
        </dependency>
        <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>6.9.10</version>
      <scope>test</scope>
    </dependency>
        <dependency>
        <groupId>com.jayway.restassured</groupId>
        <artifactId>rest-assured</artifactId>
        <version>2.9.0</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-firefox-driver</artifactId>
            <version>${version.selenium}</version>
        </dependency>
        <dependency>
        <groupId>org.zeroturnaround</groupId>
        <artifactId>zt-zip</artifactId>
        <version>1.7</version>
        </dependency>
        <dependency>
        <groupId>javax.mail</groupId>
        <artifactId>mail</artifactId>
        <version>1.4.7</version>
    </dependency>
            <dependency>
                <groupId>org.seleniumhq.selenium</groupId>
                <artifactId>selenium-java</artifactId>
                <version>${version.selenium}</version>
                <scope>test</scope>
            </dependency>
            <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-server</artifactId>
        <version>${version.selenium}</version>
    </dependency>
            <dependency>
                <groupId>ru.yandex.qatools.allure</groupId>
                <artifactId>allure-testng-adaptor</artifactId>
                <version>${allure.version}</version>
                <exclusions>
                    <exclusion>
                        <groupId>junit</groupId>
                        <artifactId>junit</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
            <dependency>
                <groupId>org.hamcrest</groupId>
                <artifactId>hamcrest-all</artifactId>
                <version>1.3</version>
            </dependency>
         <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-api</artifactId>
                <version>${version.slf4j}</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
                <version>${version.slf4j}</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>jul-to-slf4j</artifactId>
                <version>${version.slf4j}</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>jcl-over-slf4j</artifactId>
                <version>${version.slf4j}</version>
                <scope>test</scope>
            </dependency>
             <dependency>
                <groupId>org.aspectj</groupId>
                <artifactId>aspectjtools</artifactId>
                <version>${aspectj.version}</version>
              </dependency>
               <dependency>
                       <groupId>org.aspectj</groupId>
                       <artifactId>aspectjrt</artifactId>
                        <version>${aspectj.version}</version>
                        </dependency>     
        </dependencies>
        <build>
    <pluginManagement>
         <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.1</version>
                    <configuration>
                        <source>${compiler.version}</source>
                        <target>${compiler.version}</target>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.19.1</version>
                    <configuration>
                     <suiteXmlFiles>
                        <suiteXmlFile>testng.xml</suiteXmlFile>
                     </suiteXmlFiles>
                        <argLine>-javaagent:${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar</argLine>
                    </configuration>
                    <dependencies>
                        <dependency>
                            <groupId>org.aspectj</groupId>
                            <artifactId>aspectjweaver</artifactId>
                            <version>${aspectj.version}</version>
                        </dependency>
                    </dependencies>
                </plugin>
                <plugin>
                    <groupId>org.eclipse.jetty</groupId>
                    <artifactId>jetty-maven-plugin</artifactId>
                    <version>9.2.10.v20150310</version>
                    <configuration>
                        <webAppSourceDirectory>${project.build.directory}/site/allure-maven-plugin</webAppSourceDirectory>
                        <stopKey>stop</stopKey>
                        <stopPort>1234</stopPort>
                    </configuration>
                </plugin>
            </plugins>
    </pluginManagement>
        </build>
        <reporting>
            <excludeDefaults>true</excludeDefaults>
            <plugins>
                <plugin>
                    <groupId>ru.yandex.qatools.allure</groupId>
                    <artifactId>allure-maven-plugin</artifactId>
                    <version>2.5</version>
                      <configuration>
                        <reportVersion>${allure.version}</reportVersion>
                    </configuration>
                </plugin>
            </plugins>
        </reporting>
    </project>
    

    Again, reminding you that the tests work perfectly fine windows machine. OSX version : 10.11.4 . I hope I can get a fix for this.

    Also, Right clicking the testng.xml and clicking on run as testng suite IS WORKING FINE.

    3rd-party-issue 
    opened by hemanthsridhar 59
  • ConcurrentModificationException

    ConcurrentModificationException

    When running a test class using 6.9.13.8 the tests pass, but the following error is printed. This does not occur in 6.9.12. (Running via Eclipse)

    [TestNG] Reporter org.testng.reporters.JUnitReportReporter@2b552920 failed
    java.util.ConcurrentModificationException
        at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
        at java.util.ArrayList$Itr.next(ArrayList.java:851)
        at org.testng.reporters.JUnitReportReporter.getNextConfiguration(JUnitReportReporter.java:220)
        at org.testng.reporters.JUnitReportReporter.generateReport(JUnitReportReporter.java:105)
        at org.testng.TestNG.generateReports(TestNG.java:1175)
        at org.testng.TestNG.run(TestNG.java:1102)
        at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
        at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:152)
        at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:57)
    
    opened by ben-manes 49
  • [GITHUB-2664] Restore DependsOnGroups Behaviour.

    [GITHUB-2664] Restore DependsOnGroups Behaviour.

    Fixes #2664 .

    Did you remember to?

    • [x] Add test case(s)
    • [x] Update CHANGES.txt

    We encourage pull requests that:

    • Add new features to TestNG (or)
    • Fix bugs in TestNG

    If your pull request involves fixing SonarQube issues then we would suggest that you please discuss this with the TestNG-dev before you spend time working on it.

    opened by martinaldrin 47
  • Problem of combination dependsOnMethods and RetryAnalyzer

    Problem of combination dependsOnMethods and RetryAnalyzer

    Hi guys,

    when a test failed and passed after retry using IRetryAnalyzer implementation, all dependent tests are skipped. This seems to be a bug. Have a look at my simple example below.

    TestNG Version

    latest (6.12)

    Expected behavior

    1. a() failed //i==0
    2. a() passed //i==1
    3. b() passed //i==2

    Actual behavior

    1. a() failed //i==0
    2. a() passed //i==1
    3. b() skipped // this is the problem

    Is the issue reproductible on runner?

    yes

    • [ ] Shell
    • [ ] Maven
    • [ ] Gradle
    • [ ] Ant
    • [x] Eclipse
    • [ ] IntelliJ
    • [ ] NetBeans

    Test case sample

    public class Example {
        int i = 0;
    
        public static class Retry implements IRetryAnalyzer {
            @Override
            public boolean retry(ITestResult result) {
                return true;
            }
        }
    
        @Test(retryAnalyzer = Retry.class)
        public void a() {
            Assert.assertTrue(i++ == 1);
        }
    
        @Test(dependsOnMethods = "a", retryAnalyzer = Retry.class)
        public void b() {
            Assert.assertTrue(i++ == 2);
        }
    }
    
    Feature: retry-analyzer Feature: dependsOn 
    opened by qwikso 41
  • Split modules

    Split modules

    Note: below tricks might help to hide file movements: A) GitHub UI: scroll the diff ("files changed") to the very end, and type the following to the browser console: $$('div.empty').forEach(function(e) { e.parentElement.parentElement.style.display='none'; })

    B) You can watch individual commits (e.g. open Commits ) and review individual ones

    C) You can use git log -M30% -l0 --stat -p -G. on the command line to show non-trivial diffs only (hide movements)

    D) You can use git diff -M30% -l0 --stat -p -G. origin/master to see the full diff

    Note GitHub has a limit on the number of renamed files, so sometimes it shows a file as "deleted and inserted" even if that was a simple rename.

    -l0 can be used to set "unlimited" number of "renamed files" in git diff / git log.

    The true diff (excluding file movements) is 85 files changed, 1617 insertions(+), 812 deletions(-)


    Open questions:

    • none

    Probably resolved:

    • Should testng-* be published (see https://github.com/cbeust/testng/pull/2564#issuecomment-852469267)? -- not for now
    • Should multi-module publishing be handled now? (see https://github.com/cbeust/testng/pull/2564#issuecomment-852534230) -- publishing to Central is included
    • Should we deal/prepare to split packages issues now? Answer: no, we could shuffle classes later.
    • Which other classes should be moved to API (or their own modules). Answer: see above, we could shuffle classes later.

    TODO:

    • [x] Publishing to Central (~ https://github.com/gradle-nexus/publish-plugin)
    • [x] testng-all (the jar that embeds testng modules in one leaving third-party dependencies as usual)
    • [x] Ensure regular (non-feature-specific) dependencies are listed in pom.xml
    • [x] testng-all-javadoc (aggregate javadoc from all the modules)
    • [x] testng-all-sources (aggregate sources from all the modules)
    • [x] OSGi
    • [x] Guice vs testng-core-api (for now guice is optional in API)
    • [x] cleanup /core/build.gradle.kts (remove object This and so on)
    • [x] testng-ant
    • [x] testng-asserts
    • [x] testng-bom
    • [x] testng-collections
    • [x] testng-core
    • [x] testng-core-api
    • [x] testng-test-kit (non-published, helper classes for test code)
    opened by vlsi 38
  • testng 7.1.0 java.lang.ClassNotFoundException: com.google.inject.Stage

    testng 7.1.0 java.lang.ClassNotFoundException: com.google.inject.Stage

    TestNG Version

    TestNG 7.1.0

    Expected behavior

    Actual behavior

    the testng-remote integration test failed https://travis-ci.org/testng-team/testng-remote/jobs/481901771

    [RemoteTestNG] revisions:
    	git.commit.id=32c8977
    	git.branch=32c8977911e409c8c8c2d4bce973e94d2acf5ca7
    	git.build.version=1.4.1-SNAPSHOT
    [RemoteTestNG] loaded class org.testng.internal.Version at file:/home/travis/.groovy/grapes/org.testng/testng/jars/testng-7.1.0.jar
    [RemoteTestNG] detected TestNG version 7.1.0
    Caught: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: com/google/inject/Stage
    java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: com/google/inject/Stage
    	at org.testng.internal.Configuration.<init>(Configuration.java:33)
    	at org.testng.TestNG.init(TestNG.java:216)
    	at org.testng.TestNG.<init>(TestNG.java:200)
    	at org.testng.remote.AbstractRemoteTestNG.<init>(AbstractRemoteTestNG.java:17)
    	at org.testng.remote.support.RemoteTestNG6_12.<init>(RemoteTestNG6_12.java:18)
    	at org.testng.remote.support.RemoteTestNGFactory6_12.createRemoteTestNG(RemoteTestNGFactory6_12.java:16)
    	at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:67)
    	at org.testng.remote.RemoteTestNG$main.call(Unknown Source)
    	at TestNGTest.run(TestNGTest.groovy:49)
    Caused by: java.lang.NoClassDefFoundError: com/google/inject/Stage
    	... 9 more
    Caused by: java.lang.ClassNotFoundException: com.google.inject.Stage
    	... 9 more
    [BaseMessageSender] Stopped receiver
    

    Is the issue reproductible on runner?

    • [ ] Shell
    • [ ] Maven
    • [ ] Gradle
    • [ ] Ant
    • [ ] Eclipse
    • [ ] IntelliJ
    • [ ] NetBeans

    Test case sample

    Please, share the test case (as small as possible) which shows the issue https://github.com/testng-team/testng-remote/blob/master/remote-test/src/test/groovy/Tester.groovy

    opened by missedone 38
  • how to enable generateTestResultAttributes using Maven

    how to enable generateTestResultAttributes using Maven

    TestNG Version

    Note: only the latest version is supported 6.13.1

    Expected behavior

    testresult attributes should be present in the XML report

    Actual behavior

    there are no testresult attributes present in the XML report

    Is the issue reproductible on runner?

    • [ ] Shell
    • [x ] Maven
    • [ ] Gradle
    • [ ] Ant
    • [ ] Eclipse
    • [ ] IntelliJ
    • [ ] NetBeans

    Test case sample

    Please, share the test case (as small as possible) which shows the issue

    How can I setup TestNG, using Maven, for setting up the generateTestResultAttributes flag to true? I tried several ways but I was unable to set it up; i'm unsure if there is a bug or not. In my pom.xml I added this kind of config:

        <build>
            <plugins>
    ....
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.12.4</version>
            <configuration>
              <properties>
                <property>
                  <name>usedefaultlisteners</name>
                  <value>flase</value> <!-- disabling default listeners is optional -->
                </property>
                 <property>
                  <name>generateTestResultAttributes</name>
                  <value>true</value>
                </property>
                <property>
                  <name>reporter</name>
                  <value>org.testng.reporters.XMLReporter</value>
                </property>
              </properties>
            </configuration>
          </plugin>
    
    Feature: report Runner: maven 
    opened by bitcoder 38
  • DependsOnMethods made parallel class use several threads

    DependsOnMethods made parallel class use several threads

    TestNG Version

    6.9.12 and 6.9.13.8

    Expected behavior

    When setting TestNG to run tests in parallel by class, all the tests in a single class should be run in the same thread.

    Actual behavior

    If one of the test method depends on another, a new thread is created, which is really annoying for Selenium case for instance (BeforeClass open the browser, AfterClass kill it)

    Test case sample

    <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
    <suite name="main" parallel="classes" thread-count="2" verbose="0" group-by-instances="false">
        <test name="all">
            <classes>
                <class name="TestFoo"/>
            </classes>
        </test>
    </suite>
    
    import org.testng.annotations.AfterClass;
    import org.testng.annotations.BeforeClass;
    import org.testng.annotations.Test;
    
    public class TestFoo {
    
    
      @BeforeClass
      public void navigateTo() {
        System.out.print("BeforeClass - ");
        printThread();
      }
    
      private void printThread() {
        System.out.println(Thread.currentThread().getName());
      }
    
      @AfterClass
      public void die() {
        System.out.print("AfterClass - ");
        printThread();
      }
    
      @Test
      public void t1() {
        System.out.print("T1 - ");
        printThread();
      }
    
      @Test(dependsOnMethods = "t1")
      public void t2() {
        System.out.print("T2 - ");
        printThread();
      }
    
    }
    
    
    BeforeClass - pool-1-thread-1
    T1 - pool-1-thread-1
    T2 - pool-1-thread-2
    AfterClass - pool-1-thread-2
    

    What I've tried:

    • singleThreaded = true
    • group-by-instance="true"
    Feature: before/after Feature: parallel Feature: dependsOn Issue: Confirmed 
    opened by thibaut-sticky 38
  • [Feature] Allow test classes to define

    [Feature] Allow test classes to define "configfailurepolicy" at a per class level

    TestNG Version

    Note: only the latest version is supported 7.7.1

    Today TestNG allows a user to define a configfailurepolicy ONLY at a global level via the suite attribute -configfailurepolicy. As a user I would like to be able to control this behavior at individual class level.

    Let's say I have a Java class that looks like below.

    public class TestFail {
        private int i=0;
        @BeforeMethod
        public void beforeMethod() {
            i++;
            if (i ==1) {
                throw new RuntimeException("failedWithException");
            }
            assert true;
        }
    
        @Test
        public void testIt() {
            assert true;
        }
    
        @Test
        public void testIt2() {
            assert true;
        }
    }
    

    It would be good if we can either

    1. Add a new attribute called continueOnFailure (would be a boolean) to the BeforeMethod| BeforeClass annotations (or)
    2. Introduce a new marker annotation @IgnoreFailure which a user can add on top of the configuration methods either at the method level (or) at the class level.

    This behaviour should honour inheritance of test classes.

    Caveats:

    • In terms of priority, this capability will have the last say over the global configFailurePolicy defined.
    • This capability cannot be toggled by any sort of annotation transformers.
    opened by krmahadevan 1
  • Feature request: Add new `fail` config failure policy

    Feature request: Add new `fail` config failure policy

    TestNG Version

    LATEST

    Problem

    Can't to fail fast CI build right after configuration method failure.

    Maven Surefire has option skipAfterFailureCount. It allows us to fail E2E tests build after first test method failure. So we can investigate issue in relevant stand state. But we can't achieve the same for configuration method failures.

    Solution

    Add new fail config failure policy. So all test methods in test class would be failed and CI build would be failed asap too.

    opened by pavetok 0
  • Listeners in xml stops test from running

    Listeners in xml stops test from running

    TestNG Version

    7.7.0

    Expected behavior

    run all tests in XML

    Actual behavior

    zero test run

    Is the issue reproducible on runner?

    • [ ] Shell
    • [x] Maven
    • [ ] Gradle
    • [ ] Ant
    • [ ] Eclipse
    • [x] IntelliJ
    • [ ] NetBeans

    After updating from 6.14.3 to 7.7.0 version, test suite run on Jenkins stopped running tests at all. Below final result from run test:

    00:00:39.379 [INFO] Results: 00:00:39.379 [INFO] 00:00:39.379 [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

    The issue happens when I have listeners defined in xml and in annotation. When I remove listener tags from xml file, all tests are running as before. Is there any reason for this behavior in newer version?

    Need: sample 
    opened by krzysztof-isbrandt 1
  • Fix sporadic failure for “firstTimeOnly” methods

    Fix sporadic failure for “firstTimeOnly” methods

    Ensure that the listeners for “firstTimeOnly” methods are executed ONLY if the config method was run else skip listener execution as well.

    The test was failing because it relied on config listeners

    Fixes the sporadic failure in https://github.com/krmahadevan/testng/actions/runs/3611418844/jobs/6085892554

    opened by krmahadevan 3
  • Streamline TestNG listener instantiation

    Streamline TestNG listener instantiation

    TestNG Version

    Note: only the latest version is supported 7.6.1

    Currently TestNG instantiates listener objects multiple times and throws away duplicate instances if the listener has been wired in. This becomes a problem for people who also use their listener objects to manage state.

    This issue is to track this discrepancy in TestNG and ensure that listeners are instantiated ONLY if they are going to be wired in.

    Please see this discussion for more context.

    Feature: listener 
    opened by krmahadevan 0
  • Discrepancies with dependsOnMethods when dependencies span across classes

    Discrepancies with dependsOnMethods when dependencies span across classes

    TestNG Version

    Note: only the latest version is supported 7.6.1

    Expected behavior

    dependsOnMethods attribute should honour upstream failures when it spans across instances rather than the same instance.

    Actual behavior

    Upstream failures are confined to ONLY the current instance but does not span across instances.

    Is the issue reproducible on runner?

    • [X] Shell
    • [X] Maven
    • [X] Gradle
    • [X] Ant
    • [X] Eclipse
    • [X] IntelliJ
    • [ ] NetBeans

    Issue summary

    Let's say we have two test classes.

    • ClassA

      • aa()
    • ClassB

      • ba()

    Lets assume that aa() has been defined as below:

    @Test(dependsOnMethods = "ClassB.ba")
    public void aa() {}
    

    Now if both ClassA and ClassB have been chosen for execution, then TestNG is currently doing the following:

    • ba() is first executed before running aa(). This assures that TestNG is honouring cross class method dependencies.

    But if ba() fails, then TestNG should be skipping ClassA.aa() (because of the dependency) but it does not do that and instead runs aa() despite there being a failure in upstream.

    The same behaviour is seen even when we use dependsOnMethods to refer to a method in a inner nested class.

    Test case sample

    Please, share the test case (as small as possible) which shows the issue

    ClassATestClassSample
    package com.rationaleemotions.dependencies;
    
    import org.testng.annotations.Test;
    
    public class ClassATestClassSample {
    
      @Test(dependsOnMethods = "com.rationaleemotions.dependencies.ClassBTestClassSample.passingMethodB")
      public void methodA() {}
    
      @Test(dependsOnMethods = "com.rationaleemotions.dependencies.ClassBTestClassSample.failingMethodB")
      public void methodB() {
      }
    }
    
    ClassBTestClassSample
    package com.rationaleemotions.dependencies;
    
    import org.testng.Assert;
    import org.testng.annotations.Test;
    
    public class ClassBTestClassSample {
    
      @Test
      public void passingMethodB() {}
    
      @Test
      public void failingMethodB() {
        Assert.fail();
      }
    }
    
    NestedTestClassSample
    package com.rationaleemotions.dependencies;
    
    import org.testng.Assert;
    import org.testng.annotations.Test;
    
    public class NestedTestClassSample {
    
      @Test(dependsOnMethods =
          "com.rationaleemotions.dependencies.NestedTestClassSample$InnerClassSample.failingMethodB")
      public void methodA() {}
    
      public static class InnerClassSample {
    
        @Test
        public void failingMethodB() {
          Assert.fail();
        }
      }
    }
    
    TestResultsGatherer
    package com.rationaleemotions.dependencies;
    
    import java.util.ArrayList;
    import java.util.List;
    import org.testng.ITestListener;
    import org.testng.ITestResult;
    
    public class TestResultsGatherer implements ITestListener {
    
      private final List<String> passed = new ArrayList<>();
      private final List<String> failed = new ArrayList<>();
      private final List<String> skipped = new ArrayList<>();
    
      public List<String> getFailed() {
        return failed;
      }
    
      public List<String> getPassed() {
        return passed;
      }
    
      public List<String> getSkipped() {
        return skipped;
      }
    
      @Override
      public void onTestSuccess(ITestResult result) {
        passed.add(result.getMethod().getQualifiedName());
      }
    
      @Override
      public void onTestFailure(ITestResult result) {
        failed.add(result.getMethod().getQualifiedName());
      }
    
      @Override
      public void onTestSkipped(ITestResult result) {
        skipped.add(result.getMethod().getQualifiedName());
      }
    }
    

    Test case

    import static org.assertj.core.api.Assertions.assertThat;
    
    import org.testng.TestNG;
    import org.testng.annotations.Test;
    
    public class IssueTest {
    
      @Test
      public void runTestWithNestedClasses() {
        TestNG testng = new TestNG();
        testng.setTestClasses(new Class[]{NestedTestClassSample.class});
        TestResultsGatherer gatherer = new TestResultsGatherer();
        testng.addListener(gatherer);
        testng.setVerbose(2);
        testng.run();
        String outer = NestedTestClassSample.class.getCanonicalName();
        String inner = outer + "$" + NestedTestClassSample.InnerClassSample.class.getSimpleName();
        assertThat(gatherer.getFailed())
            .containsExactly(inner + ".failingMethodB");
        assertThat(gatherer.getPassed())
            .isEmpty();
        assertThat(gatherer.getSkipped())
            .containsExactly(outer + ".methodA");
      }
    
      @Test
      public void runTest() {
        TestNG testng = new TestNG();
        testng.setTestClasses(new Class[]{
            ClassATestClassSample.class, ClassBTestClassSample.class
        });
        TestResultsGatherer gatherer = new TestResultsGatherer();
        testng.addListener(gatherer);
        testng.setVerbose(2);
        testng.run();
        String classB = ClassBTestClassSample.class.getCanonicalName();
        String classA = ClassATestClassSample.class.getCanonicalName();
        assertThat(gatherer.getFailed())
            .containsExactly(classB + ".failingMethodB");
        assertThat(gatherer.getPassed())
            .containsExactly(classA + ".methodA", classB + ".passingMethodB");
        assertThat(gatherer.getSkipped())
            .containsExactly(classA + ".methodB");
      }
    }
    
    
    Feature: dependsOn 
    opened by krmahadevan 0
Releases(7.7.1)
  • 7.7.1(Dec 29, 2022)

    What's Changed

    • Streamline overloaded assertion methods for Groovy by @krmahadevan in https://github.com/cbeust/testng/pull/2858

    Full Changelog: https://github.com/cbeust/testng/compare/7.7.0...7.7.1

    Source code(tar.gz)
    Source code(zip)
  • 7.7.0(Dec 9, 2022)

    What's Changed

    • Replace FindBugs by SpotBugs by @gruenich in https://github.com/cbeust/testng/pull/2781
    • Gradle: Drop forUseAtConfigurationTime() by @gruenich in https://github.com/cbeust/testng/pull/2783
    • Added ability to provide custom message to assertThrows\expectThrows methods by @anatolyuzhakov in https://github.com/cbeust/testng/pull/2793
    • Fix issue 2801 - Only resolve hostname once by @spkrka in https://github.com/cbeust/testng/pull/2802
    • [SECURITY] Fix Zip Slip Vulnerability by @JLLeitschuh in https://github.com/cbeust/testng/pull/2806
    • GITHUB-2807 - Failsafe buildStackTrace by @seregamorph in https://github.com/cbeust/testng/pull/2808
    • Prevent overlogging of debug msgs in Graph impl by @krmahadevan in https://github.com/cbeust/testng/pull/2813
    • Streamline dataprovider invoking in abstract classes by @krmahadevan in https://github.com/cbeust/testng/pull/2814
    • Streamline TestResult due to expectedExceptions by @krmahadevan in https://github.com/cbeust/testng/pull/2815
    • Unexpected test runs count with retry analyzer by @krmahadevan in https://github.com/cbeust/testng/pull/2816
    • Make PackageUtils compliant with JPMS by @krmahadevan in https://github.com/cbeust/testng/pull/2817
    • Ability to retry a data provider during failures by @krmahadevan in https://github.com/cbeust/testng/pull/2820
    • Refactoring by @krmahadevan in https://github.com/cbeust/testng/pull/2821
    • Fixing bug with DataProvider retry by @krmahadevan in https://github.com/cbeust/testng/pull/2822
    • Add config key for callback discrepancy behavior by @krmahadevan in https://github.com/cbeust/testng/pull/2823
    • Upgrading versions by @krmahadevan in https://github.com/cbeust/testng/pull/2824
    • Fix #2770: FileAlreadyExistsException on copy by @melloware in https://github.com/cbeust/testng/pull/2827
    • JarFileUtils.delete(File f) throw actual exception (instead of FileNotFound) when file cannot be deleted #2825 by @speedythesnail in https://github.com/cbeust/testng/pull/2826
    • GITHUB-2830 - Failsafe parameter.toString by @seregamorph in https://github.com/cbeust/testng/pull/2831
    • Changing assertion message of the osgitest by @krmahadevan in https://github.com/cbeust/testng/pull/2832
    • hidden spotbugs in release #2829 by @bobshie in https://github.com/cbeust/testng/pull/2833
    • Enhancing the Matrix by @krmahadevan in https://github.com/cbeust/testng/pull/2834
    • Avoid Compilation errors on Semeru JDK flavour. by @krmahadevan in https://github.com/cbeust/testng/pull/2835
    • Add addition yml extension by @speedythesnail in https://github.com/cbeust/testng/pull/2837
    • Support getting dependencies info for a test by @krmahadevan in https://github.com/cbeust/testng/pull/2839
    • Honour regex in dependsOnMethods by @krmahadevan in https://github.com/cbeust/testng/pull/2838
    • Ensure All tests run all the time by @krmahadevan in https://github.com/cbeust/testng/pull/2842
    • Deprecate support for running Spock Tests by @krmahadevan in https://github.com/cbeust/testng/pull/2846
    • Streamline dependsOnMethods for configurations by @krmahadevan in https://github.com/cbeust/testng/pull/2845
    • Ensure ITestContext available for JUnit4 tests by @krmahadevan in https://github.com/cbeust/testng/pull/2848
    • Deprecate support for running JUnit tests by @krmahadevan in https://github.com/cbeust/testng/pull/2849

    New Contributors

    • @gruenich made their first contribution in https://github.com/cbeust/testng/pull/2781
    • @anatolyuzhakov made their first contribution in https://github.com/cbeust/testng/pull/2793
    • @spkrka made their first contribution in https://github.com/cbeust/testng/pull/2802
    • @JLLeitschuh made their first contribution in https://github.com/cbeust/testng/pull/2806
    • @seregamorph made their first contribution in https://github.com/cbeust/testng/pull/2808
    • @melloware made their first contribution in https://github.com/cbeust/testng/pull/2827
    • @speedythesnail made their first contribution in https://github.com/cbeust/testng/pull/2826
    • @bobshie made their first contribution in https://github.com/cbeust/testng/pull/2833

    Full Changelog: https://github.com/cbeust/testng/compare/7.6.1...7.7.0

    Source code(tar.gz)
    Source code(zip)
  • 7.6.1(Jul 4, 2022)

    This is a bug fix release and just includes 1 bug fix in it.

    What's Changed

    • Fix Files.copy() such that parent dirs are created by @krmahadevan in https://github.com/cbeust/testng/pull/2764
    • Remove deprecated utility methods by @krmahadevan in https://github.com/cbeust/testng/pull/2762
    • Fix typos by @asolntsev in https://github.com/cbeust/testng/pull/2772

    Full Changelog: https://github.com/cbeust/testng/compare/7.6.0...7.6.1

    Source code(tar.gz)
    Source code(zip)
  • 7.6.0(May 18, 2022)

    What's Changed

    • Remove redundant Parameter implementation by @krmahadevan in https://github.com/cbeust/testng/pull/2706
    • Upgrade to JDK11 by @krmahadevan in https://github.com/cbeust/testng/pull/2705
    • Move SimpleBaseTest to be Kotlin based by @krmahadevan in https://github.com/cbeust/testng/pull/2703
    • Restore testnames when using suites in suite. by @martinaldrin in https://github.com/cbeust/testng/pull/2712
    • Moving ClassHelperTests into Kotlin by @krmahadevan in https://github.com/cbeust/testng/pull/2716
    • IHookable and IConfigurable callback discrepancy by @krmahadevan in https://github.com/cbeust/testng/pull/2713
    • Minor refactoring by @krmahadevan in https://github.com/cbeust/testng/pull/2718
    • Add additional condition for assertEqualsNoOrder by @Dymitriux in https://github.com/cbeust/testng/pull/2723
    • beforeConfiguration() listener method should be invoked for skipped configurations as well by @bj-9527 in https://github.com/cbeust/testng/pull/2732
    • #2734 keep the initial order of listeners by @asolntsev in https://github.com/cbeust/testng/pull/2737
    • SuiteRunner could not be initial by default Configuration by @bj-9527 in https://github.com/cbeust/testng/pull/2744
    • Enable Dataprovider failures to be considered. by @krmahadevan in https://github.com/cbeust/testng/pull/2748
    • BeforeGroups should run before any matched test by @velma in https://github.com/cbeust/testng/pull/2749
    • Fix possible StringIndexOutOfBoundsException exception in XmlReporter by @velma in https://github.com/cbeust/testng/pull/2750
    • DataProvider: possibility to unload dataprovider class, when done with it by @dsankouski in https://github.com/cbeust/testng/pull/2739
    • fix possibilty that AfterGroups method is invoked before all tests by @velma in https://github.com/cbeust/testng/pull/2753
    • fix equals implementation for WrappedTestNGMethod by @velma in https://github.com/cbeust/testng/pull/2755
    • Upgrade dependencies, Upgrade to JDK17 by @hduerkop in https://github.com/cbeust/testng/pull/2747
    • Wire-In listeners consistently by @krmahadevan in https://github.com/cbeust/testng/pull/2756
    • Streamline AfterClass invocation by @krmahadevan in https://github.com/cbeust/testng/pull/2757
    • Show FQMN for tests in console by @krmahadevan in https://github.com/cbeust/testng/pull/2758
    • Fix 2725 by @krmahadevan in https://github.com/cbeust/testng/pull/2759

    New Contributors

    • @Dymitriux made their first contribution in https://github.com/cbeust/testng/pull/2723
    • @asolntsev made their first contribution in https://github.com/cbeust/testng/pull/2737
    • @velma made their first contribution in https://github.com/cbeust/testng/pull/2749
    • @hduerkop made their first contribution in https://github.com/cbeust/testng/pull/2747

    Full Changelog: https://github.com/cbeust/testng/compare/7.5...7.6.0

    Source code(tar.gz)
    Source code(zip)
Owner
Cedric Beust
Creator of Android Gmail.
Cedric Beust
A sample repo to help you use relative locators for automation test in Java-TestNG on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to use relative locators for automation test in Java-TestNG on LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven wit

null 11 Jul 13, 2022
A sample repo to help you use CDP console in Java-TestNG automation test on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to use CDP console in Java-TestNG automation test on LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven with Homebrew

null 11 Jul 13, 2022
A sample repo to help you capture performance logs in Java-TestNG using CDP on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to capture performance logs in Java-TestNG using CDP on LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven with Homeb

null 12 Jul 13, 2022
A sample repo to help you intercept network with Java-TestNG on LambdaTest cloud. Run Selenium tests with TestNG on LambdaTest platform.

How to intercept network with Java-TestNG on LambdaTest cloud Environment Setup Global Dependencies Install Maven Or Install Maven with Homebrew (Easi

null 12 Oct 23, 2022
A sample repo to help you set geolocation for automation test in Java-TestNG on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to set geolocation for automation test in Java-TestNG on LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven with Home

null 12 Jul 13, 2022
A sample repo to help you emulate network control using CDP in Java-TestNG automation test on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to emulate network control using CDP in Java-TestNG automation test on LambdaTest Environment Setup Global Dependencies Install Maven Or Install M

null 12 Oct 23, 2022
A sample repo to help you handle basic auth for automation test in Java-TestNG on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to handle basic auth for automation test in Java-TestNG on LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven with Ho

null 11 Jul 13, 2022
A sample repo to help you set device mode using CDP in Java-TestNG automation test on LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

How to set device mode using CDP in Java-TestNG automation test on LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven wit

null 11 Jul 13, 2022
A sample repo to help you test in Java-TestNG-Appium for LambdaTest. Run Selenium tests with TestNG on LambdaTest platform.

Sample automation test in Java-TestNG-Appium for LambdaTest Environment Setup Global Dependencies Install Maven Or Install Maven with Homebrew (Easier

null 13 Jul 13, 2022
TestNG testing framework

Documentation available at TestNG's main web site. Release Notes 7.4.0 7.3.0 7.1.0 7.0.0 Need help? Before opening a new issue, did you ask your quest

Cedric Beust 1.8k Jan 5, 2023
Java testing framework for testing pojo methods

Java testing framework for testing pojo methods. It tests equals, hashCode, toString, getters, setters, constructors and whatever you report in issues ;)

Piotr Joński 48 Aug 23, 2022
BDD framework for automation using Selenium Cucumber and TestNg

Selenium Framework with Cucumber BDD framework for automation using Selenium Cucumber and TestNg The framework has following features Modular Design M

null 3 Jan 20, 2022
A TestNG like dataprovider runner for JUnit with many additional features

junit-dataprovider Have a look to this repositories Wiki pages for documenation, usage examples and further information. Table of contents Home What i

TNG Technology Consulting GmbH 237 Sep 7, 2022
An e-commerce automation project of Selenium TestNG using Page Object Model

Selenium-POM-TestNG Prerequisites Install jdk 8 or any LTS version Configure JAVA_HOME and GRADLE_HOME Download Allure 2.17.2 and configure environmen

Asif Shahriar 2 Aug 4, 2022
java接口自动化测试框架:java + testng + httpclient + allure + ...

设计思路、使用方式、待扩展功能等 主要技术栈:java + testng + httpclient + allure + fastjson + jsonpath + ... 设计思路 使用方式 待扩展功能 效果图 自动化测试框架(汇总) 干货分享 从测试小白到高级测试修炼之路,持续更新中:https

持之以恒(韧) 12 Dec 5, 2022
A modern testing and behavioural specification framework for Java 8

Introduction If you're a Java developer and you've seen the fluent, modern specification frameworks available in other programming languages such as s

Richard Warburton 250 Sep 12, 2022
A programmer-oriented testing framework for Java.

JUnit 4 JUnit is a simple framework to write repeatable tests. It is an instance of the xUnit architecture for unit testing frameworks. For more infor

JUnit 8.4k Jan 4, 2023
The Enterprise-ready testing and specification framework.

Spock Framework Spock is a BDD-style developer testing and specification framework for Java and Groovy applications. To learn more about Spock, visit

Spock Framework 3.3k Jan 5, 2023
Layout and functional testing framework for websites

Galen Framework master: Galen is an open-source tool for testing layout and responsive design of web applications. It is also a powerfull functional t

Galen Framework 1.4k Dec 10, 2022