This appears to work for Mockito and TemporaryFolder as evidenced. It should work for Class rules too and needs roadtesting with Spring.

It should ultimately solve #15 #49 and #34

Still WIP at the moment - demonstrated purely by a replacement of the let function.

I've reorganised the package structure, and I'm in the process of killing off all arounds/befores etc. First target - the humble let.

Code coverage will not be 100% yet as there are unused features.

Intended as a fix for #76 and #73. Eventually. Intended as a new platform on which to build #56

JUnit provides @ClassRule and @Rule annotations, which would let me use the new @SpringBootTest annotation with Spectrum. Is there any plan to provide support for @ClassRule and @Rule? Thanks!

i am not sure what you can do on your end, but thought I would report it anyhow. the following test case fails because Variables use a ThreadLocal, and awaitility by default happens to run in a separate one. The only way it works is if the user does an await().pollInsSameThread().

java.lang.AssertionError: 
Expected: not (null or an empty string)
     but: was null
Expected :not (null or an empty string)
     
Actual   :null

package test;

import com.greghaskins.spectrum.Spectrum;
import com.greghaskins.spectrum.Variable;
import org.junit.runner.RunWith;

import static com.greghaskins.spectrum.Spectrum.describe;
import static com.greghaskins.spectrum.Spectrum.it;
import static org.awaitility.Awaitility.await;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.isEmptyOrNullString;
import static org.hamcrest.Matchers.not;

@RunWith(Spectrum.class)
public class VariableTest {{
  describe("suite", () -> {
    final Variable<String> var = new Variable<>();
    it("test", () -> {
      var.set("value");
      await().until(() -> {
        assertThat(var.get(), not(isEmptyOrNullString()));
        return true;
      });
    });
  });
}}

Hi Greg,

I have been having at Spectrum and really like the way the tests get grouped together. I wondered if there was any support for mockito annotations, or if you knew a way to get them working.

I have managed to get a Mockito test by explicitly constructing the mocks and then adding a manual call to validateMockitoUsage and reset in the afterEach, it just feels quite hard work.

Normally I would add in a Mockito JUnit rule, annotate each mock and then construct the service in a before method. Having to manually construct the mocks is not too bad, but It would be great if there was some way to trigger the validate and reset at the end of each test.

Thanks,

Peter.

I'm working on a few things in this PR, and will update it as I go.

In no particular order:

• [x] Put all public APIs in one main package for easier discovery and import. Many of them will be available directly under Spectrum.class.
  • com.greghaskins.spectrum.internal.ConfiguredBlock
  • com.greghaskins.spectrum.model.BlockConfiguration.Factory
• [x] Move as many internal things to the internal packages as possible. This will clarify the real public API.
• [x] Update version numbers and deprecation notices in javadocs
• [ ] Make sure nothing internal is involved in the public API (either as entry point, parameter, or return value). Some of these will need to move to main package as a result.
• [ ] Resolve cyclic dependencies, at least involving the main package com.greghaskins.spectrum and it's classes.
• [ ] ~Reduce the logging noise from the test suite. Mainly the Spring stuff spits a bunch of stderr messages.~
• [ ] ~Determine which features should be marked and packaged as spectrum.experimental~

Unfortunately version 1.1.0 seems to have introduced funky behavior in Intellij Idea test output structure in case of exception inside tests.

First the test code that throws exception: Second how Intellij Idea displays it with Spectrum 1.0.0 Third the same test with Spectrum 1.1.0

Happens at least with Intellij Idea 2017.1 and 2016.3.5

We have found that using @Mock and @InjectMocks annotations don't work with spectrum, resulting in the mock causing a NullPointerException.

It seems that the runner needs to do all the injections and mocking pretty early in the test execution, so that all instances are injected and/or mocked.

This issue is somewhat related to this issue, but perhaps it's helpful to bring it up here separately.

Thanks for the great framework,

@ishustava && @larham

describe("after creation of attribute", () -> {
        describe( "latest()", () -> {
                it("should return 200", () -> {
                    mockMvc.perform(get("/api/latest/attribute")).andExpect(status().isOk());
                });
        });
});

if I replace latest() with latest everything works fine!

Thank you!

There are a few ways to test for some code ending in an exception.

• Use a framework like AssertJ with assertThatThrownBy

E.g.

it("expects a boom", () -> {
   assertThatThrownBy(() -> { throw new Exception("boom!"); }).isInstanceOf(Exception.class)
                                                             .hasMessageContaining("boom");
}

• Employ the ExpectedException rule from JUnit either with junitMixin or even raw with @Rule

E.g.

@Rule
public ExpectedException expectedException = ExpectedException.none();

{
   it("expects a boom", () -> {
         expectedException .expect(NullPointerException.class);
         throw new NullPointerException();
   });
}

• You could even do aroundEach

describe("These all need an exception to be thrown", () -> {
   aroundEach(block -> {
      try {
         block.run();
      } catch (DesiredException e) {
         return;
      }
     fail("Desired exception not thrown. aaagh!");
   });
});

Each of these is plausible (though the last one sucks a bit), but not quite as neat as @Test(expected=SomeException.class) public void myTest() {} that you get in JUnit. Here are two motivations for some alternative:

• A terse syntax for expecting exceptions that's part of the spec
• Ability to define exception expectations hierarchically - i.e. group together in a context or describe a bunch of things that end in the same exception

There are a couple of ways I could imagine doing this and I'd like feedback on which might be desirable:

• As a BlockConfiguration as part of the with syntax
• Using a hook, a bit like let to provide the expected exception similar to ExpectedException but more native to Spectrum

Block Configuraton Approach

This is hierarchical and can be applied anywhere in the tree

e.g.

describe("Suite", () -> {
   // all children are expected to thrown this exception, though they can specialise it
   // or even override it using the same `with` mechanism
   describe("throwing suite", with(expectedException(NullPointerException.class), () -> {
      it("Should throw", () -> {
          throw new NullPointerException();
      });
   }));

   it("doesn't have to throw", () -> {});

   it("requires its own exception", with(expectedException(RuntimeException.class, "boom!"), () -> {
       throw new RuntimeException("boom!");
   }));
});

Hook approach

Let's use ExpectedException within Spectrum, or a clone of it if we don't want it to be too JUnit. What we end up with will be similar to let but will add an expected exception hook instead of a let hook.

e.g.

describe("Some suite", () -> {
   // start with the assumption that nothing throws, but each spec can modify the expectation
   Supplier<ExpectedException> expectedException = expectException(ExpectedException.none());
   it("wants an exception to be thrown", () -> {
       expectedException .get().expect(NullPointerException.class);
       throw new NullPointerException();
   });
});

// or

describe("Some suite", () -> {
   // Start with the assumption that everything will throw
   Supplier<ExpectedException> expectedException = expectException(NullPointerException.class);
   it("wants an exception to be thrown", () -> {
       throw new NullPointerException();
   });
});

@greghaskins - your thoughts pls.

The latest released git tag is 1.0.0, but there're a lot of amazing features listed on README which are not under any tag.

Wouldn't that be better to remove those features from README until they are released or add notes telling the user those features are not under the latest released tag yet?

There are about ~ 100 commits on master branch since v1.0.0. When are you guys going to release a new tag with all those features included?

I'd like to use Spectrum in an OSGi-based project, but the JAR does currently not have an OSGi manifest. This can be easily added either in a hard-coded fashion by adding a src/main/resources/META-INF/MANIFEST.MF file, or by adding a build step to the Gradle build file that generates a manifest automatically. If there is a chance that such a change could get released to Maven Central soon, I'm happy to make a pull request with the necessary changes.

Spectrum only offers Data-Driven-Tests with the scenarioOutline-method defined in the gherkinDSL. However, while they can be nested like SpecificationDSL's describe, the JUnit report isn't correct when more than one example is used. The report shows all examples but cannot assign a status to it, i.e show that it passed or failed.

Here's a minimal example:

@RunWith(Spectrum.class)
public class NestedScenarioOutline
{
    {
        feature("A feature", () ->
            {
                scenarioOutline("The background with an example", (browser) ->
                    {
                        scenarioOutline("Eating less cucumbers than I have ", (have, eat, remaining) ->
                            {
                                final AtomicInteger cukes = new AtomicInteger();
                                given("I have " + have + " cucumbers", () ->
                                    {
                                        cukes.set(have);
                                    });
                                when("I eat " + eat + " cucumbers", () ->
                                    {
                                        cukes.addAndGet(-eat);
                                    });
                            }, withExamples(example(12, 5, 7), example(20, 5, 15)));

                    }, withExamples(example("Chrome"), example("Firefox")));
            });
    }
}

And the corresponding report in Eclipse:

The first example, Chrome, does not have any status, neither do the children of it.

In addition, if an error occurs, the test suite fails but no test shows any failure. Here's an example, which fails when the outer scenarioOutline receives "Chrome" as example:

@RunWith(Spectrum.class)
public class NestedScenarioOutline
{
    {
        feature("A feature", () ->
            {
                scenarioOutline("The background with an example", (browser) ->
                    {
                        scenario("Eating less cucumbers than I have ", () ->
                            {
                                final AtomicInteger cukes = new AtomicInteger();
                                given("I have 12 cucumbers", () ->
                                    {
                                        System.out.println("Current Browser is " + browser);
                                        cukes.set(12);
                                        if (browser.equals("Chrome"))
                                        {
                                            throw new AssertionError();
                                        }
                                    });
                                when("I eat 5 cucumbers", () ->
                                    {
                                        cukes.addAndGet(-5);
                                    });
                                then("I have 7 cucumbers left", () ->
                                    {
                                        assertThat(cukes.get(), equalTo(7));
                                    });
                            });
                        scenarioOutline("Eating less cucumbers than I have ", (have, eat, remaining) ->
                            {
                                final AtomicInteger cukes = new AtomicInteger();
                                given("I have " + have + " cucumbers", () ->
                                    {
                                        cukes.set(have);
                                    });
                                when("I eat " + eat + " cucumbers", () ->
                                    {
                                        cukes.addAndGet(-eat);
                                    });
                                then("I have " + remaining + " cucumbers left", () ->
                                    {
                                        assertThat(cukes.get(), equalTo(remaining));
                                    });
                            }, withExamples(example(12, 5, 7), example(20, 5, 15)));

                    }, withExamples(example("Chrome"), example("Firefox")));
            });
    }
}

The JUnit report as shown in Eclipse:

The report further shows, that the when and then step of the scenario in the scenarioOutline were ignored, which isn't true for the Firefox example.

Since a scenarioOutline can already contain other scenario(Outline)s, it would be nice if the report were correct. This would also add the ability to supply test data to Spectrum's fixtures (beforeEach, beforeAll, etc.) , which is often very helpful.

How I can stop queue of tests in spec if one of test is fail?

        describe("some spec", () -> {
            it("First test", () -> {});
            it("Second test", () -> { // something broken });
            it("Third test", () -> {});
        });

if second test will is broken, third test will not run.

This may just be something I'm overlooking in the docs -- or that could use an example -- but is there a way to do RSpec-style shared contexts and shared examples?

For example, I just wrote my first test using Spectrum, which looked like this:

@RunWith(Spectrum.class)
public class EnumsTest {
  {
    describe(".findValueOrThrow", () -> {
      context("when given an Enum that has no values", () -> {
        final Supplier<Class<EmptyEnum>> enumClass = let(() -> EmptyEnum.class);

        it("throws an IllegalArgumentException", () -> {
          assertThatExceptionOfType(IllegalArgumentException.class)
            .isThrownBy(() -> {
              Enums.findValueOrThrow(enumClass.get(), (value) -> true);
            })
            .withMessage(
              "No `com.rosieapp.util.EnumsTest.EmptyEnum` was found that matched the specified " +
              "filter.")
            .withNoCause();
        });
      });

      context("when given an Enum that has values", () -> {
        final Supplier<Class<Colors>> enumClass = let(() -> Colors.class);

        context("when the predicate does not match any of the values", () -> {
          final Supplier<Predicate<Colors>> predicate = let(() -> (color) -> false);

          it("throws an IllegalArgumentException", () -> {
            assertThatExceptionOfType(IllegalArgumentException.class)
              .isThrownBy(() -> {
                Enums.findValueOrThrow(enumClass.get(), predicate.get());
              })
              .withMessage(
                "No `com.rosieapp.util.EnumsTest.Colors` was found that matched the specified " +
                "filter.")
              .withNoCause();
          });
        });

        context("when the predicate matches one of the values", () -> {
          final Supplier<Predicate<Colors>> predicate = let(() -> (color) -> color.name().equals("WHITE"));

          it("returns the matching value", () -> {
            assertThat(Enums.findValueOrThrow(enumClass.get(), predicate.get()), is(Colors.WHITE));
          });
        });

        context("when the predicate matches multiple values", () -> {
          final Supplier<Predicate<Colors>> predicate = let(() ->
            (color) -> !Arrays.asList("RED", "WHITE").contains(color.name()));

          it("returns the first matching value, according to the order within the enum", () -> {
            assertThat(Enums.findValueOrThrow(enumClass.get(), predicate.get()), is(Colors.BLUE));
          });
        });
      });
    });
  }
  // ...
}

Two of those scenarios are expected to throw the same exception with nearly the same error message. In RSpec, I'd abstract that out into a shared example group and then control what's provided using let. If shared example groups are out of the question, what would be the best practice when using Spectrum for this?

Coming from Ruby and RSpec, I am used to see the texts from describe and it blocks in the error message when the test fails.

I would assume something similar to this example:

describe("Spectrum", () -> {
        it("can make assertions", () -> {
            assertThat(true, is(false));
        });
});

Output might look like

Test failed: Spectrum, it can make assertions (/thisIs/A/PathTo/MyTestFile:15)
Expected: is <false>
     but: was <true>

with "/thisIs/A/PathTo/MyTestFile:15" indicating the line in which the it block starts.

wdyt?