Noteworthy in Version 1.2.7

Summary:

BREAKING CHANGES:

Support Gherkin v6 Grammar

Grammar changes:

  • Rule concept added to better correspond to Example Mapping concepts

  • Add aliases for Scenario and Scenario Outline (for similar reasons)

FEATURE GRAMMAR (PSEUDO-CODE)
@tag1 @tag2
Feature: Optional Feature Title...

    Description?        #< CARDINALITY: 0..1 (optional)
    Background?         #< CARDINALITY: 0..1 (optional)
    Scenario*           #< CARDINALITY: 0..N (many)
    ScenarioOutline*    #< CARDINALITY: 0..N (many)
    Rule*               #< CARDINALITY: 0..N (many)

A Rule (or: business rule) allows to group multiple Scenario(s)/Example(s):

RULE GRAMMAR (PSEUDO-CODE)
@tag1 @tag2
Rule: Optional Rule Title...

    Description?        #< CARDINALITY: 0..1 (optional)
    Background?         #< CARDINALITY: 0..1 (optional)
    Scenario*           #< CARDINALITY: 0..N (many)
    ScenarioOutline*    #< CARDINALITY: 0..N (many)

Gherkin v6 keyword aliases:

Concept

Preferred Keyword

Alias(es)

Scenario

Example

Scenario

Scenario Outline

Scenario Outline

Scenario Template

Examples

Examples

Scenarios

EXAMPLE:

FILE: features/example_with_rules.feature
# -- USING: Gherkin v6
Feature: With Rules

  Background: Feature.Background
    Given feature background step_1

  Rule: Rule_1
    Background: Rule_1.Background
      Given rule_1 background step_1

    Example: Rule_1.Example_1
      Given rule_1 scenario_1 step_1

  Rule: Rule_2

    Example: Rule_2.Example_1
      Given rule_2 scenario_1 step_1

  Rule: Rule_3
    Background: Rule_3.EmptyBackground
    Example: Rule_3.Example_1
      Given rule_3 scenario_1 step_1

Overview of the Example Mapping concepts:

Cucumber: `Example Mapping`_

Hint

Gherkin v6 Grammar Issues

  • cucumber issue #632: Rule tags are currently only supported in behave. The Cucumber Gherkin v6 grammar currently lacks this functionality.

  • cucumber issue #590: Rule Background: A proposal is pending to remove Rule Backgrounds again

Tag-Expressions v2

Tag-Expressions v2 are based on cucumber-tag-expressions with some extensions:

  • Tag-Expressions v2 provide boolean logic expression (with and, or and not operators and parenthesis for grouping expressions)

  • Tag-Expressions v2 are far more readable and composable than Tag-Expressions v1

  • Some boolean-logic-expressions where not possible with Tag-Expressions v1

  • Therefore, Tag-Expressions v2 supersedes the old-style tag-expressions.

TAG-EXPRESSION EXAMPLES
# -- EXAMPLE 1: Select features/scenarios that have the tags: @a and @b
@a and @b

# -- EXAMPLE 2: Select features/scenarios that have the tag: @a or @b
@a or @b

# -- EXAMPLE 3: Select features/scenarios that do not have the tag: @a
not @a

# -- EXAMPLE 4: Select features/scenarios that have the tags: @a but not @b
@a and not @b

# -- EXAMPLE 5: Select features/scenarios that have the tags: (@a or @b) but not @c
# HINT: Boolean expressions can be grouped with parenthesis.
(@a or @b) and not @c

COMMAND-LINE EXAMPLE:

USING: Tag-Expressions v2 with behave
# -- SELECT-BY-TAG-EXPRESSION (with tag-expressions v2):
# Select all features / scenarios with both "@foo" and "@bar" tags.
$ behave --tags="@foo and @bar" features/

# -- EXAMPLE: Use default_tags from config-file "behave.ini".
# Use placeholder "{config.tags}" to refer to this tag-expression.
# HERE: config.tags = "not (@xfail or @not_implemented)"
$ behave --tags="(@foo or @bar) and {config.tags}" --tags-help
...
CURRENT TAG_EXPRESSION: ((foo or bar) and not (xfail or not_implemented))

# -- EXAMPLE: Uses Tag-Expression diagnostics with --tags-help option
$ behave --tags="(@foo and @bar) or @baz" --tags-help
$ behave --tags="(@foo and @bar) or @baz" --tags-help --verbose

Tag Matching with Tag-Expressions

Tag-Expressions v2 support partial string/tag matching with wildcards. This supports tag-expressions:

Tag Matching Idiom

Example 1

Example 2

Description

tag.starts_with

@foo.*

foo.*

Search for tags that start with a prefix.

tag.ends_with

@*.one

*.one

Search for tags that end with a suffix.

tag.contains

@*foo*

*foo*

Search for tags that contain a part.

FILE: features/one.feature
Feature: Alice

  @foo.one
  Scenario: Alice.1
    ...

  @foo.two
  Scenario: Alice.2
    ...

  @bar
  Scenario: Alice.3
    ...

The following command-line will select all features / scenarios with tags that start with “@foo.”:

USAGE EXAMPLE: Run behave with tag-matching expressions
$ behave -f plain --tags="@foo.*" features/one.feature
Feature: Alice

  Scenario: Alice.1
    ...

  Scenario: Alice.2
    ...

# -- HINT: Only Alice.1 and Alice.2 are matched (not: Alice.3).

Note

  • Filename matching wildcards are supported. See fnmatch (Unix style filename matching).

  • The tag matching functionality is an extension to cucumber-tag-expressions.

Select the Tag-Expression Version to Use

The tag-expression version, that should be used by behave, can be specified in the behave config-file.

This allows a user to select:

  • Tag-Expressions v1 (if needed)

  • Tag-Expressions v2 when it is feasible

EXAMPLE:

FILE: behave.ini
# SPECIFY WHICH TAG-EXPRESSION-PROTOCOL SHOULD BE USED:
#   SUPPORTED VALUES: v1, v2, auto_detect
#   CURRENT DEFAULT:  auto_detect
[behave]
tag_expression_protocol = v1    # -- Use Tag-Expressions v1.

Tag-Expressions v1

Tag-Expressions v1 are becoming deprecated (but are currently still supported). Use Tag-Expressions v2 instead.

Note

Tag-Expressions v1 support will be dropped in behave v1.4.0.

Select-by-location for Scenario Containers

In the past, it was already possible to scenario(s) by using its file-location.

A file-location has the schema: <FILENAME>:<LINE_NUMBER>. Example: features/alice.feature:12 (refers to line 12 in features/alice.feature file).

Rules to select Scenarios by using the file-location:

  • Scenario: Use a file-location that points to the keyword/title or its steps (until next Scenario/Entity starts).

  • Scenario of a ScenarioOutline: Use the file-location of its Examples row.

Now you can select all entities of a Scenario Container (Feature, Rule, ScenarioOutline):

  • Feature: Use file-location before first contained entity/Scenario starts.

  • Rule: Use file-location from keyword/title line to line before its first Scenario/Background.

  • ScenarioOutline: Use file-location from keyword/title line to line before its Examples rows.

A file-location into a Scenario Container selects all its entities (Scenarios, …).

Support for Emojis in Feature Files and Steps

  • Emojis can now be used in *.feature files.

  • Emojis can now be used in step definitions.

  • You can now use language=emoji (em) in *.feature files ;-)

FILE: features/i18n_emoji.feature
# language: em
# SOURCE: https://github.com/cucumber/cucumber/blob/master/gherkin/testdata/good/i18n_emoji.feature
# HINT:
#   Temporarily disabled on os=win32 (Windows) until unicode encoding issues are fixed.
#   Try with environment variable: PYTHONUTF8=1

@not.with_os=win32
📚: 🙈🙉🙊

  📕: 💃
    😐🎸
FILE: features/steps/i18n_emoji_steps.py
# -*- coding: UTF-8 -*-
# NEEDED-BY: features/i18n_emoji.feature

from __future__ import absolute_import, print_function
from behave import given


@given(u'🎸')
def step_impl(context):
    """Step implementation example with emoji(s)."""
    pass

Extension Point: Runner

behave has now an extension point to supply an own test runner. See Runners for more details.

Improve Active-Tags Logic

The active-tag computation logic was slightly changed (and fixed):

  • if multiple active-tags with same category are used

  • combination of positive active-tags (use.with_{category}={value}) and negative active-tags (not.with_{category}={value}) with same category are now supported

All active-tags with same category are combined into one category tag-group. The following logical expression is used for active-tags with the same category:

ALGORITHM: Active-Tag Expressions
category_tag_group.enabled := positive-tag-group-expression and not negative-tag-group-expression
positive-tag-group-expression := enabled(tag1) or enabled(tag2) or ...
negative-tag-group-expression := enabled(tag3) or enabled(tag4) or ...

enabled(tag) := TRUE, if positive/negative active-tag condition is TRUE.
POSITIVE TAGS: tag1, tag2  --  @use.with_{category}={value}
NEGATIVE TAGS: tag3, tag4  --  @not.with_{category}={value}

EXAMPLE:

FILE: features/active_tag.example.feature
Feature: Active-Tag Example

  @use.with_browser=Safari
  @use.with_browser=Chrome
  @not.with_browser=Firefox
  Scenario: Use one active-tag group/category

    HINT: Only executed with web browser Safari and Chrome, Firefox is explicitly excluded.
    ...

  @use.with_browser=Firefox
  @use.with_os=linux
  @use.with_os=darwin
  Scenario: Use two active-tag groups/categories

    HINT 1: Only executed with browser: Firefox
    HINT 2: Only executed on OS: Linux and Darwin (macOS)
    ...

Active-Tags: Use ValueObject for better Comparisons

The current mechanism of active-tags only supports the equals / equal-to comparison mechanism to determine if the tag.value matches the current.value, like:

NAME SCHEMA FOR: Active-tags
# -- SCHEMA: "@use.with_{category}={value}" or "@not.with_{category}={value}"
@use.with_browser=Safari    # HINT: tag.value = "Safari"
@not.with_browser=Safari    # HINT: tag.value = "Safari"

ACTIVE TAG MATCHES, if:
    current.value == tag.value  (using "@use..." for string values)
    current.value != tag.value  (using "@not..." for string values)

The equals-to comparison method is sufficient for many situations. But in some situations, you want to use other comparison methods. The behave.tag_matcher.ValueObject class was added to allow the user to provide an own comparison method (and type conversion support).

EXAMPLE 1:

FILE: features/active_tags.example1.feature
Feature: Active-Tag Example 1 with ValueObject

  @use.with_temperature.min_value=15
  Scenario: Only run if temperature >= 15 degrees Celsius
    ...
FILE: features/environment.py
import operator
from behave.tag_matcher import ActiveTagMatcher, ValueObject
from my_system.sensors import Sensors

# -- SIMPLIFIED: Better use behave.tag_matcher.NumberValueObject
# CTOR: ValueObject(value, compare=operator.eq)
# HINT: Parameter "value" can be a getter-function (w/o args).
class NumberValueObject(ValueObject):
    def matches(self, tag_value):
        tag_number = int(tag_value)
        return self.compare(self.value, tag_number)

current_temperature = Sensors().get_temperature()
active_tag_value_provider = {
    # -- COMPARISON:
    # temperature.value:     current.value == tag.value  -- DEFAULT: equals  (eq)
    # temperature.min_value: current.value >= tag.value  -- greater_or_equal (ge)
    "temperature.value":     NumberValueObject(current_temperature),
    "temperature.min_value": NumberValueObject(current_temperature, operator.ge),
}
active_tag_matcher = ActiveTagMatcher(active_tag_value_provider)

# -- HOOKS SETUP FOR ACTIVE-TAGS: ... (omitted here)

EXAMPLE 2:

A slightly more complex situation arises, if you need to constrain the execution of an scenario to a temperature range, like:

FILE: features/active_tags.example2.feature
Feature: Active-Tag Example 2 with Min/Max Value Range

  @use.with_temperature.min_value=10
  @use.with_temperature.max_value=70
  Scenario: Only run if temperature is between 10 and 70 degrees Celsius
    ...
FILE: features/environment.py
...
current_temperature = Sensors().get_temperature()
active_tag_value_provider = {
    # -- COMPARISON:
    # temperature.min_value:  current.value >= tag.value
    # temperature.max_value:  current.value <= tag.value
    "temperature.min_value": NumberValueObject(current_temperature, operator.ge),
    "temperature.max_value": NumberValueObject(current_temperature, operator.le),
}
...

EXAMPLE 3:

FILE: features/active_tags.example3.feature
Feature: Active-Tag Example 3 with Contains/Contained-in Comparison

  @use.with_supported_payment_method=VISA
  Scenario: Only run if VISA is one of the supported payment methods
    ...

  # OR: @use.with_supported_payment_methods.contains_value=VISA
FILE: features/environment.py
# -- NORMALLY:
#  from my_system.payment import get_supported_payment_methods
#  payment_methods = get_supported_payment_methods()
...
payment_methods = ["VISA", "MasterCard", "paycheck"]
active_tag_value_provider = {
    # -- COMPARISON:
    # supported_payment_method: current.value contains tag.value
    "supported_payment_method": ValueObject(payment_methods, operator.contains),
}
...

Detect Bad Step Definitions

The regular expression (re) module in Python has increased the checks when bad regular expression patterns are used. Since Python >= 3.11, an re.error exception may be raised on some regular expressions. The exception is raised when the bad regular expression is compiled (on re.compile()).

behave has added the following support:

  • Detects a bad step-definition when they are added to the step-registry.

  • Reports a bad step-definition and their exception during this step.

  • bad step-definitions are not registered in the step-registry.

  • A bad step-definition is like an UNDEFINED step-definition.

  • A BadStepsFormatter formatter was added that shows any BAD STEP DEFINITIONS

Note

More Information on BAD STEP-DEFINITIONS:

Gherkin Parser strips no longer trailing colon from step

In the past, the Gherkin parser removed a trailing colon (:) on steps that had a text or table section.

EXAMPLE:

FILE: features/parser_example.feature
Feature:
  Scenario:
    Given a file named "some_file.txt" with:
      """
      Lorem ipsum, ipsum lorem, ...
      """
FILE: features/steps/filesystem_steps.py
# -- OLD IMPLEMENTATION:
from behave import given, when, then
from pathlib import Path

@given('a file named "{filename}" with')  #< HINT: Ends without colon
def step_write_file_with_contents(ctx, filename):
    Path(filename).write_text(ctx.text, encoding="UTF-8")

The behaviour of the Gherkin parser was changed:

  • Trailing colon character is no longer removed on steps with text/table section

REASONS:

  • The new behaviour is more natural and much simpler.

  • The step writer can define whatever is needed.

  • Fixes a problem in PyCharm IDE where the lookup of the step-definition in such a case is not working.

EXAMPLE 2:

FILE: features/steps/filesystem_steps.py
# -- NEW IMPLEMENTATION:
from behave import given, when, then
from pathlib import Path

@given('a file named "{filename}" with:')  #< HINT: Ends with colon
def step_write_file_with_contents(ctx, filename):
    Path(filename).write_text(ctx.text, encoding="UTF-8")

Hint

The old behaviour of the Gherkin parser can be (re-)enabled by setting the following environment variable before using behave:

On UNIX: Using bash shell
export BEHAVE_STRIP_STEPS_WITH_TRAILING_COLON="yes"
On WINDOWS: Using cmd shell
set BEHAVE_STRIP_STEPS_WITH_TRAILING_COLON="yes"

Distinguish between Failures and Errors

behave distinguishes now between failures and errors:

  • a failure is caused by an assert-mismatch (or: AssertionError is raised)

  • an error is caused normally when an “unexpected” exception is caught.

In addition, an error occurs if:

  • a hook error occurs

  • a cleanup error occurs (and is not ignored)

  • a pending step is detected (behave.api.pending_step.StepNotImplementedError)

  • a undefined step is detected

Support for Pending Steps

behave provides now better support for pending steps.

  • A pending step has a binding between the step-pattern and its step-function.

  • Therefore, a pending step registers itself in the step registry

  • But a pending step is not yet implemented (marked-by: behave.api.pending_step.StepNotImplementedError )

A pending step looks like:

FILE: features/steps/pending_step_example.py
from behave import given, when, then
from behave.api.pending_step import StepNotImplementedError

@given('a pending step')
def step_given_a_pending_step(ctx):
    raise StepNotImplementedError("Given a pending step")

A pending step causes an error during the test run. But you can mark a scenario temporarily with the @wip tag to let any of its pending steps pass:

FILE: features/pending_step.feature
Feature: Example

  @wip
  Scenario: With @wip tag and pending step
    Given a step passes
    When a pending step is used
    Then another step passes
shell: Run behave tests
$ behave -f plain features/pending_step.feature
Feature: Example

  Scenario: With @wip tag and pending step
    Given a step passes ... passed
    When a pending step is used ... pending_warn
    When another step passes ... passed

...
1 scenario passed, 0 failed, 0 skipped
2 steps passed, 0 failed, 0 skipped, 1 pending_warn

Without the @wip marker, a scenario with pending steps causes an error:

shell: Run behave tests
$ behave -f plain features/other_pending_step.feature
Feature: Example 2

  Scenario: Without @wip tag but with pending step
    Given a step passes ... passed
    When a pending step is used ... pending

...
0 scenarios passed, 0 failed, 1 error, 0 skipped
1 step passed, 0 failed, 1 skipped, 1 pending

Note

More Information on pending steps and undefined steps:

Step Definitions with Cucumber-Expressions

Support for a step definitions with cucumber-expressions was added to behave by providing the behave.cucumber_expression module.

Cucumber-expressions:

  • Provide a simple syntax for step-parameters (aka: placeholders) compared to regular-expressions

  • Provide a simple syntax for optional or alternative (unmatched) text parts.

  • Provide support for parameter types and type conversions

  • Provide a number of predefined parameter types, like: {int}, {word}, {string}, …

  • Similar to parse-expressions that are normally used in behave (hint: parse-expressions was one of the descendants that lead to the development of cucumber-expressions)

EXAMPLE 1:

Use the use_step_matcher_for_cucumber_expressions() function to enable this step-matcher before any step definitions with cucumber-expressions are used. It is possible to do this:

  • in the features/environment.py file (as default step-matcher)

  • in each features/steps/*.py steps file

FILE: features/environment.py
from behave.cucumber_expression import use_step_matcher_for_cucumber_expressions

# -- HINT: Use StepMatcher4CucumberExpressions as default step-matcher.
use_step_matcher_for_cucumber_expressions()

In this example, we want to use the Color enum as parameter_type (placeholder) in the steps definitions:

FILE: example4me/color.py
from enum import Enum

class Color(Enum):
    red = 1
    green = 2
    blue = 3

    @classmethod
    def from_name(cls, text: str):
        text = text.lower()
        for enum_item in iter(cls):
            if enum_item.name == text:
                return enum_item
        # -- OOPS:
        raise ValueError("UNEXPECTED: {}".format(text))

We provide the necessary steps with the additional parameter_type=color by using the Color.from_name() function as type converter/transformer.

FILE: features/steps/color_steps.py
# -- REQUIRES: Python3
from behave import when, then
from behave.cucumber_expression import (
    ParameterType,
    define_parameter_type,
    # -- SIMILAR-TO: define_parameter_type_with
)
from example4me.color import Color

# -- REGISTER PARAMETER TYPES:
# OR: Use define_parameter_type_with(name="color", ...)
define_parameter_type(ParameterType(
    name="color",
    regexp="red|green|blue",
    type=Color,
    transformer=Color.from_name
))

...

After the parameter_type=color is defined, we can use it as {color} placeholder in the step definitions:

FILE: features/steps/color_steps.py (continued)
# -- STEP DEFINITIONS:
@when('I select the "{color}" theme colo(u)r')
def step_when_select_color_theme(ctx, color: Color):
    assert isinstance(color, Color)
    ctx.selected_color = color

@then('the profile colo(u)r should be "{color}"')
def step_then_profile_color_should_be(ctx, the_color: Color):
    assert isinstance(the_color, Color)
    assert ctx.selected_color == the_color
FILE: features/cucumber_expression.feature
Feature: Use CucumberExpressions in Step Definitions
    Scenario: User selects a color twice
      Given I am on the profile settings page
      When I select the "red" theme colour
      But  I select the "blue" theme color
      Then the profile color should be "blue"

EXAMPLE 2: Use TypeBuilder.make_enum()

The solution in “EXAMPLE 1” can be simplified by using the TypeBuilder class. It provides a TypeBuilder.make_enum() that generates a parse-function for an Enum class or a dict-mapping. This parse-function provides a type converter/transformer and its regular-expression pattern (as attribute), like:

MORE:

In addition, the TypeBuilder class provides support to compose parse-functions (aka: type converters) and regular-expression patterns from other parse-functions or data, like:

  • TypeBuilder.make_enum(): Builds a parse-function and regex-pattern for an Enum class or a key/value mapping (aka: dict).

  • TypeBuilder.make_choice(): Builds a parse-function and regex-pattern for a list of string values.

  • TypeBuilder.make_variant(): Builds a parse-function and regex-pattern from a list of parse-functions (and their patterns) as alternative types.

  • TypeBuilder.with_many(): Builds a parse-function and regex-pattern for many items based on the parse-function of one item

See also

parse-expressions

Note

A parameter_type can only be defined once (maybe: Use the environment-file or …).

Improved Logging Support

It is now simpler to set up the logging to a file in behave:

FILE: features/environment.py
def before_all(ctx):
    log_format = "LOGFILE.{levelname} -- {name}: {message}"
    ctx.config.setup_logging(filename="behave.log", format=log_format)

# -- NOTE: Setup with logging configuration file was needed before.

Tip

behave supports now the newer, additional format styles for log record formats:

  • f-string format style, like: {message}

  • shell placeholder format style, like: ${message}

Only the percent-string placeholder style was supported before (like: %(message)s).

Improved Capture Support

The capture of hooks is now supported (special case: before_all() hook). To better support this, the formatter(s) are now called before the before_feature/before_scenario/before_tag hooks are called. This ensures that the Feature/Scenario name is shown (as context) before the any captured output of before_feature/before_scenario/before_tag hooks is printed.

AFFECTED FORMATTERS:

  • pretty

  • plain

CHANGES (partly incompatible):

The name of capture related command line options have been changed slightly:

Option Name

Old Option Name

Description

--capture

NEW: Enable/disable capture mode for stdout/stderr/log.

--capture-hooks

NEW: Enable/disable capture of hooks.

--capture-stdout

--capture

Enable/disable capture of stdout.

--capture-stderr

--capture-stderr

Enable/disable capture of stderr.

--capture-log

--logcapture

Enable/disable capture of log-output.

The Configuration class attribute names were adapted accordingly to better correspond to the command line options:

Attribute Name

Old Attribute Name

Description

capture

NEW: Enable/disable capture mode for stdout/stderr/log.

capture_hooks

NEW: Enable/disable capture of hooks.

capture_stdout

stdout_capture

Enable/disable capture mode for stdout.

capture_stderr

stderr_capture

Enable/disable capture mode for stderr.

capture_log

log_capture

Enable/disable capture mode for log output.

The CaptureController class attribute names were renamed accordingly to better correspond to the naming scheme:

Attribute Name

Old Attribute Name

Description

capture_stdout

stdout_capture

Used to capture stdout output.

capture_stderr

stderr_capture

Used to capture stderr output.

capture_log

log_capture

Used to capture log output.

Note

A deprecating warning will be emitted if you use the old names.

Step Decorators: Support for Async-Steps

To simplify usage, the normal step decorators directly support async-steps now, like:

FILE: features/steps/async_steps.py
# -- NOW:
import asyncio
from behave import given, when, then, step

@step('a coroutine step waits "{duration:f}" seconds')
async def step_coroutine_waits_seconds(ctx, duration: float):
    await asyncio.sleep(duration)

@when('I execute the long running command', timeout=20.0)
async def step_execute_long_running_command(ctx):
    # -- HINT: Step fails if step duration exceeds 20 seconds.
    pass  # ...
FILE: features/steps/async_steps_before.py
# -- BEFORE:
import asyncio
from behave import step
from behave.api.async_step import async_run_until_complete

@step('a coroutine step waits "{duration:f}" seconds')
@async_run_until_complete
async def step_async_step_waits_seconds(ctx, duration: float):
    await asyncio.sleep(duration)

@when('I execute the long running command')
@async_run_until_complete(timeout=20.0)
async def step_execute_long_running_command(ctx):
    # -- HINT: Fails if step duration exceeds 20 seconds.
    pass  # ...

DEPRECATING @async_run_until_complete decorator

  • BETTER: Use nornmal step decorators instead.

  • The support for @async_run_until_complete decorator will be removed in behave v1.4.0.

Changes

ModelRunner:

  • Simplify signature on method run_hook(context, *args) to run_hook(*args)

NEW behave.model_type module:

  • Moved generic model classes here from behave.model_core module, like: behave.model_core.Status, behave.model_core.FileLocation.