Loading

Skip tests in Scout

There are multiple ways to ensure a Scout test or test suite does not run at all, or runs only in certain environments.

Tip

Best practices when skipping tests:

  • Skipping is usually temporary—re-enable as soon as possible.
  • Create a tracking issue in the Kibana repo (label: skipped-test) and link it in a comment next to the skip.
  • AppEx QA maintains internal documentation (for Elasticians only) on tracking skipped tests.

Scout uses Playwright, which provides test.skip() and test.describe.skip() to skip tests or suites in all environments.

If you prefer to control which environments your tests run in, use deployment tags instead.

import { test, tags } from '@kbn/scout';

// Add a comment to explain why this suite is skipped
test.describe.skip('test suite', { tag: tags.stateful.classic }, () => {
  test('test', async ({ page }) => {
    // this test will be skipped
  });
});
		
import { test, tags } from '@kbn/scout';

test.describe('test suite', { tag: tags.stateful.classic }, () => {
  // Add a comment to explain why this test is skipped
  test.skip('first test', async ({ page }) => {
    // this test will be skipped
  });

  test('second test', async ({ page }) => {
    // this test will run
  });
});

Playwright also provides test.fixme() as an alternative to test.skip(). It behaves the same way but semantically indicates the test is broken and needs to be fixed:

import { test, tags } from '@kbn/scout';

test.describe('test suite', { tag: tags.stateful.classic }, () => {
  // Add a comment to explain what needs to be fixed
  test.fixme('broken test', async ({ page }) => {
    // this test will be skipped and marked as "fixme"
  });
});

Instead of skipping entirely, control which environments suites run in using deployment tags (@{location}-{arch}-{domain}).

For example, to run only on stateful classic environments:

import { test, tags } from '@kbn/scout';

test.describe('Stateful classic feature', { tag: tags.stateful.classic }, () => {
  test('my test', async ({ page }) => {
    // this test only runs in stateful classic environments
  });
});

See Deployment tags for shortcuts and patterns.