Write a Python test

Normally we write functional tests in Python.

• We use pytest for our Python test infrastructure.
• Python tests should be added to tests/python/test_xxx.py.

For example, you've just added a utility function ti.log10. Now you want to write a test to ensure that it functions properly.

note

Before running the test launcher tests/run_tests.py, you need to install the corresponding dependencies:

pip install -r requirements_test.txt

Add a new test case

Look into tests/python, see if there is already a file suitable for your test. If not, create a new file for it. In this case, let's create a new file tests/python/test_logarithm.py for simplicity.

Add a function, the function name must start with test_ so that pytest could find it. e.g:

import taichi as ti

def test_log10():
    pass

Add some tests that make use of ti.log10 to ensure it works well. Hint: You may pass/return values to/from Taichi-scope using 0-D fields, i.e. r[None].

import taichi as ti

def test_log10():
    ti.init(arch=ti.cpu)

    r = ti.field(ti.f32, ())

    @ti.kernel
    def foo():
        r[None] = ti.log10(r[None])

    r[None] = 100
    foo()
    assert r[None] == 2

Execute python tests/run_tests.py logarithm, and the functions starting with test_ in tests/python/test_logarithm.py will be executed.

Test against multiple backends

The line ti.init(arch=ti.cpu) in the test above means that it will only test on the CPU backend. In order to test against multiple backends, please use the @ti.test decorator, as illustrated below:

import taichi as ti

# will test against both CPU and CUDA backends
@ti.test(arch=[ti.cpu, ti.cuda])
def test_log10():
    r = ti.field(ti.f32, ())

    @ti.kernel
    def foo():
        r[None] = ti.log10(r[None])

    r[None] = 100
    foo()
    assert r[None] == 2

In this case, Taichi will execute this test case on both ti.cpu and ti.cuda backends.

And you may test against all available backends (depends on your system and building environment) by simply not specifying the argument:

import taichi as ti

# will test against all backends available on your end
@ti.test()
def test_log10():
    r = ti.field(ti.f32, ())

    @ti.kernel
    def foo():
        r[None] = ti.log10(r[None])

    r[None] = 100
    foo()
    assert r[None] == 2

Use ti.approx for comparison with tolerance

Sometimes the precision of math operations could be limited on certain backends such as OpenGL, e.g., ti.log10(100) may return 2.000001 or 1.999999 in this case.

Adding tolerance with ti.approx can be helpful to mitigate such errors on different backends, for example 2.001 == ti.approx(2) will return True on the OpenGL backend.

import taichi as ti

# will test against all backends available on your end
@ti.test()
def test_log10():
    r = ti.field(ti.f32, ())

    @ti.kernel
    def foo():
        r[None] = ti.log10(r[None])

    r[None] = 100
    foo()
    assert r[None] == ti.approx(2)

caution

Simply using pytest.approx won't work well here, since it's tolerance won't vary among different Taichi backends. It'll likely fail on the OpenGL backend.

ti.approx also correctly treats boolean types, e.g.: 2 == ti.approx(True).

Parametrize test inputs

In the test above, r[None] = 100 means that it will only test that ti.log10 works correctly for the input 100. In order to test against different input values, you may use the @pytest.mark.parametrize decorator:

import taichi as ti
import pytest
import math

@pytest.mark.parametrize('x', [1, 10, 100])
@ti.test()
def test_log10(x):
    r = ti.field(ti.f32, ())

    @ti.kernel
    def foo():
        r[None] = ti.log10(r[None])

    r[None] = x
    foo()
    assert r[None] == math.log10(x)

Use a comma-separated list for multiple input values:

import taichi as ti
import pytest
import math

@pytest.mark.parametrize('x,y', [(1, 2), (1, 3), (2, 1)])
@ti.test()
def test_atan2(x, y):
    r = ti.field(ti.f32, ())
    s = ti.field(ti.f32, ())

    @ti.kernel
    def foo():
        r[None] = ti.atan2(r[None])

    r[None] = x
    s[None] = y
    foo()
    assert r[None] == math.atan2(x, y)

Use two separate parametrize to test all combinations of input arguments:

import taichi as ti
import pytest
import math

@pytest.mark.parametrize('x', [1, 2])
@pytest.mark.parametrize('y', [1, 2])
# same as:  .parametrize('x,y', [(1, 1), (1, 2), (2, 1), (2, 2)])
@ti.test()
def test_atan2(x, y):
    r = ti.field(ti.f32, ())
    s = ti.field(ti.f32, ())

    @ti.kernel
    def foo():
        r[None] = ti.atan2(r[None])

    r[None] = x
    s[None] = y
    foo()
    assert r[None] == math.atan2(x, y)

Specify ti.init configurations

You may specify keyword arguments to ti.init() in ti.test(), e.g.:

@ti.test(ti.cpu, debug=True, log_level=ti.TRACE)
def test_debugging_utils():
    # ... (some tests have to be done in debug mode)

is the same as:

def test_debugging_utils():
    ti.init(arch=ti.cpu, debug=True, log_level=ti.TRACE)
    # ... (some tests have to be done in debug mode)

Exclude some backends from test

Some backends are not capable of executing certain tests, you may have to exclude them from the test in order to move forward:

# Run this test on all backends except for OpenGL
@ti.test(exclude=[ti.opengl])
def test_sparse_field():
    # ... (some tests that requires sparse feature which is not supported by OpenGL)

You may also use the extensions keyword to exclude backends without a specific feature:

# Run this test on all backends except for OpenGL
@ti.test(extensions=[ti.extension.sparse])
def test_sparse_field():
    # ... (some tests that requires sparse feature which is not supported by OpenGL)

Require extension in tests

If a test case depends on some extensions, you should add an argument require in the @ti.test decorator.

@ti.test(require=ti.extension.sparse)
def test_struct_for_pointer_block():
    n = 16
    block_size = 8

    f = ti.field(dtype=ti.f32)

    block = ti.root.pointer(ti.ijk, n // block_size)
    block.dense(ti.ijk, block_size).place(f)

    f[0, 2, 3] = 1

    @ti.kernel
    def count() -> int:
        tot = 0
        for I in ti.grouped(block):
            tot += 1
        return tot

    assert count() == 1

Now, Taichi supports the following extensions:

Name	Extension details
sparse	Sparse data structures
quant_basic	Basic operations in quantization
quant	Full quantization functionalities
data64	64-bit data and arithmetic
adstack	For keeping the history of mutable local variables in autodiff
bls	Block-local storage
assertion	Run-time asserts in Taichi kernels
extfunc	Support inserting external function calls or backend source