Tests and solutions

You can create tests for a problem and specify files with an original solution or configure test generators and validators for self-checking when editing the problem, also assembling test suites.

Create a test

1. Go to the problem page and select Tests and solutions on the left.
2. Click Create a test.
3. Fill in the form fields: enter the test file name, test data, response file name, and response.
4. Click Add. The test file will appear in the tests directory.

See an example of test creation in a programming problem.

Solution

You need the solution for self-checking when editing the problem, generating answer files, and using the test generator.

Add a solution

1. In the Solutions section, click Add solution.
2. Click Select file and upload the ready solution. The file with the solution will appear in the directory with the problem files.
3. Select the uploaded file and click Select. The file will be added to the problem. At the moment, it's just a file marked as a solution.

Run the problem with the solution

1. In the list of solutions, select the compiler that matches the solution.
2. Click Submit. This creates a submission that will undergo all tests and comply with the set limits.

Run locally

To run your solution locally, use the yc-invoke utility. To get a sample run command for your problem, select the compiler name from the provided list.

Test generators

A generator is a special program that can be fed a value as an input. The output is a pseudorandom dataset that will be used as test data.

What makes a generator different from a simple randomizer is that its output data is constant when fed the same input data on different platforms.

Test generator environment

You can select the environment (compiler) in which test generators will be executed. The selected compiler applies to every generator in the problem.

How to select an environment

To select an environment:

1. In the Tests and solutions section, find the Generator environment field.
2. Select the desired environment from the dropdown list.

Environment behavior

If the problem has no generators yet, the list shows Not selected and the Add button is disabled.

After you select an environment, the list displays its name and the setting is immediately applied to all existing generators.

#include "testlib.h"
#include <iostream>

using namespace std;

int main(int argc, char* argv[])
{
    registerGen(argc, argv, 1);
    int n = atoi(argv[1]);
    cout << rnd.next(1, n) << " ";
    cout << rnd.next(1, n) << endl;
}

Add a generator

1. Go to the Generate tests section and click Add new generator.
2. Click Select a file and upload the file with the generator code. The file with the generator will appear in the directory with the problem files.
3. Select the uploaded file and click Select. The file will be added to the problem.

Launch test generation

To launch test generation, you need an added solution and the test generation script.

The name of the generator file is specified in the generation script. The file launch arguments and output are saved to a specific file, usually a test file:

generator_name.cpp 1 > tests/01
generator_name.cpp 5 > tests/02
generator_name.cpp 15 > tests/03

To launch test and answer generation:

1. Enter or paste the test generation script in the Edit test generation script field and click Save.
2. Select the uploaded author's solution in the list below and click Execute.

The system will launch the generator with the submitted arguments one by one and save the data to the test files. After that, the system will launch the author's solution with the generated test files one by one. The output is saved to the test answer file.

Test validators

A validator is a strict program that checks whether the input data file complies with the rules.

A validator can be written in C++ using the testlib.h library or in another supported language.

#include "testlib.h"
#include <iostream>

using namespace std;

int main(int argc, char* argv[]) {
    registerValidation();

    cout << inf.readLong(-1000000000000000LL, 1000000000000000LL);
    inf.readSpace();
    cout << " ";
    cout << inf.readLong(-1000000000000000LL, 1000000000000000LL);
    inf.readEoln();
    cout << endl;
    inf.readEof();
}

Add a new validator

1. Under Validators, click Add validator.
2. Click Select a file and upload the file with the validator code. The file with the validator will appear in the directory with the problem files.
3. Select the uploaded file and click Select. The file will be added to the problem.
4. Select the environment to run the validator from the dropdown list. By default, C++20 (GCC 14.1) is selected.
5. In the list of validators, select Yes to apply the validator to the answer file, or No to apply the validator to the input data file.

The validators table displays an Environment column showing the selected environment for each validator. For validators added before the environment selection feature was implemented, the Legacy label is displayed.

Edit a validator

To change validator settings:

1. In the validators list, click the icon in the row with the required validator.

2. In the modal form that opens, you can:

   • View the validator file name (read-only);
   • Change the Apply to answer setting (Yes/No);
   • Select a different environment to run the validator.

3. Click Save.

Run a validator

Click Run all validators or the  icon in the row with the required validator.
You will see the results of each validator for each test or answer file.

Test sets

Test suites are used to group and separate tests. By default, a problem has an All tests set. You can also add samples sets and simple numbered sets (1, 2, 3, and so on) to problems.

A samples set is one that is displayed in the problem statement. We recommend including 1-3 tests in this set so the contest participants can see what data they will be working with.

Other test sets are usually not visible to users (this can be changed in the contest settings).

To add a test set, click Add test set. To add a samples set, click Add samples.

If you want to delete a set, click in the row with the set.

Click the set name to go to settings:

• Input file pattern: This is the pattern the system uses to search for test files. The default value is the one that can be used to find tests with a standard name.

  You can use the following expressions to search for files:

  • No patterns — matches the given name.

    For example, "tests/01" — the system will search for a file named "01" in the "tests" folder.

  • Wildcard matcher — single substitution: "?" — this is any single character.

    For example, "tests/??" — the system will search for files in the "tests" folder whose names consist of two characters, "a1", "b2", "00", and so on.

  • Range matcher — range or individual values.

    For example, "tests/{1-2,5,44-}" — the system will search for files in the "tests" folder whose names consist of characters 1, 2, 5, or starting from 44 and onwards. Leading zeros are ignored, so the string "01" will be treated as 1.

• Correct answer file template: This is the template the system uses to search for the test answer file.

  You can use the following in templates:

  • * — full path to input.

    For example, test file "tests/01.txt" will correspond to "tests/01.txt".

  • %f — input file name.

    For example, test file "tests/01.txt" will correspond to "01.txt".

  • %s — input file name without extension.

    For example, test file "tests/01.txt" will correspond to "01".

  • %p — full path to input without extension.

    For example, test file "tests/01.txt" will correspond to "tests/01".

After making changes, you need to update test sets: open the desired set and click Update tests. The list of tests in the set will be updated according to the input file patterns and the correct answer file.