Checkers

A checker is a program that checks whether the output of the participant's program is correct and reports the result to the system in a format it understands.

In Yandex Contest, a checker can be:

  • An executable (compiled) file in C++.
  • A file in a different programming language indicating the startup shebang in the first row of the file.

For more information about checker types and functions, see Checker settings.

Create a checker

C++ source code files can be compiled internally using the testlib.h library.

To compile a file in the system:

  1. Open the problem and select Checker settings in the menu on the left.
  2. Choose TESTLIB_EXITCODE_CHECKER or EJUDGE_EXITCODE_CHECKER depending on which set of exitcode will determine the verdict (for more information, see Checker settings).
  3. Click Compile checker. In the window that appears, open the desired folder and select a file or upload a file from the device and select it.
  4. Click Compile. During compilation, the built-in testlib.h library is available in the file environment, and the file can access it.
  5. When compilation is complete, the compiled checker file appears in the checker files block at the bottom of the page (without the extension, if it was specified).
  6. Save the problem settings.

Only use interpreted programming languages. To run your interpreter, specify the shebang in the first row of the checker file. Available shebangs:

Shebang

Explanation

#!/usr/local/bin/python3.7

Python 3.7

#!/usr/bin/python3

Inconsistent: may be version 3.2, 3.4, or 3.7

#!/usr/bin/ruby2.2

Ruby 2.2

#!/opt/custom/nodejs8/bin/node

NodeJS 8

#!/bin/sh

Bash 4.2.4

#!/usr/bin/php

php 5.3

To upload the checker:

  1. Open the problem and select Checker settings in the menu on the left.
  2. Choose the type of checker: TESTLIB_EXITCODE_CHECKER or EJUDGE_EXITCODE_CHECKER, depending on which set of exitcode will determine the verdict (for more information, see Checker settings).
  3. Click Add file. In the window that appears, upload the file with the specified shebang and select it. You can also select a previously uploaded file. The executable file must be uploaded and selected first.
  4. Click Select. The file appears in the checker files block at the bottom of the page.
  5. Save the problem settings.

Alert

The checker file should only contain line breaks in the form \n (Unix line breaks). If you use a different kind of line break (for example, \r\n — Windows line breaks), the system will not be able to run the file.

Configuring a checker

Open the problem and select Checker settings in the menu on the left.

Checker type
  • STANDART_CHECKER: One of the standard checkers. It's selected by default.
  • EJUDGE_EXITCODE_CHECKER: You upload your own checker program which operates the Ejudge system return codes (source in Russian).
  • TESTLIB_EXITCODE_CHECKER: You download your own checker program which operates the testlib library return codes.
Checker ID

Alert

Only for STANDART_CHECKER.

Select the checker ID from the standard programs:

  • cmp_file: Comparing two files string by string. It's selected by default.
  • cmp_file_nospace: Comparing two files but ignoring duplicate spaces.
  • cmp_bytes: Comparing two files byte by byte.
  • cmp_int: Comparing two signed 32-bit integers.
  • cmp_int_seq: Comparing two sequences of signed 32-bit integers.
  • cmp_long_long: Comparing two signed 64-bit integers.
  • cmp_long_long_seq: Comparing two sequences of signed 64-bit integers.
  • cmp_unsigned_int: Comparing two unsigned 32-bit integers.
  • cmp_unsigned_int_seq: Comparing two sequences of unsigned 32-bit integers.
  • cmp_unsigned_long_long: Comparing two unsigned 64-bit integers.
  • cmp_unsigned_long_long_seq: Comparing two sequences of unsigned 64-bit integers.
  • cmp_huge_int: Comparing two integers of arbitrary size.
  • cmp_double: Comparing two double-precision real numbers with a set maximum error.
  • cmp_double_seq: Comparing two sequences of double-precision real numbers with a set maximum error.
  • cmp_long_double: Comparing two extended-precision real numbers with a set maximum error.
  • cmp_long_double_seq: Comparing two sequences of extended-precision real numbers with a set maximum error.
  • cmp_sexpr: Comparing two S-expressions (trees in the Lisp language syntax).
  • cmp_yesno: Comparing two YES or NO answers.
  • rncmp: Comparing two real numbers with a maximum absolute error of 1.5e-5.
  • rncmp1: Comparing two real numbers with a maximum absolute error of 1.0e-1.
  • rncmp2: Comparing two real numbers with a maximum absolute error of 1.0e-2.
  • rncmp3: Comparing two real numbers with a maximum absolute error of 1.0e-3.
  • rncmp4: Comparing two real numbers with a maximum absolute error of 1.0e-4.
  • rncmp5: Comparing two real numbers with a maximum absolute error of 1.0e-5.
  • rncmp6: Comparing two real numbers with a maximum absolute error of 1.0e-6.
  • rncmp7: Comparing two real numbers with a maximum absolute error of 1.0e-7.
  • rncmp8: Comparing two real numbers with a maximum absolute error of 1.0e-8.
  • rncmp9: Comparing two real numbers with a maximum absolute error of 1.0e-9.

Yandex.Contest has more checkers:

  • fcmp: Comparing two floating-point numbers using the Knuth algorithm.
  • hcmp: Comparing two signed integers.
  • icmp: Comparing two signed integers.
  • lcmp: Comparing files as token sequences in strings.
  • ncmp: Comparing ordered sequences of integers.
  • wcmp: Comparing token sequences.
  • ok: Returns OK.
  • cmp_range: Checking whether the answer is within the required range with a maximum absolute error of 1.0e-8.

You can read more about writing your own checkers on GitHub (in English).

Scoring checker

Alert

Only for EJUDGE_EXITCODE_CHECKER and TESTLIB_EXITCODE_CHECKER.

Enable this option to extend checker behavior: In addition to the return code, the checker will be able to return a number that will be interpreted by the system as test points.

Checker files

Specify checker files and files necessary for the proper operation of the checker. The first step is to add the compiled executable checker file.

Alert

We recommend compiling the checker in the Yandex.Contest executable environment. To upload checker sources, click Compile checker.

How checkers work

For checkers built on testlib.h, see the documentation and examples on the library's website.

The following is about interpreted checkers in other programming languages. Examples are given for Python.

File access

Checkers have access to the following files:

  • Test file (test input data).
  • The participant's solution output file.
  • The correct answer file.

The path to the test file is in the first argument of the checker launch. The path to the participant's solution output file is in the second launch argument. The path to the file with the correct answer is in the third launch argument.

Example:

#!/usr/local/bin/python3.7
    
import sys
    
testFile = sys.argv[1]
solutionResultFile = sys.argv[2]
answerFile = sys.argv[3]

Verdict and points

The checker informs the system of the verdict based on its check of the submission and can also set points for checking the test (optional).

The checker's exitcode is used to transmit the test verdict. The interpretation of exitcode depends on the checker settings.

If the Scoring checker option is enabled in the checker settings, the number of points for the test is transmitted as a number with a decimal fraction part and a decimal point as the separator. Points can be given for an OK verdict as well as any other verdict.

Example:

#!/usr/local/bin/python3.7
    
import sys
    
print('10.42 PE')
sys.exit(2)

 — The test will receive a PE verdict and 10.42 points (when using TESTLIB_EXITCODE_CHECKER).

Note

The points message does not need to indicate the verdict. The system takes the first number from the first row output by the checker as the points for a test. The rest is only needed as service information and, depending on the contest settings, may be shown to the user.

Example of a checker

A simple checker checks the values in the input and output files while cutting off spaces and hyphens:

#!/usr/local/bin/python3.7
    
import sys
    
solutionResultFile = sys.argv[2]
answerFile = sys.argv[3]
    
sFile = open(solutionResultFile, mode="r")
aFile = open(answerFile, mode="r")
    
a = aFile.readline().strip()
s = sFile.readline().strip()
    
if (a == s):
  print('1 OK')
  sys.exit(0)
else:
  print('0 WA')
  print(f"{s} != {a}")
  sys.exit(1)

Contact Support

Previous
Statements
Next
Postprocessors