Tools for checking and formatting source code for projects running on Aurora OS

This document translations

• English
• Russian / Pусский

Tools list

Tools list includes:

• check-code.py — checking source codes for compliance with requirements.
• check-cpp-code.py — checking C++ code for compliance with requirements.
• format-cpp-code.py — formatting source code in C++ according to requirements.
• check-qml-code.py — checking QML code for compliance with requirements qmlformat.
• format-qml-code.py — formatting source code in QML according to requirements.
• license-format.py — checking and then writing the license header to the project files.
• check-md-files.py — checking markdown files.
• check-translation.py — checking translation files.
• check-pro-files.py — checking pro files.
• remove-executable-bit.py — checking and deleting the executable bit in the project files.

Dependencies

The tools depend on the following tools:

• Python interpreter version 3.8 and higher.
• clang-format version 15.
• qmlformat, included with Qt 6 release.
• File comparison tool colordiff.
• Python 3 lxml package for parsing xml files.
• Python 3 langdetect package for file analysis AUTHORS.md.

To install dependencies on GNU/Debian 12 and Ubuntu 22.10 run:

apt-get update
apt-get install clang-format-15 colordiff python3 python3-lxml \
 python3-langdetect qt6-declarative-dev-tools qt6-l10n-tools

To install dependencies in Alt p10 branch, run:

apt-get update
apt-get install clang15.0-tools colordiff python3 python3-module-lxml \
  pip qt6-declarative-devel qt6-tools
pip3 install langdetect

In Alt you also need to set the environment variable ALTWRAP_LLVM_VERSION equal to 15.0, to use the required version of the tools.

Specifying a project to check

Scripts must be run from the project root directory, or passed the project path using an argument --directory.

An example of running the script check-code.py for the project tempProject from the root of the project being checked

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py

An example of running the check-code.py script for the tempProject project using the --directory argument:

~/$ python3 ~/auroraos-code-formatters/check-code.py --directory ~/tempProject/

Selecting directories to check

The list of directories in which C++ code and QML code should be checked can be specified using the arguments --cpp-dir and --qml-dir, respectively. If there are several directories, they should be listed separated by a space. It is necessary to specify paths relative to the project root.

An example of using the --cpp-dir and --qml-dir arguments when running the check-code.py script for the tempProject project. In this example, the C++ code will be checked only for the src/controllers and src/types directories, and the QML code - for the qml/pages and qml/components directories:

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py \
  --cpp-dir src/controllers src/types \
  --qml-dir qml/pages qml/components

A list of directories in which code should not be checked or formatted can be specified using the --except-dir argument. If there are several directories, they should be listed separated by a space. It is necessary to specify paths relative to the project root.

An example of using the --except-dir argument when running the check-code.py script for the tempProject project. In this example, the check will be carried out for all directories except the specified src/controllers and qml/pages:

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py \
  --except-dir src/controllers qml/pages

Run code formatting

To start code formatting, you need to run the code checking script check-code.py with the --format flag. When you run the script, the C++ code and QML code will be formatted, and the license headers of the project files will be adjusted. An example of using this flag:

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py \
  --license-path ../license \
  --year 2022-2023 --format

Selecting logging level

The logging level is selected using an --log-level argument. There are only 3 levels:

1. INFO -- allows you to display messages of all levels.
2. WARNING -- installed by default, allows you to display only warnings and errors.
3. ERROR -- allows you to output only errors.

Examples of using this flag:

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py --log-level INFO

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py --log-level WARNING

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py --log-level ERROR

Disabling individual tools for organizing checks

To disable individual tools from the code verification script check-code.py , it is necessary to specify a flag corresponding to script excluded from the general check when calling it.

The set of flags for disabling individual tools includes:

• --except-check-cpp — excludes C++ code check performed by the script check-cpp-code.py .
• --except-check-qml — excludes QML code check performed by the script check-qml-code.py .
• --except-check-license — excludes verification of the license header performed by the script license-format.py .
• --except-check-markdown — excludes check of markdown files performed by the script check-md-files.py .
• --except-check-translation — excludes checking of translation files performed by the script check-translation.py .
• --except-check-pro — excludes checking of pro files performed by the script check-pro-files.py .

Example of using these flags:

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py \
--except-check-cpp --except-check-qml

In this example, the tools for checking C++ code and QML code will be excluded from the general check.

Verification restriction profiles

You can also use different validation script profiles to disable certain checks. To use the profile when calling the code verification script check-code.py you must specify the --profile argument and pass the name of the profile you are interested in.

List of profiles available for use:

• OnlyCodeCheck — the profile that disables all tools except for C++ and QML code checks, allows you to get information about current errors in the formatting of the application code.
• DisableCppCheck — the profile that disables C++ code checks is used if the project requires custom formatting other than the formatting of clang-format tools.
• DisableQmlCheck — the profile that disables QML code checks is used if the project requires custom formatting other than the formatting of qmlformat tools.
• DisableCodeCheck — the profile that disables C++ and QML code checks.
• DisableMarkdownCheck — the profile that disables checking markdown files is intended for projects, where the markdown files to be checked should not be presented, as a result of which error messages during verification can be avoided.

An example of using a verification profile:

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py \
  --profile=DisableCodeCheck

In this example, the DisableCodeCheck profile will be used from the general check, which disables checks of C++ code and QML code.

Script for checking license headers of project files

To run the script to check the license header, the following arguments can be used when calling check-code.py:

1. --directory and --except-dir, described in previous sections.
2. --year - the year or period of years that is used in checking and when recording the license header, the current calendar year is used by default.
3. --license-path - path to the file with the license header text, by default the license-template file in the project directory is used, it is recommended to use {year} in the file text for subsequent substitution of the --year argument into the text.

An example of using arguments to check the license header when running the check-code.py script for the tempProject project.

~/tempProject$ python3 ~/auroraos-code-formatters/check-code.py \
  --directory ~/tempProject/ \
  --license-path ../license --year 2022-2023

To run license-format.py separately, the following arguments can be used in addition to those listed above:

1. --check-license - a flag that, when specified, will check the license header without writing it.
2. --replace-license - a flag that, when specified, will remove the old license header in the project file before writing a new one; the argument is used only in the absence of --check-license.

An example of using the license-format.py script to check the license header for the tempProject project.

~/tempProject$ python3 ~/auroraos-code-formatters/license-format.py \
  --directory ~/tempProject/ \
  --license-path ../license --year 2022-2023 --check-license

An example of using the license-format.py script to check and write the license header for the tempProject project.

~/tempProject$ python3 ~/auroraos-code-formatters/license-format.py \
  --directory ~/tempProject/ \
  --license-path ../license --year 2022-2023

An example of using the license-format.py script to check and write the license header, replacing the old license header for the tempProject project.

~/tempProject$ python3 ~/auroraos-code-formatters/license-format.py \
  --directory ~/tempProject/ \
  --license-path ../license --year 2022-2023 --replace-license

Description of license-template contents

The license-template file contains the standard license header with a built-in {year} parameter.

Tests

Unit tests have been written to verify the operation of deleting the old license header when replacing it. To run them, use the command:

~/tempProject$ python3 ~/auroraos-code-formatters/tests/license-format-tests.py

Script for checking project markdown files

This script checks for the presence of the following markdown files in the project:

1. AUTHORS.md.
2. CODE_OF_CONDUCT.md.
3. CONTRIBUTING.md.
4. LICENSE.BSD-3-CLAUSE.md.
5. README.md.

All files with the same names found in the project are checked.

For files named AUTHORS.md the following checks are also performed:

1. All file text must be in English.
2. The presence of "Authors" section.
3. The presence of the authors' first and last names (two words, without patronymic).
4. The presence of an indication of the roles of the authors and the year in the format "Role, YYYY" or "Role, YYYY-YYYY".
5. Availability of email addresses only with the @omp.ru domain.

Example of a AUTHORS.md file:

# Authors

* Ivan Petrov, <i.petrov@omp.ru>
  * Developer, 2022-2023
* Ekaterina Sokolova
  * Developer, 2023
* Dmitriy Kuznetsov, <d.kuznetsov@omp.ru>
  * Product owner, 2023
  * Reviewer, 2023
* Anna Romanova
  * Reviewer, 2022-2023

Files named LICENSE.BSD-3-CLAUSE.md are also checked for the presence of a copyright line.

Script for checking translation files

This script performs sequential checks against translation files:

1. Checking for the presence of translation files for the Russian language (-ru.ts) in the project. If the project does not have this file, the script will display a corresponding message.
2. Checking the syntax data of the translation files. This check is carried out using the lrelease tool. If an error is detected, a message like this will be displayed:

$ lrelease error: ...

3. Check for incomplete translations. An unfinished translation means a translation string with the type type="unfinished", the corresponding message looks like:

$ In the 'ts_filename' file on line 'ts_n': translation is unfinished
    The location of the translation: filename 'filename' line 'n'

where ts_filename and ts_n are the name of the translation file and the line of the translation file where the error occurred, filename and n are the file and line number of the unfinished translation.

4. Check for missing translations. Lack of translation means missing text in the <translation ></translation> tag. The corresponding message looks like:

$ In the 'ts_filename' file on line 'ts_n': there is no translation
    The location of the translation: filename 'filename' line 'n'

where ts_filename and ts_n are the name of the translation file and the line of the translation file where the error occurred, filename and n are the file and line number of the missing translation.

Script for checking project pro files

This script checks the "DISTFILES" section of the project's pro files for the presence of paths to non-existent files. If there are such paths, a warning will be displayed, which contains the path to the pro file and the line number with a non-existent path. If the "DISTFILES" section is missing, an informational message will be displayed.

Script for checking the executable bit in the project files

This script checks and then deletes the executable bit in the project files.

An example of using a script remove-executable-bit.py to check and delete the executable bit:

~/tempProject$ python3 ~/auroraos-code-formatters/remove-executable-bit.py \
  --directory ~/tempProject/

It is also possible to run the script only to check for the presence of an executable bit. To do this, the --check-executable argument is used.

An example of using a script remove-executable-bit.py only for checking the executable bit:

~/tempProject$ python3 ~/auroraos-code-formatters/remove-executable-bit.py \
  --directory ~/tempProject/ --check-executable

Setting up QtCreator

Source code formatting can be configured directly in the Qt Creator environment. The formatting style must be set to File if there is a .clang-format configuration file in the project root.

Using Docker container

The tool can be packaged inside the Docker image. Run the following command in the root of the project:

docker image build . -t project-formatter:latest

Then the tools can be run from within the project-formatter image. You can use the following syntax to start code checking:

docker container run --rm -u $(id -u):$(id -g) -v $(pwd):/app -w /app project-formatter check-code.py

This will run the container, start the tool and remove the container.