RU | EN

LLM Runner

LLM Runner provides an example of a local mobile chat application with Large Language Models (LLMs) using llama.cpp. It demonstrates DeepSeek-style chat UX with a sidebar of conversations and full local message history.

Table of Contents

• Compatibility
• Build features
• Branch info
• Install and running
• Screenshots
• Use cases
• Project Structure
• Terms of Use and Participation in Development

Compatibility

Conan is required to resolve and install project dependencies during build. The plugin has been working correctly since the 5.0.0 version of the Aurora OS.

Build features

The project is built using CMake build system. The project uses Conan package manager to obtain dependencies including llama-cpp library and its dependencies. For x86_64, rpm/ru.auroraos.LlmRunner.spec contains a temporary workaround for Conan 2.7: it creates an ldd wrapper that runs /lib64/ld-linux-x86-64.so.2 --list so conan-deploy-libraries can correctly detect and copy shared libraries for the target executable. Newer Conan versions such as 2.9 do not need this workaround, and a plain conan-deploy-libraries "$EXECUTABLE" "$CONAN_LIB_DIR" "$SHARED_LIBRARIES" call is enough.

Branch info

Application versions conform to the branch naming convention

Install and running

Installation and build are performed according to the Build example instruction.

Screenshots

Use cases

The application demonstrates local LLM chat functionality:

Chat Management

• Creating, selecting, renaming, and deleting conversations

Model Inference

• Running local LLM inference with real-time token generation using llama.cpp

Conversation History

• Full local storage of chat history with timestamps

System Prompts

• Customizable system prompts for different use cases

Project Structure

The project has a standard structure of an application based on C++ and QML for Aurora OS with LLM capabilities.

• CMakeLists.txt file describes the project structure for the CMake build system and llama.cpp dependencies.
• icons directory contains the application icons for different screen resolutions.
• qml directory contains the QML source code and the UI resources.
  • cover directory contains the application cover implementations.
  • icons directory contains the additional custom UI icons.
  • pages directory contains the application pages.
  • LlmRunner.qml file provides the application window implementation.
• rpm directory contains the rpm-package build settings.
  • ru.auroraos.LlmRunner.spec file is used by rpmbuild tool.
• src directory contains the C++ source code with llama.cpp integration.
  • types.h contains shared data structures for chats and messages.
  • chatmanager.h and chatmanager.cpp implement chat lifecycle operations (create/select/rename/delete) and active chat state.
  • storagemanager.h and storagemanager.cpp load and persist chat history and system prompt to local storage.
  • modelmanager.h and modelmanager.cpp resolve model paths and initialize llama.cpp model/context/sampler objects.
  • promptbuilder.h and promptbuilder.cpp build prompts from chat history using chat templates with fallback logic.
  • tokenmanager.h and tokenmanager.cpp track token usage, context limits, and context compression logic.
  • generationengine.h and generationengine.cpp run background token generation and stream output back to UI callbacks/signals.
  • llamacontroller.h and llamacontroller.cpp expose the main QML-facing API and coordinate all managers.
  • main.cpp is the application entry point and registers LlamaController in QML context.
• translations directory contains the UI translation files.
• models directory contains bundled GGUF models, including Falcon-H1-Tiny-90M-Instruct.Q8_0.gguf; a custom model location can also be provided via LLAMA_MODEL_PATH.
• ru.auroraos.LlmRunner.desktop file defines the display and parameters for launching the application.

Terms of Use and Participation in Development

The source code of the project is provided under license, which allows to use it in third-party applications.

To participate in the development of the project, please read the member agreement. If you plan to submit your own source code for inclusion in the project, you will need to accept the CLA terms and conditions.

Participant information is listed in the AUTHORS file.

The Code of Conduct is the current set of rules of the Open Mobile Platform Company, which informs about the expectations for interaction between community members when communicating and working on projects.