How we created our server OS: a step-by-step story of NiceOS Z

Today, NiceOS Z is a lightweight server OS without a graphical environment, which can:

• Deploy typical services (web, databases, monitoring, etc.) "out of the box".

• Support national encryption standards and key protection means (GOST, certification).

• Be conveniently installed and updated from its own repositories located in the local network.

We will tell you how we came to this and what we did at each stage.

1. How it all started: goals and initial sketches

1.1. Why create our own OS?

We were working on several large projects for state companies, where one of the requirements was the use of domestic software (or software that can be certified according to GOST). In the process, we encountered the following:

1. Ready-made distributions (Debian, CentOS, Fedora, etc.) are good, but contain a lot of unnecessary things for a typical server installation.

2. Certification under local requirements requires deep control over the system, down to kernel options and the exact version of cryptographic libraries.

3. Vendor support: we needed a platform that we could develop ourselves, without waiting for upstream.

1.2. The first prototype

We assembled a "minimal distribution" based on CentOS, removing graphical interfaces and unused packages. At that time, we were simply creating a kickstart file for automatic installation (Anaconda), so that everything unnecessary would not get into the system. But we quickly realized that we wanted more control — down to packaging the kernel, libraries, and system daemons.

2. Architecture: kernel, packages, patches

2.1. Linux kernel (LTS) and our modifications

Basic idea: we take the "vanilla" LTS branch of the kernel (for example, 6.1 LTS or 5.15 LTS - depending on what was relevant at the time of release). Then:

1. Add patches for GOST encryption. This includes support for the GOST 28147-89 algorithm, GOST R 34.11-2012 hashing, etc. Some kernel patches can be found in public repositories:
   

   • linux-gost-crypto.

   • Sometimes it is necessary to manually adapt them to the required kernel version.

2. Drivers for specific hardware. For example, we had projects that required some old RAID controllers and domestic solutions that do not always get along with the standard kernel. We had to include modules found on GitHub or obtain a driver from the manufacturer (often in the form of .ko for a specific kernel version).

3. Security. We wanted to activate SELinux (or at least AppArmor) to have mandatory access control. In the end, we chose SELinux because it is better integrated with systemd and Red Hat-like environments.

Kernel build
We use the classic combination: make olddefconfig (to transfer settings from the vanilla config), then we make changes: activate the necessary options (for example, CONFIG_CRYPTO_GOST, CONFIG_SECURITY_SELINUX=y, etc.). We have a script build_kernel.sh that:

1. Downloads the kernel sources and our patches.

2. Applies the patches.

3. Runs make menuconfig (or olddefconfig) with the saved .config.

4. Compiles the kernel and modules.

5. Builds an RPM package (rpmbuild) for further installation via DNF.

2.2. Choosing a package manager

We settled on DNF (Fork yum4) with the RPM format, as our team has historically been more closely associated with the Red Hat environment than with Debian. But we also had tests with APT/deb. In general, the algorithm:

1. Specification files (SPEC): we write them for each package, specifying dependencies, build and installation instructions.

2. Own repository: we set up a local HTTP server (Nginx), generate metadata with the command createrepo_c /path/to/repo.

3. DNF configuration: in /etc/dnf/dnf.conf we specify the addresses of our repos.

For simple packages (like zabbix-agent or nginx), we take the source code, patch it if necessary (for example, adding GOST support in OpenSSL), and then create an RPM.

Example fragment of .spec for building Nginx with GOST patches (simplified):

Name:           nginx
Version:        1.22.1
Release:        1%{?dist}
Summary:        High-performance HTTP server and reverse proxy
License:        BSD
Source0:        https://nginx.org/download/nginx-%{version}.tar.gz
Patch0:         nginx-gost.patch

BuildRequires:  openssl-gost-devel
...

%description
Nginx with GOST crypto support for Russian standards...

%prep
%setup -q -n nginx-%{version}
%patch0 -p1

%build
./configure --with-openssl=/usr/src/openssl-gost ... 
make

%install
make install DESTDIR=%{buildroot}

%files
/usr/local/nginx
...

%changelog
* Wed Dec 20 2024 NiceOS Team - 1.22.1-1
- Initial build with GOST support

Of course, real .spec files can be much larger, with scriptlets and additional options.

2.3. "Minimal" base system

To ensure the server OS does not carry gigabytes of unnecessary packages, we have formed a "minimal" meta-package (niceos-minimal), which only pulls in:

• bash, coreutils, systemd, dnf, iproute2, openssh-server, tar, xz, vim-minimal, etc.

• Essential cryptographic libraries (considering GOST).

• SELinux (for security).

Everything else is installed optionally: there are meta-packages niceos-webserver, niceos-postgres, niceos-monitoring, etc. They pull in dependencies for the corresponding role.

Thus, when installing from ISO or netinstall, the user can choose: "I need a web server" — niceos-minimal + Nginx, PHP-FPM, firewall, etc. If only basic SSH access is needed, niceos-minimal is sufficient.

3. Installer and autoconfiguration

3.1. Fork of Photon Installer

In the early stages, we repurposed the Photon Installer for our needs. It is a simple installer. Nothing extra for the server.

1. Kickstart files: we specified everything needed: package list, partition creation, network configuration.

2. Post-install step: we install GOST certificates, configure locales, and SELinux policy.

3.2. Netinstall image

For convenience, we have also collected a netinstall-ISO, which weighs ~300 MB. It contains only the basic installer and several scripts. During installation, it asks for the repository address (our local server), downloads the packages, and installs the system. This way, we save space and can quickly update the repository.

4. GOST encryption and certification

4.1. Integration of GOST in OpenSSL

To make OpenSSL "understand" Russian ciphers, we use patches from gost-engine. In the .spec file for OpenSSL, we add:

Patch999: openssl-gost-engine.patch
...

After building, the system installs libcrypto.so with GOST support.

Verification:

openssl ciphers | grep -i gost

If we see a list of GOST algorithms, everything is fine.

4.2. Certification by FSTEC/FSB

If the OS will be used in government organizations or critical information infrastructure (CII) facilities, it often needs to comply with regulations:

• FSTEC: certification for the absence of undeclared capabilities (UDC). It is necessary to provide experts with the source code, a description of the build process, and test results.

• FSB: if the system processes state secrets or contains encryption that is subject to control. Relevant licenses and expertise are required.

We prepare part of the documentation in the form of packages:

• niceos-security (which describes all SELinux policies, logging, kernel configuration).

• niceos-crypto (detailed description of cryptographic modules, certificates).

This is a large bureaucratic task: tests, audits, inspections. At the development stage, it is useful to keep in mind: "what logs we write, how we form the SELinux policy, where encryption keys are stored." The better the structure, the easier it is to certify later.

5. Build and testing infrastructure

5.1. GitLab CI/CD

We have an internal GitLab server, where:

1. We store all the code: the kernel (with patches), .spec files, scripts, auxiliary utilities.

2. Pipeline with multiple stages:
   

   • build_kernel: download vanilla sources, patch, build RPM.

   • build_base: build base packages (glibc, bash, systemd, etc.).

   • build_services: for example, nginx, openssl, all necessary daemons.

   • assemble_iso: generate an ISO image (or netinstall image) using lorax or livecd-tools.

   • test_virtual: deploy the freshly built ISO in QEMU/KVM, check that the system installs, SSH works, SELinux does not block important processes.

5.2. Automated tests

We wrote a basic "smoke-test" in Python that performs:

1. Installation: runs QEMU with the ISO, goes through the automated installation scenario (kickstart).

2. Verification:
   

   • The system booted, SELinux is enabled, the sshd service is active.

   • Locales ru_RU.UTF-8 and en_US.UTF-8 are available.

   • openssl ciphers | grep gost returns the required algorithms.

Only if all checks are "green" is the built image published to our repository.

6. Pilot projects and real operation

6.1. Case: web server for a government structure

The first real case: we deployed NiceOS as a web server for a government institution:

• Installed via netinstall image.

• Set up Nginx with GOST support for HTTPS, PostgreSQL for data storage.

• Used SELinux with a strict policy allowing only necessary ports and processes.

• Conducted an audit together with the client's security service, formalized all documents.

Result: the web server passed the check, the system performed stably under load testing.

6.2. Internal monitoring services

At the same time, we implemented NiceOS for internal CI/CD pipelines and monitoring (Zabbix, Prometheus). This helped to catch bugs faster and "live" check compatibility with popular tools (Docker, Kubernetes).

7. Documentation and user support

7.1. Documentation

We immediately try to maintain documentation so that it can be read not only by the "internal" developer but also by the external system administrator:

• Wiki based on GitLab Pages.

• Description of the entire process: installation, SELinux configuration, working with locales, enabling GOST encryption.

• Section "frequent issues": if any utility requires additional configuration for Russian standards, we provide detailed descriptions.

7.2. Support Channels

• Telegram chat: to promptly respond to questions from colleagues, testers, and potential users.

• GitLab Issues: bugs, feature requests.

• For serious clients (government institutions, businesses), we have organized a ticketing system and SLA, but that's more about business processes.

8. Conclusions and Future Plans

During the development of NiceOS, we have realized that creating (and even more so maintaining) your own server Linux system is a challenging task. It is important to lay down the following from the start:

1. Automation (CI/CD, tests, repositories) — otherwise, you will quickly get bogged down in manual assembly.

2. Strict selection of packages — the server OS should not be overloaded.

3. Consideration of local requirements (GOST, locales, certificates) from the very beginning.

Future steps:

• Add "automagic" for configuring typical roles (web, database, virtualization).

• Expand the monitoring system for installations (to centrally see the version, installed packages, SELinux status).

• Officially complete the certification process for certain industries and government structures (FSTEC/FSB).

We do not rule out that we will later create a desktop version, but for now, the priority is the server segment, where there is indeed demand for local solutions.

Case: step-by-step assembly of NiceOS — from a clean kernel to a custom distribution

Step 1. Define goals and assemble the "skeleton" of the system

1. Defining requirements:
   

   • Server OS without a graphical environment,

   • Minimum set of programs and libraries (basic utilities, network tools, SSH),

   • Support for Russian cryptography standards (GOST),

   • Ability for a "clean" (or semi-automatic) installation,

   • RPM packages + DNF as the package manager.

2. Choosing the kernel:
   

   • Start with an LTS kernel (for example, 6.1 LTS) in the form of vanilla sources: download the tarball from kernel.org.

   • Immediately prepare a set of patches, including support for GOST cryptography, necessary drivers (if there is exotic hardware), and security options (SELinux).

3. Creating a basic rootfs manually:
   

   • Create a "sandbox" (usually via chroot on a virtual machine or in a directory), where we place everything needed: bash, coreutils, iproute2, systemd, openssh.

   • The first version of rootfs can simply be "assembled" from binaries from some "donor" repository (for example, CentOS/ALT/SUSE) or compiled by ourselves. We chose a middle path: some packages were taken from the CentOS repository, while others (for example, OpenSSL with GOST patches) were built manually.

What is important: at this stage, we do not have a full installer and normal packages. This is just a "minimal folder with a system" that can be packed into initrd and loaded via QEMU.

Step 2. Configuring the kernel build and security foundation

1. Kernel compilation:
   

   • Unpack linux-6.x.y.tar.xz, apply our patches (with the command patch -p1 < gost.patch, etc.).

   • Run make menuconfig or copy the default .config (for example, from the Red Hat distribution), enable:
     

     • CONFIG_CRYPTO_GOST (or a similar name),

     • CONFIG_SECURITY_SELINUX=y,

     • CONFIG_DEFAULT_SELinux,

     • drivers for the required hardware.

   • make -j8 bzImage modules && make modules_install. We get the ready kernel and modules.

2. SELinux policies:
   

   • First, we decide that SELinux will operate in Enforcing mode. This means we need to enable the corresponding packages and base policies (for example, the selinux-policy-base package).

   • In the future, we will have to refine our modules for services, but for now, we take the standard ones.

3. OpenSSL with GOST:
   

   • Take the official openssl (version 1.1.1 or 3.x — depending on the time),

   • Mix in gost-engine. There are usually patches like openssl-gost.patch for this.

   • Compile, check openssl ciphers | grep GOST, make sure the algorithms are visible.

Result: there is a minimal set of components (kernel, libc, bash, systemd, SELinux, openssl-gost), and all of this is packed into our chroot/rootfs.

Step 3. "Organizing" the build: RPM + DNF

Now we want not just to have compiled binaries, but to turn them into regular RPM packages, so that any user can install or update the system via DNF.

1. Organize project structure:
   

   • Git repository (e.g., GitLab) with folders:
     

     • kernel/ (kernel sources, patches, and .spec for building),

     • base/ (glibc, bash, coreutils, systemd, etc.),

     • crypto/ (openssl-gost, libgcrypt, if needed),

     • selinux/ (rules, policies).

2. Create .spec files for each package:
   

   • For example, kernel.spec, where we specify where to get the sources, which patches to apply, how to compile, and where to install.

   • In the %post section, indicate that grub.cfg or initramfs needs to be updated.

3. Build RPM:
   

   • Either manually via rpmbuild -ba kernel.spec or automate it in CI (GitLab CI).

   • After a successful build, we get a .rpm for installation (e.g., kernel-niceos-6.1.XX.rpm).

4. Set up your repository:
   

   • Folder /var/www/html/niceos/ on the local web server,

   • Command createrepo_c /var/www/html/niceos/.

   • Now any server where baseurl=http://repo.niceos.local/niceos/ is specified can install these packages via dnf install kernel-niceos.

5. DNF and minimal meta-package:
   

   • Build the package niceos-minimal, which in Requires: lists everything needed for the base system: kernel-niceos, systemd, selinux-policy-base, bash, iproute2, openssh-server, etc.

   • Thus, the "homonymous" meta-package pulls in all basic dependencies.

Step 4. Generating the installation ISO

To prevent ordinary people from building the system manually, an installation image (ISO) is needed:

1. Tools: you can use lorax (used in Fedora/Red Hat) or livecd-tools.

2. Scripts:
   

   • We collect a minimal rootfs from our RPMs (via dnf --installroot=/folder ...).

   • We generate initrd and vmlinuz (take from our kernel-niceos).

   • We package everything into an ISO, adding a boot system (GRUB or ISOLinux).

3. Kickstart (optional):
   

   • We create ks.cfg, where we list:
     

     • %packages (contents, including niceos-minimal),

     • %post (additional scripts — for example, enabling SELinux in Enforcing mode, creating user admin, setting locales).

   • Later, during installation, Anaconda or another installer will read these instructions, and everything will proceed automatically.

Result: an "official" NiceOS build in the form of an ISO image has appeared. It can be written to a flash drive and installed on a server just like any other Linux system.

Step 5. Testing in QEMU and on real hardware

1. Automated tests (CI):
   

   • In GitLab (or Jenkins), we organize a pipeline:
     

     • After building all packages and generating the ISO, we launch QEMU, load the ISO, and automatically answer the installer’s questions (kickstart).

     • We wait for the installation to complete, check that the system has booted, SELinux is enabled, SSH is working, and the Russian locale ru_RU.UTF-8 is present.

   • We conduct a smoke test: openssl ciphers | grep GOST, check dnf install zabbix-agent, etc.

2. Test on a real server:
   

   • We take some "hardware" stand (or even a regular PC).

   • We install NiceOS from the flash drive. We check if all drivers (RAID controller, network adapters) are working.

   • Sometimes we have to modify initramfs scripts (for example, if it does not see LVM volumes during boot).

Step 6. Tuning SELinux, cryptconfig, and other nuances

At the stage of real use, small details usually come to light:

1. SELinux: sometimes services are blocked by the "wrong" policy. It is necessary to build custom .te modules. For example, if we want Nginx to access a non-standard folder /srv/data, we need to specify the contexts (semanage fcontext) and permissions.

2. GOST extension: in addition to OpenSSL, GnuTLS, libcrypto++ may be required if the client's software uses them. This means we need to similarly patch and build packages with GOST support.

3. Diagnostic utilities (dmesg, journald, systemd-analyze). It is important that everything is in place and localized.

Step 7. Documentation and regular updates

1. Documentation:
   

   • We write a Wiki (or ReadTheDocs) with screenshots on how to install NiceOS from ISO, how to configure SSH, SELinux, GOST, networks, and so on.

   • We add a "frequent issues" section: why SELinux does not allow the service to start, how to bind guest certificates, where the logs are located.

2. Regular updates:
   

   • Periodically, we move to new kernel versions (LTS), perform a rebase.

   • We build security packages (for example, if a vulnerability is found in OpenSSL).

   • We publish them in the repository, and the system (via dnf update) can update on the fly.

Result

As a result, we have a fully controlled distribution called NiceOS Z:

• Our patched kernel,

• System packages in RPM format,

• A repository on our server,

• An installation ISO with an installer and kickstart,

• A CI pipeline that automatically builds and tests all changes,

• SELinux, GOST cryptography, and Russian locale "out of the box".

This distribution is convenient to maintain for internal needs (or for clients) because we are responsible for each package and know exactly where everything is located. Yes, it is much more complicated than "just taking CentOS", but it provides complete independence and control.

What's the "catch" and what we understood

• Automation is the key to success. Without CI and repositories, everything will fall into chaos.

• SELinux is better to enable from the start than to activate later and catch hundreds of errors.

• GOST patches do not always merge smoothly — sometimes you have to manually adapt them to a specific version of OpenSSL or the kernel.

• Documentation: we spent a lot of time detailing each step. Without this, any new team member or administrator starts drowning in questions.

• Constant support: each new vulnerability, each new version — this is the work of assembling updated packages, checking that nothing is broken.

We hope our step-by-step case will be useful for those who are planning to build their own Linux distribution (or seriously modify an existing one). If you have any questions, feel free to ask, we would be happy to share our experience even further!

Conclusion

We hope our experience has been useful: we tried not just to list the steps, but to show how exactly we did them — from the kernel and package manager to CI/CD and tests. Creating your own server OS makes sense if:

• You need deep compliance with local standards and the possibility of certification.

• It is important to have full control over the lifecycle (when, how, and what to update).

• You have a team ready to handle building, updates, and technical support.

At the same time, we never tire of repeating: if your project does not require such specific conditions, it may be easier to use an already ready distribution. But when Russian specificity and certification are necessary, a custom OS (even if based on Linux) becomes an excellent solution.

Thank you for reading to the end!
If you have similar experience or questions, we would be happy to discuss them in the comments.