Insecurity of Debian

However, many in the open-source community perceived Red Hat's decision as a blatantly underhanded move.

More and more people are declaring that they will switch (or have already switched) to Debian, seeking refuge from what they consider greedy corporate influence. I completely understand this sentiment. However, there is a problem I want to talk about: security.

The bitter truth is that ensuring security is a complex task. It is tedious, unpleasant, and requires a lot of work to get it right.

Debian does not do enough to protect its users.

Red Hat has long implemented the use of SELinux (Security-Enhanced Linux). And they went beyond simply enabling this feature in their kernel. They did painstaking work to create default SELinux policies for their distribution.

These policies come enabled by default in their distribution. The policies help protect various daemons running by default in RHEL, as well as many of the most popular daemons that are commonly used on servers.

Apache, nginx, MariaDB, PostgreSQL, OpenSSH, etc. — all of them are covered by SELinux policies shipped in RHEL distributions.

Protection extends even to containers. Containers are becoming an increasingly preferred method of software deployment for developers, including myself. A common misconception is that if you run something in a container, it is inherently secure. This is completely untrue. Containers themselves do not solve the security problem. They solve the software distribution problem. They create a false sense of security for those who use them.

In Red Hat-based distributions, you can use podman, an alternative to Docker, which allows you to run containers without a daemon (unlike Docker) and provides other advantages, such as the ability to run completely rootless. But Red Hat goes even further and applies strict SELinux policies by default, which separate the container from the host OS and even from other containers!

There have been many examples of the possibility of escaping from a container and accessing the host OS or other containers. This is where tools like SELinux come into play. Applying SELinux policies to a container allows you to create a fortified "sarcophagus" for your application, which reduces the risk of unknown future exploits. And this requires almost no effort in RHEL.

Red Hat understood that if they didn't do the work on these default policies, their users simply wouldn't use this technology, and millions of servers would remain vulnerable. Because, let's be realistic, SELinux is a complicated thing. The policy language and tools are cumbersome, unintuitive, and about as appealing as filling out tax returns. Frankly, it's terrible to use—if you're manually creating your own policies.

But thanks to the work done by Red Hat, using SELinux in RHEL is mostly transparent and provides real security benefits to its users.

Debian Approach

Debian, the stronghold of the open-source community, is revered for its stability and extensive software library. I am a fan and donate to the project annually (you should too!), although I don't use it in a production environment.

Nevertheless, its default security system leaves much to be desired. Debian's decision to include AppArmor by default starting with version 10 marks a positive step towards enhancing security, but it falls short due to incomplete implementation in the system.

Debian's reliance on AppArmor and its default configurations reveals a systemic issue with its security approach. While AppArmor can provide robust security when properly configured, Debian's out-of-the-box settings do not utilize its full potential:

• Limited default profiles: Debian comes with a minimal set of AppArmor profiles, leaving many critical services unprotected.

• Reactive rather than proactive stance: Debian's security model often relies on users to implement stricter policies, rather than providing a secure default configuration.

• Inconsistent application: Unlike SELinux in Red Hat systems, which is applied uniformly across the system, AppArmor in Debian is applied fragmentarily, leading to potential security gaps.

• Lack of resources: Debian, as a community project, does not have the resources to develop and maintain comprehensive security policies comparable to those provided by Red Hat.

Very often, users run container workloads on Debian through Docker, which automatically generates and loads a default AppArmor profile for containers named docker-default. Unfortunately, this is not a very reliable profile from a security standpoint, as it is overly permissive.

This profile, while providing some protection, leaves significant attack surfaces open. For example:

  network,
  capability,
  file,
  umount,
  # Host (privileged) processes may send signals to container processes.
  signal (receive) peer=unconfined,
  # runc may send signals to container processes (for "docker stop").
  signal (receive) peer=runc,
  # crun may send signals to container processes (for "docker stop" when used with crun OCI runtime).
  signal (receive) peer=crun,
  # dockerd may send signals to container processes (for "docker kill").
  signal (receive) peer={{.DaemonProfile}},
  # Container processes may send signals amongst themselves.
  signal (send,receive) peer={{.Name}},
  deny @{PROC}/* w,   # deny write for all files directly in /proc (not in a subdir)
  # deny write to files not in /proc// or /proc/sys/
  deny @{PROC}/{[^1-9],[^1-9][^0-9],[^1-9s][^0-9y][^0-9s],[^1-9][^0-9][^0-9][^0-9/]*}/** w,
  deny @{PROC}/sys/[^k]** w,  # deny /proc/sys except /proc/sys/k* (effectively /proc/sys/kernel)
  deny @{PROC}/sys/kernel/{?,??,[^s][^h][^m]**} w,  # deny everything except shm* in /proc/sys/kernel/
  deny @{PROC}/sysrq-trigger rwklx,
  deny @{PROC}/kcore rwklx,
  deny mount,
  deny /sys/[^f]*/** wklx,
  deny /sys/f[^s]*/** wklx,
  deny /sys/fs/[^c]*/** wklx,
  deny /sys/fs/c[^g]*/** wklx,
  deny /sys/fs/cg[^r]*/** wklx,
  deny /sys/firmware/** rwklx,
  deny /sys/devices/virtual/powercap/** rwklx,
  deny /sys/kernel/security/** rwklx,

The network rule allows all network system calls without restrictions.

The capability rule, without specific prohibitions, allows most capabilities by default.

The file rule provides broad file access rights, relying on specific denial rules for protection.

AppArmor vs SELinux

The fundamental difference between AppArmor and SELinux lies in their approach to MAC (Mandatory Access Control). AppArmor operates based on a path-based model, while SELinux uses a significantly more complex type-based mandatory access control system. This difference becomes particularly evident in container environments.

SELinux applies a type to each object in the system - files, processes, ports, anything. When you run a container on an RHEL system with SELinux support, it is immediately assigned the type container_t - a strict access control mechanism. The type container_t effectively isolates the container, preventing it from interacting with any object not explicitly marked for container use.

But SELinux is not limited to mandatory type control. It takes container isolation to a new level with MCS (Multi-Category Security) labels. These labels function as an additional layer of separation, ensuring that even containers of the same type (container_t) remain isolated from each other. Each container receives its own unique MCS label, creating what is essentially a private "sandbox" within the broader container_t environment.

AppArmor, on the other hand, does not care about types or categories. It focuses on restricting the capabilities of specific programs based on predefined profiles. These profiles specify which files a program can access and what operations it can perform. While this approach is simpler to implement and understand, it lacks the granularity and systemic consistency of SELinux's mandatory type control. Almost none of the major Linux distributions ship comprehensive AppArmor profiles for all common network daemons by default.

The practical implications of these differences are quite significant. In an SELinux environment, a compromised container faces significant obstacles when accessing the host system or other containers or impacting them, thanks to the dual barriers of mandatory type enforcement and MCS labels.

This does not mean that one is universally better than the other. SELinux offers more robust isolation, but at the cost of increased complexity and potential performance overhead. AppArmor provides a simpler, more accessible security model that can still be quite effective when properly configured. The essence of my argument is that Red Hat has done the work to make using SELinux and containers seamless and easy for its users. You are not left to figure it out on your own.

Ultimately, the choice between Debian and Red Hat is not just a choice between corporate influence and community-driven development. It is also a choice between a system that assumes the best and a system that prepares for the worst. Unfortunately, in today's interconnected world, pessimism is necessary.