2024 LiveCD Memory Usage Compare

I am using pretty much the exact same setup I did in 2020. Let's see who is more efficient in a live session!

But first let's take a look at the image sizes:

Image size (in G) 001122334455UbuntuXubuntuXubuntu-minimalKubuntuLubuntuUbuntu MateManjaro 24.1 (KDE)Linux Mint 22 (Cinnamon)Fedora 40 (Gnome)Endless OS 65.840.565286906228884237.3745496805519Ubuntu3.998.51569677227016312.14438462634905Xubuntu2.5156.46610663831143367.2379472179891Xubuntu-minimal4.1214.4165165043527304.27387568468623Kubuntu3.1272.36692637039397343.62642039300044Lubuntu4330.31733623643527308.20913015551764Ubuntu Mate3.9388.2677461024765312.14438462634905Manjaro 24.1 (KDE)2.8446.21815596851775355.4321838054948Linux Mint 22 (Cinnamon)2.2504.1685658345591379.04371063048336Fedora 40 (Gnome)3.9562.1189757006003312.14438462634905Endless OS 6Image size (in G)

Charge Open Movie is what I viewed if I can make it to YouTube.

I decided to be more selective and remove those that did very porly at 1.5G, which was most.

  • Ubuntu - booted but desktop not stable, took 1.5 minutes to load Firefox
  • Xubuntu-minimal - does not include a web browser so can't further test. Snap is preinstaled even though no apps are - but trying to install a web browser worked but couldn't start.
  • Manjaro KDE - Desktop loads, but browser doesn't
  • Xubuntu - laggy when Firefox is opened, can't load sites
  • Ubuntu Mate -laggy when Firefox is opened, can't load sites
  • Kubuntu - laggy when Firefox is opened, can't load sites
  • Linux Mint 22 - desktop loads, browsers isn't responsive

Memory usage compared (in G) 000.10.10.20.20.30.30.40.40.50.50.60.60.70.70.80.80.90.9111.11.11.21.21.31.31.41.4LubuntuEndless OS 6.0Fedora 400.4557.52699314991314372.0296569207792Lubuntu1273.2532174620874286.9854710078829Endless OS 6.00.7488.97944177426166333.3732087785536Fedora 400.9120.8066856148176302.4480502647731Lubuntu1336.5329099269918286.9854710078829Endless OS 6.01.1552.2591342391661271.5228917509926Fedora 401.1184.086378079722271.5228917509926Lubuntu1.3399.81260239189635240.5977332372121Endless OS 6.01.4615.5388267040705225.13515398032192Fedora 40Memory usage compared (in G)Desktop responsiveWeb browser loads simple siteYouTube worked fullscreen

Fedora video is a bit laggy, but watchable.. EndlessOS with Chromium is the most smooth and resonsive watching YouTube.

For fun let's look at startup time with 2GB (with me hitting buttons as needed to open a folder)

Startup time (Seconds) 00101020203030404050506060707080809090LubuntuEndless OS 6.0Fedora 4033107.38104458917655401.2549765487598Lubuntu93299.13290992699183247.63515398032195Endless OS 6.045490.8847752648071370.53101203507225Fedora 40Startup time (Seconds)Seconds

Conclusion

  • Lubuntu lowered it's memory usage from 2020 for loading a desktop 585M to 450M! Kudos to Lubuntu team!
  • Both Fedora and Endless desktops worked in lower memory then 2020 too!
  • Lubuntu, Fedora and Endless all used Zram.
  • Chromium has definitely improved it's memory usage as last time Endless got dinged for using it. Now it appears to work better then Firefox.

Notes:

  • qemu-system-x86_64 -enable-kvm -cdrom lubuntu-24.04.1-desktop-amd64.iso -m 1.5G -smp 4 -cpu host -vga virtio --full-screen
  • Screen size was set to 1080p/60Hz.
  • I tried to reproduce 585M on Lubuntu 20.04 build, but it failed on anything below 1G.
  • Getting out of full screen on YouTube apparently is an intensive task. Dropped testing that.
  • All Ubuntu was 24.04.1 LTS.

Lubuntu Memory Usage and Rsyslog

In 2020 I reviewed LiveCD memory usage.

I was hoping to review either Wayland only or immutable only (think ostree/flatpak/snaps etc) but for various reasons on my setup it would just be a Gnome compare and that's just not as interesting. There are just to many distros/variants for me to do a full followup.

Lubuntu has previously always been the winner, so let's just see how Lubuntu 23.10 is doing today.

Previously in 2020 Lubuntu needed to get to 585 MB to be able to run something with a livecd. With a fresh install today Lubuntu can still launch Qterminal with just 540 MB of RAM (not apples to apples, but still)! And that's without Zram that it had last time.

I decided to try removing some parts of the base system to see the cost of each component (with 10MB accuracy). I disabled networking to try and make it a fairer compare.

  • Snapd - 30 MiB
  • Printing - cups foomatic - 10 MiB
  • rsyslog/crons - 10 MiB

Rsyslog impact

Out of the 3 above it's felt more like with rsyslog (and cron) are redundant in modern Linux with systemd. So I tried hitting the log system to see if we could get a slowdown, by every .1 seconds having a service echo lots of gibberish.

After an hour of uptime, this is how much space was used:

  • syslog 575M
  • journal at 1008M

CPU Usage on fresh boot after:

With Rsyslog

  • gibberish service was at 1% CPU usage
  • rsyslog was at 2-3%
  • journal was at ~4%

Without Rsyslog

  • gibberish service was at 1% CPU usage
  • journal was at 1-3%

That's a pretty extreme case, but does show some impact of rsyslog, which in most desktop settings is redundant anyway.

Testing notes:

  • 2 CPUs (Copy host config)
  • Lubuntu 23.10 install
  • no swap file
  • ext4, no encryption
  • login automatically
  • Used Virt-manager and only default change was enabling EUFI

Email list to Ghost site - PastaDollar.com

My brother, Nick, just launched a new website/newsletter using Ghost. Ghost is pretty unique in this space, it is:

  • Open Source
  • Backed by a Non-Profit
  • A web publishing platform
  • A newsletter publishing platform
  • A CMS for static sites
  • A member subscription / Patreon replacement

Here is Nick's experience with Ghost so far...

Pricing and Publishers

From a pricing perspective, the platform scales with your audience size. Importantly, Ghost does not take a cut of your membership subscription revenue. For this reason, the pricing incentives mean your Ghost site is more cost-effective when you have 100% paid members and 0% free members. I'm sure that ratio is a pipe dream for most publications, but that's how the business model works. Pricing is currently $9/mo for any site with 500 or fewer subscribers, including web hosting. For context, the new blog that I started in 2023 using Ghost is www.pastadollar.com. The site was originally an email newsletter from my generation of 14 cousins investing together. If you're looking for finance tips and travel hacks, Subscribe here! An email-list-to-ghost-site is a common path for Ghost users. I'm just starting, but major publications like The Atlantic, The Lever, and The Browser use Ghost.

Positives

Ghost has only a handful of themes, but on the whole, they are super clean and approachable. Picking one is as much about your content as it is about which one you list best. All of the themes are responsive and have all the standard customization options like colors and logos. Notably, there is no simple way to use custom fonts. Instead, you can choose between a serif font and a sans-serif font. There is a workaround, but the need for a long list of fonts is a relatively notable omission from a CMS or theme these days. The Ghost editor has a learning curve, but on the whole, it is excellent. I'm used to WordPress's editors, which are total crap by comparison. There are several Ghost editor keyboard shortcuts worth looking up, such as Shift+Enter to jump to the following line but to remain in the same editor block. Hitting Enter will jump to a new block, skipping more lines. The editor auto-saves your work as you go, just as you would expect any web-based tool to do these days. The saving experience is much like google docs - automatic and out of your way. However, sometimes when I haven't changed anything, a modal pops up asking me if I'm certain I want to navigate away without saving—maybe just a bug, but a bit annoying. Many tasks that take multiple steps in a WordPress site are automatic in Ghost. For example, when you add an image to a post or page, Ghost will optimize it. There is no more need to use a plugin or a site like tinypng.com to optimize the image in advance because the platform doesn't do it for you. Ghost was designed with plenty of best practices. For example, the default post URL is the post's title, without any year or dates included. Including dates is possible on other platforms but can quickly cause 404s and redirection issues if/when you update a post. The Ghost developers took the best practice, implemented it as default, and left it to you to simply create the content. It's great.

Negatives

Plugins

Ghost has a great library of plugins, which they accurately call 'integrations.' The catch is that most of these are integrations with other major companies' SaaS products, meaning many of them are paid solutions. The company Ghost seems to rely on the most to expand its functionality is Zapier. Half of the help and support articles I've read on Ghost seem to using Zapier to solve a problem. I don't think it's a stretch to say that Ghost relies heavily on Zapier, a freemium SaaS product. So basically, once you set it up and grow your site, it will eat into your bottom line. That isn't necessarily bad, provided it's worth it. But it seems like the solutions to expand your Ghost site are mostly paid solutions rather than something custom via code injection, which Ghost also supports. To be fair, I need to explore code injection solutions more than I have as of this writing.

Quality of Life Issues

URLs are a big part of any site, so I'm amazed this is an issue: When creating a hyperlink, there isn't proper data validation on the link. If you omit the 'https://www' from a new hyperlink, the Ghost editor saves it, but the URL will not actually work. Saving a link like "pastadollar.com" will show as a broken link and not work when you publish the post. This behavior is incredibly annoying because when you're in your site admin, many URLs around the dashboard omit the https://. To ensure your internal URLs will work, I've solved this by always copying them from the live site. Bullets and Numbered lists aren't an easy option I've found when using the editor. I finally found them hidden in an editor block called markdown editor card. The markdown block supports some rich text features that literally anyone who uses a computer is accustomed to. It isn't exactly user-friendly to hide them in a block, but I'm sure there was a good reason for doing so. Another simple workaround I used when I started was to simply copy and paste a bulleted or numbered list from a proper text editor. I'm guessing many writers create whole drafts outside the Ghost editor anyway.

is is saved though

Alternatives

Alternativeto.net lists more than 250+ alternatives to Ghost, but that long list includes broader solutions like WordPress. Like probably anyone who has developed a website in their lives, I've used WordPress (and continue to use it to this day). WordPress, by default, needs a lot of help from its vast plugin library to set it up, similar to how Ghost works out of the box. I don't consider WordPress a proper alternative if your goal is primarily an email newsletter and membership site. Ghost is absolutely worth the cost over WordPress in that regard - it's not even close.

Beehiiv

In my opinion, the most similar and direct competitor to Ghost is a platform called Beehiiv. Beehiiv has less favorable pricing, and the themes and designs are uglier, but it boasts a stronger feature set out of the box. For example, two big ones are subscriber referral rewards for sharing with friends and more customizable email templates. To be clear, you can probably add these features to a Ghost site; it just takes some work. Both Ghost and Beehiiv are designed to scale quickly with your audience size.

Conclusion

Ghost support has been both highly responsive and extremely helpful. For example, I reached out about getting custom domain emails, such as [email protected], and their guidance was perfect. Their < 24-hour replies led me to many paid and free services and cautioned me on a few 'gotchas' others ran into. These were not canned responses, and I feel like I can go back to them for anything, and we will be able to find a solution. Overall, I love Ghost. The admin interface is spotless and fast. I'm coming primarily from WordPress, and the speed difference is a game-changer. I would work from many tabs simultaneously in WordPress because switching between different interface pages took too many seconds to load. The Ghost admin pages usually take less than a second to load.

Comments

Add a comment via Gitlab. And yes, Ghost has much nicer integrated commentng built-in.

Small EInk Phone

Aside in 2022-05-22. it's not the same.. but there is a renewed push by Pebble creator Eric Migicovsky to show demand for a SmallAndroidPhone. It's currently at about 29,000.

Update 2022-02-26: Only got 12 responses which likely means there isn't that much demand for this product at this time (or it wasn't interesting enough to spread). Here are the results as promised:

What's the most you would be willing to spend on this? 7 - $200, 4 - $400. But that doesn't quite capture it. Some wanted even cheaper than $200 (which isn't doable) and others were will to spend a lot more.

Of the priority's that got at least 2 people agreeing (ignoring rating): 4 - Openness of components, Software Investments 3 - Better Modem, Headphone Jack, Cheaper Price 2 - Convergence Capable, Color eInk, Replaceable Battery

I'd guess about half of the respondents would likely be happy with a PinePhone (Pro) that got better battery life and "Just Works".

End Update.

Would you be interested in crowdfunding a small E Ink Open Phone? If yes, check out the specs and fill out the form below.

If I get 1000 interested people, I'll approach manufacturers. I plan to share the results publicly in either case. I will never share your information with manufacturers but contact you by email if this goes forward.

Basics:

  • Small sized for 2021 (somewhere between 4.5 - 5.2 inches)
  • E Ink screen (Maybe Color) - battery life over playing videos/games
  • To be shipped with one of the main Linux phone OSes (Manjaro with KDE Plasma, etc).
  • Low to moderate hardware specs
  • Likely >6 months from purchase to getting device

Minimum goal specs (we might be able to do much better than these, but again might not):

  • 4 Core
  • 32 GB Storage
  • USB Type-C (Not necessarily display out capable)
  • ~8 MP Front camera
  • GPS
  • GSM Modem (US)

Software Goals:

  • Only open source apps pre-installed
  • MMS/SMS
  • Phone calls
  • View websites / webapps including at least 1 rideshare/taxi service working (may not be official)
  • 2 day battery life (during "normal" usage)

Discussions: Phoronix

Linux Gaming in 2022

Quick thoughts

I'd bet the Steam Deck (and other changes) will have the following impacts on Linux overall by the end of 2022.

  1. The majority of Linux users will run Wayland over X11.
  2. Valve’s Steamdeck is going to double the number of Linux gamers per Valve's Hardware Survey.
  3. Flatpak/Flathub will ride the Deck wave - usage will double.
  4. Distros will ride the Deck wave - gaming usage will increase by about 20% (1% -> 1.20%).

1 Majority Wayland

Currently we are at less than 10% running wayland per Firefox telemetry stats on Phoronix but there are a lot of movers, namely:

  1. Ubuntu 22.04 LTS will be the first Ubuntu LTS release defaulting to Wayland.
  2. Nvidia drivers explicitly improving Wayland support.
  3. Although this is still a big list, KDE wayland support has been getting a lot of improvements recently and it's the default on some installs.
  4. Steam Deck will be using Wayland.

2 Double Steam Linux users

Right now for January 2022 1.06% of Steam users are running Linux. I estimate about 340-460k Steam Linux users (Valve published Flatpak installs for November at about 5%. There are approximately 17000-23000 users for each update).

We don't have current numbers for Steam deck reservations, but near launch it was > 100k. Selling 300k-500k seems quite within the realm of possibilities. I would also not be surprised if they sold more.

3 Flathub usage doubles

Download on Flathub not a link

Flatpak is the easiest way to install non-Steam software on a Steam deck - "Yes. You'll be able to install external apps via Flatpak or other software without going into developer mode" - Steam Deck FAQ

The key items I see at first would be MineCraft and the large collection of gaming emulators. It would also be the obvious choice if another studio wanted to bring a game to the Deck.

Valve has avoided picking sides regarding Flatpak vs Snap vs AppImage so far. They still offer a deb from their own download page. Given Steam's user base just making the Flatpak the default would likely more than double Flatpak usage.

There are more wildcards here:

  • How actually easy will flatpak be?
  • How hard are other options - snaps (requires dev mode?), AppImage (seems like it might work fine), etc?

4 Distros ride the wave

I'm expecting at least 20% increase on the Steam Hardware Survey (so 1% to 1.20%) not including Steam Deck. Right now, the Steam Linux usage is less than half what you get from other sources. That could mean multiple things:

  • Linux is less likely to be used by gamers
  • Linux users prefer playing awesome open source games (or otherwise don't use Steam)
  • Users switch to game on other platforms. Many users might take a fresh look at Steam on their existing Linux boxes with all the press from the Steam Deck.

All are likely true to some extent.

Linux distros have many options to further ride the wave:

  • Encourage Linux consumer focused pre-installs with similar AMD chips to what's in the Steam Deck (I know many have asked for more AMD preinstalls for a long while)
  • Enable Flathub/flatpak by default for easier finding apps (Steam itself is not discoverable on Ubuntu unless you enable flatpak. If flatpak takes off more, this will be quite essential)
  • Gaming on Linux has been discussed more on mainstream tech shows than at any point in my memory. This is a marketing opportunity to not pass up.
  • Help support Game studios with Linux porting and compatibility. (or show other stores they can come!)

Comments

Do you think these will all come to pass? Was I way off? Add a comment via Gitlab

Using a compressed diff instead of lines of code

Lines of code (LOC) has some known flaws, but one of its advantages is that it lets humans visualize it for a small enough number. For bigger numbers like 100,000 vs 200,000 lines of code it really doesn't help us humans picture it.

For big enough changes, you could switch to just compressing the diff and measuring that. That also nicely tracks what developers would have to actually download to get the new changes. It also helps with understanding the bandwidth requirements of contributing to a project.

Here is what it looks like for the Linux kernel since 4.1. (For Rc1s only - the other rcs are in the 30-100 KiB range)

Compressed_Only

Here is a comparison of how far apart the LOC numbers are from the compressed diff numbers - the longer the line is the further apart they are. The numbers are normalized to 0-1. As you can see, they generally line up.

Compressed_vs_LCO

(You can get the raw spreadsheet here )

Let's get some numbers from another project - say systemd.

$ git tag --list --sort=creatordate | tail

#Pick the last two major releases..
$ git diff v247 v248 |  xz -c -q | wc -c | numfmt --to=iec-i --round=nearest
1.1MiB

Conclusion

This isn't ground breaking, but it may prove to be slightly more useful than using LOCs. At the very least as an alternative, it could help put less emphasis on LOCs.

Some interesting future things to look at:

  • Better comparisons between software projects using different languages?
  • Tracking other changes to software projects in a similar way (Wikis, MLs).
  • Compare with other kinds of projects. For instance Wikipedia does track changes monthly by the GB.

Comments and Feedback

Feel free to make a PR to add comments!

Why hasn't snap or flatpak won yet?

Where win means becomes the universal way to get apps on Linux.

In short, I don't think either current iteration will. But why?

I started writing this a while ago, but Disabling snap Autorefresh reminded me to finish it. I also do not mean this as a "hit piece" against my former employer.

Here is a quick status of where we are:

Use case     Snaps   Flatpak
Desktop app  ☑️       ☑️    
Service/Server app  ☑️       🚫   
Embedded  ☑️       🚫   
Command Line apps  ☑️       🚫
Full independence option   🚫      ☑️  
Build a complete desktop   🚫      ☑️  
Controlling updates   🚫      ☑️  

Desktop apps

Both Flatpaks and Snaps are pretty good at desktop apps. They share some bits and have some differences. Flatpak might have a slight edge because it's focused only on Desktop apps, but for the most part it's a wash.

Service/Server / Embedded / Command Line apps

Flatpak doesn't target these at all. Full stop.

Snap wins these without competition from Flatpak but this does show a security difference. sudo snap install xyz will just install it - it won't ask you if you think it's a service, desktop app or some combination (or prompt you for permissions like Flatpak does).

With Embedded using Ubuntu Core it requires strict confinement which is a plus (Which you read correctly, means "something less" confinement everywhere else).

Aside: As Fedora SilverBlue and Endless OS both only let you install Flatpaks, they also come with the container based Toolbox to make it possible to run other apps.

Full independence option / Build a complete desktop

Snaps

You can not go and (re)build your own distro and use upstream snapd.

Snaps are generally running from one LTS "core" behind what you might expect from your Ubuntu desktop version. For example: core18 is installed by default on Ubuntu 21.04. The embedded Ubuntu Core option is the only one that is using just one version of Ubuntu core code..

Flatpak

With Flatpak you can choose to use one of many public bases like the Freedesktop platform or Gnome platform. You can also build your own Platform like Fedora Silverblue does. All of the default flatpak that Silverblue comes with are derived from the "regular" Fedora of the same version. You can of course add other sources too. Example: The Gnome Calculator from Silverblue is built from the Fedora RPMs and depends on the org.fedoraproject.Platform built from that same version of Fedora.

Aside: I should note that to do that you need OSTree to make the Platforms.

Controlling updates

Flatpak itself does not do any updates automatically. It relies on your software application to do it (Gnome Software). It also has the ability for apps to check for their own updates and ask to update itself.

Snaps are more complicated, but why? Let's look at the Ubuntu IoT and device services that Canonical sells:

Dedicated app store ...complete control of application versions, updates and controlled rollouts for $15,000 per year.

Enterprise app store ...control snap updates and upgrades. Ensure that all device traffic goes through an audited communications channel and determine the precise versions of snaps used inside the business.

Control of the update process is one of the ways Canonical is trying to make money. I don't believe anyone has ever told me explicitly that this is why Snaps update work this way. it just makes sense given the business considerations.

So who is going to "win"?

One of them might go away, but neither is set to become the universal way to get apps on Linux at least not today.

It could change starting with something like:

  • Flatpak (or something like it) evolves to support command line or other apps.
  • A snap based Ubuntu desktop takes off and becomes the default Ubuntu.

Either isn't going to get it all the way there, but is needed to prove what the technology can do. In both cases, the underlying confinement technology is being improved for all.

Comments

Maybe I missed something? Feel free to make a PR to add comments!

Let's keep time like it is in the summer

If you are in the USA - Please use my new site KeepSummerTime.com to write to your congresspeople asking for summer time all year long.

The USA has an active bill in congress to keep us from changing the clocks and stay on time like it is in the summer year round (also called permanent DST). Changing the clocks has not been shown to have substantial benefits and the harms have been well documented.

For global communities - like FLOSS -

  • It makes it that much harder to schedule across the world.
  • The majority of the world does not do clock switching. It's generally EU/US specific.

If you are in the USA - Please use my new site KeepSummerTime.com to write to your congresspeople asking for summer time all year long.

If you want to help out

  • the site is all available on Github although the actual contact congress bit is from ActionNetwork.
  • I'd be very happy to make this site global in nature for all of us stuck with unstable time. Please get in touch!

What packages are really required for Debian?

I used 2 of the variants supported by mmdebstrap to illustrate the different small build options.

Thanks to Dan at EndlessOS for showing me the much easier way:

$ grep-aptavail -n -s Package -F Essential yes
$ grep-aptavail -n -s Package -F Priority required
$ grep-aptavail -n -s Package -F Priority important

Essential

Uncompressed tarball size 94M

For when you don't even want to have apt.

base-files
base-passwd
bash
bsdutils
coreutils
dash
debconf
debianutils
diffutils
dpkg
findutils
gcc-10-base:amd64
grep188M
init-system-helpers
libacl1:amd64
libattr1:amd64
libaudit-common
libaudit1:amd64
libblkid1:amd64
libbz2-1.0:amd64
libc-bin
libc6:amd64
libcap-ng0:amd64
libcom-err2:amd64
libcrypt1:amd64
libdb5.3:amd64
libdebconfclient0:amd64
libgcc-s1:amd64
libgcrypt20:amd64
libgmp10:amd64
libgpg-error0:amd64
libgssapi-krb5-2:amd64
libk5crypto3:amd64
libkeyutils1:amd64
libkrb5-3:amd64
libkrb5support0:amd64
liblz4-1:amd64
liblzma5:amd64
libmount1:amd64
libnsl2:amd64
libpam-modules:amd64
libpam-modules-bin
libpam-runtime
libpam0g:amd64
libpcre2-8-0:amd64
libpcre3:amd64
libselinux1:amd64
libsmartcols1:amd64
libssl1.1:amd64
libsystemd0:amd64
libtinfo6:amd64
libtirpc-common
libtirpc3:amd64
libudev1:amd64
libuuid1:amd64debian-requirements.md
zlib1g:amd64

Added in minbase

Uncompressed tarball size 123M

adduser
apt
debian-archive-keyring
e2fsprogs
gcc-9-base:amd64
gpgv
libapt-pkg6.0:amd64
libext2fs2:amd64
libffi7:amd64
libgnutls30:amd64
libhogweed6:amd64
libidn2-0:amd64
libnettle8:amd64
libp11-kit0:amd64
libseccomp2:amd64
libsemanage-common
libsemanage1:amd64Added in minbase
libxxhash0:amd64
logsave
mount
passwd
tzdata

Added in default variant

Uncompressed tarball size 188M

Theoretically all Priority: Important packages.

This is where items start to get a bit redundant IMHO. Mostly because I prefer the built-in systemd options as opposed to ifupdown, rsyslog/logrotate and cron.

apt-utils
cpio
cron
debconf-i18n
dmidecode
dmsetup
fdisk
ifupdown
init
iproute2
iputils-ping
isc-dhcp-client
isc-dhcp-common
kmod
less
libapparmor1:amd64
libargon2-1:amd64
libbpf0:amd64
libbsd0:amd64
libcap2:amd64
libcap2-bin
libcryptsetup12:amd64
libdevmapper1.02.1:amd64
libdns-export1110
libedit2:amd64
libelf1:amd64
libestr0:amd64
libfastjson4:amd64
libfdisk1:amd64
libip4tc2:amd64
libisc-export1105:amd64
libjansson4:amd64
libjson-c5:amd64
libkmod2:amd64
liblocale-gettext-perl
liblognorm5:amd64
libmd0:amd64
libmnl0:amd64
libncurses6:amd64
libncursesw6:amd64
libnewt0.52:amd64
libnftables1:amd64
libnftnl11:amd64
libpopt0:amd64
libprocps8:amd64
libreadline8:amd64
libslang2:amd64
libtext-charwidth-perl
libtext-iconv-perl
libtext-wrapi18n-perl
libxtables12:amd64
logrotate
nano
netbase
nftables
procps
readline-common
rsyslog
sensible-utils
systemd
systemd-sysv
systemd-timesyncd
tasksel
tasksel-data
udev
vim-common
vim-tiny
whiptail
xxd

Learning through breaking

I run Steam in a flatpak for convenience and confinment reasons. One day my Steam install failed with

32 bit libarires not installed

My first instinct is to check to make sure libc6:i386 is actually installed - it is. Then I check to see if there are flatpak updates, but with the 32-bit libraries I find more errors:

        ID                                                  Branch        Op        Remote         Download
 1. [] org.freedesktop.Platform.GL32.nvidia-460-39         1.4           i         flathub        178.7 MB / 178.7 MB

Error: While trying to apply extra data: apply_extra script failed, exit status 40704
error: Failed to install org.freedesktop.Platform.GL32.nvidia-460-39: While trying to apply extra data: apply_extra script failed, exit status 40704

Journal log

Feb 26 08:18:24 desktop polkitd(authority=local)[641]: Registered Authentication Agent for unix-process:3535:65589 (system bus name :1.75 [flatpak install org.freedesktop.Platform.GL32.nvidia-460-39], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)
Feb 26 08:18:26 desktop flatpak[3535]: libostree pull from 'flathub' for runtime/org.freedesktop.Platform.GL32.nvidia-460-39/x86_64/1.4 complete
                                       security: GPG: summary+commit 
                                       security: SIGN: disabled http: TLS
                                       delta: parts: 1 loose: 3
                                       transfer: secs: 0 size: 349.8 kB


Feb 26 08:18:54 desktop flatpak[3535]: system: Pulled runtime/org.freedesktop.Platform.GL32.nvidia-460-39/x86_64/1.4 from flathub
Feb 26 08:18:55 desktop audit[3583]: SECCOMP auid=1000 uid=0 gid=0 ses=2 subj==unconfined pid=3583 comm="apply_extra" exe="/app/bin/apply_extra" sig=31 arch=40000003 syscall=122 compat=1 ip=0x80a933d code=0x0

This is where I remember that I've been testing a lot of systemd confinement changes (including limiting SystemCalls) and one of the services I modified was gpg-agent. However, reverting that change doesn't help but I'm getting closer. (Aside: Great time to guess what config change I made that caused the errors..)

I then run:

sudo flatpak repair

to verify all the files in flatpak but nothing needed fixing.

I then ran:

$ sudo dpkg -V
...
/etc/systemd/system.conf
...

Oh, shoot I did setup

SystemCallArchitectures=native

This is saying I only want native syscalls to be run, but why is it applying to an application! I would have thought it just applied to services or other things systemd runs.

Sure enough disabling that option fixes it, Steam works, and the 32-bit NVidia via Flatpak install too.

But.. why?

Flatpak runs apps in a systemd scope (if available).

$ systemctl status --user app-flatpak-com.valvesoftware.Steam-6702.scope  app-flatpak-com.valvesoftware.Steam-6702.scope
     Loaded: loaded (/run/user/1000/systemd/transient/app-flatpak-com.valvesoftware.Steam-6702.scope; transient)
  Transient: yes
     Active: active (running) since Wed 2021-03-03 12:16:23 PST; 1min 2s ago
      Tasks: 113 (limit: 38415)
     Memory: 352.0M
        CPU: 16.066s
     CGroup: /user.slice/user-1000.slice/[email protected]/app.slice/app-flatpak-com.valvesoftware.Steam-6702.scope
             ├─6702 bwrap --args 41 /app/bin/steam-wrapper
             ├─6706 bwrap --args 4But what does 1 xdg-dbus-proxy --args=43
             ├─6707 xdg-dbus-proxy --args=43
             ├─6711 bwrap --args 41 /app/bin/steam-wrapper
             ├─6713 bash /home/bryan/.local/share/Steam/steam.sh
            ....etc

I want to explore inside this scope more and I stumble upon some Sandbox docs, but using flatpak run just creates it's own scope:

$ systemctl status --user app-flatpak-com.valvesoftware.Steam-7616.scope  app-flatpak-com.valvesoftware.Steam-7616.scope
     Loaded: loaded (/run/user/1000/systemd/transient/app-flatpak-com.valvesoftware.Steam-7616.scope; transient)
  Transient: yes
     Active: active (running) since Wed 2021-03-03 12:20:21 PST; 33s ago
      Tasks: 6 (limit: 38415)
     Memory: 2.8M
        CPU: 61ms
     CGroup: /user.slice/user-1000.slice/[email protected]/app.slice/app-flatpak-com.valvesoftware.Steam-7616.scope
             ├─7616 bwrap --args 42 bash
             ├─7620 bwrap --args 42 xdg-dbus-proxy --args=44
             ├─7621 xdg-dbus-proxy --args=44
             ├─7624 bwrap --args 42 bash
             └─7626 bash

But this is an awesome way to see what the Flatpak actually has access to (and the package icon is just such a nice touch)

$ flatpak run --command=bash com.valvesoftware.Steam 
[📦 com.valvesoftware.Steam ~]$ ls
Music  Pictures  cache  config  data
[📦 com.valvesoftware.Steam ~]$ pwd
/home/bryan

I totally forgot that Steam has a built-in music player. Let's turn that off.

flatpak permissions-show or list doesn't seem to do anything.

flatpak info --show-permissions com.valvesoftware.Steam is the right answer (thanks!)

filesystems=xdg-run/app/com.discordapp.Discord:create;xdg-pictures:ro;xdg-music:ro;
persistent=.;

I then decide to just install Flatseal to review those and end up disabling all the default file permissions.

$ flatpak run --command=bash com.valvesoftware.Steam 
[📦 com.valvesoftware.Steam ~]$ ls
Music  Pictures  cache  config  data

Hmm.. Did I do something wrong?

$ ls Music/ Pictures/
Music/:

Pictures/:

Nope, those directories are now empty. Previosly they were my actual music and pictures. Better confinement and a better understanding of how it works. Nice!

Have a comment or did I make a mistake? Add it via Gitlab.