All Things UNIX/Unix-Like/FOSS/etc.

[Manny]

Here's a thread for discussing all things free & open source. It appears we have some experienced users and at least one developer on the forum at this present time. Feel free to share any news/info that could be of interest!

[fearedbliss]

So let me make the first post here (excluding OP of course haha).

So what you are seeing in the above screenshot is my new poudriere set up running in my FreeBSD 13.0-RELEASE server (Which is hosting xyinn.org and these forums). I normally use binary packages through "pkg" to upgrade and install packages on my server and only use ports a little bit for some specific things, however mixing ports/binary packages is not really recommended because there could be problems with the generic build configuration packages that are provided by the upstream build server (All binary packages provided by FreeBSD are built with the default flags and the default configurations) and the configuration options you may have set locally. For a while, this has affected me in that FreeBSD 13.0 uses postgres 13 as its default postgres implementation, however, I'm using postgres 14 on this server. This also means that there are particular binary packages that I installed (like let's say php74-pgsql driver for php that links to postgres 13 since that's the default in FreeBSD 13 and it's a driver I need for this forum to operate). This means that I have had to:

1. Do some local tweaking to make postgres 14 the default.
2. I need to use ports to build the php psql driver (databases/php74-pgsql) that is linked to my local default postgres of version 14 (ports will install this as its own binary package as well so "pkg" will register and detect that correctly).
3. I need to watch out for updates that try to remove or downgrade my postgres implementation due to these differences between binary packages.

Now in this case, the problem is still there in that when I do another upgrade to the system, whenever it detects that the php74-pgsql package has changed by upstream, it will try to downgrade or just flat out fail the build because of these conflicts. The solution recommended by the FreeBSD community is to use poudriere to automatically build all of these ports packages for me, for my specific freebsd version, using my specific configuration, and it will automatically create binary packages in a directory structure that is the exact same as what "pkg" uses. Afterwards, I can simply point my host's package server to target that poudriere location (hosted over http), and that's it. The host won't fight anymore with upstream, because upstream is actually my own build server that has the same exact build configuration that I wanted to have (same as upstream but with psql 14 as my target default postgres version).

A lot of this reminds me of stuff I used to do when I ran Gentoo and had to set up my own binary package server (PORTAGE_BINHOST), but the FreeBSD one is a lot better, simpler, stable, and I don't have to do as much of the work that I used to do given that back when I was on Gentoo, I was actually pushing development for these types of features, primarily in relation to my ZFS work.

Also poudriere uses jails to isolate all of the build instances, I have it integrated with ZFS as well so it will automatically create datasets to put its files in. It can even cross compile! (amd64 <> i386 for instance). Poudriere is also what FreeBSD themselves use to build all of their packages from what I know. So that's pretty cool. Example: FreeBSD Package Status.

I also found this PDF by the creator of poudriere, Bryan Drewery. I haven't finished reading it yet but it's pretty cool.

This is also an example of having two separate build directories, fully isolated, from two separate FreeBSD branches on the same box. The first one in this case will be building packages for 13.0 RELEASE amd64, and the second is building for CURRENT (HEAD) on i386. I haven't gotten into STABLE/CURRENT stuff yet, but I foresee I'll need to do this soon if I want to switch my framework laptop from Linux Mint to FreeBSD, which I've been dying to do, and support is slowly coming along (It's pretty much mostly running it seems but there are some kinks).

Code: Select all

poudriere jail -c -j 130amd64 -v 13.0-RELEASE -a amd64
poudriere jail -c -j current -v head -a i386

All of these jails can either have shared "make.conf" configuration or can even have their own set of configs, or a combination of them. There is a defined hierarchy which you can take a look at here, search for "Create optional make.conf".

- fearedbliss

[hedgesparrow]

Compiling from source makes a lot of sense. You cut out the middle-man between you and the author. You can often reduce the number of dependencies during build time. You understand *exactly* what is installed on your machines and it's configured perfectly. The problem, of course, is you are taking on the job of the package managers so the cost is time and lots of it!

I was working on a POSIX shell script that automated building a heavily customised KISS Linux. The idea was to not even have a package manager. I would compile exactly what I need, and rebuild the whole system every six months to include the latest versions of everything.

I had problems with musl libc though. Chromium would crash occasionally (oh boy compiling Chromium yourself is a massive job in itself!). musl also doesn't support multilib in the same way, which is needed for Wine. I couldn't get wine64 to run at all on musl and wine32 needs multilib.

So I was going to do the same using multilib glibc but I decided to try Void Linux again. It does support musl but, as I found, glibc seems more stable, probably as it's tested a lot more. Void Linux does let me avoid systemd and keep my system pretty simple. I have the option of compiling individual packages from source, if I want to customise it further.

Here's my pstree, it's very lean compared to something like out-of-the-box Ubuntu. I'm running: a window manager, PCem for Quake, deluge for torrents, Firefox, several terminals and several services for logging etc:

Code: Select all

runit─┬─at-spi-bus-laun─┬─dbus-daemon
      │                 └─2*[{at-spi-bus-laun}]
      ├─at-spi2-registr───2*[{at-spi2-registr}]
      ├─bash───firefox─┬─Privileged Cont───20*[{Privileged Cont}]
      │                ├─RDD Process───2*[{RDD Process}]
      │                ├─Web Content───23*[{Web Content}]
      │                ├─Web Content───14*[{Web Content}]
      │                ├─3*[Web Content───20*[{Web Content}]]
      │                ├─Web Content───19*[{Web Content}]
      │                ├─Web Content───21*[{Web Content}]
      │                ├─WebExtensions───19*[{WebExtensions}]
      │                └─189*[{firefox}]
      ├─bash───pcem───10*[{pcem}]
      ├─bash───deluge-gtk───6*[{deluge-gtk}]
      ├─dbus-daemon
      ├─dbus-launch
      ├─gpg-agent
      ├─runsvdir─┬─4*[runsv]
      │          ├─runsv───login───bash───sx─┬─Xorg───4*[{Xorg}]
      │          │                           └─dwm─┬─st───bash───vim
      │          │                                 ├─st───bash───pstree
      │          │                                 ├─st───bash
      │          │                                 └─sxrc───sleep
      │          ├─runsv───pause
      │          ├─runsv───udevd
      │          ├─2*[runsv───agetty]
      │          ├─runsv───nanoklogd
      │          ├─runsv───dhcpcd
      │          ├─runsv─┬─logger
      │          │       └─openntpd───openntpd───openntpd
      │          └─runsv─┬─socklog
      │                  └─svlogd
      └─ssh-agent

I'm planning on customising Firefox to remove the dbus dependency. Keeping it simple means I actually understand what I'm running on my system, it's not just a philisophical choice!

As Void Linux is doing a good job there is no point me investing many more hours in chasing my utopia of a Linux distribution .

[fearedbliss]

Lol trust me, as a retired Gentoo Linux Developer, I know EXACTLY what you mean LOL. The reason I choose Gentoo in the past wasn't because I liked compiling from source or because I wanted to "rice" my box, but because I really really wanted to run Linux on top of OpenZFS as my root filesystem and I spent years working on perfecting that, bliss-initramfs being the main result of that. I also made sure that my init system worked with and without systemd (when systemd eventually became popular / standard). Eventually I left Gentoo and switched to Debian stable for a little because, but I was no longer able to use / on ZFS in the way that I wanted to. I actually ended up splitting my 6x 1 TB zpool into 1 TB for Debian and the other 5 TBs on OpenZFS just cause I didn't want to deal with running a debootstrap and doing all the other stuff. At least on Gentoo, you are encouraged to explore and run "non-standard" configurations, where as in Debian you really should be running what apt has or the "Debian way" of doing something. I think I either stayed for a little bit and then went back to Gentoo before switching to FreeBSD, or I switched from Debian to FreeBSD directly, but either way, once I heard that FreeBSD 12 (IIRC) was going to be rebasing their OpenZFS implementation on top of ZFS on Linux (Thus creating OpenZFS), I was like that's it! I was able to get what I finally wanted without having to continue maintaining almost the entire stack. FreeBSD also has binary packages in addition to ports, so for 99% of the stuff, I can just use binary packages exactly as you would in Debian. On top of that, there were a few other bonuses I was getting:

• A complete Operating System rather than just Linux Kernel + GNU/Userland. No longer needing to deal with a fragmented type of amalgamation of components that distributions are in charge of basically trying to get to work together. This type of strategy has it's pros and cons for sure. It's not all bad but it's not all good either. I personally prefer a complete OS with consistent and complete documentation. FreeBSD provides all of this.
• BSD Licensed OS. I was finally able to further distance myself from the "Free Software" and "FSF" folks since I really don't believe in that philosophy at all and I try to avoid contributing to GPLed projects. Not because I don't think they are not good, but because I just don't support that type of social contract and I feel like I'm wasting my time. I personally believe in the rights of the developer rather than the rights of the user. Of course I want the user to have the code as well (for most projects I've published), but I want the developer to have the decision on what they want to do with their forks and releases. All I really care about is that _my_ copy of the code remains Open Source, which is guaranteed because I'm not changing the license on my version of the codebase. If people want to fork and close it or sell it, whatever I don't care. If they want to contribute back up, ok.. I'll take a look at it but there is no guarantee I will accept your changes because I might not like what you've done or the direction you've taken it. This is why I encourage people to just fork my stuff cause I'm very picky about what lands in my projects and the design direction that I want to take my projects in. I'm perfectly fine with people forking my code and implementing their dream with it lol. You then also have the whole legal aspect of it which is something I don't even want to spend my time on (going to courts and getting lawyers). If I want something to be open, then I'm gonna open source it completely, do whatever you want. If I want something to be proprietary, then I'll make it proprietary (Singling). I don't have a fundamental ethical crusade against proprietary software even though I of course heavily lean on open source and will use it whenever possible, but if there are tools I need that are good that are worth buying and are proprietary (Alcohol 120%, Windows XP / 7 / 10), then I'll get it (I'll at least buy the software once to support them.. don't get me started about this pay per seat situation, especially for non commercial home users LOL). There are situations where you may want to open source something and prevent people from closing it, I suppose in these cases if _you_ really want to take on that fight, fine strong copyleft it, I don't personally have the need to do that and I might spend some thinking of just going with maybe an MPL model so it's more of a compromise approach, but yea.. All of my code is either released as "Apache License 2.0" or has been relicensed to it. I might have gone full BSD/MIT for the simplicity, but there is an argument to be made about software patents are not good and I do agree that we probably should protect our projects against patent trolls. I'll still need to spend some more time researching BSD people's position on this issue (I've done it before but don't remember atm). I'm also a big fan of the Mozilla Public License 2.0 if we need a File-Level Copyleft license. MIT / BSD 2-3 / ISC are good as well. Alpaca was the only exception to this rule and that's because Yohann released PlugY under the GPLv3 (Which is the worst one if anything.. I prefer (L)GPLv2 if I had to choose).. so Alpaca is the only project I have that's GPLv3. At the end of the day, no license is inherently bad or wrong, each developer needs to decide for themselves how they want to release a body of work and if they even want to use or not use someone else's body of work as well.
• No systemd. systemd honestly wasn't my main reason for leaving and I used it for a few years and it worked fine. But there's a bunch of stuff about it that I don't like so if possible it's nice to have a system without it. But I wouldn't die on this hill kinda like the Devuan folks. Although respect to them for continuing their beliefs, I'm a big fan of people that truly believe something and still pursue it regardless of the opinion of others.
• Ports / Binary Package flexibility. As you read before, it was nice being able to have binary packages completely provided by FreeBSD but then also being able to easily access and use Ports in a native (and actually encouraged - even above binary packages depending on your use case) vs source compilation being an after thought (deb-src).
• OpenZFS. I mean you already know this was the main reason I even decided to spend the time, energy, and resources to actually learn FreeBSD (After many years of glossing over it and occasionally MAYBE trying out an ISO and then running away lol!). But luckily everything worked out and my system has been freaking rock solid for at least a year or two now. Upgrading between minor and major releases has been an absolute pleasure.

In this case, I'm actually happy that I was able to ride the binary package wave on FreeBSD for this long since it helped me quickly get the system up and running and now I feel a lot more confident to start exploring deeper parts of the system. To the point where I feel that compiling from source in this type of context is a true blessing and I don't mind it at all given I'm not really "forced" to do it but I want to do it because of some minor annoyances that I just want to completely eliminate. And like I posted, the whole point of poudriere is to use the ports system to _easily_ and _automatically_ create all of the binary packages you want in an isolated jail and produce binary packages for you. After that I just re-use them myself. At the same time, since my server is already fully running and set up, the updates and compilations are happening on a live system so I don't really notice any downtime or slowdown. I just have a tmux session running, leave it updating and building the new packages. Leave... come back later and just install all those binary packages. You'll really only feel this impact if you are trying to set up the system for the first time and want to have the binary packages immediately.. but in that case I would say to go ahead and actually just pull the binary packages already provided by FreeBSD directly, and then once you are up and running, if you feel like you actually need to compile from source, then switch to ports (At least if you are planning on running and sticking to -RELEASE). If you want to run -STABLE or -CURRENT well it's source / ports only at the point I believe so.. Or at least I think you can do source for -STABLE and -CURRENT w/ the default FreeBSD Binary packages.. that seems to be more of the correct way of looking at it from my current understanding.

[hedgesparrow]

I entirely agree with your thoughts on BSD and the BSD License. I use OpenBSD on my PC Engines router and love it. However, the driver situation for OpenBSD is dire and that is what rules out using it as a desktop and gaming OS. The driver situation is mainly down to man hours, there are plenty of open drivers for Linux that could be ported. Also, once you move away from the excellent base the quality goes downhill.

I was going down the route of trying to have a truely free laptop: Devuan, Libreboot etc. One thing that was always in the back of my mind was games. Games have never been free and open and I never had a problem with that. I actually want to buy a black box that provides me with an experience I explore and enjoy for what it is. It's cool if the developer relases source code, think Quake, but it's not a requirement for me to enjoy the game. I then researched the FSF and realised it's more of a philosophy, if not religion, than a real need for everything to be free and open.

[fearedbliss]

Yup exactly. Look at what happened with the FSF and RMS... He did all that stuff and then they kicked him out and then brought him back despite public pressure not to do so. Then they say "We can't continue without him".. lol what? GPL and FSF as an idea and goal should be clear enough to understand by this point to not need RMS anymore. But whatever lol.

Regarding gaming, after many years of experimenting, I pretty much ended up concluding that the best gaming experience for me is on Windows and I don't need to make Linux something it isn't. On top of that, I look at my gaming computer (and Windows) the same way I look at my Nintendo Switch, DS Lite, GB/GBC/GBA.. it's just a gaming box, that's it. No more no less. I don't do anything personal on it and I don't /really/ care if it's proprietary. So that's why my personal laptop is my framework laptop running open source software (everything except gaming) and my gaming laptop: Razer, is the "Wintendo". It works out great. I can still play and back up all of my classic games on pretty much their original platform (at least Windows has done an amazing job with backwards compatibility) and I can use Alcohol 120% to back up those exact games. No online DRM, and Alcohol 120% allows me to essentially bypass the old school SecuROM (CD based DRM) by being able to efficiently back up those dirty bits in its awesome .MDS / .MDF format. I really think that's the best of both worlds. If I can logically accept my GB, DS, Switch and all these other consoles I've used throughout my whole life, I don't see why I should treat Windows and PCs when it comes to gaming exclusively as something different. If anything, we actually have it really good in the PC master race world since the system is definitely way more open and definitely way more backwards compatible than a lot of these other systems (Switch, GB, DS, etc).

UNIX philosophy right? Do one thing and do it well. So Windows can stay as a gaming machine only. Haha.

[hedgesparrow]

That makes sense, a gaming machine runs Windows and anything else runs Linux or whatever fits best. However, what do you do with a machine that is both for gaming and home office? You're now stuck on Windows for office stuff too.

Linux is good for VMs, you can even run Linux programs in a seamless mode which integrates nicely with Windows. You're out of luck for OpenBSD though as they don't support any drivers or guest tools yet. Is FreeBSD better in a VM?

[fearedbliss]

I don't have office and games on the same machine lol, that's a workflow I tested extensively and that ultimately I decided I wanted to isolate myself from Windows as much as possible when it comes to my personal data so using a dedicated gaming machine for Windows was the best lifestyle and approach for me since I hate dual booting. But if you were in that situation that you described, dual booting would be your only option if you want full access to the underlying hardware.. if your game is simple enough to run with more basic 3d acceleration, a Windows XP VM might be good enough (Or another Windows version).

I think FreeBSD behaves fairly well in a VM, I haven't done anything production related in a FreeBSD VM other than spinning something up quickly in virtual box, testing a few commands for some things I cared about doing on my actual bare metal FreeBSD box, and then applying those findings on the bare metal one.

I did that a lot when I was trying to record my findings for my OpenZFS partition layout for manually installing FreeBSD with my own partitioning scheme rather than using the automated ZFS on / install (it works amazingly well and it's basically almost a one click install haha, even with geli encryption - as opposed to native OpenZFS encryption which I don't believe is yet supported in the FreeBSD bootloader), but I didn't like the default dataset scheme..

How to partition your drive with OpenZFS and continue using the FreeBSD Installer

[fearedbliss]

So this is a screenshot of another popular and modern ports builder for FreeBSD (and DragonflyBSD) called "Synth". It's primarily aimed at regular users and it's technically simpler and "faster" to set up. Synth is primarily focused on just building packages on your current system and using them on this current system (Although you could just share the packages over the network either via NFS or exposing it via HTTP to your other machines) via pkg. Below you can see Synth running on the top left tmux window. I have my existing ccache that I use for poudriere also being used in Synth as well for this test. You can also see the repository structure currently generated on the bottom right tmux window (Before the final htop window on the bottom). Overall so far Synth is pretty cool and I love the interface. Poudriere is amazing as well and I like that it's used officially in FreeBSD and that it's in a way more or less some sort of standard I guess. Once I got poudriere up and running and now Synth, I think they are both easy to set up. Poudriere uses ZFS and Jails for build isolation / clean environment and Synth uses chroots and nullfs mounts for its build isolations / clean environment.

[fearedbliss]

Two posts I wrote recently regarding my adventures with FreeBSD 14.0-CURRENT on the Framework Laptop. Definitely bleeding edge but I'm really trying to switch my laptop to FreeBSD away from Linux. It's rough the support but I'm happy that there are a few FreeBSD developers trying to get better support for this machine and also the FreeBSD project itself seems to be working to add support for the machine as well.

• How to partition your drive using OpenZFS w/ GELI Encryption and continue using the FreeBSD Installer
• FreeBSD on the Framework Laptop
• Q1 2022 Software Development Projects Update