{"id":704,"date":"2022-04-19T04:11:29","date_gmt":"2022-04-19T04:11:29","guid":{"rendered":"https:\/\/www.buddroyce.com\/?p=704"},"modified":"2022-04-19T04:11:29","modified_gmt":"2022-04-19T04:11:29","slug":"linux-from-scratch","status":"publish","type":"post","link":"https:\/\/www.buddroyce.com\/?p=704","title":{"rendered":"Linux From Scratch"},"content":{"rendered":"\n<p>I actually attempted to build my own Linux distro from scratch waaaaaaaaay back in the early 2000&#8217;s when I got out of college and started working for IBM. I just couldn&#8217;t get it to work. Maybe it was because I was younger and impatient or maybe I just didn&#8217;t know as much as I thought I did, but either way, I never finished it and has been one of those elusive things that I wished I did much like getting my CISSP.<\/p>\n\n\n\n<p>Now that I have my CISSP cert, I figured I&#8217;d give this another go and see if I can finally conquer the challenge I set for myself almost 20 years ago.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"637\" height=\"498\" src=\"https:\/\/www.buddroyce.com\/wp-content\/uploads\/2022\/04\/LFS_booted.png\" alt=\"\" class=\"wp-image-707\" srcset=\"https:\/\/www.buddroyce.com\/wp-content\/uploads\/2022\/04\/LFS_booted.png 637w, https:\/\/www.buddroyce.com\/wp-content\/uploads\/2022\/04\/LFS_booted-300x235.png 300w\" sizes=\"auto, (max-width: 637px) 100vw, 637px\" \/><\/figure>\n\n\n\n<p><em>(Yes I know eth0 doesn&#8217;t exist. I need to figure out the right drivers for this Virtual Machine)<\/em><\/p>\n\n\n\n<p><strong>The Host<\/strong><\/p>\n\n\n\n<p>To build LFS I&#8217;ll need a host OS to build all the toolchain and initial environment. The entire process is very much like cross-compiling software for a different architecture except not.<\/p>\n\n\n\n<p>LFS doesn&#8217;t list any recommended distros and while I&#8217;m partial to Debian as I&#8217;ve been running my servers with it for quite literally decades now, I decided to give a different more modern distro a go simply because I wanted to make sure that all the libraries were new enough to meet or exceed the LFS requirements. After looking around for a bit, I found a great guide from Kernotex on youtube who was running EndeavourOS so I figured I&#8217;d give it a go.<\/p>\n\n\n\n<p>The idea was to use a EndeavourOS&#8217;s pre-installation live environment as my temporary host on a Hyper-V VM. Normally I use Virtual Box but I figured I might as well give Hyper-V a go.<\/p>\n\n\n\n<p>I ran into some minor hiccups but eventually got it to work. The fixes are outlined in my post here: <a rel=\"noreferrer noopener\" href=\"https:\/\/www.buddroyce.com\/?p=687\" target=\"_blank\">https:\/\/www.buddroyce.com\/?p=687<\/a><\/p>\n\n\n\n<p>Hyper-V VM Settings were:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Generation 2 VM <em>(I totally should have used Generation 1, will cover why later in this post)<\/em><\/li><li>6 Cores assigned <em>(I originally had 24 cores assigned but it didn&#8217;t work that well. Will cover it later in this post)<\/em><\/li><li>8gb-16gb of RAM <\/li><li>50gb of HDD space<\/li><li>TURN SECURE BOOT OFF!! (otherwise EndeavourOS wouldn&#8217;t boot)<\/li><\/ul>\n\n\n\n<p>Why did I use EndeavourOS? Not going to lie, I was just following the Kernotex video on youtube but even though he was building LFS 10.1 and I was building 11.1, EndeavourOS had all the up to date binaries needed with next to no additional software required, which is awesome.<\/p>\n\n\n\n<p>Unlike my attempt at trying to build Unikernels with Unikraft, getting things up and running for my LFS build was waaay smoother.<\/p>\n\n\n\n<p><strong>The Build<\/strong><\/p>\n\n\n\n<p>I know for years whenever I looked at doing a LFS build, I always ran into problems. Usually versioning and dependency problems that I just didn&#8217;t want to sort out or just difference between what was in the LFS book and what I was actually getting.<\/p>\n\n\n\n<p>Thankfully LFS 11.1 came out pretty recently so a lot of the software versions from the book were still valid and most problems were addressed in the book. Between the book and Kernotex&#8217;s video, I didn&#8217;t find this build all that daunting.<\/p>\n\n\n\n<p>Overall the whole process could have been done over a weekend but I made some screw ups that caused me to back track a little which I&#8217;ll cover. <\/p>\n\n\n\n<p><strong>Issues<\/strong><\/p>\n\n\n\n<p>Initially I ran into problems booting into a Linux host OS. EndeavourOS didn&#8217;t want to work and the Ubuntu pre-installation environment didn&#8217;t want to work either but those aren&#8217;t build specific issues. My build specific issues were<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Errors Compiling &amp; Testing<\/li><li>Cleaning too many things up<\/li><li>Not enough system ram<\/li><li>Unable to get LFS to boot at all<\/li><\/ul>\n\n\n\n<p><strong>Issue 1: Errors Compiling<\/strong><\/p>\n\n\n\n<p>Originally I had the VM using the Hyper-V quick create which automatically defaulted to using 24cores and whatever ram I had.<\/p>\n\n\n\n<p>I set the MAKEFLAG=&#8221;-j24&#8243; as an environment variable to make full use of the 24cores but for some strange reason some of my code compiles started spitting out errors.<\/p>\n\n\n\n<p>Reducing it down to 20 cores seem to help but I had to bring it down to 4-6 cores for code to finally compile without any errors.<\/p>\n\n\n\n<p>I don&#8217;t know what the cause was but it&#8217;s likely to do with the code not being very parallel compile friendly. Concurrency is always an issue and it&#8217;s likely that with 24 cores, some code was done compiling but errored out when other bits of code that normally gets done in a 1-4 core system doesn&#8217;t exist.<\/p>\n\n\n\n<p>If you run into similar problems. Try reducing the number of assigned cores.<\/p>\n\n\n\n<p>make -j4 will assign 4 cores while make -j1 will assign just a single core to a compile job and there were definitely instances where I had to drop down to a single core to compile and test properly.<\/p>\n\n\n\n<p><strong>Issue 2: Cleaning up too many things<\/strong><\/p>\n\n\n\n<p>I seriously blame myself for being impatient and just copying and pasting stuff here.<\/p>\n\n\n\n<p>During my first run through on my 3rd day of building LFS. I finished installing all the binaries and reached chapter 8.77 (Stripping). For some stupid reason I decided that it was a good idea to do the optional stripping of debugging symbols from binaries and libraries to save space. Little did I know that that would screw things up for me two chapters later when I tried to build the Linux Kernel.<\/p>\n\n\n\n<p>Ended up redoing the whole thing again and learning from my mistakes. If it&#8217;s optional, it&#8217;s probably best not to do it until AFTER I get a couple of successful builds in.<\/p>\n\n\n\n<p><strong>Issue 3: Not enough system RAM<\/strong><\/p>\n\n\n\n<p>I should clarify that the VM has more than enough ram. My system has 64gb of ram but 32gb of it is allocated as a ram cache for my 3 drives. That meant that I had 32gb of ram left which was more than enough&#8230; until I try to do something else while code is compiling.<\/p>\n\n\n\n<p>I found that while I could watch youtube videos while I had the VM running, I couldn&#8217;t do much else. Firing up any other video games on the host windows machine would cause things to really lag.<\/p>\n\n\n\n<p>It turned out that the VM and my ram caching solution were conflicting a bit. Reducing my ram cache down from 32gb to 16gb solved the issue.<\/p>\n\n\n\n<p>I have an additional 64gb of ram on order to bring my machine to 128gb of ram to fully avoid this even with several VMs running in the background.<\/p>\n\n\n\n<p><strong>Issue 4: Unable to get LFS to boot at all<\/strong><\/p>\n\n\n\n<p>This literally took me a day or two to figure out. After getting everything all reinstalled and building out my kernel, I was ready to reboot the system and all I got was&#8230; nothing.<\/p>\n\n\n\n<p>It turns out that the base LFS build was meant for legacy systems and not systems running UEFI.<\/p>\n\n\n\n<p>The Hyper-V VM I was using defaulted to generation 2 which meant it supports UEFI and not a legacy bios. The thing is.. once you create a VM and set it&#8217;s generation type, you can&#8217;t change it again, so I thought I was shit outta luck&#8230; until&#8230;. I decided to create a new VM using a generation 1 setting and using the existing vhdx hard drive file.<\/p>\n\n\n\n<p>Since the .vhdx file is literally just a disk image, I figured I could use that the same way I could a real hard drive. If everything was physical, I would have just taken the hard drive with LFS on it and shoved it into a system with a legacy bios.<\/p>\n\n\n\n<p>To my surprise, it actually booted.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"635\" height=\"499\" src=\"https:\/\/www.buddroyce.com\/wp-content\/uploads\/2022\/04\/LFS_grub.png\" alt=\"\" class=\"wp-image-706\" srcset=\"https:\/\/www.buddroyce.com\/wp-content\/uploads\/2022\/04\/LFS_grub.png 635w, https:\/\/www.buddroyce.com\/wp-content\/uploads\/2022\/04\/LFS_grub-300x236.png 300w\" sizes=\"auto, (max-width: 635px) 100vw, 635px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>I actually attempted to build my own Linux distro from scratch waaaaaaaaay back in the early 2000&#8217;s when I got out of college and started working for IBM. I just couldn&#8217;t get it to work. Maybe it was because I was younger and impatient or maybe I just didn&#8217;t know as much as I thought [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"class_list":["post-704","post","type-post","status-publish","format-standard","hentry","category-it-and-tech"],"_links":{"self":[{"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=\/wp\/v2\/posts\/704","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=704"}],"version-history":[{"count":2,"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=\/wp\/v2\/posts\/704\/revisions"}],"predecessor-version":[{"id":708,"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=\/wp\/v2\/posts\/704\/revisions\/708"}],"wp:attachment":[{"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=704"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=704"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.buddroyce.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=704"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}