Dyson — Debian GNU/kOpenSolaris

Starting from the beginning of January 2024 (not 2023) the site will be only available as osdyson.ru.
This includes all subdomains, for example apt.osdyson.org will become apt.osdyson.ru.

Previous domain osdyson.org will be free.
Until then redirects will be set up.

It is possible to spin up a VirtualBox machine using Vagrant. See https://git.osdyson.ru/vagrant/dyson

Some things do not work yet (e. g. mounting /vagrant), but basically it is usable.

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'dyson' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Box file was not detected as metadata. Adding it directly...
==> default: Adding box 'dyson' (v0) for provider: virtualbox
    default: Downloading: https://dl.osdyson.org/vagrant/2020-01-02/dyson.box
    default: Calculating and comparing box checksum...
==> default: Successfully added box 'dyson' (v0) for 'virtualbox'!
==> default: Importing base box 'dyson'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: vagrantfiles_default_1577965263075_52776
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: No guest additions were detected on the base box for this VM! Guest
    default: additions are required for forwarded ports, shared folders, host only
    default: networking, and more. If SSH fails on this machine, please install
    default: the guest additions and repackage the box to continue.
    default: 
    default: This is not an error message; everything may continue to work properly,
    default: in which case you may ignore this message.
==> default: Mounting shared folders...
    default: /vagrant => /home/pashev/tmp/vagrantfiles
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

sudo /sbin/mount -F vboxfs -o uid=`/usr/xpg4/bin/id -u vagrant`,gid=`/usr/xpg4/bin/id -g vagrant` vagrant /vagrant

Stdout from the command:

Stderr from the command:

bash: line 4: /usr/xpg4/bin/id: No such file or directory
bash: line 4: /usr/xpg4/bin/id: No such file or directory
mount: Operation not applicable to FSType vboxfs

$ vagrant provision 
==> default: Running provisioner: shell...
    default: Running: inline script
    default: 
    default: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
    default: Hit:1 http://apt.osdyson.org testing InRelease
    default: Reading package lists...
    default: Building dependency tree...
    default: 
    default: Reading state information...
    default: All packages are up to date.
    default: 
    default: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
    default: Reading package lists...
    default: Building dependency tree...
    default: 
    default: Reading state information...
    default: Calculating upgrade...
    default: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Built LLVM 8 (moving towards Rust). The Patch.

In addition to lldb disabled in previous builds, this build disables LLVM's OpenMP. It is sad how portability-unfriendly code could be in LLVM, for no reason, even in trivial case like parsing command line options... After some my efforts, I think both can be ported, but both need so much to update. Still a lot of assumptions and quick hacks, countless things like #if defined(linux) || defined(freebsd) || defined(kfreebsd) ..., which could be done by proper configure system like GNU autoconf or CMake.

Anyway:

clang++-8 gcd.cpp -stdlib=libc++ -o gcd-cpp -v
clang version 8.0.1-4+dyson1 (tags/RELEASE_801/final) (llvm/llvm-toolchain-8.git de3800437459c8153299ae0e16ef827e8d000049)
Target: x86_64-pc-solaris2.11
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/4.9
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/4.9.1
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/5
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/5.2.1
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/6
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/6.3.0
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-solaris2.11/4.9
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-solaris2.11/4.9.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-solaris2.11/5
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-solaris2.11/5.2.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-solaris2.11/6
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-solaris2.11/6.3.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-solaris2.11/9
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
 "/usr/lib/llvm-8/bin/clang" -cc1 -triple x86_64-pc-solaris2.11 -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name gcd.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /usr/lib/llvm-8/lib/clang/8.0.1 -internal-isystem /usr/lib/llvm-8/bin/../include/c++/v1 -internal-isystem /usr/include/clang/8.0.1/include/ -internal-isystem /usr/local/include -internal-isystem /usr/lib/llvm-8/lib/clang/8.0.1/include -internal-externc-isystem /usr/include/x86_64-illumos -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir /home/pashev/gcd -ferror-limit 19 -fmessage-length 225 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /var/tmp/gcd-4b480d.o -x c++ gcd.cpp -faddrsig
clang -cc1 version 8.0.1 based upon LLVM 8.0.1 default target x86_64-pc-solaris2.11
ignoring nonexistent directory "/usr/local/include" 
ignoring duplicate directory "/usr/include/clang/8.0.1/include" 
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/llvm-8/bin/../include/c++/v1
 /usr/include/clang/8.0.1/include
 /usr/include/x86_64-illumos
 /usr/include
End of search list.
 "/usr/bin/x86_64-pc-solaris2.11-ld" --hash-style=both --build-id --eh-frame-hdr -m elf_x86_64_sol2 -o gcd-cpp /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9/../../../x86_64-illumos/crt1.o /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9/../../../x86_64-illumos/crti.o /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9/../../../x86_64-illumos/values-Xa.o /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9/crtbegin.o -L/usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9 -L/usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9/../../../x86_64-illumos -L/lib/x86_64-illumos -L/usr/lib/x86_64-illumos -L/usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9/../../.. -L/usr/lib/llvm-8/bin/../lib -L/lib -L/usr/lib /var/tmp/gcd-4b480d.o -lc++ -lgcc_s -lc -lgcc -lm /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9/crtend.o /usr/bin/../lib/gcc/x86_64-pc-solaris2.11/9/../../../x86_64-illumos/crtn.o

The most serious issue was operation priorities :D

$ pypy --version
Python 2.7.13 (7.2.0+dfsg-1+dyson1, Dec 04 2019, 16:40:34)
[PyPy 7.2.0 with GCC 9.2.1 20191109]

$ cat gcd.py
import sys
import functools

def gcd2(a, b):
    if b == 0:
        return a
    else:
        return gcd2(b, a % b)

def gcdn(ns):
    return functools.reduce(gcd2, ns)

ints = map(int, sys.argv[1:])
gcd = gcdn(ints)
print(gcd)

$ pypy gcd.py 11 22 121
11

After reading https://guix.gnu.org/blog/2018/bootstrapping-rust/ I have discovered a Rust compiler written in C++ - mrust. It only needed a few changes to make it work on Dyson. I have committed them to https://git.osdyson.ru/rust/mrust.

When building a few troubles raised, the most important is that it needs a lot of memory, A LOT :). At some step there was a C file of size 760 MiB. I could not build it on a server with 12 GiB RAM and 40GiB swap (!), the server just hung. Then I spun up a VirtualBox instance with about 18GiB RAM (on a 24 GiB RAM server) plus 20 GiB swap. It took a while to compile, but finally it succeeded.

Here is a screenshot of the process:

mrust currently builds Rust 1.29, I have to add extra patch to make Rust work on Dyson. It adds new target (x86_64-pc-solaris) which seems to appear in later Rust versions, and disables Rust version detection in the regex crate, because it fails to find the rust compiler for some reason.

What I have now:

root@dyson:~/mrust# cat ~/main.rs
fn main ()
{
    println!("hello");
}
root@dyson:~/mrust# run_rustc/output/prefix/bin/rustc --version
rustc 1.29.0-stable-mrustc
root@dyson:~/mrust# run_rustc/output/prefix/bin/rustc ~/main.rs -o /tmp/main
root@dyson:~/mrust# ldd /tmp/main
        libsocket.so.1 =>        /lib/x86_64-illumos/libsocket.so.1
        libresolv.so.2 =>        /lib/x86_64-illumos/libresolv.so.2
        libgcc_s.so.1 =>         /lib/x86_64-illumos/libgcc_s.so.1
        libc.so.1 =>     /lib/x86_64-illumos/libc.so.1
        libm.so.2 =>     /lib/x86_64-illumos/libm.so.2
        libnsl.so.1 =>   /lib/x86_64-illumos/libnsl.so.1
        libmd.so.1 =>    /lib/x86_64-illumos/libmd.so.1
        libmp.so.2 =>    /lib/x86_64-illumos/libmp.so.2
root@dyson:~/mrust# file /tmp/main
/tmp/main: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/amd64/ld.so.1, BuildID[sha1]=2c927fc72b5c16174196c35914d80a1f0cc8a75b, not stripped
root@dyson:~/mrust# /tmp/main
hello

All tests are passing except one for locales, which is definitely a problem with libc.

# perl --version

This is perl 5, version 30, subversion 0 (v5.30.0) built for x86_64-pc-solaris2.11-thread-multi-64
(with 48 registered patches, see perl -V for more detail)

Copyright 1987-2019, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

Both versions of fakeroot (tcp and sysv) work like a charm. For some reason faked wasn't even trying to bind on port 0 (default). This might be working on linux. but not on illumos/solaris. But binding on port 0 is perfectly legit:

--- fakeroot.git.orig/faked.c
+++ fakeroot.git/faked.c
@@ -1434,7 +1434,6 @@ int main(int argc, char **argv){
   if (setsockopt(sd, SOL_TCP, TCP_NODELAY, &val, sizeof (val)) < 0)
     fail("setsockopt(TCP_NODELAY)");

-  if (port > 0) {
     memset((char *) &addr, 0, sizeof (addr));
     addr.sin_family = AF_INET;
     addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
@@ -1442,7 +1441,6 @@ int main(int argc, char **argv){

     if (bind(sd, (struct sockaddr *) &addr, sizeof (addr)) < 0)
       fail("bind");
-  }

   if (listen(sd, SOMAXCONN) < 0)
     fail("listen");

# gcc-9 --version
gcc-9 (Debian 9.2.1-19+dyson1) 9.2.1 20191109
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The Stack Smashing Protector (SSP) is disabled by default (enable with -fstack-protector switches). This is because libc does not support SSP yet, and thus libssp from GCC is required. But when SSP is enabled by default, the libssp library is not passed to the linker. Additionally, when enabled by default, SSP may be required when building GCC itself.

The /lib directory is removed from default library path. All development libraries have to reside under /usr/lib, but /lib contains some for illumos linker (sunld) specifically (e. g. libc.so, while libc.so under /usr/lib is a GNU linker script). Removal of /lib ensures that the normal linker will not pickup the wrong library.

As usually, documentation and man pages are included.

With GNU Modula-2:

# cat test.mod 
MODULE Hello;
FROM StrIO IMPORT WriteString;
BEGIN
  WriteString("Hello World!");
END Hello.

# gm2-9 test.mod -o test

# ./test 
Hello World!

$ gforth --version
gforth 0.7.3

$ gforth ./gcd.fs 11 22 121
11

$ cat gcd.fs
: gcd ( a b -- d )
  2dup > if swap endif
  over mod
  dup 0 <> if recurse else drop endif
  ;

: gcdn ( a1 a2 .. an n -- d )
  dup 1 >
  if
    1 - rot rot
    gcd
    swap
    recurse
  else
    drop
  endif
  ;

\ This is gforth-specific.
\ Usage:
\ # gforth ./gcd.fs 11 22 33 121
\ 11

: main
  0 >r
  begin
  next-arg 2dup 0 0 d<> while
  s>unumber? if drop else abort endif
  r> 1 + >r
  repeat
  2drop
  r> gcdn . cr
  ;

main bye

$ dpkg -s gforth
Package: gforth
Status: install ok installed
Priority: optional
Section: interpreters
Installed-Size: 481
Maintainer: Peter Pentchev <roam@ringlet.net>
Architecture: illumos-amd64
Multi-Arch: foreign
Version: 0.7.3+dfsg-4+dyson1
Depends: libc1 (>= 5.10.18.git.2a44663-1), libffcall1 (>= 1.10+2.41), libffi6 (>= 3.0.13-4+dyson1), libltdl7 (>= 2.4.6), emacsen-common (>= 2.0.7), gforth-common (= 0.7.3+dfsg-4+dyson1), gforth-lib (= 0.7.3+dfsg-4+dyson1)
Conffiles:
 /etc/emacs/site-start.d/50gforth.el 1629d9314f2e496f2c476df52bf64e80
Description: GNU Forth Language Environment