Posts tagged with “documentation”

man.cgi with lighttpd

This post describes setting up OpenBSDs man.cgi(8) with lighttpd on Gentoo, with OpenRC.

Reasons for choosing lighttpd over nginx:

  • Native CGI binary support
  • Native chroot support

Both of these are going to turn out to be very important for man.cgi(8).

Installing man.cgi

First step would be to install a working man.cgi static binary. For this, we need to install to app-text/mandoc with the cgi USE flag:

$ USE="cgi" emerge --ask --verbose app-text/mandoc

This will install a static binary /var/www/cgi-bin/man.cgi.

Installing lighttpd

The default flags for lighttpd are enough to bring in CGI wrapper support.

$ emerge --ask --verbose www-servers/lighttpd

Set up /etc/lighttpd/lighttpd.conf

I am going to assume that this server is only for serving man pages and for nothing else, in which case the following /etc/lighttpd/lighttpd.conf should suffice:

# chroot to directory (defaults to no chroot)
server.chroot            = "/var/www"

server.username          = "lighttpd"
server.groupname         = "lighttpd"

var.basedir              = "/"
var.logdir               = "/var/log/lighttpd"
var.statedir             = "/var/lib/lighttpd"

server.errorlog          = var.logdir + "/error.log"
accesslog.filename       = var.logdir + "/access.log"

server.document-root     = "/"
server.pid-file          = "/var/run/lighttpd.pid"

# NOTE: the order of modules is important.
server.modules = (
    "mod_access",
    "mod_cgi",
    "mod_accesslog"
)

server.indexfiles        = ("index.php", "index.html",
                            "index.htm", "default.htm")

server.follow-symlink    = "enable"

server.use-ipv6          = "enable"

# (extensions that are usually handled by mod_cgi, mod_fastcgi, etc).
static-file.exclude-extensions = (".php", ".pl", ".cgi", ".fcgi")

$HTTP["url"] =~ "^/cgi-bin/" {
	cgi.assign = (
		".cgi"   => "/cgi-bin/man.cgi"
	)
}

include "mime-types.conf"

Don't start the server yet, it will fail

Set up the /var/www chroot

From the above configuration, it is clear that we need some extra directories in /var/www and with proper permissions.
I am going to be a bit lax about permissions, they can be hardened if need be.

$ mkdir -p /var/www/var/{{lib,log}/lighttpd,run,tmp} /var/www/man/system /var/www/dev
$ chown -R lighttpd:lighttpd /var/www/var
$ chmod 777 /var/www/var/{run,tmp}

Set up the /etc/init.d/lighttpd service

Lighttpd currently needs a /dev/null to be present inside the chroot at /var/www which is not there by default.
We will modify the lighttpd init.d file to mount a devtmpfs on /var/www/dev at each start.
You can of course do it a different way; make it into a separate init, modify existing devtmpfs mounts, etc. This is good enough for local servers.

#!/sbin/openrc-run
# Copyright 1999-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

extra_started_commands="reload graceful"

LIGHTTPD_PID="$(grep pid ${LIGHTTPD_CONF} | cut -d '=' -f 2 | tr -d \\\" | tr -d [:space:])"

depend() {
	need net
	use mysql logger spawn-fcgi ldap slapd netmount dns
	after famd
	after sshd
}

checkconfig() {
	if [ ! -f "${LIGHTTPD_CONF}" ] ; then
		ewarn "${LIGHTTPD_CONF} does not exist."
		return 1
	fi

	if [ -z "${LIGHTTPD_PID}" ] ; then
		eerror "server.pid-file variable in ${LIGHTTPD_CONF}"
		eerror "is not set. Please set this variable properly"
		eerror "and try again"
		return 1
	fi
	/usr/sbin/lighttpd -t -f ${LIGHTTPD_CONF}
}

start() {
	checkconfig || return 1
	# Glean lighttpd's credentials from the configuration file
	# Fixes bug 454366
	LIGHTTPD_USER="$(awk '/^server.username/{s=$3};{sub("\"","",s)};END{print s}' ${LIGHTTPD_CONF})"
	LIGHTTPD_GROUP="$(awk '/^server.groupname/{s=$3};{sub("\"","",s)};END{print s}' ${LIGHTTPD_CONF})"
	checkpath -d -q -m 0750 -o "${LIGHTTPD_USER}":"${LIGHTTPD_GROUP}" /run/lighttpd/
	
	mount -t devtmpfs none /var/www/dev

	ebegin "Starting lighttpd"
	start-stop-daemon --start --exec /usr/sbin/lighttpd \
		--pidfile "${LIGHTTPD_PID}" -- -f "${LIGHTTPD_CONF}"
	eend $?
}

stop() {
	local rv=0
	ebegin "Stopping lighttpd"
	start-stop-daemon --stop --pidfile "${LIGHTTPD_PID}"
	umount -R /var/www/dev
}

reload() {
	if ! service_started "${SVCNAME}" ; then
		eerror "${SVCNAME} isn't running"
		return 1
	fi
	checkconfig || return 1

	ebegin "Re-opening lighttpd log files"
	start-stop-daemon --pidfile "${LIGHTTPD_PID}" \
		--signal HUP
	eend $?
}

graceful() {
	if ! service_started "${SVCNAME}" ; then
		eerror "${SVCNAME} isn't running"
		return 1
	fi
	checkconfig || return 1

	ebegin "Gracefully stopping lighttpd"
	start-stop-daemon --quiet --pidfile "${LIGHTTPD_PID}" \
		--signal INT
	if eend $? ; then
		rm -f "${LIGHTTPD_PID}"
		start
	fi
}

Enable and start lighttpd.

$ mkdir -p /var/www/dev
$ rc-update add lighttpd default
$ rc-service lighttpd start

Set up the man pages

We will copy all the man pages into /var/www/man/system and be done with the whole setup.

$ cp -r /usr/share/man/.  /var/www/man/system
$ cd /var/www/man/system
$ makewhatis -T utf8 .
$ echo "system" > /var/www/man/manpath.conf

AND WE ARE DONE

Visit http://localhost/cgi-bin/man.cgi/ to check out them sexy man pages.

OpenBSD backups with restic and /etc/daily.local

This is yet another one of the guides on how to do backups for a server, this time with restic and OpenBSD. This sets up a daily backup and mails the changes to root.

Install restic

This is the simplest part:

$ pkg_add restic

Setup a restic repository

From the official instructions, the example here sets up an sftp repository over ssh. For this to work, root must have access to the backup server using its ssh key.

$ cat /root/.ssh/config
Host truenas
	HostName 192.168.4.2
	User restic
	IdentityFile /root/.ssh/id_ed25519

$ restic --repo sftp:truenas:/mnt/Media/backups/fwall init

Store the password that was used in plain text in /root/.ssh/restic:

$ ls -lh /root/.ssh/restic
-rw-------  1 root  wheel    18B Dec 24 20:32 /root/.ssh/restic

$ cat /root/.ssh/restic
SomeReallyCoolResticPassword

NOTE: Make sure that the ssh user, restic, has full access to the backup folder.

Setup /etc/daily.local

Create a new file /etc/daily.local.restic with the contents:

# backup restic
R_FILE="/root/.ssh/restic"
R_REPO="sftp:truenas:/mnt/Media/backups/fwall"
env RESTIC_PASSWORD_FILE="${R_FILE}" \
HOME="/root" \
/usr/local/bin/restic --repo ${R_REPO} \
	--verbose backup \
	--exclude-if-present=no_restic \
	--exclude-file=/etc/restic.exclude \
	--files-from=/etc/restic.include \
	--tag="$(date +%c)"

# list changes
PREV=$(env RESTIC_PASSWORD_FILE="${R_FILE}" HOME="/root" \
	/usr/local/bin/restic --repo ${R_REPO} \
	snapshots --compact | tail -4 | head -1 | awk '{print $1}')
LAST=$(env RESTIC_PASSWORD_FILE="${R_FILE}" HOME="/root" \
	/usr/local/bin/restic --repo ${R_REPO} \
	snapshots --compact | tail -3 | head -1 | awk '{print $1}')

RDIFF_FILE="$(mktemp)"
env RESTIC_PASSWORD_FILE="${R_FILE}" HOME="/root" \
	/usr/local/bin/restic --repo ${R_REPO} \
	diff ${PREV} ${LAST} > ${RDIFF_FILE}

NLINES=$(wc -l "${RDIFF_FILE}" | awk '{print $1}')
if [ ${NLINES} -gt 108 ] ; then
	head -n 100 "${RDIFF_FILE}"
	printf "======= SNIP ======\n"
	tail -n 8 "${RDIFF_FILE}"
else
	cat "${RDIFF_FILE}"
fi
rm -f "${RDIFF_FILE}"
unset R_REPO R_FILE RDIFF_FILE NLINES

And in the /etc/daily.local, add a line which sources /etc/daily.local.restic:

...
{other personal scripts}
...
. /etc/daily.local.restic

This sets up the daily script, which backs up the whole system. We still need to create the config files /etc/restic.include and /etc/restic.exclude.

/etc/restic.include:

/etc
/root
/home
/usr
/bin
/sbin
/var

/etc/restic.exclude:

/var/run
/var/spool
/var/tmp

Any folder which contains a file named no_restic is also excluded. For example, if a user has multiple git repositories in /home/aisha/GIT/ and they wish to avoid backups of this folder, create a file /home/aisha/GIT/no_restic, an empty file is fine, it can also contain the reason for excluding this folder.

$ locate no_restic
/home/aisha/GIT/no_restic

$ cat /home/aisha/GIT/no_restic
no need to do backups of online version controlled folders

Test backup

To test it out, first do a full back of the system:

$ env -i R_FILE="/root/.ssh/restic" \
    R_REPO="sftp:truenas:/mnt/Media/backups/fwall" \
    RESTIC_PASSWORD_FILE="${R_FILE}" \
    HOME="/root" \
    /usr/local/bin/restic --repo ${R_REPO} \
    --verbose backup \
    --exclude-if-present=no_restic \
    --exclude-file=/etc/restic.exclude \
    --files-from=/etc/restic.include \
    --tag="$(date +%c)"

Now do a test run of /etc/dailly.local.restic with:

$ sh /etc/daily.local.restic
open repository
repository f719d564 opened successfully, password is correct
lock repository
load index files
using parent snapshot 05b3f0dd
start scan on [/etc /root /home /usr /bin /sbin /var]
start backup on [/etc /root /home /usr /bin /sbin /var]
scan finished in 2.230s: 35818 files, 2.111 GiB

Files:           0 new,    10 changed, 35808 unmodified
Dirs:            0 new,    12 changed,  1859 unmodified
Data Blobs:      8 new
Tree Blobs:     11 new
Added to the repo: 817.516 KiB

processed 35818 files, 2.111 GiB in 0:05
snapshot 11d4745c saved
comparing snapshot 05b3f0dd to 11d4745c:

M    /root/.local/share/nvim/shada/main.shada
M    /var/cron/log
M    /var/cron/log.0.gz
M    /var/cron/log.1.gz
M    /var/cron/log.2.gz
M    /var/log/daemon
M    /var/log/dhcpd
M    /var/log/messages
M    /var/log/pflog
M    /var/log/rad

Files:           0 new,     0 removed,    10 changed
Dirs:            0 new,     0 removed
Others:          0 new,     0 removed
Data Blobs:      8 new,     8 removed
Tree Blobs:     10 new,    10 removed
  Added:   815.139 KiB
  Removed: 821.155 KiB

Congratulations

This sets up proper daily backups for the server, with blazing speed.