Building a package repository for FreeBSD with poudriere and pkgng


This post covers building a server that will be used to update both ports and the base system on FreeBSD backend servers that don’t have access to the Internet. For ports it will use poudriere combined with the new pkgng package manager in order to build packages that will be distributed to the other servers using Nginx. Since Nginx is already there it will be used as a forward proxy, as opposed to reverse proxy as it’s usually used, to and This covers freebsd-update and installing pkg on FreeBSD 9.X. For no good reason, just to do something different, there’s a cron job that runs every day and fetches auditfile.tbz from if it’s changed, so that pkg audit can do it’s job properly. In order to remotely install the servers they are PXE booted into a net install image, mfsbsd, base install files are mirrored on the package repository and served by Nginx. With that, the environment is complete, servers can be installed and kept up to date without giving them any kind of access to the Internet.

Server – poudriere

The Documentation. Poudriere knows how to use ZFS (used to need it, that’s not longer the case), we installed FreeBSD on a ZFS mirror using this script.

To compile packages for different releases and architectures from different ports trees poudriere uses “jails”, which are actually just chroot, not FreeBSD jails. Each jail or ports tree is a different zfs filesystem so they can be easily added/removed/restored, etc. When compiling packages any combination of jail+ports can be used. A jail represents a base system used to compile the ports. For example, we could have FreeBSD 8 and FreeBSD 9, some of them using the latest php version from a portstree fetched with portsnap while others use an year old version from a portstree checked out by svn. By chrooting inside a jail any modification to the base system can be made to mirror the real server environments, although that’s probably rarely necessary. So yes, it’s very versatile.

Packages are stored in different repositories under /poudriere/data/packages/$JAIL-$PORTS, using our configuration. If WITH_PKGNG is defined each repository will have repo.txz created, which is an sqlite database that pkgng downloads and uses to see what packages are available on the repository and check for updates.

First steps

When I first wrote this, it was on FreeBSD 9, with gcc and ZFS was mandatory. Things changed, so I give two versions below. Note that ZFS or not ZFS has nothing to do with the version of FreeBSD, both work in either version.

For FreeBSD 9 and ZFS:

Like the documentation says, install poudriere, pkgng if necessary and ccache from ports. Make sure dialog4ports is installed too, it will be needed in order to set options. If it isn’t pulled in as a dependency, install it. Copy poudriere.conf.sample to poudriere.conf and edit as necessary. Our config looks like this, comments filtered out for brevity:


FreeBSD 10, no ZFS:

There are two things that are different in FreeBSD 10 (ZFS isn’t one of them). One is that the default compiler changed to clang, the other is that pkgng is in base now. The default compiler affects the use of ccache. I am no expert, but clang is not officially supported in ccache 3.1.9, the current stable version, because there are some issues with it. In my opinion the risk of a port not compiling because of ccache doesn’t justify the benefits. If, however, someone would like to use it, pay attention to the port config option that installs symlinks for clang and the message after install. There is also a shell script called ‘’, which creates links for different compilers. As for pkgng, it is now in base in FreeBSD 10, so doesn’t need to be installed anymore. However, ports might have a newer version.


Using this configuration there are two directories of note, $BASEFS, which is /poudriere in the ZFS version and /usr/local/poudriere in the non-ZFS version, and /usr/local/etc/poudriere.d. From this point forward $BASEFS will be assumed to be /poudriere. The working directory is $BASEFS (/poudriere), logs and packages are written here, while /usr/local/etc/poudriere.d is where various settings are, like make.conf options and ports options for each jail. We also store the lists of packages to be compiled and cron scripts for updates here.

Create a jail:

# poudriere jails -c -j srv_91r_amd64 -v 9.1-RELEASE -a amd64

Create make.conf for this jail:

# echo "WITH_PKGNG=yes" > /usr/local/etc/poudriere.d/srv_91r_amd64-make.conf
# echo "WITHOUT_X11=yes" >> /usr/local/etc/poudriere.d/srv_91r_amd64-make.conf

When compiling, the contents of this file will be added to the contents of the /etc/make.conf file that is inside the jail, in this case /poudriere/jails/srv_91r_amd64/etc/make.conf. There is also /usr/local/etc/poudriere.d/make.conf that can be used for options that should be applied to ALL jails.

Create a ports tree:

# poudriere ports -c -p default

Passing -p name creates a tree with that particular name. If -p is not specified, a default ports tree called “default” will be used. This is true for other poudriere commands, too, like poudriere bulk, used to build packages.

The -m parameter stands for “method” and it can be used to specify how that ports tree will be created and updated. For a ports tree that will be downloaded by svn for example, -m svn+http could be used. Obviously, subversion needs to be installed in that case.

Create a list of packages to be compiled by poudriere:

# vim /usr/local/etc/poudriere.d/srv_91r_amd64.pkglist

This file accepts comments marked with a hash tag (#).

Set compilation options for all ports in pkglist:

# poudriere options -f /usr/local/etc/poudriere.d/srv_91r_amd64.pkglist -j srv_91r_amd64

Build the packages:

# poudriere bulk -f /usr/local/etc/poudriere.d/srv_91r_amd64.pkglist -j srv_91r_amd64

Enjoy watching the damn thing actually do some work for a change!

Daily usage

Adding a package to the list and setting options for the port and it’s dependencies. Like nagios:

# echo "net-mgmt/nagios" >> /usr/local/etc/poudriere.d/srv_91r_amd64.pkglist
# poudriere options -j srv_91r_amd64 net-mgmt/nagios

Changing options for a single port:

# poudriere options -c -j srv_91r_amd64 net-mgmt/nagios

Removing options for a port and all dependecies:

# poudriere options -r -j srv_91r_amd64 net-mgmt/nagios

Use -n to set/remove options for nagios only. Note that, at least at the time of this writing, -r removes options for all dependencies, even if they are not direct dependencies of the respective port. For example, for nagios, it also removes options for apache22, because php5 depends on it and nagios depends on php5. But when using -c only options for things nagios directly depends on are set, like php5, but not apache22.

Updating the ports tree and rebuilding ports that were updated:

# poudriere ports -u -p default
# poudriere bulk -f /usr/local/etc/poudriere.d/srv_91r_amd64.pkglist -j srv_91r_amd64 -p default

As noted before, if -p is omitted a ports tree named “default” is assumed.

Passing -c or -C to poudriere bulk will clean all built packages or packages from the list respectively before compiling.


To have crontab update our repositories we use this script:





if [ -z "$JAILNAME" ]; then
    printf "Provide a jail name please.\n"
    exit 1

if [ -z "$PORTSTREE" ]; then

# check that the ports tree exists
poudriere ports -l | grep "^$PORTSTREE" > /dev/null
if [ $? -gt 0 ]; then
    printf "No such ports tree ($PORTSTREE)\n"
    exit 2

## check that there's a list of packages for that jail
if [ ! -f "$POUDRIERE_DIR$JAILNAME.pkglist" ]; then
    printf "No such file (list of packages: $POUDRIERE_DIR$PORTSTREE)\n"
    exit 3

# check that the jail is there
poudriere jails -l | grep "^$JAILNAME" > /dev/null
if [ $? -gt 0 ]; then
    printf "No such jail ($JAILNAME)\n"
    exit 4

# update the ports tree
poudriere ports -u -p $PORTSTREE

# build new packages
poudriere bulk -f /usr/local/etc/poudriere.d/$JAILNAME.pkglist -j $JAILNAME -p $PORTSTREE 

It’s run from /etc/crontab at 1:30 every night:

# update pkg jails
30       1      *       *       *       root    /usr/local/etc/poudriere.d/scripts/ srv_91r_amd64 default

A few hours later the other servers update their local repo.txz and check for updates.

Server – nginx

Nginx is used to make the compiled packages available to the other servers. The default website is set to /poudriere/data/packages and autoindex is turned on, that’s pretty much all there is to it. In addition to serving packages from that website it also mirrors whatever distribution we need from, like auditfile.tbz is also on this website, updated every day using this crontab entry:

# update the audit file for 'pkg audit'
# 15 minutes before @daily
45      23      *       *       *       root    cd /poudriere/data/packages/ && fetch -mq

It’s also used as a forward proxy for and Apparently the guys behind don’t like mirrors for security reasons and are likely to block repeated calls from wget and the like, using a proxy is the recommended method. Nginx is already installed and it does the job just fine. Another reason is that freebsd-update uses phttpget and squid&co might not play well with it. could be mirrored, just like base.


#user  nobody;
worker_processes  4; # adjust to the number of CPU cores

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/;
events {
    worker_connections  1024;

http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    server {
        listen       80 default;
        server_name "";

        #access_log  logs/host.access.log  main;

        location / {
            root   /poudriere/data/packages/;
            index  index.html index.htm;
            autoindex on;

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/local/www/nginx-dist;

    # proxy connections to for
    # servers that don't have internet access
    server {
        listen      80;

        location / {
            proxy_http_version  1.1;
            root                /;

    # proxy connections to for
    # servers that don't have internet access
    # used for pkg bootstrap on new FBSD9 servers
    server {
        listen      80;

        location / {
            proxy_http_version  1.1;
            root                /;

Nginx has three vhosts defined,, and These names need to be defined in’s DNS and they should point to this server.

Clients – pkgng

Since pkg 1.2, the format of the config file changed radically, basically pkg.conf contains only a few general settings and each repository gets it’s own config. Also, pkg now uses VuXML file by default. This article has not been updated to reflect that, partially because it seems that the dust hasn’t settled yet as I’m writing this.

FreeBSD 10 will have pkg installed in base by default, but in FreeBSD 9 it’s still in ports. In order to install it, a bare ‘pkg’ is provided which will download the real files from Nginx is proxy-ing connections to that site, so in order to bootstrap pkg in FreeBSD 9.1, assuming cshrc, the default root shell:

# setenv PACKAGEROOT ""
# pkg

Hopefully it’s clear enough.

Next, edit /usr/local/etc/pkg.conf:


Ready to install packages now:

# pkg update
Updating repository catalogue
repo.txz         100% 1836     1.8KB/s   1.8KB/s   00:00
# pkg install sudo vim-lite

Cron job to check for updates every night:

# check for updated packages
30      5       *       *       *       root    pkg update -q && pkg version -vRL=

Check UPDATING before upgrading packages:

# pkg updating

by default it looks for /usr/ports/UPDATING which has no reason to be there on client computers, but it can easily be given out by Nginx on the pkg builder.

When pkgng is installed it also creates the periodic script /usr/local/etc/periodic/security/410.pkg-audit, which runs pkg audit, a replacement for portaudit. This is why we need auditfile in Nginx and the reason for the PORTAUDIT_SITE setting in pkg.conf.

Mandatory link to pkgng page in FreeBSD’s handbook

Clients – freebsd-update

In /etc/freebsd-update.conf the line


should be replaced with


That’s all, freebsd-update should be working now. Provided the DNS entry is in place, of course.


# check for OS updates
@daily                                  root    freebsd-update cron