I’ve used the popular, tried and trusted phpBB forum software for the XL Toolbox forum for many years. It’s free, it does its job well, and many people around the internet are familiar with it. However, many times it was also a pain in the butt for me. Spamming is a real problem, and I could not get the countermeasures to work effectively (maybe my fault). Updating was cumbersome, the automatic updater repeatedly tries to update an already updated version, and with a manual update (i.e., installation of a new version), I had to take extra care to set up the site logo again, which is buried several levels deep in the style directories.

It is very well possible that all my issues with phpBB are my own fault, rather than the software’s. However, a little while ago I encountered a neat, modern forum software (here and here) that whetted my appetite: Discourse.

Discourse is a Ruby on Rails application. If you are not familiar with Ruby and Ruby on Rails, it might be worthwhile reading up about them a bit.

Rails applications are not as straightforward to set up and get running as a PHP application such as phpBB would be. The team behind Discourse therefore offer Docker images for Discourse. Docker is a technology that bundles up applications with entire operating systems and all requirements to facilitate deployment. Using the Discourse Docker image, you can set up your Discourse forum in a matter of minutes. Alternatively, you can use Discourse as a hosted service for a fee. The fee generates some income for the Discourse team, and it’s great altruism that they make this software available for free as well.

In my particular case, neither the Docker image nor the paid service were viable options for me.

The Docker image has a high memory demand, I tried it and my server weeped. In addition, the setup script requires port 80 and port 443 to be available. Since I host a number of sites on my machine, I had to circumvent this requirement by hacking the script (possible, but a bit of a nuisance of course).

The paid service was not really an option because I already pay for a server and I cannot affort more costs.

Therefore, I set out to install Discourse as a regular Rails web app on my server. The Discourse team strongly discourage that and offer no support for this. However, I am familiar with Ruby on Rails applications, and I already host one on the very same server.

Here I describe the steps to install and run Discourse on a Ubuntu 14.04.5 box with Apache and Phusion Passenger, which serves as a link between the web server Apache and the Ruby on Rails application, Discourse.

As noted at the end of this post, my low-spec server still could not properly handle Discourse, prompting me to upgrade the machine after all.

Sources

Here are a couple of my most important sources:

• https://github.com/discourse/discourse/blob/master/docs/DEVELOPER-ADVANCED.md
• https://www.babaei.net/blog/2016/04/29/discourse-as-a-blog-comment-service-on-freebsd-without-docker
• https://www.digitalocean.com/community/tutorials/how-to-deploy-a-rails-app-with-unicorn-and-nginx-on-ubuntu-14-04
• http://davidanguita.name/articles/setting-up-a-cheap-redmine-server-using-unicorn-and-apache

Clone the repository and install the bundle

First, clone the Discourse repository. You probably won’t be interested in the entire history, so use the --depth 1 option, which will reduce the download size to about 15 MB.

git clone --depth 1 git@github.com:discourse/discourse.git
cd discourse

Some of the Ruby gems that are required by Discourse need additional packages on the system, notably the PostgreSQL server and its development libraries and Redis server:

sudo apt install postgresql postgresql-contrib libpq-dev redis-server
bundle install

If you want to use the Unicorn web server, install additional packages to enable image optimization by Unicorn:

# Only if you want to use the Unicorn server:
# sudo apt install optipng jhead jpegoptim libjpeg-turbo-progs gifsicle
# curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
# sudo apt install nodejs
# sudo npm install -g svgo

In this post however, I describe how I set it up with Apache and Phusion Passenger on a Ubuntu Server 14.04.5 machine.

Basic configuration of Discourse

Discourse reads the basic site configuration including the database connection from a simple text file in the config directory. A sample is included as config/discourse_defaults.conf. Copy that file to config/discourse.conf and edit the copy as needed. Note: Currently, there is also a file config/discourse.config.sample in the repository. This is a sample Upstart file that has nothing to do with basic site configuration. Also note that the extension of the basic configuration file is .conf, not .config.

The basic site configuration is mostly self explaining. You basically only need to supply the site domain (e.g. ‘forum.example.com’), the e-mail sender domain (e.g. ‘example.com’) and the details of your mail server. If you use your own mail server (as I do, but note that the Discourse folks strongly discourage that because it is very complicated), you don’t even need to alter the SMTP connection details. Otherwise, supply them, too.

It is important to get the database user name right. The database user name should be the user that maintains the Ruby environment (e.g., with rbenv), i.e. it is your own login name that you use to install Ruby and prepare Discourse. This ensures that all Ruby-related commands can be executed all right. Postgres expects the database user name to be the same as the system user name; otherwise it will complain that ‘peer authentication failed’. Avoid trouble with either the Ruby environment or the Postgres server by using your normal login.

Note: You can set developer_emails to your master admin account’s e-mail address, which will give your account super-powers. However, you will still need the interactive Rails console to activate your account and grant it admin rights, because Discourse will use a bogus e-mail address by default to send account activation e-mails, and most mail servers will refuse mails from bogus sender addresses. You can configure the sender address in Discourse’s advanced setup that is stored in the database – but of course, in order to do that, you need to be able to log into Discourse somehow. To get out of this Catch-22 situation, use the Rails console as described below.

Create a postgresql role and database

As mentioned above, the default Postgres setup on Ubuntu takes the credentials of the logged in user to grant access to a database. We want this to be the same user that also issues all Ruby-related commands.

sudo -u postgres createuser daniel # replace 'daniel' with your own user name

Next, create the discourse database along with extensions. Creating the database can also be accomplished with rake db:create, but this won’t take care of the database extensions, causing the subsequent migration to fail.

# In the line below, replace 'daniel' with your own user name
sudo -u postgres psql -c 'create database discourse owner "daniel";'
sudo -u postgres psql -d discourse -c "CREATE EXTENSION hstore;"
sudo -u postgres psql -d discourse -c "CREATE EXTENSION pg_trgm;"

Prepare the database and compile assets

Every time you pull the Discourse repository, you need to migrate the database and precompile the assets (images etc.):

bundle install # don't use 'update', it may cause conflicts
RAILS_ENV=production bundle exec rake db:create
RAILS_ENV=production bundle exec rake db:migrate
RAILS_ENV=production bundle exec rake assets:precompile

Asset precompilation may take a long time (~15 min on my single-core box) and place a heavy load on your server’s CPU. On my new box with dual-core Xeon, 6 GB DDR4 RAM and an SSD, it’s a matter of 3 minutes.

It may be more convenient to Capify the project – I’ll leave that exercise for later.

Configure Apache

At this point, Discourse itself is set up and ready, albeit with no admin user yet. We’ll get to that later. Next, we will need to tell the web server to serve Discourse. I advise to use a dedicated subdomain with a Let’s Encrypt SSL certificate. I initially tried to run Discourse in a subfolder, but this did not work out for me, as many links were broken.

My preferred approach is a Apache web server in conjunction with Phusion Passenger. There are lots of different ways to accomplish the same goal, various web servers and application servers. For me, this combination works well. You can easily find lots of information on the alternatives on the internet.

# /etc/apache2/sites-available/discourse.conf
<VirtualHost *:80>
	ServerName forum.example.com
	Redirect permanent / https://forum.example.com/
	ServerAdmin admin@example.com
</Virtualhost>
<VirtualHost *:443>
	ServerName forum.example.com

  # I keep my aliased SSL certificate definition in one file
  # Include /etc/apache2/sites-available/letsencrypt.inc

  # In Germany, don't log all access due to privacy concerns...
	ErrorLog ${APACHE_LOG_DIR}/discourse-error.log
	CustomLog /dev/null combined

	ServerAdmin admin@example.com
	DocumentRoot /var/discourse/public

	PassengerRuby /home/daniel/.rbenv/versions/2.3.1/bin/ruby
	PassengerLogFile ${APACHE_LOG_DIR}/discourse-passenger.log
	RailsEnv production
	SetEnv RUBY_GC_MALLOC_LIMIT 90000000

	<Directory /var/discourse/public>
			Options +FollowSymLinks
			Require all granted
			AllowOverride FileInfo
	</Directory>

	#SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
	<FilesMatch "\.(cgi|shtml|phtml|php)$">
		SSLOptions +StdEnvVars
	</FilesMatch>
	<Directory /usr/lib/cgi-bin>
		SSLOptions +StdEnvVars
	</Directory>

	BrowserMatch "MSIE [2-6]" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
	# MSIE 7 and newer should be able to use keepalive
	BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
</VirtualHost>

Don’t forget to enable the site and reload the server configuration:

sudo a2ensite discourse.conf
sudo service reload apache2

Run Discourse server as a service

Discourse heavily relies on asynchronous operations. It uses Sidekiq to accomplish this. If Sidekiq is not running, Discourse won’t work properly. So you have to ensure Sidekiq is running at all times. That is one of the reasons why the Discourse team advocate using their Docker images or their paid service.

On the other hand, Linux makes it quite easy to add services that are automatically started on system boot and respawned if they crash. Ubuntu 14.04 uses the Upstart system to manage services; later versions use the more modern systemd. I had the pleasure to play with both because my server needs an Upstart job, and my development laptop (with latest Ubuntu) works with systemd.

Upstart jobs (Ubuntu 14.04 Trusty Tahr)

The Sidekiq examples actually comprise two Upstart jobs. The master job (‘worker’ in Sidekiq’s example) manages a certain number of slave jobs (‘sidekiq’ in Sidekiq’s example). I have set the number of Sidekiq slave jobs to 1 and could have just as well combined the scripts into one, but in case one wants to increase the number of Sidekiq instances, this is an easy way to do.

The master job:

# /etc/init/discourse.conf
# Adapted from
# https://github.com/mperham/sidekiq/blob/master/examples/upstart/sidekiq.conf

description "Manage the Discourse Sidekiq instances"
env NUM_WORKERS=2
start on runlevel [2345]
stop on runlevel [06]

pre-start script
  for i in `seq 1 ${NUM_WORKERS}`
  do
    start discourse-job index=$i
  done
end script

post-stop script
  for i in `seq 1 ${NUM_WORKERS}`
  do
    stop discourse-job index=$i
  done
end script

The slave job:

# /etc/init/discourse-job.conf
# Adapted from
# https://github.com/mperham/sidekiq/blob/master/examples/upstart/workers.conf

description "Single Discourse Sidekiq job"

# This script is not meant to start on bootup, discourse.conf
# will start all sidekiq instances explicitly when it starts.
#start on runlevel [2345]
#stop on runlevel [06]

# change to match your deployment user
setuid daniel
setgid daniel
env HOME=/home/daniel

respawn
respawn limit 3 30

# TERM is sent by sidekiqctl when stopping sidekiq. Without declaring these as
# normal exit codes, it just respawns.
normal exit 0 TERM

# Older versions of Upstart might not support the reload command and need
# this commented out.
reload signal USR1

# Upstart waits 5 seconds by default to kill the a process. Increase timeout to
# give sidekiq process enough time to exit.
kill timeout 15
instance $index

script
exec /bin/bash <<'EOT'
  # Initialize rbenv
  PATH="$HOME/.rbenv/bin:$PATH"
  eval "$(rbenv init -)"
  # Logs out to /var/log/upstart/sidekiq.log by default
  cd /var/discourse
  exec bundle exec sidekiq -i ${index} -e production -q critical -q default -q low --pidfile /var/discourse/shared/tmp/pids/sidekiq-${index}.pid
EOT
end script

(Updated 2017-08-08: It’s absolutely mandatory to tell Sidekiq which queues to process using the -q critical -q default -q low flags, otherwise essential Discourse features such as sending out sign-up e-mail will never work. It’s also a good idea to add the --pidfile flag to facilitate stopping and restarting Sidekiq using Capistrano; see separate blog post.)

Start Discourse’s Sidekiq instance with

sudo service discourse start

It will also be automatically started on system boot.

Systemd job (Ubuntu 16.04 Xenial Xerus)

The systemd service definition files live in /lib/systemd/system. The following has been adapted from the Sidekiq author’s example:

# /lib/systemd/system/discourse.service
# This is for Ruby environments that are managed by 'rbenv'
# See the examples at
# https://github.com/mperham/sidekiq/blob/master/examples
# for other approaches

[Unit]
Description=sidekiq
After=syslog.target network.target

[Service]
# It is important to explicitly load rbenv
# Don't forget to adjust the Ruby version number (here: 2.3.3)
ExecStart=/bin/bash -lc 'export PATH="$HOME/.rbenv/bin:$PATH"; eval
"$(rbenv init -)"; rbenv shell 2.3.3; bundle exec sidekiq -e production -q critical -q default -q low --pidfile /var/discourse/shared/tmp/pids/sidekiq-${index}.pid'

# Must use the proper Discourse directory, of course
WorkingDirectory=/var/discourse/current

Type=simple
User=daniel
Group=daniel
UMask=0002
RestartSec=1
Restart=on-failure
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=sidekiq

[Install]
WantedBy=multi-user.target

Once you have this file in /lib/systemd/system, you can start the service like so:

sudo service discourse start

The service will also be automatically started on system boot.

Prepare the first user

First, register on the site. If you previously entered the e-mail address under developer_emails in the config/discourse.conf, Discourse will automatically make you an admin and send you an activation link using a hard-coded bogus e-mail address noreply@unconfigured.discourse.org. This bogus sender address likely won’t get through your SMTP mail server. Once you successfully log into the forum as an admin, you can change the sender mail address to something that will be accepted by the mail server.

The initial user however needs to be manually activated. Start the Rails console in the Discourse subfolder:

rails c -e production

And activate the first user:

u = User.find_by_id 1
u.activate
u.grant_admin!

Exit the rails console by pressing CTRL+D.

Configure Discourse

Now you have a running Discourse instance and an admin server. Log into your Discourse forum and adjust the database-backed settings as needed. Importantly, you need to enter a proper sender e-mail address for the forum, otherwise no notifications will be sent out to users.

Some troubleshooting

There are of course the usual zillions of things that may go wrong. Here are solutions to a few of the problems that might occur:

• HTTP 500 error when accessing the forum: Make sure you have the package ‘imagemagick’ installed.

• Unable to start Postgres server, get no PostgreSQL clusters exist error? This may occur of your locales were not set properly when postgresql was installed. Fix it by generating the locales, then creating a database cluster like so:

  sudo locale-gen 'de_DE.UTF-8'
  sudo update-locale
  sudo pg_createcluster 9.3 main --start
  

Still resource hungry

It should be noted that Discourse is still rather resource hungry, even if you do not use a Docker container. For my single-core server with 2 GB RAM and conventional (spinning) hard disk, the addition of the Discourse forum was a bit too much.

Because Discourse created not the only bottleneck on the machine (MediaWiki was another one), I migrated to a new machine with dual-core Xeon CPU, 6 GB DDR4 RAM and SSD.