Grandstream VoIP over OpenVPN

asterisk, gxp1625, vpn, settings, config, nat

Grandstream VoIP telephones are very popular because of their high build quality compared to an excellent price. In some cases you want to use an encrypted communication channel between your device and the PBX (e.g. asterisk). The current grandstream firmware includes basic OpenVPN support (client mode, tun) which allows you to tunnel the whole SIP/RTP traffic over an encrypted channel. This is also the best solution to avoid any kind of NAT/routing issues because all devices are directly accessible within the virtual ip subnet.

OpenVPN Server Config#

Use the following (minimal) configuration as template. The important options are set to work with the current grandstream firmware (1.0.4.106). Certificate based authentication is preferred for security (login/password not needed)!

tls-server
dev tunX
topology subnet
server 172.16.1.0 255.255.255.0
port 10111
proto udp

# cert based auth
pkcs12 server.p12

# 1024 and 2048 bit dh params are supported
dh dh2048.pem
keepalive 10 120
script-security 2

# bh-cbc as well as aes-128-cbc are supported by the current firmware
cipher aes-256-cbc

# well sha1 is a bit weak but its set within grandstream firmware
auth sha1

# compression has to be enabled
comp-lzo

tun-mtu 1500
mtu-disc yes

# custom logging
verb 3

# retain TOS flags (VoIP)
passtos

# internal network (VOIP Server)
push "route 10.16.0.1 255.255.255.0"

Notes#

  • Don’t forget to alter your firewall rules. The new OpenVPN subnet needs to be accessible by your VoIP Server (e.g. asterisk) and vice versa
  • Add Quality-of-Service rules to your router which matches the OpenVPN port set above. The traffic should be marked with class EF (realtime, expected forwarding) to avoid package lost. Default VoIP rules will not match because of the encrypted channel!

Using ejs as template-engine within express.js default configuration can be very annoying – you have to pass a dedicated variable set to each response.render() call. But for a lot of tasks it is required to use some kind of global variables in your templates, e.g. page title, resources and much more.

The most reliable solution is a custom template renderer which invokes ejs in the way you want.

Custom Template Engine/Renderer Function#

const _ejs = require('ejs');

// example: global config
const _config = require('../config.json');

// custom ejs render function
module.exports = function render(filename, payload={}, cb){
    // some default page vars
    payload.page = payload.page || {};
    payload.page.slogan = payload.page.slogan || _config.slogan;
    payload.page.title = payload.page.title || _config.title;
    payload.page.brandname = payload.page.brandname || _config.name;

    // resources
    payload.resources = payload.resources || {};

    // render file
    // you can also pass some ejs lowlevel options
    _ejs.renderFile(filename, payload, {
        
    }, cb);
}

Usage#

const _express = require('express');
const _webapp = _express();
const _path = require('path');
const _tplengine = require('./my-template-engine');

// set the view engine to ejs
_webapp.set('views', _path.join(__dirname, '../views'));
_webapp.engine('ejs', _tplengine);
_webapp.set('view engine', 'ejs');

// your controller
_webapp.get('/', function(req, res){
   // render the view using additional variables
   res.render('myview', {
     x: 1,
     y: 2
   });
});

 

Use EnllighterJS with marked

markdown, gfm, javascript, nodejs

marked is one of the most popular markdown parsers written in javascript. It’s quite easy to integrate EnlighterJS within, just pass a custom highlight function as option.

Promised based highlighting#

File: markdown.js

const _marked = require('marked');
const _renderer = new _marked.Renderer();

// escape html specialchars
function escHtml(s){
    return s.replace(/&/g, '&')
            .replace(/"/g, '"')
            .replace(/</g, '&lt;')
            .replace(/>/g, '&gt;');
}

// EnlighterJS Codeblocks
_renderer.code = function(code, lang){
    return `<pre data-enlighter-language="${lang}">${escHtml(code)}</pre>`;
};

const _options = {
    // gfm style line breaks
    breaks: true,

    // custom renderer
    renderer: _renderer
};

// promise proxy
function render(content){
    return new Promise(function(resolve, reject){
        // async rendering
        _marked(content, _options, function(e, html){
            if (e){
                reject(e);
            }else{
                resolve(html);
            }
        });
    });
}

module.exports = {
    render: render
};

 

Usage#

const _markdown = require('markdown');

// fetch markdown based content
const rawCode = getMarkdownContent(..);

// render content
const html = await _markdown.render(rawCode);

 

Comparing the content of two directories binary-safe is a common used feature especially for data synchronization tasks. You can easily implement a simple compare algorithm by generating the sha256 checksums of each file – this is not a high-performance solution but even works on large files!

const _fs = require('fs-magic');

// compare directoy contents based on sha256 hash tables
async function compareDirectories(dir1, dir2){
    // fetch file lists
    const [files1, dirs1] = await _fs.scandir(dir1, true, true);
    const [files2, dirs2] = await _fs.scandir(dir2, true, true);

    // num files, directories equal ?
    if (files1.length != files2.length){
        throw new Error('The directories containing a different number of files ' + files1.length + '/' + files2.length);
    }
    if (dirs1.length != dirs2.length){
        throw new Error('The directories containing a different number of subdirectories ' + dirs1.length + '/' + dirs2.length);
    }

    // generate file checksums
    const hashes1 = await Promise.all(files1.map(f => _fs.sha256file(f)));
    const hashes2 = await Promise.all(files2.map(f => _fs.sha256file(f)));

    // convert arrays to objects filename=>hash
    const lookup = {};
    for (let i=0;i<hashes2.length;i++){
        // normalized filenames
        const f2 = files2[i].substr(dir2.length);
        
        // assign
        lookup[f2] = hashes2[i];
    }

    // compare dir1 to dir2
    for (let i=0;i<hashes1.length;i++){
        // normalized filenames
        const f1 = files1[i].substr(dir1.length);

        // exists ?
        if (!lookup[f1]){
            throw new Error('File <' + files1[i] + '> does not exist in <' + dir2 + '>');
        }

        // hash valid ?
        if (lookup[f1] !== hashes1[i]){
            throw new Error('File Checksum of <' + files1[i] + '> does not match <' + files2[i] + '>');
        }
    }

    return true;
}

await compareDirectories('/tmp/data0', '/tmp/data1');

 

TravisCI: Use custom Node.js version within container based builds

nodejs binary, custom version, second language

Sometime you may need a special version of Node.js or a recent version within a foreign build environment. But in the modern container-based infrastructure it is not possible to use apt to install custom packets which are not whitelisted. As an workaround, you can download pre-build binaries via wget into your build directory and add the bin/ dir to your PATH. This allows you to use any pre-build third party software without installation.

Example: PERL with javascript testcases#

os: linux

language: perl

perl:
  - "5.24"
  - "5.14"

# skip perl (cpanm) dependency management
# install nodejs into home folder
install: 
  # fetch latest nodejs archive
  - wget https://nodejs.org/dist/v8.8.1/node-v8.8.1-linux-x64.tar.gz -O /tmp/nodejs.tgz
  # unzip
  - tar -xzf /tmp/nodejs.tgz
  # add nodejs binaries to path - this has to be done here!
  - export PATH=$PWD/node-v8.8.1-linux-x64/bin:$PATH
  # show node version
  - node -v
  - npm -v
  # install node dependencies
  - npm install

script:
  # syntax check
  - perl -Mstrict -Mdiagnostics -cw rsnapshot
  # run javascript based tests
  - npm test

 

 

TravisCI: Setup MySQL Tables+Data before running Tests

test, mysql, mariadb, travis, continuous integration, before_install

In case your projects make use of external databases like MySQL/MariaDB you need to setup your continuous integration tests with dedicated testcases including application specific database structures. This requires some initial steps to load the database dump before starting the tests. Thanks to travisci.org you do’t need to do this kind of stuff within your application – just use the test configuration!

Travis+MySQL Server#

First of all, we add MySQL Server as service within our .travis.yml file. This initializes a dedicated database instance for testing. Additionally we hook into the before_install action to initialize our database structure. In this example all SQL commands are loaded from an external file located in our test directory.

language: node_js
node_js:
  - "7"
  - "7.6"
  - "8"
services:
  - mysql
before_install:
  - mysql -u root --password="" < test/travis.sql

Initial Database Setup#

Our Test Database structure is definied within a dedicated SQL file in test/travis.sql. It contains all necessary commands to add a new user, create demo database, create demo tables and finally add some test-data.

# Create Testuser
CREATE USER 'dev'@'localhost' IDENTIFIED BY 'dev';
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP ON *.* TO 'dev'@'localhost';

# Create DB
CREATE DATABASE IF NOT EXISTS `demo` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
USE `demo`;

# Create Table
CREATE TABLE IF NOT EXISTS `users` (
  `user_id` int(11) NOT NULL,
  `created_on` timestamp NULL DEFAULT NULL,
  `modified_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `username` varchar(50) DEFAULT NULL,
  `salt` varchar(20) DEFAULT NULL,
  `password` varchar(50) DEFAULT NULL,
  `email` varchar(150) DEFAULT NULL,
  `firstname` varchar(50) DEFAULT NULL,
  `lastname` varchar(50) DEFAULT NULL,
  `dob` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

ALTER TABLE `users`
  ADD PRIMARY KEY (`user_id`);

ALTER TABLE `users`
  MODIFY `user_id` int(11) NOT NULL AUTO_INCREMENT;

# Add Data

pfSense: Persistent OpenVPN Connection to Strato-HiDrive

openvpn, tls-auth, key-direction, smb, encryption

You may use Strato HiDrive as an external storage for team collaboration, internal file-sharing or remote backups. In such cases it can be very useful to establish the secure connection via your corporate UTM Gateway instead on each client. This only requires an additional HiDrive Account which has the ability to connect via VPN (Login allowed via OpenVPN).

This solution requires the HiDrive business plans with advanved protocol featureset!

Step 1 – Download the OpenVPN Config Package#

First of all, you have to download the official openvpn config package directly from the Strato Website. It contains the CA, TLS-Auth Key as well as a poor OpenVPN Config.

Step 2 – Upload the CA#

Go into the pfSense Webinterface and add a new CA – paste the content of the ca.drive.strato.com.crt file into the textarea and save it. The CA is now available within the OpenVPN Client config.

Step 3 – Create a new VPN Client Instance#

Finally goto VPN -> OpenVPN -> Clients and create a new instance.

Connection Settings#

User-Authentication#

You should use a dedicated HiDrive User-Account which has the ability to connect via VPN. Use this account credentials for the OpenVPN User Authentication.

Ciphers#

By default, OpenVPN uses BF-CBC as cipher with SHA1 auth – not AES as set in pfSense GUI.

TLS-Auth#

This is the most tricky/weak part..Strato is using the TLS-Auth Key in bidirectional mode, which is not recommended. Normally the key-direction 0 will be used for servers, 1 for clients and pfSense is not offering an option to change this via the GUI.

But its possible to add the TLS-Auth Key as inline statement under “Advanced Configuration -> Custom Options” without a key-direction (bidirectional by default). Just paste the following code. It contains the TLS-Auth key as of Mai 2017.

Manual TLS-Auth Config

tls-auth [inline]<tls-auth>-----BEGIN OpenVPN Static key V1-----
aa21c5facb594491bc40959f73ff1f79
be815195f2b9b22d3c672e9580db574a
ada6fa5e22a9be42c744b91fe988f4fc
582480ae85ea00fd59b60757d7cf859d
ceb3ca0f35b0ba9af1947521de78a917
7947ffb55a1fef3d800779fac89d2879
fe9f8fd87a99b1f82561ab9b2e91e5c2
e788f92016f5f47ba1ff158897a26a5c
79f627de5c48a9828c0ca4df34b5ef40
eab016669f28e0c84ed6c7974a12ec19
15a16e213d4a70832c85b59ccc74277f
da3309006c90289f1fca2c726eada188
3a86299a865149b7e178ad6235e1a153
775896594296ff24d2ab63f881021a2b
8a23e100df2153e332c0d0bb555185eb
2da94fd9b2b1b950acd075044607eddf
-----END OpenVPN Static key V1-----</tls-auth>

Node.js: Log static file requests with expressjs serve-static middleware

nodejs, express, static, logfile, analytics, statistics, download counter

In most cases, every web-application requires some kind of request logging. Especially package downloads will be counted for statistic purpose. By using expressjs, static content is served by the middleware module serve-static.

To count the successfull requests handled by this module, you can hook into the setHeaders callback which is invoked each time a file is ready for delivering (file exists, file is accessible).

Example#

// utility
const _path = require('path');

// expressjs
const _express = require('express');
let _webapp = _express();

// your statistic module
const _downloadStats = require('./download-counter');

// serve static package files
_webapp.use('/downloads', _express.static(_path.join(__dirname, 'downloads'), {
    // setHeaders is only called on success (stat available/file found)
    setHeaders: function(res, path, stat){
        // count request: full-path, file-stats, client-ip
        _downloadStats(path, stat, res.req.connection.remoteAddress);
    }
}));

 

systemd: Start your Firewall before network interfaces coming up

linux debian, ubuntu, systemd, networking, uptables

There are a serveral “tutorials” and code snippets out there but they wont work on modern systemd versions and may cause fatal errors! In case you want to start your firewall before the network interfaces will be initialized, you have to hook into the special systemd target network-pre.target. It is a passive target which is invoked before any network services has been started.

Additionally, you have to explicit set the DefaultDependencies=no option – otherwise systemd automatically adds dependency of the type After=basic.target to your service and your firewall is invoked AFTER networking has been started!

Systemd Service File#

The following service file assumes that your firewall script is located in /usr/sbin/myfirewall.sh

[Unit]
Description=MyFirewall

# Start before Network Interfaces coming up
Before=network-pre.target
Wants=network-pre.target
After=local-fs.target

# Do not start after basic.target!
DefaultDependencies=no

[Service]
ExecStart=/usr/sbin/myfirewall start
ExecStop=/usr/sbin/myfirewall stop

# Just Execute the shell script
Type=oneshot
RemainAfterExit=yes

Debugging Service Startup#

The systemd-analyze utility provides a really cool way to show the system startup. Finally you should verify that your firewall is executed before networking has started!

# dump the service startup
systemd-analyze plot > /root/systemd_startup.svg

Example#

Startup of the Firewall and Networking

As of WordPress 4.6 it is possible to hook into the wp_send_new_user_notifications action to disable the new user notifications send to site admins or the new user.

// The Parameter "$behaviour" can be set to:
// "none" (no notifications are send); 
// "default" (no changes); 
// "admin" (notifications send to admin only); 
// "user" (notification send to user only); 
// "both" (notifications send to admin + user)
public static function limitNewRegistrationNotifications($behaviour){
    // do nothing
    if ($behaviour == 'default'){
        return;
    }

    // handle user registrations (self registered users)
    remove_action('register_new_user', 'wp_send_new_user_notifications');

    // new users added via wp-admin are created using add_user() -> edit_user() chain, NOT register_new_user()
    // @see https://developer.wordpress.org/reference/functions/add_user/
    remove_action('edit_user_created_user', 'wp_send_new_user_notifications', 10, 2);

    // notifications disabled ?
    if ($behaviour == 'none'){
        return;
    }

    // add custom callback and override the $notify setting with custom behaviour
    add_action('register_new_user', function($user_id) use ($behaviour){
        // trigger notification
        wp_new_user_notification($user_id, null, $behaviour);
    });
    add_action('edit_user_created_user', function($user_id) use ($behaviour){
        // trigger notification
        wp_new_user_notification($user_id, null, $behaviour);
    });
}

 

This Tweak is available as part of the Tweakr WordPress Plugin.