úterý 29. září 2015

Central MQTT broker – mosquitto

Let's continue in configuring central MQTT broker. In previous article I described the OpenVPN part. Now is time for MQTT broker itself.

Broker implementation is mosquitto. It is open source software, compliant with MQTT version 3.1 and 3.1.1. First of all, lets install it:

root@buben-vps:~# apt-get install mosquitto

Configuration

Central broker should be configured with following attributes:

  • Listen on VPN interface – This is what I discussed if previous post. To ensure that only authorized clients can access broker, it must listen on tunnel interface only.
  • Client authentication – Even if key owners has access to broker, I like to add username/password protection. Credentials are send in plain text, but the are encrypted in VPN tunnel for outside world.

Broker configuration file should look like this:

#persistance setting
persistence true
persistence_location /var/lib/mosquitto/

# listen on localhost interface
bind_address localhost

# listen on VPN interface
listener 1883 10.9.0.1

# User authentication
password_file /etc/mosquitto/mosquitto.passwd
allow_anonymous false

irst part specifies persistence directory. This is useful when clients uses messages with retain feature and broker is restarted for some reason. Persistence will save retain messages over broker restarts.

Next part defines listen interfaces. Default listener listen on localhost interface, second one listen on VPN interface. I want to listen on localhost for some data processing services which runs on the same machine.

Finally, last part defines user authentication. I use this feature for possibility to grant access to my VPN tunnel for other devices, but ensure that they cannot mess my MQTT data exchange. Unfortunately, authentication is required to clients from all interfaces. It would be great to require authentication for clients at VPN interface only.

Password file

To authenticate MQTT users, we need to store user names and passwords somewhere. Mosquitto comes with mosquitto_passwd utility for managing password files. To create new file issue following commad:

root@buben-vps:~# mosquitto_passwd -c /etc/mosquitto/mosquitto.passwd testuser

Command asks you for password and store it in hashed form in appropriate file. Unfortunately, this utility cannot accept password via command line argument, so using some generated password is quite complicated.

I also noticed, that command can accept -U option, which updates password file to use hashed password instead of plain text. I don't know why, but it re-hash already hashed passwords. This makes this feature little bit useless.

systemd unit

By default, mosquitto doesn't use standard systemd unit. It is started using traditional System V script located at /etc/init.d/mosquitto. You can check this by following command:

root@buben-vps:~# systemctl status mosquitto
● mosquitto.service - LSB: mosquitto MQTT v3.1 message broker
   Loaded: loaded (/etc/init.d/mosquitto)
   Active: active (running) since Tue 2015-09-29 14:41:55 CEST; 9s ago
   CGroup: /system.slice/mosquitto.service
           └─3037 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

I like modern systemd units, which runs services in foreground. So let's create is. Stop current mosquitto service at first:

root@buben-vps:~# systemctl stop mosquitto
root@buben-vps:~# update-rc.d mosquitto remove
root@buben-vps:~# rm /etc/init.d/mosquitto

Now create systemd unit file:

root@buben-vps:~# nano /etc/systemd/system/mosquitto.service

And paste following content:

[Unit]
Description=MQTT v3.1 message broker
After=network.target openvpn-server@buben-vps.service
Requires=network.target openvpn-server@buben-vps.service

[Service]
Type=simple
ExecStart=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
Restart=always

[Install]
WantedBy=multi-user.target

Lets talk about systemd unit configuration directives little bit. Requires directive specifies that this unit needs active network.target unit and openvpn-server@buben-vps.service unit. If any unit listed there is stopped or fails, this unit will be stopped as well. After directive defines activation order. Activation of this unit is delayed until all units listed at After directive are started up.

Both After and Requires directives has same values. network.target tells that service should be started after network related stuff is initialized (such as initialization of TCP/IP stack). This dependecy is much more important for shut-down procedure. It instructs systemd that this service should be terminated before network connectivity. This ensures that broker can nicely close all its connections. openvpn-server@buben-vps.service defines dependency on service which provides VPN connectivity. Mosquitto is configured to listen at VPN tunnel IP address, so this service must be started up before mosquitto gets activated.

Other configuration directives are self explanatory. If you are not sure about their meaning, check systemd documentation.

After unit file is created, reload systemd configuration, enable unit and start it up:

root@buben-vps:~# systemctl daemon-reload
root@buben-vps:~# systemctl enable mosquitto
root@buben-vps:~# systemctl start mosquitto

And that's it. Central MQTT broker is up and running. In next articled I'll descibe building of different kinds of local MQTT brokers.

čtvrtek 17. září 2015

Central MQTT broker – configure OpenVPN server

In following articles I'll describe another part of my IoT project – building infrastructure for data exchange. Remember from my first post, this infrastructure consists of two main parts: central MQTT broker and bunch of local brokers.

First few articles will be about configuring central MQTT broker. It requires a computer with public IP address. Rent a VPS is pretty good choice for this purpose. Operating system which runs on your VPS doesn't really matter. In my case, I'm using Debian 8 with systemd. Configuration steps may differ for other system setup.

Encrypted connection

Security is one of my major concerns. Each local MQTT broker must establish a connection with central broker over SSL encrypted tunnel. There are basically two possibilities:

  1. Configure mosquitto to use encrypted connection.
  2. Establish VPN tunnel between hosts and configure central broker to listen on VPN interface only.

Second way have more benefits. This approach delegates responsibility for encryption into another piece of software. It is also possible to run other services at VPN interface, such as HTTP server for some diagnostic applications. Only downside is that VPN means more complex configuration.

For this tutorial, I use virtual system running in VirtualBox. I assigned two network interfaces to it. One in NAT mode (access to the Internet) and second connected to host-only network. This is for reason that I don't want to use my real public IP addresses in this tutorial. I can access my virtual machine at 192.168.56.102.

Install OpenVPN

First step is install VPN daemon. Best choice for me is to use OpenVPN. It is open source and well known VPN solution with huge community. To install OpenVPN issue following command:

root@buben-vps:~# apt-get install openvpn

And that's it. VPN server software is installed. Now comes the hard part.

Certificates and keys

To establish an encrypted connection with VPN server, we need an encryption key. We also needs some third party authority which verifies, that encryption key belongs to authorized user.

In another words, to establish connection we have to create encryption key for both server and client, create certificate authority (CA) to sign keys and create certification requests. This is generally done using openssl library tool with the same name – openssl. But create all these things with openssl command requires lot of commands with many parameters. For that reason EasyRSA was created, which simplifies this procedure.

Following steps should be done on some secured system dedicated for storing CA and key signing. It is recommended to keep this computer in locked room and completely disconnected from the network. For purpose of this article, I create all files on my /tmp directory.

First of all download EasyRSA from GitHub:

buben@ca:/tmp$ git clone https://github.com/OpenVPN/easy-rsa

Next, change working directory and create configuration file:

buben@ca:/tmp$ cd easy-rsa/easyrsa3
buben@ca:/tmp/easy-rsa/easyrsa3$ cp vars.example vars
buben@ca:/tmp/easy-rsa/easyrsa3$ nano vars

Now edit configuration values. Most important are following variables:

#set_var EASYRSA_REQ_COUNTRY     "US"
#set_var EASYRSA_REQ_PROVINCE    "California"
#set_var EASYRSA_REQ_CITY        "San Francisco"
#set_var EASYRSA_REQ_ORG "Copyleft Certificate Co"
#set_var EASYRSA_REQ_EMAIL       "me@example.net"
#set_var EASYRSA_REQ_OU          "My Organizational Unit"

Just uncomment them and assign them appropriate literals. If you want to omit some of these values (EASYRSA_REQ_OU in my case) fill it with “.” (dot).

After EasyRSA is configured, you can initialize public key infrastructure (PKI):

buben@ca:/tmp/easy-rsa/easyrsa3$ ./easyrsa init-pki

After that, empty pki should look like this:

buben@ca:/tmp/easy-rsa/easyrsa3$ tree pki/
pki/
├── private
└── reqs

Building key files

Now you can build certificate authority and key files. First thing is CA:

buben@ca:/tmp/easy-rsa/easyrsa3$ ./easyrsa build-ca

Note: using Easy-RSA configuration from: ./vars
Generating a 2048 bit RSA private key
.......................+++
.....+++
writing new private key to '/tmp/easy-rsa/easyrsa3/pki/private/ca.key.cAsM68O5hi'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Common Name (eg: your user, host, or server name) [Easy-RSA CA]:buben-ca

CA creation complete and you may now import and sign cert requests.
Your new CA certificate file for publishing is at:
/tmp/easy-rsa/easyrsa3/pki/ca.crt

Command build-ca creates CA files encrypted with pass phrase. You can append nopass argument to create plaintext CA. I recommend you to protect it. Command also asks for common name. It should be some meaningful string, as the prompt suggests.

Next, create server key and certificate request file:

buben@ca:/tmp/easy-rsa/easyrsa3$ ./easyrsa gen-req buben-vps nopass

Note: using Easy-RSA configuration from: ./vars
Generating a 2048 bit RSA private key
.+++
..............+++
writing new private key to '/tmp/easy-rsa/easyrsa3/pki/private/buben-vps.key.3pklh1AD07'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Common Name (eg: your user, host, or server name) [buben-vps]:

Keypair and certificate request completed. Your files are:
req: /tmp/easy-rsa/easyrsa3/pki/reqs/buben-vps.req
key: /tmp/easy-rsa/easyrsa3/pki/private/buben-vps.key

In this case I recommend you to use nopass argument. Otherwise you will have to manually type a pass phrase every time you want to start a VPN server, which is nonsence.

Command again asks for common name. In case of server key, it must match the server hostname. This is the way how the client verifies that it is authenticating with correct machine. If common name doesn't match to server hostname, certificate verification fails.

Next, sign server request with your CA:

buben@ca:/tmp/easy-rsa/easyrsa3$ ./easyrsa sign-req server buben-vps

Note: using Easy-RSA configuration from: ./vars


You are about to sign the following certificate.
Please check over the details shown below for accuracy. Note that this request
has not been cryptographically verified. Please be sure it came from a trusted
source or that you have verified the request checksum with the sender.

Request subject, to be signed as a server certificate for 3650 days:

subject=
    commonName                = buben-vps


Type the word 'yes' to continue, or any other input to abort.
  Confirm request details: yes
Using configuration from /tmp/easy-rsa/easyrsa3/openssl-1.0.cnf
Enter pass phrase for /tmp/easy-rsa/easyrsa3/pki/private/ca.key:
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'buben-vps'
Certificate is to be certified until Sep 12 14:53:09 2025 GMT (3650 days)

Write out database with 1 new entries
Data Base Updated

Certificate created at: /tmp/easy-rsa/easyrsa3/pki/issued/buben-vps.crt

If CA is encrypted, command asks for its pass phrase.

Last step is create Diffie-Hellman parameters for initial key exchange:

buben@ca:/tmp/easy-rsa/easyrsa3$ ./easyrsa gen-dh

Note: using Easy-RSA configuration from: ./vars
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
........................... [ … snip … ] ........++*++*

DH parameters of size 2048 created at /tmp/easy-rsa/easyrsa3/pki/dh.pem

And that's all! PKI hierarchy now should look like this:

buben@ca:/tmp/easy-rsa/easyrsa3$ tree pki
pki
├── ca.crt
├── certs_by_serial
│   └── 01.pem
├── dh.pem
├── index.txt
├── index.txt.attr
├── index.txt.old
├── issued
│   └── buben-vps.crt
├── private
│   ├── buben-vps.key
│   └── ca.key
├── reqs
│   └── buben-vps.req
├── serial
└── serial.old

Configure OpenVPN

Okay, VPN server is installed on the VPS and we also have generated CA and key files on our dedicated system. Now is time to create configuration and start VPN tunnel.

OpenVPN configuration files can't be stored anywhere. Best practice is store it in /etc/openvpn directory. I like to create separate directories for CA, keys and configuration files (both server and client, if necessary). My configuration skeleton looks like this:

root@buben-vps:~# tree /etc/openvpn
/etc/openvpn
├── ca_certificates
├── certs
└── server

Copy CA, DH and key files to VPS:

buben@ca:/tmp/easy-rsa/easyrsa3$ scp pki/ca.crt pki/dh.pem root@192.168.56.102:/etc/openvpn/ca_certificates
buben@ca:/tmp/easy-rsa/easyrsa3$ scp pki/issued/buben-vps.crt pki/private/buben-vps.key root@192.168.56.102:/etc/openvpn/certs

Now create configuration file:

root@buben-vps:~# nano /etc/openvpn/server/buben-vps.conf

And paste following lines:

port 1194
proto tcp
dev tun

ca /etc/openvpn/ca_certificates/ca.crt
cert /etc/openvpn/certs/buben-vps.crt
key /etc/openvpn/certs/buben-vps.key
dh /etc/openvpn/ca_certificates/dh.pem

server 10.9.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt

keepalive 10 120
comp-lzo
persist-key
persist-tun
verb 4

Now you can run a server from console:

root@buben-vps:~# openvpn --cd /etc/openvpn/server --config buben-vps.conf

Systemd unit

By default, openvpn package creates its systemd unit file at /lib/systemd/system/openvpn@.service which looks like this:

root@buben-vps:~# cat /lib/systemd/system/openvpn@.service
[Unit]
Description=OpenVPN connection to %i
PartOf=openvpn.service
ReloadPropagatedFrom=openvpn.service

[Service]
Type=forking
ExecStart=/usr/sbin/openvpn --daemon ovpn-%i --status /run/openvpn/%i.status 10 --cd /etc/openvpn --config /etc/openvpn/%i.conf
ExecReload=/bin/kill -HUP $MAINPID
WorkingDirectory=/etc/openvpn

[Install]
WantedBy=multi-user.target

Personally, I don't like forking services. So I created alternative unit file at /etc/systemd/system/openvpn-server@.service:

root@buben-vps:~# cat /etc/systemd/system/openvpn-server@.service
[Unit]
Description=OpenVPN service for %I
After=network.target
Documentation=man:openvpn(8)
Documentation=https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage
Documentation=https://community.openvpn.net/openvpn/wiki/HOWTO

[Service]
Type=simple
ExecStart=/usr/sbin/openvpn --cd /etc/openvpn/server --config %i.conf
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

This unit file runs openvpn process at foreground, changes its working diretory at /etc/openvpn/server and loads appropriate configuration file. It also doesn't create any status file. If service crashes for any reason, it will be restarted after 5 seconds.

After creating this file, issue following command to reload all systemd units:

root@buben-vps:~# systemctl daemon-reload

Now you can instantiate and enable service to automatically start at boot up:

root@buben-vps:~# systemctl enable openvpn-server@buben-vps.service

And start the service:

root@buben-vps:~# systemctl start openvpn-server@buben-vps.service

Congratulations! VPN server is up and running.You can check for tunnel interface using following command:

root@buben-vps:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:c5:b6:d8 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
       valid_lft 86381sec preferred_lft 86381sec
    inet6 fe80::a00:27ff:fec5:b6d8/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:e5:ef:bf brd ff:ff:ff:ff:ff:ff
    inet 192.168.56.102/24 brd 192.168.56.255 scope global dynamic eth1
       valid_lft 1203sec preferred_lft 1203sec
    inet6 fe80::a00:27ff:fee5:efbf/64 scope link
       valid_lft forever preferred_lft forever
4: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100
    link/none
    inet 10.9.0.1 peer 10.9.0.2/32 scope global tun0
       valid_lft forever preferred_lft forever

Test the tunnel

If you want to test the VPN tunnel, you can configure OpenVPN test client for it. At firs, create key files:

buben@ca:/tmp/easy-rsa/easyrsa3$ ./easyrsa gen-req test nopass
buben@ca:/tmp/easy-rsa/easyrsa3$ ./easyrsa sign-req client test

Now create client configuration skeleton and copy appropriate key files. Let's say at /tmp/vpntest. Configuration skeleton with empty config file should look like this:

buben@test:~$ tree /tmp/testvpn/
/tmp/testvpn/
├── ca_certificates
│   └── ca.crt
├── certs
│   ├── test.crt
│   └── test.key
└── client
    └── test.conf

Note that client doesn't use dh.pem file.

Next edit client configuration file:

buben@test:~$ nano /tmp/testvpn/client/test.conf

And paste following content:

client
dev tun
proto tcp
remote 192.168.56.102 1194
resolv-retry infinite
nobind
persist-key
persist-tun

ca /tmp/testvpn/ca_certificates/ca.crt
cert /tmp/testvpn/certs/test.crt
key /tmp/testvpn/certs/test.key

comp-lzo
verb 4

Once you have your client configured, you can establish a VPN tunnel:

buben@test:~$ sudo openvpn --cd /tmp/testvpn/client/ --config test.conf

And verify that your tunnel is working:

buben@test:~$ ping -c 5 10.9.0.1
PING 10.9.0.1 (10.9.0.1) 56(84) bytes of data.
64 bytes from 10.9.0.1: icmp_seq=1 ttl=64 time=0.413 ms
64 bytes from 10.9.0.1: icmp_seq=2 ttl=64 time=0.412 ms
64 bytes from 10.9.0.1: icmp_seq=3 ttl=64 time=0.413 ms
64 bytes from 10.9.0.1: icmp_seq=4 ttl=64 time=0.425 ms
64 bytes from 10.9.0.1: icmp_seq=5 ttl=64 time=0.394 ms

--- 10.9.0.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.394/0.411/0.425/0.020 ms

To be continued

Now we have created VPN connection for IoT network. In the next article I'll describe MQTT broker configuration.

pátek 4. září 2015

MQTT node software

In last article I described a hardware aspects of my IoT endpoint device. Lets look at it software now.

Software must handle DHT22 sensor for measurements, implement enc28j60 chip driver, TCP/IP stack for it and MQTT client for sending data to network. My first requirement is to use plain C instead of using any Arduino software. Reason for this is it gives me lot more possibilities, smaller code footprint (I assume) and more convenient way to fixing a bugs. Also once a software is done, it can be very easily reused in another projects.

TCP/IP stack

Most challenging part of node software is TCP/IP stack. Fortunately, there is already quite good implementation called avr-uip, which is uIP project ported to AVR platform.

uIP is minimalistic TCP/IP stack written by Adam Dunkels designed for very small embedded systems. It is designed as event-based framework. It requires some learning, but it's not too hard.

My friend from our local hackerspace already made very similar project which uses this TCP/IP stack. So his work can be used as good reference point.

Only downside is that avr-uip is very outdated, so I had to make some changes to it and fix some bugs. But it works now.

Get it work

With working TCP/IP stack is very easy to get data from the AVR over the network. I found some another projects which implement enc28j60 driver, MQTT client and DHT22 driver, forked them and merged them into my software. I had to create man state machine and after some refactoring and bug fixes my node started to sending MQTT data.

You can find the code on the projects GitHub page. If you want to build your own MQTT node, please read attached readme file for configuration instructions.

Future features

I want to create more user-friendly device, which doesn't need to be manually configured for each build. Main features should be some DHCP client code to dynamically assign node IP address.

Another major feature should be DNS client to resolve local broker IP address from DNS server.

This could allow to develop very cheap plug and play device which doesn't require any configuration. One challenging aspect is very limited resources of used MCU, which has only 2kB of RAM.