# Intro

This Thanksgiving, I was at my Mom’s and wanted to tackle a lightweight monitoring solution for tracking various Speedtest.net metrics. Having recently moved and changed ISPs – she had experienced particular issues in having multiple outages on a daily basis, throughput issues, and the like.

To help with this, the first thing I did was configure a Raspberry Pi Compute Module 4 with the DFRobot IOT Router Board to act as router, gateway, firewall, dns, etc – and just move her consumer Orbi to just Access Point mode. This is a great little board perfect for lightweight routing – Jeff Geerling has a [great write up on the board](https://www.jeffgeerling.com/blog/2021/two-tiny-dual-gigabit-raspberry-pi-cm4-routers) and I would highly recommend replacing your off-the-shelf router with one if you can get your hands on a Compute Module board.

On top of this, I wanted a keep-it-simple-stupid monitoring solution: one that’s lightweight, not full of countless dependencies, as deploying it as my Mom’s, I would prefer a “set and forget” solution rather than one requiring constant maintenance (I’m extremely paranoid to the point of a failed SD card rendering the box non-functional – so I really don’t want a lot running on the box).

InfluxDB Cloud has a great free tier that can maintain logs for 30 days that’s perfect for these small use cases. This is ideal, not having to host my own dedicated InfluxDB stack just for monitoring – I can just scrape and send a few metrics from the box.

# Speedtest CLI Configuration

Speedtest CLI now has precompiled binaries that are statically linked, and can be run on OpenWrt out of the box – so this is much more ideal than running the Python based CLI, which is less performant and requires a whole stack of dependencies. And luckily, it’s compiled for aarch64, so it works perfectly with the Raspberry Pi Compute Module 4 (luckily, because unfortunately Speedtest has some “secret sauce”, and cannot open source the tool for your own compilation).

Installation is straightforward, download the tarball from Speedtest and untar. I just leave it in /root as I persist this directory for Openwrt – but it can also be added to standard /bin directories:

“`bash

wget https://install.speedtest.net/app/cli/ookla-speedtest-1.2.0-linux-aarch64.tgz

tar xvzf ookla-speedtest-1.2.0-linux-aarch64.tgz

“`

Once you have the binary, run the Speedtest CLI and follow the prompts:

“`bash

./speedtest

==============================================================================

You may only use this Speedtest software and information generated

from it for personal, non-commercial use, through a command line

interface on a personal computer. Your use of this software is subject

to the End User License Agreement, Terms of Use and Privacy Policy at

these URLs:

https://www.speedtest.net/about/eula

https://www.speedtest.net/about/terms

https://www.speedtest.net/about/privacy

==============================================================================

Do you accept the license? [type YES to accept]: YES

“`

First usage of the tool will require you to interactively accept the EULA – after that, the record is stored in your user profile and the CLI can be run without interaction.

To run the tool, I decided to use the .csv output format, because it is extremely easy to parse into InfluxDB, which I will dive into:

“`bash

./speedtest -f csv

“`

# InfluxDB Implementation

Similarly to Speedtest, InfluxDB can also be installed as a single binary. For the Compute Module 4, grab the arm64 binary, and also untar just the same:

“`bash

wget https://dl.influxdata.com/influxdb/releases/influxdb2-client-2.5.0-linux-arm64.tar.gz

tar xvzf influxdb2-client-2.5.0-linux-arm64.tar.gz

“`

Once you have the binary, you need to create an API token, and create a InfluxDB configuration. The easiest way to do this is walked through within the InfluxDB onboarding docs, which will do the work of creating an API token and providing the configuration command like so:

“`bash

./influx config create –config-name onboarding \

–host-url “https://your-platform-url&/#8221; \

–org “your-org” \

–token “api-token” \

–active

“`

Once you have configured the profile, you can verify the configuration by listing profiles:

“`bash

./influx config

Active Name URL Org

* onboarding https://your-platform-url/ your-org

“`

To collect data to InfluxDB, we will create a bucket called speedtest to send speedtest data to:

“`bash

influx bucket create \

–name speedtest \

–description “OpenWrt Speedtest.net Bucket”

“`

With the bucket created, we can begin sending the csv data to the bucket. InfluxDB can utilize an annotated csv format in order to capture tags, measurements, and datatypes that should be indexed by InfluxDB.

So, we need to figure out how the Speedtest csv data should be annotated to properly index. To get this, I ran the csv output with header information, and mapped each field to the InfluxDB data types as so:

“`

#constant measurement,speed

#datatype tag,tag,double,double,double,long,long,long,long,tag,long,double,double,double,double,double,double,double,double,double,double

“server name”,”server id”,”idle latency”,”idle jitter”,”packet loss”,”download”,”upload”,”download bytes”,”upload bytes”,”share url”,”download server count”,”download latency”,”download latency jitter”,”download latency low”,”download latency high”,”upload latency”,”upload latency jitter”,”upload latency low”,”upload latency high”,”idle latency low”,”idle latency high”

“`

The secret sauce here is basically:

– Line 1: Set a constant measurement for all data to be called speed

– Line 2: Define each csv column header as the data type (longs and doubles where decimal). For the server related attributes, I left them as tags to be indexed.

– Line 3: These csv headers came direct from the speedtest cli output

With the annotation complete, the data is ready to send. To run it, I created a simple shell script to capture the annotations, run the speedtest, and send the full output through standard in to the influx command like so:

“`bash

#!/bin/sh

# define annotation headers

a0=”#constant measurement,speed”

a1=”#datatype tag,tag,double,double,double,long,long,long,long,tag,long,double,double,double,double,double,double,double,double,double,double”

a2='”server name”,”server id”,”idle latency”,”idle jitter”,”packet loss”,”download”,”upload”,”download bytes”,”upload bytes”,”share url”,”download server count”,”download latency”,”download latency jitter”,”download latency low”,”download latency high”,”upload latency”,”upload latency jitter”,”upload latency low”,”upload latency high”,”idle latency low”,”idle latency high”‘

# run speedtest

m=$(/root/speedtest -f csv 2>&1)

# echo to influxdb

echo -e $a0″\n”$a1″\n”$a2″\n”$m | /root/influx write –bucket speedtest –format csv –

“`

Finally, to automate the process, I added the following hourly cron job to OpenWrt to run the shell script, and restarted cron to apply:

“`bash

0 * * * * /root/speedtest_influx.sh

“`

After taking some time to let the data populate, I created a simple dashboard to show various speed stats – showing the latest download & upload in a dial, and chart the speeds over time. The result looks like so:


From here, there are many additional enhancements that could be implemented, such as custom alerting upon a certain threshold, or a dead man hook to alert on absence of ingest. For now – I will leave it simply, and just monitor the speed data over time. I do plan to implement additional monitoring capability here when I get to it – perhaps integrate into my existing LibreNMS stack, Healthchecks.io, or capture some logs to my Grafana Cloud instance. As it stands, I’m pretty satisfied to be able these metrics with just two single binaries and a cron job.

For more details on the project & script artifacts, please visit my git repo here:

[https://github.com/zarguell/influxdb-speedtest](https://github.com/zarguell/influxdb-speedtest)

## Overview

If vendor marketing is any indication, “Zero Trust” is the biggest buzzword in security lately. But not without reason; a well implemented security architecture with strong identity verification and zero trust principles is the foundation of good security today. It’s amsuing to me how many breaches in the past few months reported in the news have had the same things; valid account, compromise of phishable 2-factor credentials, compromise of enterprise VPN, establish foothold, move laterally.

As Cloudflare tags their practice of [“dogfooding”](https://blog.cloudflare.com/dogfooding-from-home/) Cloudflare Access, for me “dogfooding” is using Cloudflare Acccess to adopt a zero trust approach to my self hosted services. For a while, I have been using Cloudflare Tunnels (formerly known as Argo Tunnels) with identity providers as sort-of a “front line” identity verification to services. This is cool and all, but it doesn’t fully automate the identification process, and breaks a lot of “app” functionality. For example, built in functionality of Home Assistant or Bitwarden breaks, due to Cloudflare access prompting for authentication. Plus, it opens the door to a further enhanced security model (device posture checks, malware inspection, step up authentication, and more).

In this post, I’ll assume you already have [Cloudflare for Teams](https://blog.cloudflare.com/introducing-cloudflare-for-teams/), with apps configured with public hostnames configured via Cloudflare Tunnels (and perhaps some policies to govern access already). What I want to highlight is the addition of “gateway” access by configuration of managed devices within a team/organization through the “Warp” client.

## Initial Setup

In order to confgiure apps for ZTNA access through the Warp client, you will need clients to utilize Cloudflare’s Secure Web Gateway (SWG). For more information on developing Gateway Policies, see the [Cloudflare Docs](https://developers.cloudflare.com/cloudflare-one/policies/filtering/).

The important part here is not necessarily on what Gateway Policies you implement, but that the Cloudflare SWG must be able to inspect HTTP traffic (ie, DNS inspection through forwarding from a defined location is NOT engough), which [requires the WARP client, with trusted certificate](https://developers.cloudflare.com/cloudflare-one/policies/filtering/initial-setup/http/).

Prior to enrolling device, I would recommend configuring a basic device enrollment policy for the WARP client. This is found under “`Settings -> WARP Client -> Device Enrollment Permissions“`

Similar to access policies, the enrollment policy defines what devices can enroll in your Cloudflare team. Below is a “family members” policy for which I included a list of list of authorized family member emails. In an enterprise scenario, this could be something more aligned to organization policies, such as “allow all *@acme.organization.com.”


You will also want to configure a device posture check under “`Settings -> WARP Client -> Device Posture“`. For my scenario, I have configured the [“Require Gateway”](https://developers.cloudflare.com/cloudflare-one/identity/devices/warp-client-checks/require-gateway/) check. Per Cloudflare, “Require Gateway will only allow requests coming from devices whose traffic is filtered by your organization’s Cloudflare Gateway configuration. This policy is best used when you want to protect company-owned assets by only allowing access to employees” – exactly what we desire to enable ZTNA.


Note that device posture checks open up a door to enhance security; allowing more granular access to services based on risk, including enforcing a certain (up-to-date) OS version, precense of security controls, and more.

## Certificate Trust

Next in the setup process, you will need to add the Cloudflare for Teams certificate as a trusted root certificate in the device you are enrolling for access. Cloudflare provides standard .crt and .pem files in the docs linked [here](https://developers.cloudflare.com/cloudflare-one/connections/connect-devices/warp/install-cloudflare-cert), and should be installed as instructed per device vendor.

Installation of this certificate is necessary to allow for HTTP inspection. Platforms such as Android may provide warning that installation of a certificate will allow third parties to view private data sent to websites. This is true – Cloudflare SWG, when performing HTTP inspection, is an “adversary-in-the-middle” for all HTTP requests on the client. In our scenario, this is desirable, as it is only through HTTP inspection that we can direct apps to private gateway access, as well as enforce DLP controls, malware inspection, lockdown allowed tenants, and more.

## Certificate Pinning

For those familiar with TLS interception, whether through transparent proxy, or SWG platform – these issues are nothing new. But I believe it is worth pointing on the “issue” that TLS interception, and the introduction of “adversary-in-the-middle” has on various services, particularly client applications.

One of the biggest hurdles to properly implementing TLS/HTTP inspection across the board is the wide usage of certificate pinning through various vendors. For large detail on the pros/cons of certificate pinning, I recommend checking out this [Digicert article](https://www.digicert.com/blog/certificate-pinning-what-is-certificate-pinning) on the topic – but the TLDR for this scenario is that certificate pinned applictaions indicate a scenario in which, in some shape or form, an application developer has forced their app to be “pinned” to a particular certificate of theirs. When using WARP gateway, or any TLS interception scenario, the previously trusted certificate inspecting the traffic will be presented. This will completely break app functionality, as the backend will be met with “certificate not trusted”, or similar errors. Even though the intercepting certificate is trusted at the system level, the certificate pinning of the app ignores this trust.

Luckily, Cloudflare provides a simple “one-click” button. When onboarding on Teams, the first “create HTTP” policy that is pre-templated is a “certificate pinning” policy (pictured below).


The policy defines a list of applications correlating to domains known to cause issue with cert pinning, and defines a policy to bypass, or not inspect the traffic. Implementing this HTTP policy will resolve most common certificate pinning issues. I saw most – because it is not an end-to-end list. As you utilizing the gateway – you will notice normal traffic breaking, often due to certificate pinning. By observing logs and testing bypass of domains, you can walkthrough and remediate some of these issues.

## Device Enrollment

Now that you’ve trusted the Cloudflare teams certificate, and understand and have bypassed the requisite apps for certificate pinning, you are ready to enroll the client and begin passing HTTP traffic through Cloudflare gateway.

For small homelab usage, I’m going to assume you’re running manual deployments of the WARP client (although instructions for deploying via Intune, JAMF, etc exist. And I would love to develop a procedure for managed deployment of an organization WARP client through JumpCloud free tier… but that is a topic for another post…)

The Cloudflare Docs for [manual installation](https://developers.cloudflare.com/cloudflare-one/connections/connect-devices/warp/deployment/manual-deployment/) describe the process; either CLI or GUI based per platform. Ultimately, the WARP client will need to be configured to use the “Cloudflare for Teams” login, and you will authenticate to meet the defined device enrollment criteria via configured identity providers, and be granted access to the Cloudflare team via WARP.

## Policy Configuration

So you have your device enrolled, and you’re inspecting HTTP traffic through the gateway. The final step is enable the “requirement” of the gateway to an access application, and use that as a condition to bypass authentication.

For this scenario, I have added a policy “bypass authentication through gateway” for my bitwarden (vaultwarden) Access application.


Within the policy, the only condition I have is the “require gateway”, which was defined under WARP Client device posture checks in the initial setup. Again, depending on risk level, additional device posture checks could be enforced as need, so that even a “managed” device could be denied access to bitwarden if not up to patching, missing antivirus, etc.


After saving the policy – try testing out access. For the Bitwarden scenario, I changed my hostname from an internal DNS name to my public access domain (previously behind an authentication layer). Now, I can sync the Bitwarden app whenever, creating a software defined perimter for access to any of my self hosted services!