In Part 1 of this post I talked about the history of monitoring the Quality of Experience of the Falkland Islands’ broadband service – The enigma of monitoring the quality of the Falkland Islands’ broadband service – Part 1: Looking backwards.
In Part 2, I will look at the current status of monitoring and look forward to how this could be delivered in the face of the specific challenges discussed in Part 1.
Sure’s current broadband speed test
Let’s start by looking at Sure’s Broadband Speed Test which the company launched around 2014/15. The service was launched specifically to allow its customers to check whether the maximum download speeds of their bronze, Silver or Gold broadband packages had been provisioned correctly.
What I mean by that are the download speeds that Sure advertise as the maximum download speeds associated with each package are as shown below for recent years.
In late 2017 an old Ookla Speedtest download speed test page was replaced by a new utility – a lightweight freeware tool that can be downloaded from GitHub. However, the results of the speed tests run by this software should be used with caution as it can produce inaccurate results as it is uses a cut-corner shortcut that estimates download speed. The results it reports can be browser dependent. It also reports the results in MB not MiB (i.e. using 1000, rather 1024), resulting in an inbuilt error of 4.6%.
Sure’s current broadband speed test.
The freeware comes with a number of usage caveats.
-
- Only modern browsers are supported (IE11, latest Edge, latest Chrome, latest Firefox, latest Safari).
- The ping/jitter test is measured by seeing how long it takes for an empty XHR to complete. It is not an actual ICMP ping. Different browsers may also show different results, especially on very fast connections on slow devices.
Why would Sure use a lightweight speed test tool rather than a full-blown professional one like Global Ookla Speedtest whose results can really be fully trusted?
Incidentally, download speed tests should certainly never be run over Wi-Fi as this might not give accurate results. If I remember rightly, this was not indicated on Sure’s speed test page.
Implementing a download speed monitoring capability going forward.
In Part 1 of this post, I outlined why it is challenging to implement full Quality of Experience monitoring. The first is that there is really are only two companies that can undertake monitoring such as this from an independent perspective and working in the Falklands would be a commercial challenge. However, there is a way forward if a little compromise can be accepted.
It is irrelevant to attempt to measure Sure’s broadband performance on networks that they do not have some responsibility i.e the global Internet. There are two critical aspects of download and upload performance that need to be measured in the Falkland Islands – maximum and minimum speeds.
1 Local network monitoring: Measuring maximum download speeds on Sure’s Falkland Islands network prior to traffic being forwarded to the Intelsat geo-stationary satellite. This is the capability described above and would enable FIG and Sure customers to:
-
- Check that the appropriate download speed for the Internet package they have bought is correct.
- Check that Sure’s local network is not congested and that the Sure’s chosen contention ratios are working as expected.
This maximum download speed metric should be used as an official FIG maximum performance KPI (not just a milestone KPI) that must be met by Sure as they have 100% control of this element of the Internet service.
2 Post satellite monitoring: This is the most crucial metric as this is where all Falkland Islands’ Internet traffic hits the satellite congestion point that slows down IP traffic as explained in Part 1 of this post. This would allow FIG and Sure customers to:
-
- Check that minimum download speeds are being maintained in the face of contention and congestion at certain times of the day managed by Sure with quotas, package access rates and, most importantly by Sure’s active Traffic Shaping equipment.
The satellite upload / download equipment in the UK is located at Whitehill, Oxfordshire and is fully managed by Sure , so as with local network monitoring, this minimum download speed metric could be used as an official FIG minimum performance KPI (not just a milestone KPI) that must be met by Sure even though there is a third party supplier who own the satellite, Intelsat, involved. Sure owns and manages the equipment in Whitehill that connects to their ‘up-stream’ provider, Vodafone (used to be C&W), who then connect to the global Internet.
Maintaining Minimum download speeds performance is vitally important as it is the only way Sure can maintain a Fair Usage policy and ensure that all customers can access the internet even when there is high congestion – even if this results in a very slow service.
The graph below, although based on a rather crude calculation, provides an idea of how download speeds degrade as more and more users concurrently use bandwidth hungry services. It also shows that download speeds would tend to zero without active Traffic Shaping
How can maximum and minimum broadband speed metrics be monitored going forward.
There is a way forward which, most importantly, has the benefits of not involving an expensive 3rd provider of such services and which could be fully managed by Falkland Islands based organisation independently of Sure to varying degrees. This approach would not provide the detailed analysis as supplied by Actual Experience, but would provide easy-to-understand download and upload data.
This is through the use of the well-known Ookla Custom Speedtest service. This was initially used by Sure before they adpoted the freeware software.
Speedtest Custom uses highly configurable, mobile-friendly HTML5-based testing solution and monitoring tools. With this capability a programme can be set up that will:
-
- Analyse and download all historical test result data.
- Filter test results to gain detailed insights.
- Spot trends with easy-to-read graphs.
- Connect to consumers and businesses via a web page so they can see the download speed metrics.
Speedtest Custom can be used to monitor fixed ADSL packages as well as mobile phone download speeds. This would could be used to provide objective data on 4G download speeds.
To quote Ookla: “Get started now and publish your [FIG] branded test within minutes. Choose from a free subscription or unlock the full suite of configuration and analytics tools .
Speedtest Custom Premium comes with advanced dashboards and analytics that deliver insights into your network performance based on the results of real, consumer-initiated tests.”
Yes really, a free service could be set up initially as a trial which can be upgraded for Speedtest Custom Premium. It couldn’t be easier and cheaper and it could all be managed by a local Falkland Islands organisation. Up to three ADSL packages would need to be bought depending on what configurations are chosen and two Ookla servers would need to be hosted.
To avoid the issue of excessive download data gobbling up quotas as discussed in Part 1, the number of download tests must be limited in any 24-hour period as to not exceed quotas. Higher quotas could be agreed with Sure for the monitoring service.
Monitoring local network maximum download speeds
To measure this metric, a private Ookla server is required to be located in Stanley. There are two approaches to this depending on the amount of independence of Sure that is deemed appropriate.
Option 1 with Sure’s involvement: Sure reinstate their Ookla Speedtest server in Stanley.
There are pros and cons taking this approach:
Pros: Only the customer’s ADSL link is in the path of the test to make it 100% accurate.
Cons: Sure would manage the server and thus the monitoring would not be truly independent. A bigger issue is that Sure would not allow a 3rd party to manage a server on their network for security reasons – a standard policy in all carriers.
Option 2 fully independent of Sure: A local organisation would set up a Ookla Speedtest server in Stanley.
Pros: Fully independent of Sure.
Cons: The Ookla server ADSL link is also in the path of the monitoring path leading to the possibility of inaccurate results.
Although it would need to be fully evaluated, I would suggest that Option 2 would be be the optimal path to take.
Monitoring post-satellite minimum download speeds
To measure this key metric, an Ookla server must be installed in the UK. This will enable the monitoring of the Sure Network, the Intelsat satellite link and Sure’s equipment in the UK.
Option 1, with Sure’s involvement: Sure would install a server in their equipment rack located in Whitehill earth station in the UK.
Pros: Only the customer’s ADSL link is in the path of the test.
Cons: Sure would manage the server and thus the monitoring would not be truly independent. A bigger issue is that Sure would not allow a 3rd party to manage a server on their network for security reasons.
Option 2, fully independent of Sure: An Ookla server would need to be installed in a UK datacentre with minimum delay to Sure’s equipment in the Whitehill earth station. As Vodafone (this used to be Cable & Wireless) is the upstream network provider for Sure, using one of their data centres would be optimal.
Pros: Fully independent of Sure.
Cons: The UK Vodafone link is also in the path of the test leading to the possibility of inaccurate results.
Again, although it would need to be fully evaluated, I would think Option 2 would be the optimal path to take.
Conclusions
FIG has already has much experience of monitoring the islands’ broadband services and this should provide a good foundation to move forward. Setting up a broadband monitoring scheme as outlined here could be achieved quite quickly independently of Sure using a Falkland Islands organisation. It would not be that expensive to provision and would have low annual maintenance operating expenditure; however up to three broadband ADSL links and datahosting in the Falkland Islands and the UK would need to be paid for.
The local organisation could manage the Ookla servers and software and provide on-line public access to the data together with publishing a monthly report to stakeholders. Moreover, the Ookla services could made public or entirely private to the Falkland Islands. The latter option would be better to minimise additional Ookla traffic on the satellite.
All a monitoring programme achieves is to provide objective data which can trigger restorative activities. Without these, the expense of investing in independent monitoring of the broadband service would be questionable.
In particular, monitoring the broadband service would throw light onto ‘events’ where download speeds have deteriorated for some reason. Each of these must be investigated by a FIG resource to fully understand the cause and to make sure the changes are made to prevent a reoccurance.
There are so many benefits, it would be good to introduce independent broadband monitoring again sooner rather than later.
Chris Gare, March 2019 Copyright: OpenFalklands
