RAW DATA: TrueNet February 2015 Broadband Report - ADSL Broadband Speed is More Consistent

(TrueNet is contracted to the Commerce Commission to monitor broadband performance. It uses a network of probes attached to 400 volunteers' internet connections to rate ISPs — Editor.)

Summary of Performance Measures

The best performed in our quarterly report on LFCs is UFF which, in spite of its top placing, has seen its 100Mb/s service drop at 9pm from 101Mb/s in October, to 84Mb/s in January.

Other LFCs 100Mb/s services have also deteriorated in the last 3 months. Enable 100Mb/s service drop at 9pm from 96Mb/s in October, to 82Mb/s in January. Chorus from 91Mb/s in October to 77Mb/s in January

The best overall performance for NZ and Australian Live Webpages in January 2015 is Snap’s 100Mb/s service.

The 9pm speed remains consistent, with ADSL maintaining 9Mb/s for the last three months. The Australian ISP iiNet has slowed this month to 5Mb/s down from 6Mb/s last month.

Fibre 100Mb/s services continue to show a wide spread of speeds at 9pm.
Snap’s 9pm speed has remained consistent over the last 3 months at 92Mb/s, while Orcon has slowed since November from 87Mb/s, to 85Mb/s in January.
Vodafone 100Mb/s service has a consistently poor performance at 9pm which is slowly improving; from 35Mb/s in November, to 47Mb/s in January.  A single 100Mb/s connection on UFF is performing well at a median of 100Mb/s even during peak hour, otherwise all connections are on Chorus.

Webpage Download Time

Slow webpage downloads can be one of the most frustrating experiences when using the Internet.

TrueNet uses a combination of Live NZ and Australian internet Webpages to measure website performance, measuring the full time taken to download all the files on the page.

NZ Live Webpages

The ranking of quickest ISPs in each technology category has shuffled around over the past few months, showing that ISPs are actively competing.

Chart 1: Amongst the ADSL suppliers Slingshot maintains its top spot for the third month, with the time to download a Live Webpage at 4.1 seconds. BigPipe and Orcon have been competing hard for second and third spots over the last two months.

Vodafone is the quickest VDSL supplier which has been the trend for the last three months, although the performance of all VDSL suppliers is improving with the bottom three in November taking 4 seconds, compared to all well under 4 seconds in January.

The results confirm again that there is very little difference between 30Mb/s, and 100Mb/s fibre results in this test.

Australian Live Webpages

Chart 2: The Live Australian Webpage test has similar results by technology compared to NZ Live Webpages, but TrueNet tests fewer pages so the elapsed time is less.

Vodafone 100Mb Fibre moved just in front of Snap this month, but their results are virutally equal. With this one exception, the ranking of every ISP is the same as last month, thus implying the relative performance is more fixed in these international-based tests. This could be for numerous reasons, including peering, latency, and international link capacity.

Speed (File Download Performance)

For TrueNet's speed tests, each panelist's probe regularly downloads a 1MB file from Auckland, Wellington, Dallas and Sydney.

For national tests, the best Auckland, or Wellington download speed result is used from each test run to compute a median hourly speed for each panelists connection.  International tests represent the median hourly speed of each connection from Dallas & Sydney.

International File Download Speed

TrueNet has servers in Sydney and Dallas to measure performance from our international neighbours.
This test provides a comparison that can be used to understand the performance of each ISP when downloading a sample file from servers in similar locations.  We ensure the file is not held in New Zealand (cached) so that the test is international in nature. The speed test results selected are taken at  9pm, in the evening busy period when congestion is likely to be worst.

Fibre & Cable

The Orcon 100Mb/s Sydney test results improved above 60Mb/s in peak hour, re-taking the top position.
Vodafone 100Mb/s has the best performance to Dallas for the third month in a row despite being third for Sydney.

ADSL & VDSL

We compare the 9pm peak average speed as a percentage of the best measured average speed. This measure is used because international performance is very dependent on the speed of the connection, which in turn depends on the distance to the exchange, a factor the ISPs cannot influence.

VDSL -  Slingshot overtook Vodafone as best to Sydney, but Vodafone is still best to Dallas, but down from just over 60% last month
ADSL - Vodafone is the best performing ISP when downloading international websites from both Sydney and Dallas. Both Vodafone and Slingshot improved slightly this month, particularly the Vodafone Dallas result reaching over 90%.

NZ File Download Speed

Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the internet during the peak hours of 8pm to 10pm.  TrueNet uses the best of each pair of Auckland and Wellington downloads to calculate the median results by hour.

High Speed (Fibre - Cable - VDSL) Comparison

TrueNet's comparison of the higher speed broadband offerings in Chart 5, is intended as a visual check of the relative speeds and Time-of-Day performance.

Vodafone Cable 100Mb/s is well ahead of other high speed plans, and is showing consistently higher speeds off-peak than in December, while remaining above 100Mb/s during peak hours. Each of the Vodafone cable services achieve at, or near the advertised service speeds.

The Fibre services offer very consistent performance throughout the day with Snap the fastest of the 100Mb/s providers, and Orcon's performance being fairly similar.
Vodafone’s Fibre 100Mb/s service is achieving far below their advertised speed, with a dip in the evening high usage period, and only a small amount above Cable 50Mb/s results.

All 30Mb/s providers had comparable results, and are very stable by time of day.  

TrueNet has 88 fibre panelists on 8 different service speeds (100/50, 50/20, 30/10 etc).  With the growing diversity of service speeds being offered on fibre, an alternative view is to compare performance against the Advertised Speed.

Chart 6, shows the speed of the larger ISPs as a percentage of advertised speed. Service rates of 30Mb/s, 50Mb/s, and 100Mb/s are included.
By this measure, Spark and Snap have the most consistent Time-of-Day performances, Orcon close behind, Vodafone and Slingshot are much slower. Note that the Vodafone and Slingshot results also show a dip in the evening period, while the top three show greater consistency in speed throughout the day.

VDSL File Download Speed

Chart 7: Another month with a close field vying for best performance, and the combined results are excellent. The service speed stayed above 90% for all ISPs for each hour, and most were well above the 95% mark.

ADSL File Download Speed

A year ago, TrueNet compared ISP performance on ADSL over a 2 year period, noting major improvements by Time-of-Day between a range of 92% to 97% in December 2013 from the 2011 range from 50% to 90+%.  ISPs have improved consistency so much since December 2013 that Slingshot's performance at 93% stands out as low, while 5 out of 8 major providers are achieving better than 97%.

Chart 8: Time of Day performance has improved again this month, with a tightly packed group staying above 96% of max speed at all times of day.

Bigpipe leads the field this month with a perfect 100% of advertised speed, though Flip remains close behind at 99%.

Slingshot’s evening performance has improved compared to recent past performance.

Upload Speed Performance

Upload speed is important when sending files and content to the internet.  Upload speed impacts the time taken to upload a photo or synchronise data with the cloud.

TrueNet's upload test sends a 1MB file to our server in Wellington, and records the results using a similar method to the download tests, but measured from our Wellington server only.

By Technology - Chart 9

• the Fibre 50Mb/s Service improved a small amount from last
  month to a combined average above 44Mb/s

• the 20Mb/s Service average, relative to advertised rate, is very
  similar 

• the 10Mb/s fibre and Cable are near, or equal to the advertised rate

• ADSL averages below 1Mb/s - too slow to move large content efficiently

Upload Speed by ISP

The sample size of some Fibre products has become too small to compare all ISPs on all speed options.  

Chart 10: Snap maintains the lead at 50 Mb/s, and for 20Mb/s service is the only ISP with sufficient volunteers to report. Orcon's 50Mb/s service improved this month to average just over 40Mb/s.

Each of the 10Mb/s ISP Upload services achieved close to the advertised speed, with Vodafone cable results matching the advertised speed.
 

Quarterly Feature - LFC Speed Performance

The government's Ultra Fast Broadband (UFB) programme continues to be rolled-out by Local Fibre Companies (LFCs), and is approaching the half way point (43% as of December). With availability spreading, the uptake is reported at above 11%.

In this report, TrueNet compares download speeds of connections on each LFC (Chorus, Enable, and UltraFast Fibre). It is important to note that performance will be dependent on the service that ISP networks provide to our Volunteers.n  However, the achievable download speed can also be directly affected by LFC networks, and anti-congestion measures.

Overall, speeds are down from our July & Oct reports, although Enable and UFF performances have remained very similar as shown in Chart 11. (we have changed the report to the fastest hour rather than the average of all hours to represent "achievable download speed")
The 100Mb/s service results are significantly down on previous reports.  Most of the change appears to be the major increase in Vodafone 100Mb/s fibre panelists, the Chorus probes perform in the 50 to 80Mb/s range, with one on UFF delivering 100Mb/s.

LFC speed by Time of Day

Chart 12: Fibre services don't show a strong Time-of-Day congestion, but the Afternoon and Evening periods show reduced speeds, and is consistent across each of the LFCs.  The inclusion of 200Mb/s connections that rarely reach the advertised speed further impacts these results.