RAW DATA: Broadband congestion issues resolved

(TrueNet is contracted to the Commerce Commission to monitor broadband performance. It uses a network of probes attached to 400 volunteers' internet connections to rate ISPs — Editor.)

New Zealand data use soared immediately Netflix launched in mid March 2015, according to Spark, and over the next two months there was not enough capacity for supply to meet demand on some ISPs connections we monitor.  But by May the shortfall in supply was mostly satisfied, just a month and a half after the Netflix launch, demonstrating the highly competitive nature of the ISP market in New Zealand.

Rural Broadband is reported this month with a fresh look at Satellite which, despite high latency, manages to achieve website downloads competitive with ADSL.

Summary of Performance Measures

The evening peak period congestion of the past two months has all but vanished.  

The performance improvement in response to a sudden increase in usage was very quick, showing the highly competitive nature of the NZ Internet supplier market.   For all LFCs and ISPs, ensuring enough capacity is available for a 29% increase in demand over 3 months, as recorded by Spark, would be an expensive and time consuming activity.  

Rural Broadband is the feature of the Quarterly Report this month, in which we present preliminary results for panelists on Satellite. 

 Webpage Download Time

Responsive website browsing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.

TrueNet tests internet browsing by downloading a selection of Live NZ and Australian Webpages, measuring the time to fully download all files on the page.  These pages are changed from time-to-time.

This month the charts have been updated to indicate the amount of variation by time of day, with each "line" having markers showing the Best Hour (least time), the Peak Hour (most time), and the Average of all hours.   We have used average as our standard reporting measure in the past. 

NZ Live Webpages

Amongst the Fibre services, the average peak to off-peak speed varies little, and the averages are very similar to last month.  Notice that the average webpage is delivered in less than 0.2 of a second, ie it is extremely quick and any small changes would never be noticed by most users.  Even ADSL delivery times are very quick.  We plan to change pages to select some more busy pages next month.

ADSL, some VDSL, and Cable have peak hours different to the overall average.  For Cable, local congestion at peak hour drives this; with ADSL or VDSL  it may be due to evening period congestion at a small number of cabinets.   However, it is noticable that Spark and Snap, have a very small variation between peak and off-peak performance.  Looking deeper we see that almost none of their connections show speed variations, unlike other ISPs with similar numbers of panelists.

Australian Live Webpages

Websites downloaded from Australia require a clear path from Auckland to Sydney or Melbourne.  Some ISPs show little apparent congestion (Flip, Slingshot and Vodafone), while with other ISPs we see greater variation in time of day results.  From closer examination of our data we see some differences between Sydney and Melbourne, but we have not researched which ISPs this impacts.

US Live Webpages

Reporting live US Webpages is new this month, and uses a sample of 4 live Webpages.

The spread of performance is much more constrained in the US Webpage results than the NZ and Australian results, most likely due to the distance to US servers.   This is a smaller sample of panelists, but there is no direct evidence of congestion to the US during peak hour for almost all ISPs.   

Our panelist results also show very little difference in Webpage load times between Fibre speeds.  ADSL & VDSL on Snap, Bigpipe and Spark take slightly longer than other suppliers for the same technologies.

Speed (File Download Performance)

For TrueNet's speed tests each panelist's probe regularly downloads a 1MB or 2MB file from Auckland, Wellington, Dallas and Sydney.  

New Zealand tests take the best of each test run from Auckland or Wellington.   International tests take the result from each test run from Dallas, and Sydney. 

We compute a median hourly speed for each panelist's connection, then take the average of all probes to report the actual hourly speed. 

International File Download Speed

TrueNet tests download speeds from our servers in Dallas & Sydney to measure performance to our international neighbours.

We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance. The results are selected from 9pm measurements, in the evening busy period, when congestion is likely to be observed.

FIbre & Cable

Compared to previous results, this month shows greater difference in the 100Mb/s services, with Snappanelists achieving the fastest speed, followed by Vodafone, Orcon, then "Other ISPs". The results from the USA were all very similar. The 30Mb/s fibre results for Sydney are much closer, and the Spark results to the USA are well down on last month, though are still the best in this months results.

ADSL & VDSL

For ADSL and VDSL we compare the speed to Sydney and Dallas against the average speed of the respective ISP.   Vodafone and Slingshot VDSL panelists averaged the fastest for Sydney and Dallas combined. 

Snap and Flip ranked quickest in ADSL from Sydney, but most of the ISPs had roughly similar performances.   Slingshot and Vodafone ADSL panelists continue to achieve the fastest average speed from Dallas. 

NZ File Download Speed

Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the internet during the evening hours of 8pm to 10pm.  TrueNet uses the best of a pair of Auckland and Wellington download tests to calculate the median results by hour over the month for each monitored connection.  We take the average of all median results with each ISP for each hour.

Fibre - Cable - VDSL Comparison

The Time of Day performance of Fibre, Cable and VDSL (all ISPs, including minor ISPs) is shown in Chart 6.  Time of Day variances from all ISPs on VDSL and Fibre have all disappeared compared to last month.  The outstanding change is from Orcon.  A hardly relevant ToD variation is visible on Snap, but this is simply because they increased the average speed at all other times of the day to a record average 100Mb/s for their large number of 100Mb/s fibre connections we monitor.

Vodafone Cable, as is typical, has the highest peak speed of the measured services, with a sharp reduction in speed in the evening, dropping below 50Mb/s as it has in the past few months.  This speed drop is evident on every single connection we monitor.

There is no standout difference amongst the 30Mb/s fibre services, with VDSL very closely shadowing their performance.

The market for fibre products is expanding into many factions with different upload and download speeds being offered by each ISP.  No longer is the original 100/50Mb/s and 30/10Mb/s products standard in the industry.  This change is accelerating, and has led us to introduce a change to the way we plan to report in future.   Chart 7 shows a potential version of our new report.

An extra factor that impacts the final speed at 100Mb/s is that LFC's, the wholesale suppliers such as Chorus, Enable etc, have different wholesale models.  100Mb/s was initially launched as the standard product, but the product label of 100Mb/s was for the wholesale speed so only 94Mb/s was achievable at retail.  Most LFCs corrected this by making the wholesale speed 106Mb/s or more, but some LFCs charge ISPs extra for this service, and some ISPs are yet to purchase the extra speed for earlier customers - many of our panelists.

Chart 7 demonstrates performance as a percentage of the Advertised Speed for both Fibre and Cable.   We are including a grouping of "Other ISPs" outside of those named on the chart, this is a long list of ISPs with one or two panelists each.   Spark and Snap fibre panelists are averaging very near the advertised speed, with Orcon next, Vodafone at about 75%, and then "Other Fibre ISPs" well down at 50%.

Vodafone Cable - especially the 100Mb/s product - exhibits the typical evening performance due to congestion, slowing to just 50% of peak speed during the busy hour. 

DSL Performance by Time of Day (ToD)

VDSL File Download Speed

Compared to last month, the evening performance has improved for almost all providers.  TrueNet time of day variation for VDSL now shows Orcon, Snap and Spark all achieving better than 95% at all times, with Vodafone close behind at 93%.  Only Slingshot continues to have work to do, and we expect to see this resolved quickly.  

We hear concern that a lot of the congestion in the recent few months may have been due to Chorus cabinet congestion.   Users cannot request that a Local Fibre Company (LFC is the wholesaler) such as Chorus, resolve congestion within their network.   Only ISPs (retailers) can approach their LFC provider to correct exchange/cabinet congestion.   Users have no contact with, or influence over wholesalers; they have no idea who's fault it is, or what the fault is, and are very dependant on their ISPs to solve issues that arise beyond their ISPs networks.

ADSL File Download Speed

It is evident that the evening period is much improved on last month where three ISPs slowed to an average of 80% of maximum speed. Spark, Flip (Auckland), Orcon, Bigpipe and Snap all achieved our target of 95% variation in May.

Slingshot did not achieve 95%, but are considerably closer than their result in April, suggesting that they are overcoming any congestion.   The Vodafone dip is the worst, down to just 83%, but even this is better than the 80% achieved in April.

Upload Speed Performance

Upload speed is important to users sending large amounts of data through the internet, or loading files to the Cloud.

TrueNet's upload test sends a 1MB file to our Wellington server, and records the results using a similar method to the download tests, but measured from Wellington only.  Australian panelists connections test to our Sydney server. 

By Technology

The upload speed of Fibre and cable Technologies changes very little month by month, as expected. The 10Mb/s and 20 Mb/s services on average appear to deliver on the advertised speed, while 50Mb/s fibre averages about 40Mb/s.   If you have the choice, selecting VDSL rather than ADSL will improve your experience significantly due to the vastly superior upload speed of about 10Mb/s, compared to ADSL 1Mb/s.  Upload speed is very important for many services, especially video, but also for saving large files into the cloud.

Upload Speed ToD by ISP

ISPs are introducing more upload speeds as options in fibre, so from next month we will change to reporting upload speed as a percentage of advertised speed.  

This month we have included performance of "Other ISPs" at 20Mb/s and 50 Mb/s, which include any other ISPs not explicitly stated in the chart.

Following the trend showing in this report, the Time of Day variations in Upload Speeds are not so pronounced this month.  

Orcon quickly resolved the upload variation seen in the April report, and speeds are now constant at just above 40Mb/s for the 50Mb/s service.  Snap however are showing some congestion issues for the first time, but for most of the day their 50Mb/s product is close to the advertised speed.    "Other" ISPs show some congestion in upload speeds for their 20Mb/s product.

The 10Mb/s upload results are almost identical, with an evening dip for Snap being the only noticeable difference. 

Quarterly Feature - Rural Broadband

Our Quarterly review of rural performance is based on the three months period (March-May 2015). TrueNet reported growing signs of congestion in March and April, and while rural panelists continue to experience an evening slowdown, their results are not greatly different from the previous quarter. Since the last quarter, Spark and Vodafone have announced plans for 4G services to assist in rural areas, and if they can deliver 4G speeds to these areas it could mean a major increase in speed.

The same measurement methodology is used in Rural and Urban results:  the best of Auckland & Wellington 1MB file downloads for speed tests; the same set of NZ, Australian, and US Live Webpages within the limits of test data caps from the panelists.

This period, Satellite is included in the technology mix, giving an indication of how it compares to the existing ADSL/VDSL options in rural areas, although we are limited in the amount of testing we can do on the small caps Satellite users have. We have a small group of five Satellite connections - all from WirelessNation with very similar results for each connection.

TrueNet is working to get sufficient RBI Wireless panelists to include in reports.

Overall, RBI funded ADSL and Satellite based probes performed better than Rural ADSL. Satellite has it's own characteristics by time of day as shown in the results below.  Our sample size is however small for Satellite and just OK for RBI ADSL, however satellite connection results are highly consistent providing confidence that the measured connections represent a typical connection.

Live Webpage Performance

Webpage loading times for NZ, Australian, and USA pages are all shown in Chart 12. As with the Urban data, there were 8 NZ pages. 4 Australian pages, and 4 US pages included in the test. Times are the average time to load a webpage (e.g. averaged over the 8 pages for NZ Webpages). Satellite panelists tend to have lower data caps and hence are not yet included in the international tests.

The results show a trend for Rural ADSL to have a greater spread of Peak to Best hour across all 3 destinations, while Urban ADSL looks quite compact in comparison.

Despite the common concern about Satellite's delay impacting browsing, NZ pages "average"  time for our Satellite sample is comparable to Urban ADSL.  Satellite has a larger spread between best and peak hours than Rural ADSL, and RBI funded ADSL, but the difference is small.   For the international webpages, RBI funded ADSL was quicker with less time of day spread than rural ADSL.

Rural File Download Speed 

We would like to separate out older technology Conklin exchanges from our data to enable a fairer comparison before comparing ISPs, so this quarter we have removed that comparison.

Satellite

We find these connections all have almost identical performance patterns so we believe the small sample is likely to reflect typical performance.  Notice they have a slow period after midnight, this is simply because Wireless Nation allow an increased data allowance from midnight to 7am, and many users set updates for midnight, slowing speeds for an hour or two.

Satellite is a good option if your connection requires a very long copper line to the nearest exchange equipment.  Off-peak speeds are typically 9-10Mb/s and sometimes better, with peak speeds managing to stay above some ADSL connections although dropping to just 3Mb/s on average for our test lines at 9pm.

Rural ADSL RBI Funded

The RBI project is a major government initiative to update older equipment such as Conklin exchanges, and long copper lines typical of rural connections.  This project is having a major impact on the rural community internet access, improving average speeds, but the range of speeds remains large.  If a connection remains a long way (>2km) from the exchange equipment in a roadside cabinet, then performance may still be very slow.

Rural ADSL All

This group of connections include any connection in a rural environment that has not had an upgrade from the RBI project.  Some connections in this group are actually close to the exchange equipment with excellent speeds, but many are not so an average speed of just 4-5Mb/s hides the few with speeds below 1Mb/s as shown in Chart 12.

Time of Day in Rural

A Time of Day comparison of average speed per technology is given in the chart below. Satellite has a striking time of day pattern, noticable in every panelists results. In general Satellite is faster than rural ADSL, but the busy evening period is noticeably affected. Panelists on RBI-funded ADSL averaged higher speeds than those on Rural ADSL, one would suspect this is caused by the improvement in distance from the exchange equipment as a direct result of the RBI investment.