Summary

SRG just completed its 24th 5G NR benchmark study.  For this endeavor we collaborated with Accuver Americas and Spirent Communications to conduct an independent benchmark study of the Rakuten LTE, 5G, and 5G mmWave networks in Tokyo, Japan

Key Highlights from our Study

Our Thanks. We did this study in collaboration with Accuver Americas (XCAL-Solo and XCAP) and Spirent Communications (Umetrix Data). SRG is responsible for the data collection and all analysis and commentary provided in this report.

Our Methodology. Testing took place in April in Tokyo, Japan over a period of three days. We leveraged five smartphones, including two smartphones with DM logging capabilities. We conducted full buffer downlink and uplink data transfers, using HTTP and UDP protocols. Tests involved walk tests and stationary tests. We also did latency / jitter stress tests using a low bit rate UDP data transfer to the Umetrix Data server, which was also located in Japan.

Why it matters. The 5G portion of the the Rakuten network leverages the Open RAN network architecture. From the perspective of this testing, the Open RAN architecture didn't define the results we obtained, but the architecture did enable the inclusion of new vendors, which are not typically associated with today's 5G networks. This study largely focuses on the performance of these respective components and their respective vendors.

Key Features Supported. We found that many of the key features commonly found with today's traditional LTE and 5G networks are supported in the Rakuten network. There was concurrent LTE and 5G data transfers in the downlink and uplink directions (PDCP combining), including with 5G mmWave. Likewise, there was 256QAM and 4x4 MIMO on both LTE and mid-band 5G. There weren't features, such as LTE carrier aggregation - the operator only has a single LTE band - while mmWave uplink was limited to a single 100 MHz channel, as are most networks in the world.

Room for Improvement. Although the network supported many key features, we found opportunities for improvement. Schedule inefficiencies, in particular with the HTTP protocol, resulted in 5G spectral efficiency which lagged that of the operator's LTE network and which was well below what we've experienced in other networks we've tested. LTE performance, when operating in parallel with 5G (PDCP combining) also underperformed relative to LTE when there wasn't a 5G radio bearer. 5G mmWave performance was consistent with the RF conditions, but the RF conditions where we found mmWave signals were never optimal.