Multiple network benchmarking tests

A case study in managing multiple network benchmarking test projects

Julian Lowe
Mar. 15 2021

Here we are again, two seasoned partners in the mobile network testing space: Infovista, home of TEMS; and umlaut, the global, full-service, cross-industry, end-to-end company that offers advisory and fulfilment services to clients all over the world – including those with network testing needs.

It seems recent in our minds, but it was five years ago that umlaut chose Infovista’s TEMS Investigation for its wide-reaching field work, selected as they felt it offers the best drive testing solution for data collection of air interface measurement.

Last year, having enjoyed sustained success from such a good fit, the two companies expanded their relationship to 5G support.

So, where now for the trendsetting partnership?

In recent months, we’ve taken it even further with TEMS Paragon and TEMS Discovery. The move represents an exciting shift towards more efficiently and effectively managing multiple network benchmarking test projects, which are becoming one of umlaut’s strongest suits.

In this blog post, we meet senior representatives of both companies to talk about:

  • How this project arose;
  • What their shared goals are; and
  • How things are going now that it’s up and running.

Tying in with this short video, our discussions will reveal some of the best-practice thinking on mobile network testing generally and on the specific subject of benchmarking for multiple networks.

But first, a quick note on the tech behind these activities…

Underpinning technology

TEMS Paragon is the first component offered by Infovista to help umlaut achieve operational excellence and efficiency when managing multiple network benchmarking testing projects.

A de facto standard requiring a de facto, high-value, best-in-class benchmarking ‘just-drive’ solution, TEMS Paragon is part of a consolidated platform and vision that support umlaut’s unique scoring methodology, including:

  • TEMS Discovery: highly configurable and automated, this is used for true benchmarking, rollout, analysis and troubleshooting activities;
  • Teams alignment: to break silos and work in an automated agile environment, and to increase operational efficiency; and
  • The promise of a future-proofed solution that supports 5G and seamlessly connects to the umlaut platform.

Enter umlaut

Such a package was always likely to appeal to Hakan Ekmen, CEO Telecommunication at umlaut. Based in Germany, the patent-holding executive has been at the company for 14 years.

What does his company do, we asked?

We’re “a global, full-service, cross-industry, end-to-end company that offers advisory and fulfilment services to clients all over the world,” he explained. “Our interdisciplinary capabilities enable us to add value, quality and focus to their organisations and produce.

Describing umlaut as “an able and agile group of consultancies and engineering firms,” he highlighted that the company employs 4,500 specialised experts and engineers who “provide innovative solutions across all industries and their various intersections as well as serving the public sector and developing organisational cultures, structures and processes.”

It’s all about added value “on top of [our clients’] services, products and entire organizational fabric, all the way through our partnership,” says Hakan. “Our special mark is to help our clients all over the world to change the quality of their technological capabilities and organisational culture for the better.”

The company provides independent technical and management consulting services. This includes network planning and optimization, security audits, international mobile and fixed network benchmarking, device testing and acceptance services. Indeed “umlaut addresses all industries with connectivity needs,” says Hakan. “Our clients include network operators, equipment vendors, device manufacturers, public safety organizations and regulatory authorities around the world.”

But what of network benchmarking?

Network benchmarking the umlaut way

Hakan considers umlaut to be the global industry-leader. “The framework we use is based on a rule set that is applied in all countries,” he says. “It provides a fair, transparent and independent evaluation of all networks. We call it the umlaut Score.”

“Today, more than 200 mobile networks in more than 120 countries are being evaluated by our unique scoring methodology, being widely recognized as the de facto industry standard.”

Tell us more about this score.

“The umlaut score delivers a robust scale to assess network quality and performance,” Hakan explains. “It’s coupled with a unified measurement methodology for true international comparability. Our benchmarks help network operators to demonstrate how well they are delivering wireless connections to consumers, business users and enterprises. Operators, managements and supervisory boards know where they stand in international comparison.”

Does anyone else do it like this?

“Our approach is unique,” Hakan attests. “We combine drive, walk and crowd tests, thus giving operators a full 360-degree view of their networks. We use our analytical skills to slice and dice the data in a way that produces actionable insights into network performance and the user experience, helping operators optimize service delivery and make better, more informed investment decisions.”

How do you keep up with new trends in this space?

“We constantly adjust the testing methodology and scoring models to the progressing technical development,” Hakan explains. “Still, most operators manage to improve their performance. This clearly demonstrates that umlaut’s benchmarking contributes to the improvement of the networks.”

How Infovista helps with umlaut’s work

Managing Director & SVP EAA for Infovista, Faiq Khan and his team were keen to facilitate umlaut’s advances in this space.

How is he helping?

“Through their exclusive use of Infovista’s TEMS portfolio of solutions for testing, benchmarking, and optimizing network service performance and subscriber experience, umlaut is able to efficiently and cost-effectively help mobile operators enhance and evolve their networks to address the growing demand for more data and more connections as the industry matures into 5G,” Faiq explains.

It’s all about building on an existing relationship of more than five years, he adds. The new era into which the two companies have entered together capitalizes on “synergies that will address the growing demands of the industry to provide a competitive mobile network and deliver superior customer experiences,” says Faiq.

“TEMS Paragon is the premier drive test system, optimized for mobile network benchmarking,” he adds. “Through close cooperation with chipset manufacturers and scanner vendors, TEMS Paragon has led the way to 5G and provides support for LTE-A, IOT, VOLTE and VILTE, and verification of 5G radio.”

Continues Faiq: “We’re proud to be a part of the ‘best network’ services provided by umlaut to dozens of mobile operators around the world.”

The nuts and bolts of network benchmarking

Getting down to detail, it’s clear that Hakan shares this positivity, describing the partnership as “providing the most comprehensive network benchmarking services globally”.

He explains that “umlaut is using Infovista TEMS Paragon solution as its primary source of data collection to provide network benchmarking and optimization services, which helps umlaut to improve efficiency and simplify development, project management, and software updates”.

How has umlaut’s programme been enhanced and what does the future hold?

“We’ve extended our testing schedule with a crowdsourced component rolling out our metrics for coverage, connectivity, latency, speed and performance,” he says. “Over the top content services, technologies like carrier aggregation or voiceover LTE and IoT will become more important in the upcoming years. This will make the future benchmark results even more exciting.”

Adds Hakan: “We supplement our proprietary services with Infovista’s complementary network testing and data collection tools – which is a perfect fit. Through this alliance, we are excited to take advantage of the TEMS brand’s highly reputable service quality insights, in addition to Infovista’s international footprint which grants us exceptional market access across the globe.”

Infovista and the umlaut score

And this dovetails with your umlaut score-centric work?

In a word, yes. TEMS Paragon is now umlaut’s “primary network benchmarking solution used to generate the industry standard umlaut score for network testing,” says Hakan. “Through this cooperation, network operators gain the insight and support they need to improve quality of service. With this agreement, TEMS Paragon will become an essential element in generating the industry-standard umlaut score based on its proprietary measurement methodology used worldwide in network testing.”

The umlaut score tells you what type of network quality can be expected in almost any country, as well as what it’s possible to achieve. Not only does it allow everyone to compare their performance over time, but, as the tests are identical, it’s possible to compare with peers anywhere in the world. So you can see how well mobile networks perform in the UK compared with France, or in Germany compared with Canada.

Says Faiq: “From all the data published by umlaut some interesting details can be seen. One that might be slightly counterintuitive is the relative importance that audio quality makes towards the differences in overall score. Audio quality is often used as a litmus test for predicting future network quality problems, as many of the obstacles to good voice quality – latency, jitter, delay, loss – will also affect new features like video and gaming. So it’s interesting to see that these relative performances differ so much.”

Adds Hakan: “The use of TEMS Paragon will also help umlaut achieve significant efficiency improvements through consolidation of disparate tools and simplification in areas such as tool deployment, application development, project management, and software updates.”

Beyond network benchmarking

Of course, neither company is a one-trick pony. What related activities are should we be aware of?

“At Infovista, our portfolio is constantly evolving to make network operations, including network testing, more efficient through remote administration, real-time analytics, and fully-automated capabilities,” says Faiq. “We are committed to umlaut’s success as they use our TEMS and Planet solutions to help mobile operators worldwide provide a competitive mobile network and deliver an exceptional customer experience.”

Expanding on his answer, Faiq adds: “We strongly believe our next generation solutions and our ideas to cooperate in smart network planning with the use of umlaut crowd-source data will strengthen the automation vision of Infovista. We believe automation throughout the network lifecycle will be part of a wider strategic partnership with umlaut.”

This chimes with Hakan, who explains that, in addition to its TEMS adoption, “umlaut started using InfoVista Planet as RF planning software and optimization solutions, and is complementing the planning, testing and optimization lifecycle for its customers.”

“The world is getting more and more connected,” he adds, “and digitization at home becomes an important key for economic success. With our unique approach, umlaut helps to solve these upcoming challenges.”

Infovista is only too happy to help umlaut and companies like them meet their potential in this ever-evolving landscape.

Watch Hakan and Faiq discuss these topics in this short video.