Index

Publications

Mind Your Outcomes: Quality-Centric Systems Development

March 2022

Published in Computers

Abstract: This paper directly addresses a critical issue that affects the development of many complex distributed software systems: how to establish quickly, cheaply and reliably whether they will deliver their intended performance before expending significant time, effort and money on detailed design and implementation. We describe ΔQSD, a novel metrics-based and quality-centric paradigm that uses formalised outcome diagrams to explore the performance consequences of design decisions, as a performance blueprint of the system. The ΔQSD paradigm is both effective and generic: it allows values from various sources to be combined in a rigorous way, so that approximate results can be obtained quickly and subsequently refined. ΔQSD has been successfully used by Predictable Network Solutions for consultancy on large-scale applications in a number of industries, including telecommunications, avionics, and space and defence, resulting in cumulative savings of $Bs. The paper outlines the ΔQSD paradigm, describes its formal underpinnings, and illustrates its use via a topical real-world example taken from the blockchain/cryptocurrency domain, where application of this approach enabled an advanced distributed proof-of-stake system to meet challenging throughput targets.

Downloadable Paper

A PDF of the paper is available here.

Measuring Network Impact on Application Outcomes Using Quality Attenuation

September 2021

Published in Measuring Network Quality for End-Users, 2021

Abstract: Starting from an objective view of how the network impacts a distributed application's performance, we develop a quantitative measure of network quality based on the notion of `observations' that is tractable and composable. This guides practical measurement implementations based on constructing observable events.

The mathematical properties of this method produce high spatial and temporal localisation from low-impact measurements that can be run continuously. This can be used to both assure end-to-end conformance and localize `breaches' to specific network segments, enabling both service and contractual assurance. In laboratory and field trial settings, it can be used for testing configurations, properties of firmware, differences between different access technologies, etc.

After using this approach in several successful technical engagements, the authors presented the general concepts to the Broadband Forum in late 2018. The work was developed as the QED (Quality of Experienced Delivered) project within the PEAT (Performance, Experience and Application Testing) work stream, culminating in an extensive study document. A series of technical documents followed from this, in particular TR-452.1 from September 2020. This standard has enabled multiple implementations, deployed to solve real problems. Several vendor implementations were demonstrated at Broadband World Forum in 2019. The initiative is ongoing.

Downloadable Paper

A PDF of the paper is available here.

Flexible Formality Practical Experience with Agile Formal Methods

October 2020

Published in TFP 2020: Trends in Functional Programming

Abstract: Agile software development and Formal Methods are traditionally seen as being in conflict. From an Agile perspective, there is pressure to deliver quickly, building vertical prototypes and doing many iterations/sprints, refining the requirements; from a Formal Methods perspective, there is pressure to deliver correctly and any change in requirements often necessitates changes in the formal specification and might even impact all arguments of correctness.

Over the years, the need to “be agile” has become a kind of mantra in software development management, and there is a prevalent prejudice that using formal methods was an impediment to being agile. In this paper, we contribute to the refutation of this stereotype, by providing a real-world example of using good practices from formal methods and agile software engineering to deliver software that is simultaneously reliable, effective, testable, and that can also be iterated and delivered rapidly. We thus present how a lightweight software engineering methodology, drawing from appropriate formal methods techniques and providing the benefits of agile software development, can look like. Our methodology is informed and motivated by practical experience. We have devised and adapted it in the light of experience in delivering a large-scale software system that needs to meet complex real-world requirements: the Cardano blockchain and its cryptocurrency ada.

The cryptocurrency domain is a rather new application area for which no clear engineering habit exists, so it is fitting well for agile methods. At the same time, there is a lot of real monetary value at stake, making it a good fit for using formal methods to ensure high quality and correctness. This paper reports on the issues that have been faced and overcome, and provides a number of real-world lessons that can be used to leverage the benefits of both agile and formal methods in other situations.

Downloadable Paper

A PDF of the paper is available here.

Towards a RINA-Based Architecture for Performance Management of Large-Scale Distributed Systems

June 2020

Published in the Special Issue Post-IP Networks: Advances on RINA and other Alternative Network Architectures of the journal Computers, Vol 9 no. 2.

Abstract: Modern society is increasingly dependent on reliable performance of distributed systems. In this paper, we provide a precise definition of performance using the concept of quality attenuation; discuss its properties, measurement and decomposition; identify sources of such attenuation; outline methods of managing performance hazards automatically using the capabilities of the Recursive InterNetworking Architecture (RINA); demonstrate procedures for aggregating both application demands and network performance to achieve scalability; discuss dealing with bursty and time-critical traffic; propose metrics to assess the effectiveness of a performance management system; and outline an architecture for performance management.

Downloadable Paper

A PDF of the paper is available here.

Towards a performance management architecture for large-scale distributed systems using RINA

June 2020

Published in 2020 23rd Conference on Innovation in Clouds, Internet and Networks and Workshops (ICIN).

Abstract: Performance management of distributed systems is becoming increasingly important for the smooth functioning of society. In this paper we provide a precise definition of performance using the concept of quality attenuation, identify sources of such attenuation, outline methods of managing them automatically using the capabilities of the Recursive InterNetworking Architecture (RINA), and propose metrics to assess the effectiveness of a performance management system.

Downloadable Paper

The paper is available from the IEEE here.

Performance Contracts in SDN Systems

May 2017

Published in IEEE Softwarization eNewsletter.

Abstract: SDN virtualizes connectivity and access to the underlying bearers. This enables more variety of routes and new ways to share the bearers to meet customer demands at lower cost. However customers will need assurances about the fitness for purpose of the delivered service for their critical applications. This requires new ways to quantify their requirements and to measure the delivered service that go beyond simple notions of bandwidth/capacity.

Downloadable Paper

A PDF of the paper is available here. The paper is also available from the IEEE (possibly for a small charge) here.

Challenges of Network Slicing

January 2017

Published in IEEE Softwarization eNewsletter.

Abstract:There are a lot of technology buzzwords about ‘network slicing’, but what do they really mean? In particular, what does a good ‘network slicing’ solution look like? What do we want from it? What capabilities are needed to deliver these requirements?

Downloadable Paper

The paper is available from the IEEE (for a small charge) here.

Assuring QoS Guarantees for Heterogeneous Services in RINA Networks with ΔQ

December 2016

Sergio Leon Gaixas, Jordi Perelló, Davide Careglio, Eduard Grasa, Miquel Tarzan-Lorente, Neil J. Davies and Peter Thompson
Presented at NetCloud 2016, Dec. 12, 2016

Downloadable Paper

A PDF of the paper is available here.

Smart networks for smart grids and smart cities; New network science: RINA and ∆Q

March 2016

Paper delivered at the 2nd International Workshop on Smart Grid Technology and Data Processing 18 March 2016 in Suzhou, China

Abstract—Smart grids, smart cities and the Internet of Things all require connectivity that is substantially more secure, resilient, predictable, scalable and efficient than packet networks are today. New approaches are required: here we consider the Recursive InterNetwork Architecture (RINA) and the quality attenuation calculus (∆Q).

Downloadable Paper

A PDF of the paper is available here.

A Study of Traffic Management Detection Methods & Tools

August 2015

Ofcom has published research from Predictable Network Solutions Ltd on the technical feasibility of detecting broadband "neutrality violations". We applied a rigorous scientific framework in order to independently evaluate the efficacy of existing traffic management detection (TMD) techniques, and TMD in general.

The specific issues of TMD are just one part of the general issue of analysing interactions between components and players in digital supply chains, which we are uniquely equipped to resolve. As such, this framework can be applied to the quantitative analysis of wider broadband policy issues, enabling better-informed management and policy decisions. Whilst this is the first time this framework has been applied to inform national policy, it has been successfully used for over a decade to solve many large scale practical problems.

Further details

A further discussion of the significance of this report is here.

Downloadable Version

A PDF of the technical report is available here.

Engineering Quality of Experience: A Brief Introduction

November 2012

Connecting the quality of user experience to parameters a network operator can directly measure and control is a challenge, but one that is of fundamental importance to the successful, efficient and ultimately sustainable operation of packet-switched networks. This paper presents a simple, scientific, solution to this problem based on a quality, rather than bandwidth, centred approach. Quality attenuation is a property that can be directly measured end-to-end across a network, and can also be tied to application outcomes. This permits a systematic approach to delivering good QoE called AREA™, connecting: user/application Aspirations; network performance Requirements; traffic Execution; and outcome Assurance.

Downloadable Presentation

A PDF of the technical report is available here.

NetHealthCheck™ Packet Flow Performance Optimisation

July 2012

NetHealthCheck is a service to uncover performance impairments and sub-optimal aspects of the way that a network is configured, managed and structured (recognising that optimality has different meanings for different Network Operators and even for the same Network Operator at different points in the market cycle).

Downloadable Presentation

A PDF of the white paper is available here.

BT Operate Case Study

August 2012

By working with PNSol, BT Operate gained a new quantitative understanding of how end-user QoE is affected in the network and also the factors, beyond overall utilisation, that might put its SLA at risk. This gives flexibility to optimise planning rules to extract economic benefits, such as a 30% increase in the utilisation of key resources. BTO estimates this to be worth £2.3M, a significant saving of operating budget, and a ROI >20.

Downloadable Presentation

A PDF of the case study is available here.

Emergent Properties of Queuing Mechanisms

November 2005

Delivering multiple assured services requires that there is sufficient control in the QoS mechanisms to deliver on the per-application network quality (delay and loss) requirements over a range of link speeds.

This report is a worked example which takes several applications, each with different network quality and criticality requirement, assessing how the per-hop behaviours available in today's routers can deliver upon the quality requirements. Each per-hop behaviour is evaluated in terms of its effectiveness in delivering the per-application traffic within the appropriate quality constraints.

Downloadable Technical Report

A PDF of the technical report is available here.

Delivering Predictable Quality in Saturated Networks: Technical Report

September 2003

To assure that a network application behaves predictably under all network loads, there is a need to transport its data packets with at least a minimum quality. Although delivering such quality is not usually an issue for a single application instance on a closed network, adding more instances, applications and users soon raises the potential for the demand on the network to substantially outstrip the supply. The implication is that there is always the potential for saturation to occur - a potential which is all too often realized. The attached whitepaper ("Delivering Predictable Quality in Saturated Networks") explicitly considers saturation of the network and its associated consequences.

Downloadable Technical Report

A PDF of the technical report is available here.