As telcos roll out Internet protocol television (IPTV) services, they experience new challenges in delivering a quality experience to their customers. Providing high picture quality, clear audio, and instantaneous channel changes becomes more complex when video is delivered using Internet protocol (IP).

This paper focuses on some of the system and design issues associated with providing a comprehensive customer experience through a purpose-built service assurance system for IPTV. It discusses quality of service (QoS) in terms of each of the elements of an end-to-end video and multimedia service chain and how these elements integrate to build a complete system for assuring the customer experience.

As telcos roll out IPTV services, they experience new challenges in delivering a quality experience to their customers. Providing high picture quality, clear audio, and instantaneous channel changes becomes more complex when video is delivered using IP. At a minimum, these essential features must meet the quality levels currently experienced with traditional broadcast delivery formats. And to compete with cable and satellite providers, telcos will strive to differentiate their service with an even higher-quality experience. As telcos add the many multimedia entertainment, information, and communications services—all internetworking and displayed on the same device (a TV screen)—a quality customer experience will be even more critical.

Managing service quality is familiar to telcos in the commercial market, where performance service-level agreements (SLAs) are quite common. Often these SLAs are custom-built to meet the needs of each corporate customer, with financial penalties to the telco if they fail to deliver on the SLA promises. But IPTV is a consumer service, characterized by large scales and the necessity of well-defined and highly automated processes to deliver and maintain the services. The familiar commercial experience may not translate well to the mass market.

IPTV combines and expands the assurance challenges that are already present in delivering Internet voice and data services and cable video services. Like cable, IPTV has all of the issues in subscription management, data rights, and video content quality. Many of these issues are magnified for IPTV because it will provide more personalized video content delivery.

In addition, IPTV raises several new challenges, including locating content to allow efficient bandwidth usage and instant channel-changing capabilities. While great advances have been made in bringing broadband to customers, bandwidth is still a "dear" resource, especially "behind" the access network, where bandwidth and bandwidth demand is aggregated (sometimes called the aggregation network). Personalized TV means unique content views for individual users, which will rapidly consume bandwidth in the shared portion of the network unless content sources and routing are chosen wisely. In IPTV, even channel changing (surfing) is a network function, which means it is subject to the response of a traffic-shared network. The quality of the customer experience can be significantly affected by the service performance in each of these areas, and the service and network must be carefully monitored to ensure the quality experience in these aspects that customers expect.

The quality issue is further compounded by the business objectives and competitive environment that IPTV service providers find themselves in. In almost all cases, IPTV is being offered as an alternative to an existing TV service, whether it is over the air, cable TV, or satellite. IPTV must exceed the quality of these existing and well-understood services, or else newly gained customers will switch back (churn) to their former provider, and it is likely that word of mouth will keep other potential customers away. Since IPTV is perceived to be the main attractor to the broadband multimedia services that telcos especially are counting on for revenue growth, the quality of the IPTV experience is directly linked to corporate financial success.

IPTV services extend the range of classical telecommunications service assurance [1]. The move from analog or time division multiplexing (TDM) production and distribution to packetized systems requires coordinated and sophisticated management, surveillance, and testing. As additional multimedia services are added, the entire service package moves from a one-dimensional customer experience model to a multidimensional experience where there is far greater interdependency of elements.

In addition, video delivery over an IP network raises issues in assuring QoS that are analogous to, but often more difficult than, those of VoIP and data services. IPTV will use broadband access facilities based on ADSL2+, VDSL2, or passive optical networks (PONs) to bring content to the customer premises. Thus, IPTV service assurance also assumes all of the challenges in assuring broadband access networks.

The key to successful management of IPTV service quality is correct configuration, management, surveillance, and testing of the end-to-end service. Feeding this must be a customer experience model describing the success and satisfaction criteria on which telcos will base their network design, business processes, and monitoring models.

As shown in Figure 1, telcos will need to control and monitor elements of an end-to-end video and multimedia service chain consisting of subscriber information, video and other content servers, network services, and set-top boxes (STBs). This is in contrast to classic network-based SLAs that specify network performance and uptime without reference to the user-visible services running above the network. The IPTV service chain is also far more complex than past service architectures because it involves many more places where the customer experience can break down. These include partner links, the storage network delivering an on-demand movie, the authorization/ordering system, the broadband connection, the power, the STB, and the middleware software.

View larger image

Figure 1 High-Level IPTV Service Model

For telcos, the service-level management process for video services is a challenging technical task. This is also true for cable providers who will upgrade their video delivery infrastructure for IP multiservice support. The quality management process applies to the end-to-end video traffic path and, as such, it requires that all network elements on this path should support simple network-management protocol (SNMP) management information bases (MIBs) or similar TL1-accessible data for accumulating management information as in other IP network service management processes. In the case of triple-play services, and especially video delivery, special attention should be given to newly introduced devices such as Ethernet and IP-enabled digital subscriber line access multiplexers (DSLAMs) as well as STBs and residential gateways.

IPTV networks may present an array of problems whose resolution requires consistent multilayer monitoring and analysis functionality. The basic requirements to detect, isolate, and resolve such complex problems are (1) monitoring a multitude of appropriate performance metrics at multiple IPTV network layers and (2) efficient correlation functionality.

For the telco, quality of IPTV services is predominantly centered on delivering a heightened QoS. However, managing service quality in an IPTV network goes beyond what is required for traditional service offerings. The very nature of IPTV means that both subjective video quality measurements and objective network measurements must be considered. In addition to the typical IP network metrics, which include delay, latency, jitter, and packet loss, the following metrics are of importance when managing service quality in IPTV:

• Picture quality—Picture quality is a well-understood issue-from the perspective of the viewer at least. Digital transmission usually results in improved picture quality, but the typical packet network impairments of packet loss and delay, as well as packet jitter, will have new effects on the perception of picture quality.
• Zap Time—Zap time represents the delay encountered when a user selects a particular channel. It is a particularly new concern with TV distribution because of the nature of the IPTV protocol. IPTV uses IP multicasting and the IP group management protocol (IGMP). This avoids the need to distribute broadcast channels to every subscriber and saves bandwidth while allowing a potentially unlimited number of channels to be provided. But this arrangement means that channel switching is performed in the network, not locally at the individual STB. The IGMP join-and-leave delays in the network are now a factor in the customer experience. The number of simultaneous join requests impacts the ability of the network to include the subscriber in the multicast group. The load on the service management server in the network dictates the time it takes to verify the accessibility of the channel. The integrity of the video stream dictates how quickly the STB can begin decoding the channel.
• Control command response time—Control command response time is a key factor in the acceptance of video on demand (VoD) and personal video recorder (PVR) functionality. The responsiveness of the IPTV infrastructure to pause, rewind, and fast-forward control commands is an essential IPTV quality metric.
• STB startup time—STB startup time can be a user annoyance, especially if reboots are caused by loss of network synchronicity. The problem can be compounded when the IPTV infrastructure has to cope with a large number of simultaneous authentication requests.
• Audio-to-video synchronization delay—Audio-to-video synchronization delay, often referred to as "lip synch," becomes more severe as the video and audio undergo further processing through the components of the IPTV network.
• Authentication errors—Authentication errors can occur each time the STB starts up or verifies the validity of the subscriber.

They can also occur when selecting premium channels that require verification that the subscriber is properly subscribed.

Taking a customer-centric view means being able to monitor all the important quality measurements throughout the network in real time. This also means services must be viewed end-to-end and, where required, use nontraditional data for determining the absolute customer experience. Furthermore, telcos must be in a position where they can adapt to new services and service-level offerings through the ability to design their own measurement criteria.

The customer and service provider both want problems to be resolved quickly, with as little human intervention as possible. This can be achieved using automated operations systems that ensure sufficient equipment and network capacity at service provisioning. These systems must include automated network surveillance to spot and correlate troubles to find the root cause before customers start complaining, and have trouble ticketing integrated with automated test and workforce administration to rapidly and accurately fix troubles.

As illustrated in Figure 2, systems should be linked to exploit synergies. Automated service quality management (SQM) integrates with fault surveillance, which integrates with automated test as well as with performance data from network elements and element management systems (EMSs) and analyses operations data, including trouble ticketing and workforce administration.

Figure 2: Integrated Network Management System (NMS) for IPTV Service Assurance

Among the key challenges in an end-to-end IPTV architecture is the ability to correlate the impact of networking and information technology (IT) (storage, video server, application) events and metrics at each of the IP/multiprotocol label switching (MPLS), Ethernet, and access layers to the IPTV service.

Performance Metrics
Some necessary performance metrics for IPTV surveillance can be collected from Layer 2 (gigabit Ethernet and/or asynchronous transfer mode [ATM]) and the IP/MPLS network elements and element management systems, including the network traffic loss/delay/jitter statistics. Broadcast service surveillance requires continuous health monitoring of IP multicast router standard SNMP MIBs and multicast reachability maps. Higher-layer monitoring such as Moving Pictures Experts Group (MPEG) stream analysis and real-time transfer protocol (RTP)/real-time conferencing protocol (RTCP) stream statistics can provide more insight on the video stream transport quality and detect potential problems such as frozen or skipped video frames. User-perceived quality should also be monitored using video picture quality metrics. Other useful service metrics include multicast join/leave latencies and join rates that can be derived by analyzing activity and logs of IPTV servers handling authorization and processing of subscriber channel change requests.

Performance metric violations and hard faults trigger alarms that must be analyzed to report, identify, and troubleshoot existing or upcoming problems. The main functions of service-level surveillance include integrated IPTV performance and fault event correlation, root-cause analysis, and service impact analysis.

To implement such functionality, service models are used that keep the information of dependencies among IPTV network components and events in different layers. As a first step of automation, service models allow the network administrators to navigate on-screen from end-effects to potential root causes. More advanced tools are enabled to automatically correlate large event streams and determine a minimum number of potential root causes. Another usage of service models is the combination of multilayer metrics to evaluate key quality indicators and key performance indicators (KQIs and KPIs). These summarizing indicators allow IPTV network administrators to have an integrated view of the overall service health and inspect specific metrics only when deemed necessary.

Video Quality Metrics
There is not necessarily a simple relationship between the quality of the service provided by the network and the quality of the video as perceived by the end customer. Moreover, elimination of load- and network-related defects has a direct impact on the capital cost of the infrastructure. Thus, to assure video quality and maintain a reasonable cost structure, the service provider may need additional tools for assessing the user-perceived video quality of experience (QoE). QoE can be affected by many things, including original content quality, encoding quality, service delays and errors, picture impairments due to packet loss, and audio. Production and encoding are generally outside the scope of network service delivery, so video quality metrics (VQM) may compare the user-perceived video quality to the quality of the originally encoded video source.

Video mean opinion score (MOS) is analogous to voice MOS scores and is obtained by averaging the subjective ratings of a panel of viewers. Like the analogous voice scores, it is rated on a scale of 1 to 5, where 5 is the best possible score. Obtaining a video MOS score is impractical in an operational network, so service assurance needs to use an objective video quality metric (VQM) that is calculated algorithmically and can be integrated into automated test and analyses. In general, these methods utilize measures such as mean square error (MSE) and peak signal-to-noise ratio (PSNR) quality assessment metrics that are relatively easy to compute. The MSE is the average squared difference between the original and received video. The PSNR is the ratio of the peak signal to the root mean square (RMS) noise observed between a reference composite video signal and a captured test signal, and objectively measures degradation primarily of the luminance signal.

Audio quality has a pronounced effect on perceived video quality. Models combining audio and video quality into a single metric have been presented [1]. A frequent problem with digital video is for audio to arrive out of synchronization with video, called lip synch. Lip synch can be intolerable if it is greater than about 185 ms [4].

Objective VQM measurements [5] may be made by comparing the received video to a full reference version of the original video [6], to a partial reference signal [7, 8], or be calculated using only network quality statistics with no reference. This is a key metric the telco organization needs to decide so enforcement/SQM policies can be set. This needs to be done across several categories.

VQM with No Reference Data
MPEG-encoded video typically sends a group of pictures (GOP) with an anchor intra-frame (I-frame) twice per second; other frames (P- and B-frames) are interpolated starting from the I-frame. The loss of an I-frame header data may lose an entire GOP and cause severe video impairment, while losing packets containing an interpolated frame may be barely noticeable.

Reference [9] discussed the impact of cell or packet loss on video frame loss and reached several conclusions: Packet or cell loss due to network congestion tends to be bunched, so that a disrupted video frame can be modeled as having all of its packets disrupted. The probability of losing different video frame types (I, P, B) is proportional to the proportion of bits carried by each frame type. The number of lost P or B frames is accurately modeled as being uniformly distributed proportionally between GOPs that are useless because they have no I-frame, and good GOPs.

Other references [10, 11] have presented models that input parameters such as the probability of loss events and the number of consecutive lost packets in an individual loss event, and estimate the resulting loss in video PSNR or MSE.

IPTV Network and Service Quality Metrics
Given that we have a model of the effect of lower-layer impairments on video quality, network performance can be monitored directly to ensure video service delivery. Network transmission statistics that can help evaluate received video quality include the following:

• Packet loss ratio (PLR)
• Packet latency
• Packet jitter (equals the deviation from the long-term bit rate)
• Errored packets (may be considered lost)
• Out-of-order packets
• Bit-error rate (BER)
• Loss of frame
• Loss of signal
• Percent error-free seconds (EFS)

In addition to these transport performance metrics, IPTV requires the monitoring and correlation of numerous unique performance indicators. Some key indicators are as follows:

• Broadcast and VoD number of video freeze events
• Broadcast and VoD number of skip events
• Join and leave latency
• Interchannel latency
• Play response delay
• Fast-forward, pause, and rewind response delay
• Broadcast and VoD service quality index

These indicators come from measurements made by many probes at different points in the service architecture, ranging from the head end to the edge router and even the STB. It is the role of the SQM system to pull all these measures together and display a dashboard showing the QoS at aggregation levels ranging from an entire regional network to an individual customer's service. Figure 3 shows how the various metrics come together and relate in the determination of an overall service level.

Figure 3: IPTV Service Quality Metrics Correlation

The pressure is on telcos to maintain their competitiveness and increase their financial success. In response to this pressure, telcos are developing new IPTV service offerings for the consumer market. The challenge is to rapidly deploy these exciting new services while carefully managing the quality of the customer experience. This requires an extension of the SQM practices currently employed in the commercial market to the mass-scale consumer market, which in turn requires additional degrees of automation to manage.

Telcos are looking at new service models and quality metrics that directly relate to the customers' video experience. There are myriad variables to measure and control in order to deliver an uninterrupted stream of broadcasts with a rich clear signal and crisp sound. If telcos neglect the management aspect of this new opportunity, they will fail to build a complete system for assuring the customer experience. One of the determining factors in their continued success will be an overall SQM solution that taps into the larger service fulfillment and assurance operations.