QUPER - Guidelines

QUPER - Overview, Publications, Download

Step 1 – Identify candidate QR
When defining QR, it is important to consider relevant features, market segment, competitor, and hardware platform capability. Once such feature has been identified, the consequences for the particular QR should be consider, for example:
• Different mobile phones offered to different market segments may have different requirements on image quality
• A competitor may recently have released a mobile phone with better gaming performance changing the perception of gaming quality
• Today's hardware is not the same as tomorrows, features may run much faster
• Users' evolving expectations, expects better performance in the latest mobile phones

If several QR have been identified, it may not be useful to apply QUPER's steps on all of them. Quality requirements where QUPER may not be relevant include, for example:
• Quality requirements that refers to a certain standard
• Quality requirements where a certain level of quality is always the same, e.g. in mobile TV where 28 frames per second is standard

It may be wise to prioritize the top-n (n might be decided to be, e.g. 10) QR using a defined criterion of “importance” (e.g. using simple priority grouping or based on expert judgment), in order to focus the QUPER efforts on the most important QR. Figure 6 illustrates an example of the considered QR Time shift buffer size for the feature Mobile TV Time Shift.

Step 2 – Define scale and unit
For selected (most important, e.g. top 10 from step 1) QR, define a scale and a measurement unit that can be used to express the level of quality of QR. A scale can for example be “time” and the measurement unit can be “minutes”. Figure 7 shows the defined scale and unit for Time shift buffer size.

Step 3 – Identify reference levels
For each QR, it is useful to identify reference levels based on actual products. Reference levels can be based on competing as well as own products (Qref). Estimates can be given in three forms, depending on how the potential uncertainty in the estimates should be captured:
• Point estimates including a single figure, e.g. 3 minutes
• Interval estimates including a [min, max] interval, e.g. 3-4 minutes
• Triangle distribution estimates including a three-tuple of [low bund, most probable, high bound] figures that show the estimated probability distribution, e.g. low: 3 minutes, high: 5 minutes, probable 4 minutes.

Although three forms of estimates can be given, point estimates was the most common form in previous case studies [13]. The reference levels further calibrate the estimates and provide objective measures to relate the QR to. Figure 8 illustrates added reference levels for Time shift buffer size.

Step 4 – Elicit quality breakpoints
When all reference levels have been identified, for each QR, the market expectations should be defined in terms of the values of quality breakpoints. First, determine the utility breakpoint, which is the lowest acceptable value on the market for a given segment.

How to judge what is lowest acceptable value:
• Is it possible to sell this feature at this quality? If not, then below utility
• Will this quality generate a too high return rate? If yes, then below utility

Then, determine the saturation breakpoint, representing quality levels that are clearly considered excessive by the market.

How to judge what is excessive quality:
• Over this breakpoint will not sell any more products
• Over this breakpoint will not give any market advantages
• Will enhance the user experience

Finally, the differentiation breakpoint somewhere between utility and saturation is determined. Values above this quality level gives market advantage compared to the current products of your competitors.

How to judge differentiation quality:
• The quality will be better than competitors
• The quality can be used in marketing the product

Similar to step 3 (Identify reference levels), estimates can be given in three forms; however, point estimates are the preferred form (see Step 3 for more details). Figure 9 shows the identified quality breakpoints for Time shift buffer size, while Figure 10 illustrates the roadmap view of Time shift buffer size with reference levels and the three (utility, differentiation, and saturation) quality breakpoints.

Step 5 – Estimate cost barriers
When market expectations have been identified, for each QR, estimate the cost in terms of the values of cost barriers (CB). To identify the cost barrier, practitioners with good domain and architectural knowledge may be needed. If possible, identify similar quality requirements' cost barriers from previous projects and use as input.

Although it is possible to identify and estimate one, two, or several cost barriers for each QR, the recommended number of cost barriers is two. The first cost barrier is mainly related to software changes, while a second cost barrier is mainly related to new hardware components, or affects the entire software architecture.

First, estimate the first CB in terms of cost (C1) and at what quality level (Q1) where an increase in quality has a high cost penalty.

How to identify the first cost barrier:
• Q1: May relate to software changes, for example, requires a change in one or a few parts of the architecture, extensive optimization of code, or a major re-work of the code
• Q1: May only affect your own and/or closely related projects' code/architecture
• C1: Represents the cost penalty of raising the quality level from the current quality level (Qref) to Q1

Then, estimate the second CB in terms of cost (C2) and at what quality level (Q2) where an increase in quality has a high cost penalty.

How to identify the second cost barrier:
• Q2: May affect major (if not all) parts of the entire products' architecture
• Q2: The hardware's physical constraints may be used as Q2
• Q2: May require major infrastructure (e.g. code optimization) changes in several projects
• C2: Represents the cost penalty given that the C1 investment has been made, when raising the quality from Q1 to Q2

In Figure 11, cost barriers have been identified for Time shift buffer size, while Figure 12 illustrates the added cost barriers in the roadmap view.

Step 6 – Set candidate requirements
Now, make estimations, propose candidate requirements, discuss and decide actual requirements for coming releases, where estimates can be given in three forms (as in Step 3):
• Point estimates including a single figure
• Interval estimates including a [min, max] interval
• Triangle distribution estimates including a three-tuple of [low bund, most probable, high bound] figures that show the estimated probability distribution

One way to specify a requirements quality interval is by using both a Good and a Stretch target. The actual QR is the interval that is specified by the two targets. It is possible to define the requirement interval in the following ways:
• With both a Good target and a Stretch target
• With only a Stretch target, which means the highest value is specified
• With only a Good target, which means the lowest accepted value is specified

Figure 13 shows the identified target, as an interval using Good and Stretch, for Time shift buffer size. Figure 14 illustrates the roadmap view for Time shift buffer size when the actual requirement (level of quality) for the next release is defined.