Accounting for respondent’s preference uncertainty in choice experiments

ABSTRACT

Preference uncertainty is an important aspect affecting respondents’ choices and attribute valuation. However, elicitation of preference uncertainty and its modelling is strongly restricted within choice experiments. This paper applies modelling techniques to account for the preference uncertainty data to evaluate road traffic noise. The paper argues that modelling the preference uncertainty data to examine the error structure can shed significant light on the potential causes of preference uncertainty. The results also reveal that accounting for preference uncertainty data within modelling can have important implications for the valuation exercise. It is found that the nested logit model can examine significant correlation between similar preference certainty levels arising from choice-set characteristics while the error components logit model can be used to examine the effect of inherent respondent uncertainty and stochastic factors on preference uncertainty. The paper therefore recommends treating and accounting for preference uncertainty within choice experiments and thereby examine its impact on any subsequent valuations.

KEYWORDS:

• Preference uncertainty
• road traffic noise
• environmental valuation
• nested logit
• error components logit

Introduction

Discrete choice models commonly elicit respondents’ preferences by assuming that respondents have known and consistent preferences which can be modelled using standard error assumptions. However, respondents are known to be unsure of their preferences or treat them strategically (March 1988), causing inconsistency in preferences. This respondent preference uncertainty can also be caused by choice-set characteristics in the choice experiment (CE) (Jia, Luce, and Fisher 2004; Kosenius 2009; Olsen et al. 2011). Respondent preference uncertainty has significant implications for the willingness to pay estimate and thus affects valuation (Akter, Bennett, and Akhter 2008; Dekker et al. 2016). While choice experiment theory assumes that individual’s preferences are known with certainty (Ben-Akiva and Lerman 1985), it is nonetheless important to allow respondents to state any level of preference uncertainty they might have during the decision-making process.

CE commonly ask respondents to choose one of the alternatives from the choice set although several causes of respondents’ preference uncertainty have been outlined in the literature. The causes of preference uncertainty have been argued to be due to descriptive and procedural invariance, low utility differences across the given attributes or alternatives, respondents’ inability to deal with the choice scenario and respondents’ latent decision uncertainty (Slovic 2000; Restle 1961; Fowkes and Wardman 1988; Dekker et al. 2016).

In order to elicit respondents’ preference uncertainty, several preference elicitation techniques have been applied in the contingent valuation literature. These include follow-up questions, polychotomous choices and test-retest method (Li and Mattsson 1995; Akter, Bennett, and Akhter 2008; Broberg and Brannlund 2008; Boman 2009; Brown et al. 2006). The Multiple Bounded Dichotomous Choice (MBDC) question, known as the polychotomous choice (PC) is a commonly applied method to elicit respondents’ level of preference certainty along with post-decisional certainty measure, within contingent valuation method. The MBDC or PC in the contingent valuation literature elicit respondent’s preferences over a five point scale from ‘Definitely Yes’ to ‘Definitely No’ for a particular willingness to pay bid (Boman 2009). The post-decisional certainty method requires the respondent to state their level of certainty to the willingness to pay amount following a yes/no question to the WTP bid (Li and Mattsson 1995). In CE however, relatively fewer studies can be found that explicitly allow for respondents to state their level of preference certainty. Studies that have elicited preference uncertainty, have asked respondents to indicate their level of certainty (on a five- or six-point scale) after choosing their preferred alternative (Olsen et al. 2011; Lundhede et al. 2009).

The preference information gathered from the preference elicitation techniques has been modelled by recoding, elimination or explicitly treating the preference uncertainty data by incorporating a heteroskedastic scale parameter to account for the variance (Lundhede et al. 2009). However, fewer studies are found in the choice experiment literature that analyse the preference uncertainty data without recoding and elimination. This paper examines the effect of different preference elicitation techniques on the ability to capture preference uncertainty data, along with the effect of different modelling techniques for treating the preference uncertainty data. Two types of preference elicitation techniques are applied in this paper – the one stage five-point Likert scale and the two stage Likert elicitation method.

Two main reasons for respondents’ preference uncertainty are assumed in this paper: respondents’ inherent attitude towards the good and their level of choice commitment based on known choice-set characteristics and, preference uncertainty caused by random effects or stochastic factors. Based on the reason of respondents’ preference uncertainty, different modelling techniques are considered to be more suitable. The nested logit (NL) and error components logit (ECL) models are used to analyse the preference data in this paper.

Methods

In order to examine the effect of different preference elicitation techniques on their ability to capture preference information, a choice experiment survey was conducted in Telheiras, Lisbon, in the context of residential choice to evaluate road traffic noise. The choice experiment survey followed from a previous noise valuation survey conducted by Arsenio (2002) in the same area. The Telheiras area consists of three main traffic roads, Avenue Norton de Matos, Eixo Norte Sul and Avenue Padre Cruz, which surround the residential area, with resulting noise levels at some parts of the main traffic road higher than 70 dB(A) (Dave 2011). The residential area consisted of blocks with some façades facing the main traffic road and other façades facing a quieter area. The residential area thus provided a good study site for conducting a choice experiment with varying levels of noise attribute within the survey. While Arsenio (2002) conducted a computer-based personal interview with pivotal design in the same area, the current study used a face-to-face interview with two different methods of attribute representation and three methods of preference elicitation.

Residential ‘view’, ‘noise’, ‘sunlight’ and ‘housing service charge’ were selected as the main attributes for the choice experiment survey. The choice experiment survey was conducted sequentially based on the method of attribute representation. Within the first phase of the choice experiment survey, residential ‘view’, ‘noise’ and ‘sunlight’ attributes were represented based on the location of the apartments within the block. Apartments at the third and sixth floor of the residential block, on the façade facing the main traffic road as well as on the façade opposite the main traffic road were selected as attribute levels for this phase of the choice experiment survey. The ‘view’, ‘noise’ and ‘sunlight’ attributes with the location representation method were thus represented as – sixth floor on the façade facing the main traffic road (6F), sixth floor on the façade opposite the main traffic road (6 T), third floor on the façade facing the main traffic road (3F) and third floor on the façade opposite the main traffic road (3 T). The attribute levels for the first phase of the choice experiment were based on the pilot survey as well as the previous study conducted in the area (Dave 2011).

Respondents during the first phase of the survey were asked to provide a numeric rating and a linguistic perception of the attribute levels for each of the ‘view’, ‘noise’ and ‘sunlight’ attributes. Based on the numeric ratings and linguistic categories obtained for the attribute levels during the first phase of the survey, the linguistic levels for the ‘view’, ‘noise’ and ‘sunlight’ attributes were formed for the second phase of the choice experiment. The second phase of the choice experiment survey thus comprised of the linguistic representation of residential ‘view’, ‘noise’ and ‘sunlight’ attributes within the survey. For the linguistic representation method, the attribute levels were defined as ‘good’ and ‘neither good nor bad’ for the ‘view’ attribute, ‘noisy’, ‘neither noisy nor quiet’ and ‘quiet’ for the ‘noise’ attribute and ‘very good’, ‘good’ and ‘neither good nor bad’ for the ‘sunlight’ attribute. The ‘charge’ attribute for both the location and linguistic representation methods was defined as Euro/month. An orthogonal fractional factorial main effects design was generated for both the phases of the choice experiment, using the attribute level differences. In order to eliminate dominant choice problem in the second phase of the survey, variations were made in the sign of the housing service charge attribute difference for some of the scenarios. A total of sixteen choice scenarios were generated for each of the choice experiment surveys. For both the phases of the choice experiment survey, binary choice, one stage Likert question and two stage Likert question were used as preference elicitation methods in the survey.

The binary choice question asked the respondents to choose between alternatives A and B. The one stage Likert elicitation method, provided respondents with a five-point Likert scale, where respondents were asked to indicate their level of choice certainty over a five-point scale of ‘Definitely A’, ‘Probably A’, ‘Uncertain’, ‘Probably B’ and ‘Definitely B’. In case of the two stage Likert elicitation, respondents were asked to choose between alternatives A and B, and then indicate their level of choice certainty as – ‘Absolutely certain’ or ‘Not so certain’.

A split-sample design was adapted where the choice experiment questions comprised of half binary and half one or two stage Likert question. One half of the sample received binary and one stage Likert questions while the other half received binary and two stage Likert questions. Within each of the sub-samples, the ordering of the binary and Likert elicitation methods were changed for half of the respondents. This was done to avoid any ordering bias. Figure 1 provides an overview of the survey methodology used along with the different phases and experiments of the survey:

Figure 1. Survey methodology – phases and experiments.

The survey was thus divided into four different experiments – the first half of the first sub-sample received eight binary and eight one stage Likert elicitation questions while the second half of the sub-sample received eight one stage Likert elicitation and eight binary questions. The same method was repeated for the binary and two stage Likert elicitation methods.

The choice scenario for both the Likert elicitation methods took the following form (Tables 1 and 2).

Table 1. Example of a five point Likert scale choice scenario with the Location method.

Option A			Option B
View: 6F			View: 6F
Noise: 3F			Noise: 6F
Housing service charge: € 40			Housing service charge: € 75
Sunlight: 6F			Sunlight: 3T
Definitely A	Probably A	Uncertain	Probably B	Definitely B

Table 2. Example of a two stage Likert scale choice scenario with Linguistic method.

Option A	Option B
View: Good	View: Good
Noise: Noisy	Noise: Noisy
Housing service charge: € 70	Housing service charge: € 50
Sunlight: Very good	Sunlight: Good
A	B

The main survey was conducted between February and April 2008. The location and linguistic representation surveys were conducted sequentially. Different respondents participated in the two surveys; however, the surveys were conducted in the same area (Dave, Toner, and Chen 2018).

Model structure

Choice models are based on random utility theory where individuals are expected to follow utility maximisation. The random utility model consists of a deterministic utility component and a stochastic error. For a choice over two alternatives, the random utility model of choosing alternative i over alternative j can be given as: (1) $U_i>U_{j}for\mathrm{\forall }i\ne j$(1) Equation (1) implies that alternative i will be chosen over alternative j if the random utility of alternative i is greater than alternative j (Bates 1988; Meyer and Miller 1984).

The random utility $U_n$ can be decomposed as: (2) $U_n=V_n+{\epsilon }_n$(2) where

$U_n$ = random utility for alternative n

$V_n$ = deterministic utility for alternative n

${\epsilon }_n$ = stochastic error associated with alternative n (Ben-Akiva and Lerman 1985)

The main assumption underlying random utility theory is that the error terms are independent and identically distributed (Train 2003). Within choice models, these error terms are assumed to be Type-1 Gumbel distributed to apply the standard multinomial logit model (Train 2003). With further relaxation of the error terms, advanced model structures can be assumed and applied (Ben-Akiva and Lerman 1985; Train 2003).

The error structure in terms of correlation and heterogeneity can provide further insight on the causes of preference uncertainty. This paper will explicitly treat the preference uncertainty data by incorporating it within the choice model. While previous studies have applied the heteroskedastic scale parameter to estimate the error variance associated with the different preference certainty levels (Dekker et al. 2016), this paper will analyse the preference uncertainty data based on the heterogeneity and correlation in the error structure. Two main reasons for respondents’ preference uncertainty assumed in this paper are respondent’s known level of choice commitment from choice-set characteristics and preference uncertainty arising from respondent’s inherent uncertainty or other stochastic factors. In order to examine the cause of preference uncertainty, the preference elicitation data is analysed to capture heterogeneity and correlation in the error structure, caused due to respondent’s level of choice commitment or other stochastic effects.

The correlation between the error structures is examined to analyse whether any choice certainty level is provided by the respondents due to choice-set characteristics and an inherent similarity in how the choice sets are perceived. Under this model, it is thus assumed that when respondents perceive the choice-sets in a particular light, based on the choice-set characteristics, they would choose a particular level of preference certainty. Thus, in this case, a correlation between error structures is expected due to similarity in the perception of choice set characteristics. The correlation between the error structures caused due to choice-set characteristics is analysed using the nested logit (NL) model. The NL model relaxes the IIA assumption by grouping alternatives that are similar to each other in terms of unseen characteristics, than they are to other alternatives (Koppelman and Bhat 2006). The nested logit model is based on the argument that the commonality of certain characteristics across alternatives can result in some parts of the random error to be correlated. The nested logit model is formed by partioning the choice set such that the alternatives that share some common characteristics are grouped together, resulting in the independent and identically distributed error assumption being relaxed, as the error component of the alternatives are correlated within a nest but not across nests. This model is applied to examine correlation between error terms associated with similar preference uncertainty levels across the alternatives. Thus, the NL model captures the similarity in the way alternatives are perceived when they belong to the same level of preference certainty. The NL model implies that the respondents are relatively certain of their chosen preference certainty level and they have selected a particular preference certainty level based on their level of choice commitment, for a particular choice scenario. In this case, the NL model is expected to model the respondent’s level of choice commitment by estimating the correlation between the similar preference uncertainty levels.

Thus, when respondents’ preference certainty level is not due to stochastic factors but respondents’ level of commitment, the NL model can be applied to analyse the preference uncertainty data. Using the NL model, it is therefore anticipated that if respondents regard the choice scenario in a particular light based on the choice-set characteristics, some correlation should be observed between the ‘Definitely A’ and ‘Definitely B’ options and the ‘Probably A’ and ‘Probably B’ options of the one stage, five point Likert preference elicitation method and the ‘Absolutely certain’ alternatives as well as the ‘Not so certain’ alternatives of the two stage Likert preference elicitation method.

In a five point Likert scale of choosing between options, ‘Definitely A’, ‘Probably A’, ‘Uncertain’, ‘Probably B’ and ‘Definitely B’, the choice of alternative ‘Definitely A’ can be given using the nested logit model as follows –

The composite utility of selecting ‘Definitely A’ is: (3) $U_{D,A}=U_D+U_{A|D}$(3) (4) $U_{D,A}=V_{D,A}+{\epsilon }_{D,A}$(4) (5) $U_{D,A}=V_D+V_{A|D}+{\epsilon }_D+{\epsilon }_{A|D}$(5)

The joint probability of selecting ‘Definitely A’ can be given as – (6) $P_{D,A}=P_D\cdot P_{A|D}$(6) The nested logit model takes the following structure, as provided in Figure 2, considering pooling across the two Likert elicitation methods.

Thus, based on the nested logit structure provided in Figure 2, correlation between the ‘Definitely’ alternatives are examined along with the ‘Probably’ alternatives for the one stage Likert preference elicitation and correlation between the ‘Absolutely certain’ alternatives are examined for the two stage Likert preference elicitation method.

Figure 2. Nested Logit Structure for the preference uncertainty data.

The λ parameter in the NL model estimates the scale parameter across the two pooled models, thus signifying the difference between the one and two stage Likert elicitation methods. The μ parameter estimates the nest parameter for each of the preference uncertainty levels. Thus, it denotes the correlation between the same preference uncertainty levels across the two alternatives. For example, the correlation between Definitely A and Definitely B preference levels, is captured by the nest coefficient μ for the nest ‘Definitely’.

The utility function under the NL specification took the following form for the location ratings model of the one stage Likert preference elicitation (7) $\begin{array}{c}{U}_{DA}=AS{C}_{DA}+\alpha VA+\beta NA+\gamma SA+\eta CA\\ U_{PA}=AS{C}_{PA}+\alpha VA+\beta NA+\gamma SA+\eta CA\\ \begin{array}{c}{U}_U=AS{C}_U\\ U_{PB}=AS{C}_{PB}+\alpha VB+\beta NB+\gamma SB+\eta CB\\ U_{DB}=\alpha VB+\beta NB+\gamma SB+\eta CB\end{array}\end{array}$(7) where

U_DA and U_DB are the utility functions for ‘Definitely A’ and ‘Definitely B’ preference uncertainty level

U_PA and U_PB are the utility functions for ‘Probably A’ and ‘Probably B’ preference uncertainty level

U_U is the utility function for ‘Uncertain’ preference level

V is the view attribute

N is the noise attribute

S is the sunlight attribute

C is the housing service charge

ASC is the alternative specific constant

Under the two stage Likert elicitation method and the linguistic representation method, the NL specification utility forms can be given as follows: (8) $\begin{array}{rl}& U_{AcA}=AS{C}_{AcA}+{\alpha }_{1}V{A}_1+{\beta }_{1}N{A}_1+{\beta }_{2}N{A}_2+{\gamma }_{1}S{A}_1+{\gamma }_{2}S{A}_2+\eta CA\\ & U_{NsA}=AS{C}_{NsA}+{\alpha }_{1}V{A}_1+{\beta }_{1}N{A}_1+{\beta }_{2}N{A}_2+{\gamma }_{1}S{A}_1+{\gamma }_{2}S{A}_2+\eta CA\\ & U_{AcB}={\alpha }_{1}V{B}_1+{\beta }_{1}N{B}_1+{\beta }_{2}N{B}_2+{\gamma }_{1}S{B}_1+{\gamma }_{2}S{B}_2+\eta CB\\ & U_{NsB}=AS{C}_{NsB}+{\alpha }_{1}V{B}_1+{\beta }_{1}N{B}_1+{\beta }_{2}N{B}_2+{\gamma }_{1}S{B}_1+{\gamma }_{2}S{B}_2+\eta CB\end{array}$(8) where

U_AcA and U_AcB are the utility functions for ‘Absolutely certain A’ and ‘Absolutely certain B’

U_NsA and U_NsB are the utility functions for ‘Not so certain A’ and ‘Not so certain B’ preference uncertainty levels

The NL model utility functions for the location representation and two stage Likert preference elicitation as well as the linguistic representation and one stage Likert preference elicitation followed a similar structure as given in Equations (7) and (8) based on their respective attribute representation techniques, with alternative specific constant held ‘fixed' for ‘Definitely B' alternative and ‘Absolutely certain B' alternative for the one and two stage Likert elicitation respectively.

While the NL model assumes homoskedastic error variance, the error components logit (ECL) model allows for heteroskedastic error structure in the choice model. Heterogeneity and correlation in the error variances can be captured by the ECL model. This can be applied to incorporate preference uncertainty caused due to random effects, within the choice model. By examining the error structure for each of the preference levels, respondent’s uncertainty due to stochastic effects can be examined using the ECL model.

The utility of the respondent under the ECL model can be given as follows: (9) $U_{in}=V_{in}+({\xi }_{in}+{\epsilon }_{in})$(9) where $\xi {}_{in}$ is the random error with zero mean and distribution dependent on the underlying parameters and ${\epsilon }_{in}$ is random error with zero mean and independent and identically distributed (i.i.d.) error component across the different alternatives for an individual n (Hensher and Greene 2001). Based on the definition of the ECL model, various forms of correlation and heteroskedasticity can be examined from the preference uncertainty data.

When preference uncertainty is caused by stochastic factors, this can be captured using the ECL model with heteroskedastic and alternative specific error component. For each of the preference uncertainty level and for each alternative, an alternative specific error component with $\sim N(0,{\sigma }^2)$is incorporated in the utility function.

The error components logit model specification thus takes the following utility form for the linguistic dummy model for the one stage Likert elicitation method: (10) $\begin{array}{rl}& U_{DA}=AS{C}_{DA}+{\alpha }_{1}V{A}_1+{\beta }_{1}N{A}_1+{\beta }_{2}N{A}_2+{\gamma }_{1}S{A}_1+{\gamma }_{2}S{A}_2+\eta CA\\ & U_{PA}=AS{C}_{PA}+{\alpha }_{1}V{A}_1+{\beta }_{1}N{A}_1+{\beta }_{2}N{A}_2+{\gamma }_{1}S{A}_1+{\gamma }_{2}S{A}_2+\eta CA+{\xi }_{PA}\\ & U_U=AS{C}_U+{\xi }_U\\ & U_{PB}=AS{C}_{PB}+{\alpha }_{1}V{B}_1+{\beta }_{1}N{B}_1+{\beta }_{2}N{B}_2+{\gamma }_{1}S{B}_1+{\gamma }_{2}S{B}_2+\eta CB+{\xi }_{PB}\\ & U_{DB}={\alpha }_{1}V{B}_1+{\beta }_{1}N{B}_1+{\beta }_{2}N{B}_2+{\gamma }_{1}S{B}_1+{\gamma }_{2}S{B}_2+\eta CB+{\xi }_{DB}\end{array}$(10) where

$\xi$ is the alternative-specific error component

In case of the location ratings, two stage Likert elicitation method, the utility functions can be given as: (11) $\begin{array}{rl}& U_{AcA}=AS{C}_{AcA}+\alpha VA+\beta NA+\gamma SA+\eta CA\\ & U_{NsA}=AS{C}_{NsA}+\alpha VA+\beta NA+\gamma SA+\eta CA+{\xi }_{NsA}\\ & U_{AcB}=\alpha VB+\beta NB+\gamma SB+\eta CB+{\xi }_{AcB}\\ & U_{NsB}=AS{C}_{NsB}+\alpha VB+\beta NB+\gamma SB+\eta CB+{\xi }_{NsB}\end{array}$(11) The utility functions for the one stage Likert elicitation method and location representation method followed the structure provided in Equation (7) but with additional alternative specific error component fixed for the ‘Uncertain’ level and estimated for the other preference certainty levels. In case of the two stage Likert elicitation and linguistic representation method, the additional alternative specific error component was fixed for the ‘Absolutely certain A’ alternative and estimated for the other preference certainty levels while the basic utility function remained as described in Equation (8).

In order to estimate the difference between the one and two stage Likert data in their ability to capture preference uncertainty, the data from the two elicitation methods were pooled and a scale parameter was estimated for both the NL and ECL model, which captures the difference between the one and two stage Likert data. The paper thus examines the ability of different preference elicitation methods on their ability to capture the preference uncertainty data.

The base multinomial logit (MNL) model was conducted to allow for the comparison in model improvement by advanced models that explicitly examine the preference uncertainty error structure. Under the MNL model specification, each of the different preference uncertainty levels acted as a separate alternative within the one and two stage Likert elicitation methods. The utility function for the preference uncertainty levels under the MNL model followed a similar structure as provided for the NL model, without any underlying nesting structure. The error terms with the MNL model were assumed to be independent and identically distributed with Gumbel distribution.

The base multinomial logit model, nested logit as well as the error components logit model in the study incorporated panel effects, which accounts for correlation in choices within individuals (Yanez et al. 2010). The panel error component was incorporated in (n-1) utility functions for each of the one and two stage Likert preference elicitation methods, across the different models. All models were estimated in BIOGEME 2.0 (Bierlaire 2003; Bierlaire 2008) using Modified Latin Hypercube Sampling (MLHS) procedure (Hess, Train, and Polak 2006) to generate 500 draws.

As different data input methods are suitable for each of the attribute representation method (Dave, Toner, and Chen 2018), this paper provides a comparison between the location ratings and the linguistic dummy models. In case of the location representation method, the attribute values for view, noise and sunlight attributes were incorporated in the data analysis based on the numeric ratings provided by the respondents. However, in case of the linguistic representation method, the attribute values were incorporated in the choice analysis based on the linguistic levels of the attributes which were incorporated in the analysis as dummy levels.

Results and discussion

The paper reports the results obtained from the one and two stage Likert elicitation methods. The binary questions along with the recoded binary response from the Likert questions are given in Dave, Toner, and Chen (2018). This paper specifically incorporates the Likert response in the modelling, without recoding the preference uncertainty data. Nested logit and error components logit models are mainly used to analyse the preference data from the Likert responses. Results from the nested and error components logit model is compared with the base multinomial logit model to estimate the improvement in the model fit.

The percentage of respondents that choose each of the preference certainty level under the two Likert elicitation methods is given in the following table (Table 3):

Table 3. Percentage of preference levels elicited across the two Likert methods for location and linguistic attribute representation.

Respondent characteristics	Location method (percentage)	Linguistic method (percentage)
One stage Likert
Definitely A	25.6	33.7
Probably A	30.9	22.7
Uncertain	8.4	4.5
Probably B	23.7	18.0
Definitely B	11.4	21.1
Two stage Likert
Absolutely certain A	43.2	48.0
Not so certain A	18.3	13.7
Not so certain B	12.2	8.9
Absolutely certain B	26.3	29.3

It is seen that for the location representation method, a relatively higher percentage of respondents choose the ‘Probably A’ and ‘Probably B’ alternatives with the one stage Likert method, compared to the ‘Definitely’ alternatives. Under the two stage Likert method, more respondents choose the ‘Absolutely certain’ alternatives over the ‘Not so certain’ alternatives. In case of the linguistic representation method, it is seen that across the one and two stage Likert methods, a higher percentage of respondents choose the ‘Definitely’ and ‘Absolutely certain’ alternatives compared to the less certain alternatives. For the location representation method, it is seen that relatively lower percentage of respondents choose the ‘Uncertain’ alternative compared to the other preference certainty levels.

The nested logit model introduced correlation in the error structure based on the level of preference uncertainty. Thus, alternatives with similar preference certainty levels were nested together. The scale parameter estimated the difference between the two preference elicitation methods while the nest parameter estimated the correlation between alternatives with similar preference certainty levels. Conducting the NL analysis on the preference uncertainty data, the following results are obtained across the location and linguistic representation methods (see Table 4):

Table 4. NL results across location and linguistic representation methods.

Location Attributes	Location ratings (t-statistics)	Linguistic Attributes	Linguistic dummy (t-statistics)
ASC		ASC
One Stage		One Stage
Def. A	0.184 (1.60)	Def. A	0.113 (1.33)
Prob A	0.349*** (4.22)	Prob A	−0.354*** (−3.35)
Uncertain	0.491** (2.29)	Uncertain	−1.54*** (−4.93)
Prob. B	0.173** (2.15)	Prob. B	−0.440*** (−4.86)
Def. B	Fixed	Def. B	Fixed
Two Stage		Two Stage
Abs. Certain A	0.374*** (3.26)	Abs. Certain A	0.125 (1.50)
N.s. Certain A	−0.297*** (−2.97)	N.s. Certain A	−1.34*** (−9.48)
Abs. Certain B	Fixed	Abs. Certain B	Fixed
N.s. Certain B	−0.642*** (−5.73)	N.s. Certain B	−1.64*** (−11.07)
View	0.0140*** (5.70)	View
		Good	0.389*** (4.19)
		Neither
Noise	0.0125*** (5.45)	Noise
		Noisy	−1.07*** (−6.24)
		Neither	−0.486*** (−4.20)
		Quiet
Sunlight	0.0185*** (6.14)	Sunlight
		Very good	0.524*** (4.84)
		Good	0.401*** (2.98)
		Neither
Charge	−0.0231*** (−13.51)	Charge	−0.0156*** (−6.01)
ρ^2 w.r.t. 0	0.125	ρ^2 w.r.t. 0	0.131
adjusted ρ^2	0.121	adjusted ρ^2	0.126
Scale Parameter		Scale Parameter
(w.r.t. 1)		(w.r.t. 1)
1 Stage	1.42*** (3.07)	1 Stage	2.41*** (2.71)
2 Stage	1.00 (fixed)	2 Stage	1.00 (fixed)
Nest Coefficient		Nest Coefficient
(w.r.t. 1)		(w.r.t. 1)
1 Stage		1 Stage
Definitely	1.57*** (2.57)	Definitely	2.23*** (3.17)
Uncertain	1.01 (fixed)	Uncertain	1.01 (fixed)
Probably	1.00 (0.00)	Probably	1.00 (0.00)
2 Stage		2 Stage
Absolutely certain	1.00 (0.00)	Absolutely certain	2.28*** (2.90)
Not so certain	1.01 (fixed)	Not so certain	1.00 (fixed)
FLL	−3413.46	FLL	−3114.94
no. of obs.	1775	no. of obs.	1632
no. of individuals	222	no. of individuals	204
MLHS draws	500	MLHS draws	500

Coefficient estimate significant at *10%, **5%, ***1%, FLL – final log-likelihood

The coefficient estimates for each of the attributes across the two models, give expected signs and good statistical significance. As the noise attribute for the location representation method is based on the ratings (0-100) where lower value depicts poor noise quality (high traffic noise) and higher rating value depicts higher level of quietness, a positive coefficient estimate is obtained for this attribute, as expected. Thus, in this case the coefficient estimate depicts the value of quietness.

In case of the linguistic representation method, it is found that ‘good’ view has a high and significant coefficient estimate compared to ‘neither good nor bad’ view level. For the noise attribute, as expected, ‘noisy’ and ‘neither noisy nor quiet’ noise levels are considered a disutility compared to the ‘quiet’ noise level. As per theoretical expectations, ‘very good’ and ‘good’ sunlight levels have high and significant coefficient estimates compared to ‘neither good nor bad’ sunlight level.

Examining the alternative specific constants (ASCs) obtained from the analyses, it can be observed in case of the one stage Likert elicitation for the location method, that respondents prefer the ‘Probably A’ and ‘Probably B’ alternatives over the ‘Definitely B’ alternative as the ASCs have a positive and significant parameter estimates for these preference levels compared to the base ‘Definitely B’ alternative. This result is also reiterated in the preference elicitation descriptive statistics (see Table 3) which show that more respondents choose the other alternatives over the ‘Definitely B’ alternative. In case of the two stage Likert elicitation method for the location representation method, it can be observed that the ‘Absolutely certain A’ alternative is more preferred over the ‘Absolutely certain B’ alternative, while the ‘Not so certain A’ and ‘Not so certain B’ preference levels are less preferred.

For the linguistic representation method it is observed that for both the preference elicitation methods, the ‘Probably’ and ‘Not so certain’ alternatives are less preferred to the ‘Definitely’ and ‘Absolutely certain B’ alternative. Thus, in this case it is seen that the respondents are more certain of their choices. Compared to the ‘Definitely B’ alternative, the ‘Definitely A’ alternative is slightly more preferred, though the statistical significance is found to be low. The same finding is obtained in the case of the two stage Likert elicitation method.

Examining the scale parameters obtained, it is seen that across both the location and linguistic representation methods, there is a marked difference between the one and two stage Likert elicitation methods. Thus, both these preference elicitation methods capture the preference information differently under the NL model.

Within each of the preference elicitation methods, it is observed that across both the attribute representation methods, there is a higher correlation between the ‘Definitely’ alternatives for the one stage Likert elicitation, as given by the nest coefficient. Moreover, a high correlation is also obtained in case of the ‘Absolutely certain’ alternatives for two stage Likert elicitation for the linguistic representation method while no significant correlation is obtained for the ‘Absolutely certain’ alternatives for the location representation method. Across both the attribute representation methods, no correlation is obtained in case of the ‘Probably’ and ‘Not so certain’ alternatives. The results thus indicate that the respondents view the choice scenarios in a particular similar light when they have a higher level of certainty in their choice. This is especially found to be the case within the one stage Likert elicitation method.

The NL model is the first step to relax the IIA assumption and is based on the theory that the commonality of alternative characteristics causes some parts of the random error to be correlated. The NL model is thus applied when there are some shared unobserved error components associated with different choice dimensions (Ben-Akiva and Lerman 1985). Using the NL model, the homoskedastic assumption of the MNL model is maintained, which implies that though the error variance is correlated for alternatives within a nest, the overall error variance of all the alternatives remain constant (Shen 2005). Using the NL model thus, the error structure and error correlation patterns between preference certainty levels can be examined. The theoretic nest parameter lies between 0 and 1, where unity signifies little correlation in the error variance and nest parameter equal to zero indicates a degenerate model (Louviere, Hensher, and Swait 2000). However, the models reported in this paper are estimated with BIOGEME 2.0 where the nest parameter is the inverse of the theoretic nest parameter (Bierlaire 2009). A high value of nest parameter in this case thus indicates a higher correlation between the alternatives within the nest.

Results from the NL model indicate that across the location as well as the linguistic methods, there is a significant difference between the one and two stage Likert elicitation methods based on the scale parameters. A high and significant correlation is also obtained for the higher preference certainty levels with this model. This is especially the case for one stage Likert elicitation method across the location and linguistic representation models, where the granularity in preferences is adequately modelled within the nested logit model.

The error components logit model was conducted to examine whether there are any differences in the error variance across the different preference certainty levels. The ECL model was formed by adding (n-1) error components across the different preference certainty levels. The following table (Table 5) provides the results from the ECL model:

Table 5. ECL results across location and linguistic representation methods.

Location Attributes	Location ratings (t-statistics)	Linguistic Attributes	Linguistic dummy (t-statistics)
ASC		ASC
One Stage		One Stage
Def. A	−0.0627 (−0.36)	Def. A	0.191 (1.17)
Prob A	0.0232 (0.11)	Prob A	−0.474** (−2.36)
Uncertain	0.997*** (3.71)	Uncertain	−3.48*** (−6.80)
Prob. B	0.337** (2.06)	Prob. B	−1.44*** (−3.42)
Def. B	Fixed	Def. B	Fixed
Two Stage		Two Stage
Abs. Certain A	0.907*** (6.12)	Abs. Certain A	0.539*** (2.92)
N.s. Certain A	−0.972*** (−3.12)	N.s. Certain A	−1.59*** (−3.56)
Abs. Certain B	Fixed	Abs. Certain B	Fixed
N.s. Certain B	−0.221 (−1.24)	N.s. Certain B	−1.13*** (−3.95)
View	0.0191*** (6.96)	View
		Good	0.750*** (5.39)
		Neither
Noise	0.0156*** (5.85)	Noise
		Noisy	−2.09*** (−11.09)
		Neither	−0.811*** (−4.33)
		Quiet
Sunlight	0.0267*** (7.37)	Sunlight
		Very good	0.971*** (5.98)
		Good	0.744*** (3.04)
		Neither
Charge	−0.0272*** (−12.02)	Charge	−0.0286*** (−8.99)
ρ^2 w.r.t. 0	0.125	ρ^2 w.r.t. 0	0.131
adjusted ρ^2	0.12	adjusted ρ^2	0.125
Scale Parameter		Scale Parameter
(w.r.t. 1)		(w.r.t. 1)
1 Stage	1.06 (0.67)	1 Stage	1.01 (0.07)
2 Stage	1.00 (fixed)	2 Stage	1.00 (fixed)
Error Component		Error Component
(w.r.t. 1)		(w.r.t. 1)
Definitely A	0.0713 (0.37)	Definitely A	Fixed
Probably A	1.56*** (6.06)	Probably A	0.129 (0.51)
Uncertain	Fixed	Uncertain	0.551 (1.07)
Probably B	0.733*** (2.70)	Probably B	2.09*** (4.95)
Definitely B	0.0239 (0.12)	Definitely B	0.119 (0.53)
Abs. Certain A	Fixed	Abs. Certain A	Fixed
N.s. Certain A	1.47*** (4.35)	N.s. Certain A	1.34*** (2.98)
Abs. Certain B	−1.20*** (−5.37)	Abs. Certain B	0.786*** (2.95)
N.s. Certain B	0.182 (0.53)	N.s. Certain B	0.678 (1.61)
FLL	−3414.9	FLL	−3114.93
no. of obs.	1776	no. of obs.	1632
no. of individuals	222	no. of individuals	204
MLHS draws	500	MLHS draws	500

Based on the scale parameter obtained, it is observed that there is not much difference between the two Likert elicitation methods across the different attribute representation methods. However, the error components for each of the preference certainty levels indicate a high and statistically significant variance for the ‘Probably’ alternatives in case of the one stage Likert elicitation method. For the two stage Likert elicitation method, it is found that the lower preference certainty levels have a high and significant error variance in case of the ‘Not so certain’ alternatives, however, a high error variance is also found in case of the ‘Absolutely certain B’ alternative. Examining the error components for each of the preference certainty levels, it can be observed that there is a higher error variance in case of ‘Probably’ and ‘Not so certain’ alternatives across the one and two stage Likert elicitation methods. The ECL model thus shows that largely there is higher error variance in case where respondents are less certain of their preferences. This is in line with the theoretical expectations.

Comparing the one and two stage Likert elicitation methods, it can be seen that theoretically consistent variation in error components can be found across the different error variances in case of the one stage Likert elicitation method. In this case, as theoretically expected, a higher error variance is obtained for lower preference certainty levels. In case of the two stage Likert elicitation method however, it can be observed that a high error variance is also obtained for the more certain preference levels. While the scale parameters indicate that for the ECL model, there is no significant difference between the one and two stage preference elicitation methods, examining the error variances indicate that the one stage Likert preference elicitation method could be more suitable in capturing preference uncertainty information.

The attribute coefficient estimates have correct sign and high statistical significance. The ASCs show that in case of the one stage Likert elicitation for the location representation method, ‘Uncertain’ and ‘Probably B’ alternatives are more preferred over the ‘Definitely B’ alternative. In case of the two stage Likert elicitation, the ‘Absolutely certain A’ alternative is preferred over the other alternatives for both the location and linguistic representation methods.

The Likelihood ratio (LR) test signifies the differences between the models and whether the estimated model brings a statistically significant improvement over the base model. In order to examine whether the NL and ECL models bring an improvement by relaxing the error assumption, the LR test was conducted with an equivalent base multinomial logit (MNL) model for each of the representation methods. The LR test is given by the following formula – (12) $\mathrm{LR}=\text{-}2(\mathrm{L}{\mathrm{L}}_{\mathrm{base}}\text{–}{\mathrm{LL}}_{\mathrm{estimated}})$(12) where

LL_base = log-likelihood of the base MNL model

LL_estimated = log-likelihood of the estimated NL or ECL model

The value obtained from the LR test is compared to the χ² estimate obtained with the degrees of freedom equivalent to the number of additional parameters in the estimated model. The following table provides the LR test for NL and ECL models (Table 6).

Table 6. Likelihood ratio test.

	Location Method		Linguistic Method
	NL model	ECL model	NL model	ECL model
LR test statistic	23.612	20.742	34.294	34.318
degrees of freedom	3	7	3	7
Significance at α = 0.05	NL better	ECL better	NL better	ECL better

Comparing the LR test across the location and linguistic representation methods, it is seen that the NL as well as the ECL models provide a significant improvement in model fit over the MNL model. This result thus shows that accounting for preference uncertainty by incorporating the preference uncertainty levels as endogenous factors through the MNL model can be enriched by examining the error structure of the preference certainty levels.

While the MNL model provides some treatment of the preference uncertainty data over recoding and elimination, the NL and ECL model provides further insight into how the alternatives are treated and which factors influence the choice of a particular preference certainty level. The results from the NL model reveal that there is some uncaptured correlation in the higher preference certainty levels while the ECL model shows that at lower preference certainty levels, there is a higher stochastic error variance. This finding is in line with the results obtained from the previous literature (Li and Mattsson 1995; Lundhede et al. 2009; Caussade et al. 2005) which shows higher associated error variance when respondents are less certain of their choices.

Capturing preference uncertainty also has implications for the willingness to pay (WTP) estimates. Table 7 provides the WTP estimates obtained from the NL and ECL models, in comparison to the recoded BL model specified in Dave, Toner, and Chen (2018):

Table 7. Willingness to pay estimates.

Attribute	Location Method			Linguistic Method
	BL model	NL model	ECL model	BL model	NL model	ECL model
View	0.665*** (7.99)	0.61*** (5.43)	0.70*** (6.95)
Good				22.14*** (7.34)	24.94*** (6.23)	26.22*** (6.17)
Neither
Noise	0.886*** (9.69)	0.54*** (5.16)	0.57*** (5.73)
Noisy				−74.8*** (−14.6)	−68.59*** (−10.7)	−73.08*** (−10.28)
Neither				−23.9*** (−6.05)	−31.15*** (−6.16)	−28.36*** (−5.19)
Quiet
Sunlight	0.829*** (7.82)	0.80*** (5.82)	0.98*** (7.38)
Very good				31.44*** (7.70)	33.59*** (6.20)	33.95*** (5.83)
Good				22.26*** (3.59)	25.71*** (3.22)	26.01*** (2.99)
Neither

Note: Coefficient estimate significant at *10%, **5%, ***1%.

In case of the location rating model, the WTP estimates are obtained for a unit change in the attribute ratings while in the case of the linguistic dummy model, the WTP estimates are obtained for a change in the linguistic category of the attributes. Thus, a difference in the magnitude between the WTP estimates is obtained for the location and the linguistic methods.

Comparing the WTP estimates obtained from the NL and ECL models with those obtained from the binary logit (BL) model which recoded the preference uncertainty levels to ‘Alternative A’ and ‘Alternative B’ as given in Dave, Toner, and Chen (2018), it is found that in case of the linguistic dummy model, the attributes with NL and ECL models are largely valued higher when explicitly accounting for preference uncertainty. It is found that a higher WTP estimate is obtained for ‘view’ and ‘sunlight’ attributes when accounting for preference uncertainty, especially with the ECL model. It has been noted in Dave, Toner, and Chen (2018) that the location method is easier to understand for the ‘view’ attribute and the linguistic method is better for the ‘noise’ attribute; however, for each of these methods, it can be noted that explicitly accounting and modelling for preference uncertainty results in different WTP estimates compared to the recoded BL model. Comparing the NL and ECL models, it can be seen that the WTP estimates are higher and more significant in case of the ECL model for the location method, compared to the NL model. In case of the linguistic representation method, the NL model shows a slightly higher statistical significance of the WTP estimates.

Preference uncertainty has significant implications for valuation (Akter, Bennett, and Akhter 2008; Li and Mattsson 1995); with many methods developed to capture and model respondent’s uncertainty. This paper shows that explicitly accounting for respondent’s preference uncertainty in modelling results in valuation estimates different than those obtained from recoding the preference uncertainty responses. While the linguistic representation method showed a higher WTP estimate when preference uncertainty was explicitly accounted, the location representation method provided different WTP estimates for the various attributes under the NL and ECL models, compared with the recoded binary model. The WTP estimates thus varied based on the different treatment of the preference uncertainty data, for each of the attributes. It is therefore important for the valuation exercise to account for respondents’ preference uncertainty in modelling as well as to examine the structure of the error variance to understand the potential causes of preference uncertainty.

Conclusions

This paper has aimed to elicit preference uncertainty in choice experiments and treat the preference uncertainty levels from the respondents within the modelling process by examining the error variance of the preference uncertainty data for road traffic noise. Capturing respondents’ preference uncertainty is gaining increasing importance in the stated preference literature, with significant studies undertaken within the contingent valuation method. However, eliciting preference uncertainty and modelling the uncertainty data within the valuation framework is relatively lesser applied in CE.

This paper provided respondents with two different Likert scale questions to indicate their level of preference uncertainty. Based on the modelling technique used, the one and two stage Likert questions were found to capture respondent’s preference uncertainty differently. Under the nested logit model where nests were formed based on the level of preference uncertainty, it was found that there is a marked difference in the preference information obtained from the one and two stage Likert questions. It was also found applying the nested logit model, that respondents view the alternatives with higher preference levels in a particular similar light. Thus, there is higher error correlation among selected alternatives when respondents are more certain of their choices.

With the error components logit model, a higher error variance was obtained for the lower preference certainty levels and no significant difference was found in the preference information obtained from the two preference elicitation methods. Moreover, the ECL model reveals that when respondents are less certain of their preferences, there is a higher error variance associated with their choice. Results from these methods imply that the effect on error structure could be different based on the cause of the preference uncertainty. When preference uncertainty is caused due to choice-set characteristics, there is a higher error correlation between choices with similar preference certainty level. This implies that respondents perceive the choice-set characteristics in a particular light, inducing correlation between the alternatives when they are more ‘certain’ of their choices. However, when preference uncertainty is caused due to respondent’s inherent uncertainty or stochastic factors, this is reflected in higher associated error variance in the lower preference uncertainty levels.

The multiple bounded dichotomous choice question and the post-decisional certainty question (similar to the one stage Likert and two stage Likert elicitation methods respectively) are commonly known methods of preference elicitation within the contingent valuation method. While both methods can provide some insight on the error structure and decision-making process of the respondents, where the modelling process allows for the analysis of granular choice data, the one stage Likert question can be preferred over the two stage Likert elicitation due to greater detail on choice and preference uncertainty. However, where relevant, the two stage Likert elicitation method can also be used to reaffirm the choice certainty of the respondents and capture any underlying preference uncertainty not otherwise elicited through the standard binary choice question.

Results from this study thus affirms that explicitly incorporating the preference uncertainty data in choice experiment survey and modelling can provide significant insights into respondents’ decision-making and can be useful in any subsequent valuations.

Acknowledgements

The paper is dedicated to Dr Jeremy Toner who left for the eternal spiritual abode during the final stages of the preparation of this manuscript. Jeremy always provided insightful guidance and was full of wit. Although a self-realized soul may live in various material bodies while in this world, experiencing their various qualities and functions, he is never entangled, just as the wind which carries various aromas does not actually mix with them (Srimad Bhagavatam 11.7.41). Jeremy will always be dearly missed.

Disclosure statement

No actual or potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Marie Curie EST Fellowship.