Give us feedback

Project part-financed by the European Union (European Regional Development Fund)

The Interreg IVB North Sea Region Programme

The authors are solely responsible for the content of this report. Material included herein does not represent the opinion of the European Community, and the European Community is not responsible for any use that might be made of it.
Back to overview reports

Ecosystem Service Assessment of TIDE Estuaries

3c. TIDE Survey methodology & reliability

Survey one: Demand survey


In order to select focal ecosystem services and obtain an estimate of service demand, the value (sensu Costanza 2000: appraised value or importance for society) of these services was qualitatively assessed. This corresponds to the concept of assigned values (Brown, 1984; Lockwood, 1999) as applied by Bryan et al (2010). As advised by Granek et al (2010) this implied decision makers and the public.
The first basic objective was to determine which ecosystem services were important to be considered in TIDE, to narrow down the further research to the relevant services.
The second objective was to obtain a qualitative estimate of the importance of the ecosystem services, comparable over space (estuaries, salinity zones) and time.


Respondents were asked to score the importance of 46 ecosystem services for every salinity zone in their estuary, for past, present and future times. Salinity zones were based on the zonation report (ref „Zonation of the TIDE estuaries”).2 Past and future were defined as ca. 1930 and ca. 2050 respectively. The scoring values were:

Score Description
1 don’t know
2 unimportant ecosystem service
3 less important
4 important
5 very important

The survey thus yielded 48 ecosystem service lists (four estuaries, four zones, three time steps), each containing 46 scores. The respondent groups were asked to provide a debated consensus scoring per estuary.

Reliability of the survey

Reliability of the survey was evaluated by checking consistency and accordance of survey results between estuaries. Although a high accordance adds to the confidence that can be put in the results, a low accordance could well be the result of differences in score interpretation or resulting from actual differentiation in importance among estuaries.
Visually, this can be evaluated by the mean, standard error and standard deviation of scores per service (Figure 2). Standard error never exceeds categories, standard deviation rarely exceeds one category (Figure 2). This rather high observed accordance adds a third argument to the confidence in results of the first survey.

Statistically, the demand survey has an acceptable reliability (alpha = 0.798), inter-estuarine consistence (ICC-c = 0.798; F-Test F(183,549) = 4.95 , p = 6.39e-48; 95%-CI 0.746 < ICC < 0.842) and agreement (ICC-A = 0.792; F-Test F(183,459) = 4.95 , p = 7.75e-44; 95%-CI 0.737 < ICC < 0.838). Estuaries have also a high similarity in ES-demand (Omega-H = 0.77)

Respondents and their affiliations

As the confidence and legitimacy of the results clearly depends on the representativeness of the respondents, this survey was conducted in the regional working groups in each estuary. “These groups consisted of several experts being very familiar with the characteristics of the […] estuary. In this context it was the aim to bring people with different expertise together, e.g. ecology, hydrology, sediment management, engineering, experience in the implementation of various European directives, etc.. Furthermore the experts should come from different institutions […]” (pers.comm Kirsten Wolfstein).
The number of respondents per consensus group ranges widely (3 – 9), and their expertise level from student to professor or engineer. Their experiences vary from coordination of navigation projects, to estuarine ecology research, morphological modeling, drafting of conservation plans, polluted sediment remediation, etc. For the first survey, respondents’ affiliations and expertise are depicted in appendix I.

Additionally, a basic check was performed to verify whether affinities of the respondents are sufficiently broad. The respondents assigned their own personal or professional affinity to three categories (‘transport’, ‘safety’ and ‘ecology’, fig…). This test indicated a reasonable representation of basic issues in estuarine management for all estuaries, except for the “safety” aspect which is not represented in the Elbe. Also, the Scheldt group was very small. However, earlier tests indicate this did not influence inter-estuarine consistency or accordance of scorings.

Conclusions demand survey reliability

Based on the acceptable level of consistence, accordance and the similarity among expert groups scorings, the broad selection of stakeholders, their expertise and their differentiation over estuarine topics; the results of the demand survey can be assigned a high confidence, despite large differences in group size. The legitimacy of this demand survey would surely benefit from a repetition with an even broader selection of stakeholders, including non-technical experts and broader user groups.

Survey two: habitats and supply of ecosystem services


The first objective was to obtain a qualitative supply score for every service by every habitat; “how important is habitat x in delivery of ES y?”.
The second objective was to obtain this information over different salinity zones.
The third objective was to obtain qualitative scorings on the state of the habitats concerning ES, more precisely on overall functional quality (to deliver the bundle of services) and current (decades) trend of the habitats, again in each zone and estuary .


The approach of Burkhard et al (2010, 2012) was adapted for use in the TIDE estuaries. Similarly to their approach, a habitat x ecosystem service matrix was created. For this analysis, six habitat types were distinguished and described (ref Interestuarine comparison: hydro-geomorpology) . Compared to the four TIDE habitats, two low-dynamic habitats were distinguished since these exhibit different functioning. The conditions were that these habitat categories should:
  • cover the entire estuarine gradient
  • allow mapping in all four TIDE estuaries
  • be explicitly and clearly described
  • be easily understandable by a broad selection of stakeholders
  • form ecologically distinguishable functional units
These six habitats were defined as physiotopes based on elevation and slope (see 3.2 and ref “Interestuarine comparison: hydro-geomorphology”) . These habitats were regarded as service providing units (Luck et al 2003, 2009) and used as a basis for scoring the supply of ecosystem services. It should however be kept in mind that these are categories distinguished within a continuous gradient when observing sharp shifts in supply values. Since habitats are regarded per zone there are ecological differences within them along the salinity gradient.

Marsh habitat above mean high water (MHW)
Intertidal steep habitat Between MHW and MLW, slope > 2.5%
Intertidal flat habitat between MHW and MLW, slope < 2.5%
Subtidal shallow habitat between MLW and 2m beneath MLW
Subtidal moderately deep habitat between 2m and 5m beneath MLW
Subtidal deep habitat >5m beneath MLW

This allows to obtain a common habitat definition along the estuaries and salinity gradient.
The habitat x ES matrix consisted of 6 habitat types and 20 ecosystem services (120 intersections) and was scored for each salinity zonation in each estuary yielding in total 16 matrices of 120 scorings each (1920 combinations).

Score Habitat has…in supply of ES
1 no importance
2 very low importance
3 moderate importance
4 Important
5 Essential importance

Additionally, functional quality of habitats and current (decades) trend was scored along estuaries and zones using following respective questions (for every habitat, zone and estuary):

Score Description: This habitat has…
1 very bad supply quality in this zone
2 bad supply quality in this zone
3 moderate supply quality in this zone
4 good supply quality in this zone
5 very good supply quality in this zone

Score Description: This habitat is…
1 quickly disappearing in this zone
2 decreasing in this zone
3 stagnant in this zone
4 increasing in this zone
5 quickly increasing in this zone

Reliability of the supply survey

Visually, accordance can be tentatively evaluated on fig. 4: standard deviation of ES supply scores per habitat and among zones and estuaries rarely exceeds more than one score category. Although this is implies large variation (e.g. from moderate to low), this indicates that there is no large disagreement on importance of habitats in ES delivery.

The supply survey has an acceptable reliability (alpha = 0.7479), inter-estuarine consistency (ICC-C = 0.748; F-test F(479,1437) = 3.97 p = 5.2e-90; 95%-CI = 0.709 < ICC < 0.783) and accordance (ICC-a = 0.71; F-test F(479,1437) = 3.97 p = 1.44e-15; 95%-CI 0.613 < ICC < 0.779) (R package multilevel version 2.3)
Omega Hierarchical (Omega-H = 0.73 ; 3 factors, minimum residual OLS factor solution) not only confirms the previous tests, but also indicates that the estuaries can be regarded as similar in supply, which adds to the validity of the data used.

Quality and trend habitat surveys have a ‘questionable’ and ‘unacceptable’ consistency (alpha 0.687 and 0.157 respectively).

Respondents and expertise

The respondent group for the supply survey consisted of a smaller professional expert group of 12 people, locally selected based on professional knowledge, acquaintance with the ES concept and direct (field) involvement in the estuary (see appendix II).

Evaluation of physical reality of patterns from supply scores.

By performing a principal component analysis (PCA), variability in scores along the spatial units (zones, habitats) of all 20 ecosystem services is evaluated using two axes which display the maximal variation. The pattern which emerges can be evaluated to learn whether scorings represent a physical reality. For the supply scores, result of this PCA analysis is shown in fig. 5.

From this analysis, two main conclusions were drawn: (1) Scores did not differ between salinity zones. This shows that only few services are perceived to be delivered differently over the salinity gradient (see 5.5), but also that inter-zone variations in habitat surfaces did not influence scorings (e.g. proportion of marsh surface differs between zones, which could have influenced importance scorings).This again means scores can be applied in mapping further calculations involving surface. (2) The habitats display a distinct functional pattern along the tidal gradient (marsh to subtidal deep from left to right along first component). This confirms that supply scores represent a physical understanding of the system. Results are further discussed in section 4.

Conclusions supply survey reliability

The supply survey has an acceptable reliability, consistency and agreement (alpha, ICC and McDonalds scores >0.7) regardless the severity of the applied statistical procedure. The results indicate that scores were given for habitat supply importance regardless their local surface and can considered to be based on a physical understanding of the system.
However, the quality and trend scores are to be approached with great caution and data are not further used in TIDE calculations. Although estuaries are similar in ES supply, the differences in quality and trend might be very large between estuaries, yielding these low consistencies. Yet, it is not possible to distinguish differences between respondent groups for this question from existing inter-estuarine differences. Internal double checks or different rater groups within estuaries would have been needed for this.

Conclusion: valid use of results of the surveys

Use within TIDE

The data of the demand and supply survey are used within the TIDE project to derive general patterns and perform comparisons at the scale of salinity zones. Also, screenings of estuarine measures are performed, using the general conclusions from the surveys. This screening however is strictly indicative. When scaling down to actual local contexts, e.g. for quantifying or valuing services of a specific measure, data have to be used with much caution. Actual local demand as well as supply might differ from the average scores assigned to the zone level. This means that a different set of focal ES might be selected as being important locally, and that supply of services by habitats is increased or hampered by specific local features. The data and derived maps should be interpreted as such, and not be used as a basis for specific local management decisions. However, as test results above have proven, the survey methodology yields reliable results on large (estuary) scale.

Linking ES with scientific evidence

Survey data should be regarded as perceived importance of services and habitat supply. This perception, although consistent among respondents of different estuaries and fields of scientific expertise, is still a perception as is, dependent on the personal expertise and variable scientific background of the respondents. This is why statistical and argumentative checks are indispensable to interpret the results.
As respondents did not argument their choices for scorings, it is hard to evaluate the amount of scientific evidence available for each score. The amount of evidence can vary from “absent” to “abundant”, and the accordance of evidence can vary from “generally accepted” to “scientific dispute”.
Determining the amount and agreement of scientific evidence behind these perceptions is necessary to integrate system understanding among different expertise fields, determine knowledge gaps and discussion points for further fundamental research and increase rooting of ecosystem service research in natural sciences.

Use of survey method and results in similar estuaries

The survey data are valid for evaluation of spatial patterns on the salinity zone scale and indicative screenings of potential trade-offs and synergies, potential effects of measures on overall ES bundle delivery and deriving general supply scores for ES bundles per habitat. The ‘acceptable’ level of inter-estuarine accordance between estuaries suggest that these data could be representative for other estuaries, provided they are similar to the TIDE estuaries. Moreover, the survey methodology is a valid tool to be applied in order to obtain proper assessments.

Back to top