Ribeiro, Krainski, Carvalho, and Pina: The influence of socioeconomic deprivation, access to healthcare and physical environment on old-age survival in Portugal

The influence of socioeconomic deprivation, access to healthcare and physical environment on old-age survival in Portugal

Abstract

Spatial inequalities in old-age survival exist in Portugal and might be associated with factors pertaining to three distinct domains: socioeconomic, physical environmental and healthcare. We evaluated the contribution of these factors on the old-age survival across Portuguese municipalities deriving a surrogate measure of life expectancy, a 10-year survival rate that expresses the proportion of the population aged 75-84 years old who reached 85-94. As covariates we used two internationally comparable multivariate indexes: the European deprivation index and the multiple physical environmental deprivation index. A national index was developed to evaluate the access to healthcare. Smoothed rates and odds ratios (OR) were estimated using Bayesian spatial models. Socioeconomic deprivation was found to be the most relevant factor influencing old-age survival in Portugal [women: least deprived areas OR=1.132(1.064-1.207); men OR=1.044(1.001-1.094)] and explained a sizable amount of the spatial variance in survival, especially among women. Access to healthcare was associated with old-age survival in the univariable model only; results lost significance after adjustment for socioeconomic circumstances [women: higher access to healthcare OR=1.020(0.973-1.072); men OR=1.021(0.989-1.060)]. Physical environmental deprivation was unrelated with old-age survival. In conclusion, socioeconomic deprivation was the most important determinant in explaining spatial disparities in old-age survival in Portugal, which indicates that policy makers should direct their efforts to tackle socioeconomic differentials between regions.





Introduction

In high-income countries, premature mortality has plateaued at very low levels, and, consequently, old-age survival is now the mechanism that regulates life-expectancy (Rossi et al., 2013). Old-age survival is then a good general indicator of population health and development (Huisman et al., 2004). Although overall mortality variation decreased, survivors have become increasingly heterogeneous with respect to mortality risk, which suggest that mortality inequalities are shifting to older ages as survival in early life improves (Huisman et al., 2004; Engelman et al., 2010).

Despite being a relatively small country, spatial inequalities have been identified in Portugal (Santana, 2015), particularly regarding cardiovascular disease (Ferreira-Pinto et al., 2012), fractures (de Pina et al., 2008), cancer (Alves et al., 2016), suicide (Santana et al., 2015a), tuberculosis (Apolinário et al., 2017) and all-cause mortality (Santana et al., 2015b). Moreover, recently, important spatial inequalities in the distribution of old-age survival have been revealed (Ribeiro et al., 2016a, 2016b). A myriad of factors can account for the spatial inequalities in old-age survival that include a complex network of factors of different natures affects population health over the time (Ribeiro et al., 2016b).

Socioeconomic factors may play an important role in explaining these spatial differentials, as the association between socioeconomic position and health is one of the oldest and most solid findings in public health (Mackenbach et al., 2008). However, diverse studies have shown that in southern Europe the association between health and socioeconomic deprivation tends to be rather modest compared to western Europe (Mackenbach et al., 2008; Gotsens et al., 2013; Borrell et al., 2014; Hoffmann et al., 2014; Mari-Dell’Olmo et al., 2015). It is then crucial to explore the influence of other key health determinants, such as the physical environment (the material features that surrounds population and includes physical, chemical and biological agents external to the human body (Porta, 2001) and access to healthcare, i.e. the ability to obtain appropriate health services when needed (Obrist et al., 2007). Regarding, physical environment, there is considerable evidence that the characteristics of the physical environment contribute to extend or shorten life expectancy among older adults (Takano et al., 2002; Lv et al., 2011; Robine et al., 2012). From all age groups, the elderly are certainly the most affected by the impact of climate extremes (Yu et al., 2012) and air pollution (Bell et al., 2013). It is also important to note that detrimental physical environments are not randomly distributed. Several studies have found that physical and socioeconomic deprivations coincide in space (the so-called environmental injustice) (Fecht et al., 2015). Consequently, these two items should be taken into account when addressing health inequalities (Lee, 2002). Similarly, access to healthcare is a vital aspect, especially at older ages, due to the heavy burden of chronic diseases and to higher susceptibility to infections. The likelihood of surviving beyond a certain age is certainly affected by the use of healthcare resources (Vogt and Vaupel, 2015). The importance of healthcare is patent in several European studies, which reveals that the fast increase in old-age survival of the past decades can be mostly attributed to improvements in healthcare (Mackenbach et al., 2011; Peters et al., 2015; Vogt and Vaupel, 2015).

These intricate relations between population health and socioeconomic, physical and healthcare factors can only be understood with the use of theoretically sound and validated indicators that grasp the multifactorial nature of these influences (Wills and Briggs, 1995). Multivariate ecological indexes of socioeconomic deprivation are becoming common place (Department for Communities and Local Government, 2011; Guillaume et al., 2016) as epidemiology research shifts from its traditional biomedical focus to an eco-social approach. Yet, multivariate indexes about the physical environment and access to healthcare (at least specific to older populations) are still uncommon. Only recently has this kind of indexes become available in Portugal. In 2016, a multivariate index of socioeconomic deprivation was created under robust methods and theories (Guillaume et al., 2016; Ribeiro et al., 2017) and started to be used to study the link between deprivation and health outcomes at the individual level (Antunes et al., 2016) and also at the ecological level (Ribeiro et al., 2016a). In 2015, a multivariate index of physical environment deprivation (MEDIx) for Portuguese municipalities was developed using sound and internationally validated methodologies (Ribeiro et al., 2015). Significant and plausible associations between this measure and mortality were found showing its potential to understand the role of physical environment in diverse health outcomes.

In this study we aimed to evaluate the role of the socioeconomic, physical environmental and healthcare factors on old-age survival in Portugal. We derived a measure of old-age survival for the 278 municipalities of Portugal based on census data. As covariates, we used evidence-based ecological indexes.

Materials and Methods

Study area

The study was conducted in Continental Portugal (which excludes the archipelagos of Madeira and Azores) using municipalities as units of analysis. Municipalities are commonly the smallest unit for health data dissemination and, apart from the large urban areas, they tend to be homogeneous in terms of social and economic profile. Two hundred and seventy eight municipalities exist in Continental Portugal with an average population of 36,143 inhabitants in 2011.

Old age survival

Because life expectancy and mortality data in old ages was not disclosed for municipalities, we derived a measure of old-age survival that expresses the probability of the people aged 75-84 years to survive an additional ten years, i.e. surpass the average life expectancy (Ribeiro et al., 2016b):

Eq. 1
gh-12-2-581-e001.jpg

where ri is the ten-year survival rate, i(=1,…,278) the area, y the population aged 85-94 years old in 2011 and n the population aged 75-84 years old ten years before (in 2001). This and similar indicators are straightforward and understandable metrics to estimate survival at advanced ages in small areas, which solely require a time series of population census data (Poulain et al., 2004; Magnolfi et al., 2007; Ribeiro et al., 2016b).

Covariates

Socioeconomic deprivation

The European Deprivation Index (EDI) was used to classify small areas according to their level of socioeconomic deprivation. It was constructed in three steps using both individual and area level census data as has been detailed elsewhere (Guillaume et al., 2016). In brief: i) construction of an individual level indicator of deprivation based on the European Union Statistics on Income and Living Conditions (EU-SILC) information, which is available at http://ec.europa.eu/eurostat/web/microdata/european-union-statistics- on-income-and-living-conditions; ii) identification of variables available both at the individual level (EU-SILC) and at the area level (2001 national population census); and iii) determination, at the individual level, whether the set of area level variables from the census selected at step 2 were associated with the indicators of individual deprivation created in step 1.

The associated census variables were then included in the EDI formula, whose final score was based upon the weighted sum of these variables. The weights were the regression coefficients measuring the association between the indicator of individual deprivation and the variables from the census that were also available at the individual level identified in step 2. The score for Portugal was based upon the weighted sum of the following variables expressed as percent: overcrowded households; households with no bath or shower; household with no indoor flushing toilets; households occupied by non-owners; women aged ≥65; individuals with low education level; individuals in low income occupations; and individuals unemployed as discussed by Ribeiro et al. (2017).

The EDI index was normalised and then classified into ten classes (C1 – the least deprived to C10 – the most). Cut-offs for these theoretical deciles were defined based on standard deviations from the overall mean and customised so that the classes included a more even number of observations (cut-offs = -1.28, -0.84, -0.52, -0.25, 0, 0.25, 0.52, 0.84, and 1.28). This approach avoids the wellknown problems of using empirical quintiles, which assume homogeneity of risk within groups (Bennette and Vickers, 2012). The geographic EDI score distribution across Portuguese municipalities is shown in Figure 1 (classes correspond to quintiles instead of deciles to facilitate visualisation).

Access to healthcare

Some measures of access to healthcare have been developed in Portugal, but these were for the overall population (not exclusively for the elderly) and/or they only accounted for hospital services (Polzin et al., 2014; Santana, 2015). Therefore we derived an index of access to healthcare for the older population starting by retrieving all datasets on healthcare availability and accessibility (the two domains of healthcare access for which data are available). Variables were obtained at the municipality level for the year 2001 (whenever possible) and for Continental Portugal from two data sources: Hospitals and Primary Care Centers Surveys from Instituto Nacional de Estatística (INE) - National Institute of Statistics (INE, 2001a, 2001b) and Social Map from the Ministry of Solidarity, Employment and Social Security (Carta social, 2008). From 49 datasets, those with too many missing/censored and zero values were discharged (n=16 datasets selected).

We calculated the rates to express the population exposure to the variables created (n=16), which were then characterised and transformed to become more normally distributed. Subsequently, bivariate correlations were computed to identify variables excessively correlated and therefore discharged (n=10 variables selected). Finally, principal component analysis was run to derivate a summary measure expressing access to healthcare in each municipality. The three principal components that explained 72% of the variability in the latent variable access to healthcare were as follows. First, availability of long-term care and social support facilities including the capacities of the day-care centres; the nursing homes; and home care. Second, availability and geographical accessibility to healthcare facilities - population weighted mean distance to public hospitals (maternities and paediatric hospitals excluded) computed using the Network Analyst extension of ArcGIS 10.4.1 (ESRI, Redlands, CA, USA) and a street network dataset provided courtesy of ESRI; primary care centres including extensions; pharmacies; and mobile pharmacy posts. Third, availability of health professionals – medical doctors by place of residence; nurses by place of work; dentists by place of residence; and pharmacists by place of work.

For each municipality, i, each component score was multiplied by the proportion of the variation explained.

Eq. 2
HCA scorei = 0.25340 * 1st componenti + 0.23857 * 2nd componenti + 0.22924 * 3rd componenti

Similarly to socioeconomic deprivation, after standardisation, the index was categorised into 10 classes based on theoretical deciles. The geographic distribution of the index across Portuguese municipalities is shown in Figure 1 (again classes correspond to quintiles).

Physical environmental deprivation

The measure of multiple physical environmental deprivation (PT-MEDIx) was built at the municipality level using data from the years 2001 and 2011 and developed in four stages fully described elsewhere (Ribeiro et al., 2015). The PT-MEDIx covered five dimensions of the physical environment: air pollution (particulate matter, nitrogen dioxide, carbon monoxide); climate (temperature); drinking water quality (trihalomethanes and nitrates); green space availability; and industry proximity. Municipalities in the highest quintile of exposure received a score of +1 for harmful factors and -1 for beneficial factors. The PT-MEDIx of each municipality equalled the sum of these scores and ranged from -1 (the least environmental deprivation) to +4 (the most). We treated all factors as equal contributors to environmental deprivation because any weighting would be arbitrary without robust evidence (Richardson et al., 2010; Pearce et al., 2011). The geographic distribution of PT-MEDIx across the Portuguese municipalities is shown in Figure 1.

Statistical model

Bayesian Hierarchical Spatial models were used to estimate the effect of each covariate in old-age survival. We assumed that the response variable, number of survivors in each area i and gender j would (Yij) follow a binomial distribution, where pij is an unknown survival rate and nij the population aged 75-84 years old ten years before:

Eq. 3
Yij~Bin (nij, pij)

The logit of the survival rate is modelled considering gender and the interaction between the covariates and gender xij and area:

Eq. 4.1
logit(pij) = ηij = genderj + fj (xi) + si

where ηij is the linear predictor, genderj an intercept specific for each gender, fj(xi) the gender-specific effect of each covariate (which assumes the value xi for the area i), and si the area-specific effect. The function fi assumes a nonlinear effect of the covariates, which is modelled as a first order random-walk prior over the covariates’ classes, i.e. a normal distribution, whose mean at each class is an average over the neighbouring classes (Martino and Rue, 2009). As xi was categorised into classes, we can simplify fj (xi) into elj, which denotes the effect of covariate class l for gender j.

The area-specific effect si was modelled considering a Besag, York and Mollie’s (BYM) model (Besag et al., 1991) with a parametrisation as suggested by Dean and colleagues (2001):

Eq. 4.2
gh-12-2-581-e002.jpg

where υi is the structured effect and νi the unstructured effect. The υi effect was scaled in order to make the model more intuitive and interpretable, so that φ expresses the proportion of the spatial effect due to the structured part and 1/τ is the marginal variance of sj. A penalised complexity prior was considered.

Considering the model defined by equations 3 and 4.1, the exponential of the gender main effect is the ratio between the men’s odds of survival and women’s, i.e. the gender odds ratio (OR). The exponential of elj is the ratio between the odds of survival of the covariate class and gender and the overall odds for the entire population – the covariate and gender-specific OR. For example, an OR of 1.15 in a certain covariate class (e.g., the least socioeconomically deprived) and gender (e.g., women) means that for that gender and covariate class the odds of survival is 15% higher than the overall odds survival of the entire population of that gender. OR and 95% credible intervals (95% CrI) were derived from their posterior means and quantiles. An OR would be considered significantly higher or lower if its 95% CrI does not include the value 1. Posterior distributions were obtained using the Integrated Nested Laplace Approximation (INLA), which was implemented in the R INLA library (Rue et al., 2009).

The model produced by equation 4.1 was our final, but we started with a simple model, where we only included the genderspecific intercept and the spatial effect and then introduced each covariate successively. The order of entering each covariate was based on the strength and significance of associations observed in the univariable models. These models were also run to assess the unadjusted influence of each covariate. Three measures of goodness of fit, Deviance Information Criteria (DIC), Watanabe-Akaike information criterion (WAIC) and Conditional Predictive Ordinate, (CPO) were used to compare models. The relative reduction in the variance of spatial effect (τ) was also evaluated to ascertain to what extent covariates contributed to explaining the spatial variation of old-age survival. The presence of interactions between covariates was also tested. Finally, we did explore the correlation between the covariates by computing the Pearson’s correlation coefficient (r).

Results

On average, the old-age survival rates were 32.3% (maximum= 39.7; minimum=27.5) among men and 43.7% (67.2; 34.5) among women. The presence of spatial inequalities in the distribution of survival is depicted in Figure 2, showing a nearly two-fold difference between areas. In general terms, higher survival rates were concentrated in the North and Central coasts of the country and in the urban municipalities, whereas the lowest were found concentrated in the South and in the northern inland.

In Figure 3, the survival rates are represented as a function of each covariate. In general, survival rates decreased with socioeconomic deprivation in both genders. On the other hand, survival rates neither seemed to increase nor decrease in a clear fashion according to healthcare access and physical environmental deprivation. We found a significant but moderate correlation between the covariates: physical environmental deprivation was negatively associated with socioeconomic deprivation (r= -0.288, P<0.001) and access to healthcare was also negatively associated with socioeconomic deprivation (r= -0.344, P<0.001).

The results obtained with the univariable and multivariable models are shown in Table 1 (univariable), Table 2 (multivariable, men) and Table 3 (multivariable, women). Among women, in the multivariable model, a rather linear association between old-age survival and socioeconomic deprivation was observed and this association persisted even after the inclusion of the remaining covariates [least deprived areas OR=1.132 (1.064-1.207)] (Table 3). In the null model (no covariates, only spatial effect), the percent variability attributed to the spatial random effect was 65%. We then added one variable each time to assess its impact on old-age survival. The variance attributed to the spatial effect was reduced by 31.4% after including socioeconomic deprivation in the model. The inclusion of the remaining variables did neither cause much change in the adjustment parameters nor in the variance explained by the spatial effect. After adjustment for socioeconomic deprivation the association with healthcare access, which was slightly associated with survival in the univariable model (Table 1), was no longer significantly associated with survival [lowest access to healthcare OR=0.961 (0.901-1.009)]. Similarly, no significant association was found between survival and physical environmental deprivation and no interaction effects were observed between covariates. For men we reached similar results although the magnitude of the associations was considerably smaller.

Socioeconomic deprivation was significantly associated with survival among men [OR=1.044 (1.001-1.094)] (Table 2). In the null model (no covariates, only spatial effect), the percent variability attributed to the spatial random effect was 51%. Compared to what was observed for women, the reduction of spatial variance caused by the introduction of socioeconomic deprivation in men was comparatively smaller (13.1% vs 31.4% in women). As with women, neither access to healthcare nor environmental deprivation were significantly associated with old age survival for men.

Discussion

In this study we aimed at investigating the contribution of three important multidimensional determinants on old-age survival in Portuguese municipalities. We found that socioeconomic deprivation was the most relevant factor, explaining a considerable share of the spatial variance in old-age survival, especially among women. Despite evidence showing that physical environment and healthcare do affect older people’s health (Takano et al., 2002; Lv et al., 2011; Robine et al., 2012; Vogt and Vaupel, 2015), those factors did not play such a major influence as socioeconomic deprivation for older people’s chances of survival.

To date very few studies have compared the relative importance of factors pertaining to different domains, such as social, economic and physical environmental ones (Domínguez-Berjón et al., 2010; Ferreira-Pinto et al., 2012; Hood et al., 2016). These studies have used indicators (outcomes and covariates) that are not directly comparable to ours, but they all acknowledged that socioeconomic deprivation had the strongest effect. The prominent role of socioeconomic deprivation in shaping a population’s health has been the matter of discussion for centuries and its influence is still observed in our time. For instance, in the United Kingdom and the United States, socioeconomic deprivation seems to explain most of the spatial inequalities in health and life expectancy (Woods et al., 2005; Hood et al., 2016). The latter authors report the following relative contributions to health in the United States of socioeconomic factors (47%), health behaviour (34%), clinical care (16%) and the physical environment (3%). However, in our study the association between old-age survival and socioeconomic deprivation was significant but its contribution in explaining spatial effects was modest, only 31% among women and 13% in men. It is plausible that other factors, which we did not measure, counterbalanced the effects of socioeconomic deprivation (e.g., formal and informal social and economic support).

In the city of Porto in northern Portugal, we conducted a similar study (Ribeiro et al., 2016a), where we analysed the impact of the socioeconomic deprivation, built and physical environment on old-age survival across the neighbourhoods of the city. In this study, we found that more than 41% of the differences between neighbourhoods could be attributed to the socioeconomic characteristics of the neighbourhoods. We also found that the measures that described the physical environmental characteristics (built and biogeophysical) of the neighbourhoods were not associated with old-age survival at all. In other words, socioeconomic factors are also the biggest drivers of the spatial differentials in old-age survival in Porto. Although the associations were similar to those we found in the present study that entails the entire country, it is important to point out that the percent of variability explained by socioeconomic deprivation was considerably lower in Portugal as a whole (31% in women and 13% in men) than in Porto. This is likely a consequence of using municipalities as unit of analysis in the present study. These are relatively large areas that might not suffice to detect and capture spatial inequalities and associations whenever there is considerable within-area variability in outcomes and covariates.

Our current study is ecological in nature and, consequently, we could not ascertain causal relations neither the mechanism by which socioeconomic aspects affects survival. There are numerous theories trying to conceptualise that. One of the most relevant is the (neo)material model, which states that most deprived people have poorer health due to lack of material conditions at home and in the living context (work, school, neighbourhood, region) being particularly relevant (Skalická et al., 2009). However, our results do not fully accord to this theory. For instance, and contrasting to other studies (Pearce et al., 2010), we did not find socioeconomic deprivation to be directly related to physical environmental deprivation expressed by us as a combination of different exposures (green space availability, air/water pollution, climate). Indeed, we observed the exact opposite (negative correlation, r=-0.288, P<0.001) showing taht affluent areas had the poorest physical environmental conditions. But, we did find evidence that healthcare is less available in more deprived areas (negative correlation, r=-0.354, P<0.001) suggesting that some form of environmental unfairness exists in Portugal, as observed in another national study (Nogueira, 2010).

Physical environment did not affect old-age survival in our study, which was also found in the Porto study (Ribeiro et al., 2016a). Poor physical environments were concentrated in affluent urbanised areas (where there is a concentration of pollution sources, such as industry and traffic) but the positive influence of having good material resources and facilities (e.g., healthcare, jobs, housing conditions, etc.) might conceal the detrimental effects of living in a more hazardous environment. The absence of an association with physical environmental aspects might also be attributed to the mortality patterns in very old population strata; among the oldest, the top mortality cause is cardiovascular disease (CVD, responsible for over 40% of the deaths after the 85 years old). In a previous work about the development of PT-MEDIX (Ribeiro et al., 2015), we did not find a significant association between physical environmental deprivation and CVD, but, on the other hand, we observed a strong dose-response relation with cancer mortality. Therefore, the impact of physical environment might be modest among the oldest population groups that are less affected by cancer mortality.

Despite the evidence stating that access to healthcare play an important role in extending life expectancy and survival (Mackenbach et al., 2011; Peters et al., 2015; Vogt and Vaupel, 2015) we found that after controlling for socioeconomic characteristics, access to healthcare was no longer associated with old agesurvival. Indeed a dozen of studies have found that, when compared with socioeconomic factors, access to healthcare play a much smaller role (Ferreira-Pinto et al., 2012; Kim, 2014; Kim and Kim, 2014). Social and economic conditions adversely affect people’s ability to access healthcare (e.g., ability to pay for travelling and medical costs) and to understand health information (Hood et al., 2016), which may exert a much stronger influence than the availability, quantity and/or geographical accessibility to healthcare (the components of healthcare access captured in our index (Obrist et al., 2007) in the Portuguese context.

The main limitation of this study is related with the use of aggregated data. Scale might have influenced our results. We conducted this analysis at the municipality level and this unit can have as few as 1,830 inhabitants in Portugal or hold over 500,000 inhabitants, and 30% of the Portuguese population resides in municipalities with >150,000 inhabitants (INE, 2016). Consequently, we might have failed to detect important associations and inequalities. This may also explain the different proportion of explained variability we observed in Porto (41%) compared with Portugal as a whole (between 13 and 31%). In Porto, we had the opportunity of using a much smaller geographical unit, which is more appropriate for small-area studies of environment and health, since this approach minimises within-area variation, is better to control for potential confounding across areas and captures slight variations in outcomes and covariates (Elliott and Savitz, 2008). However, due to lack of high-resolution data for the entire country, we could only use municipalities as the unit of analysis. The Modifiable Areal Unit Problem (MAUP) is another potential source of bias. A different arrangement of the spatial units might have yielded different results.

Another plausible limitation was that our study was grounded on the assumption that people have lived in the same area during 10 years. However, results from census and original research support the belief that our results are not driven by migration patterns: only about 6% of the Portuguese reported to reside in another geographical unit five years ago and, according to Tatsiramos and colleagues (2006), migration of those aged ≥75 years is infrequent, especially in southern Europe (~1.0 to 1.5%). Moreover, using data from EPIPorto cohort, a landmark epidemiological cohort study in Portugal that has been ongoing for over 15 years (http://ispup.up.pt/research/researchstructures/ cohorts/), we were able to estimate the frequency of residential mobility in a large Portuguese city. The cohort was constituted in 1999-2003 comprising a representative sample of 2,485 adults (≥18 years of age) residing in Porto municipality (Ramos et al., 2004). From wave 1 (1999-2003) to wave 2 (2005-2008) of the cohort, a 6-year period, 6.5% (n=162) of the participants changed their neighbourhood of residence, but this percentage was significantly lower among the 75 years olds (n=4; mobility 2.0%). Furthermore, evidence suggests that residential mobility is most likely to cause an underestimation of spatial inequalities and socioeconomic effects (Bryere et al., 2015).

Finally, the location of nursing homes might have lower survival among their community residents, and this could have influenced the analysis (Shah et al., 2013). Presence of nursing homes, in particular in municipalities, could lead to an underestimation of old-age survival. However, we did not find any match between the spatial distribution of the areas of high and low survival and that of Portuguese nursing homes. Finally, we were not able to evaluate the role of other potentially important aspects, like social support or certain features of the built environment, which might account for the remaining spatial effect that our statistical model was not able to explain. Our study has numerous strengths as well. First, very few studies have dealt with three important determinants of human health and survival: socioeconomic deprivation, physical environment and access to healthcare. More importantly, we have used robust measures that express how advantaged or disadvantaged small areas are in terms of socioeconomic circumstances, physical environment and access to healthcare. These measures were constructed based on sound theories and methods, which allow us to be confident about the study findings. The EDI and PTMEDIX were built for other countries, with which data our results are internationally comparable. Also, multivariable indexes contribute to a better understanding and monitoring of multidimensional phenomena as they measure the cumulative burden of health detrimental factors at population level (Wills and Briggs, 1995; Corvalán et al., 2000). Statistically speaking, we have used robust spatial statistics that allowed us to account for the small number problem and extract the true spatial pattern of old-age survival in Portugal. Finally, we also accounted for spatial autocorrelation as we employed a BYM framework to model the spatial effect.

Conclusions

Important spatial inequalities in the distribution of old-age survival across Portuguese municipalities were discovered. Socioeconomic deprivation was found to be the most important determinant of old-age survival. However, further studies are needed to identify the unaccounted factors that might explain spatial differentials in old-age survival. Our results suggest policy makers should direct their efforts to tackle socioeconomic differentials between regions and guarantee equitable distribution of the healthcare resources.

Acknowledgments

This work was supported by Portuguese funds through FCT – Fundação para a Ciência e a Tecnologia in the framework of project UID/BIM/04293/2013. AIR would also like to thank to FCT – Fundação para a Ciência e a Tecnologia for the PhD grant SFRH/BD/82529/2011. MSC was supported by CNpQ (309692/2013-0) and FAPERJ (E-26/203.557/2014). The EPIUnit, Institute of Public Health, University of Porto, Portugal (UID/DTP/04750/2013) is funded by the FCT – Fundação para a Ciência e a Tecnologia.

References

1 

J Alves, C Nunes, J Perelman, 2016. Socio-economic inequalities in tobacco-related diseases in Portugal: an ecological approach. Publ Health 130:36-42.

2 

L Antunes, D Mendonça, MJ Bento, B Rachet, 2016. No inequalities in survival from colorectal cancer by education and socioeconomic deprivation - a population-based study in the North Region of Portugal, 2000-2002. BMC Cancer 16:608.

3 

D Apolinário, AI Ribeiro, E Krainski, P Sousa, Mm Duarte R Abranches, 2017. Tuberculosis inequalities and socioeconomic deprivation. Int J Tuberc Lung Dis 21:784-9.

4 

ML Bell, A Zanobetti, F Dominici, 2013. Evidence on vulnerability and susceptibility to health risks associated with short-term exposure to particulate matter: a systematic review and metaanalysis. Am J Epidemiol 178:865-76.

5 

C Bennette, A Vickers, 2012. Against quantiles: categorization of continuous variables in epidemiologic research, and its discontents. BMC Med Res Method 12:21-2.

6 

J Besag, J York, A Mollié, 1991. Bayesian image restoration, with two applications in spatial statistics. Ann Inst Stat Math 43:1-20.

7 

C Borrell, M Mari-Dell’olmo, L Palencia, M Gotsens, BO Burstrom, F Dominguez-Berjon, M Rodriguez-Sanz, D Dzurova, A Gandarillas, R Hoffmann, K Kovacs, C Marinacci, P Martikainen, H Pikhart, D Corman, K Rosicova, M Saez, P Santana, L Tarkiainen, R Puigpinos, J Morrison, MI Pasarin, E Diez, 2014. Socioeconomic inequalities in mortality in 16 European cities. Scand J Public Health 42:245-54.

8 

J Bryere, C Pornet, O Dejardin, L Launay, L Guittet, G Launoy, 2015. Correction of misclassification bias induced by the residential mobility in studies examining the link between socioeconomic environment and cancer incidence. Cancer Epidemiol 39:256-64.

9 

Social Carta, 2008. Capacidade das respostas sociais. In Gabinete de Estratégia e Planeamento, Ministério da Solidariedade, Emprego e Segurança Social, Lisboa, Portugal.

10 

C Corvalán, D Briggs, G Zielhuis, 2000. Decision-making in environmental health - from evidence to action. E & FN Spon, London, UK.

11 

MF de Pina, SM Alves, M Barbosa, H Barros, 2008. Hip fractures cluster in space: an epidemiological analysis in Portugal. Osteopor Int 19:1797.

12 

CB Dean, MD Ugarte, AF Militino, 2001. Detecting interaction between random region and fixed age effects in disease mapping. Biometrics 57:197-202.

13 

MF Domínguez-Berjón, A Gandarillas, del Pozo J Segura, B Zorrilla, MJ Soto, L López, I Duque, MI Marta, I Abad, 2010. Census tract socioeconomic and physical environment and cardiovascular mortality in the Region of Madrid (Spain). J Epidemiol Commun Health 64:1086-93.

14 

P Elliott, DA Savitz, 2008. Design issues in small-area studies of environment and health. Environ Health Perspect 116:1098-104.

15 

M Engelman, V Canudas-Romo, EM Agree, 2010. The implications of increased survivorship for mortality variation in aging populations. Popul Dev Rev 36:511-39.

16 

D Fecht, P Fischer, L Fortunato, G Hoek, K de Hoogh, M Marra, H Kruize, D Vienneau, R Beelen, A Hansell, 2015. Associations between air pollution and socioeconomic characteristics, ethnicity and age profile of neighbourhoods in England and the Netherlands. Environ Pollut 198:201-10.

17 

LM Ferreira-Pinto, F Rocha-Gonçalves, A Teixeira-Pinto, 2012. An ecological study on the geographic patterns of ischaemic heart disease in Portugal and its association with demography, economic factors and health resources distribution. BMJ Open 2:e000595.

18 

M Gotsens, M Mari-Dell’Olmo, K Perez, L Palencia, Beneito MA Martinez-, M Rodriguez-Sanz, B Burstrom, G Costa, P Deboosere, F Dominguez-Berjon, D Dzurova, A Gandarillas, R Hoffmann, K Kovacs, C Marinacci, P Martikainen, H Pikhart, K Rosicova, M Saez, P Santana, J Riegelnig, C Schwierz, L Tarkiainen, C Borrell, 2013. Socioeconomic inequalities in injury mortality in small areas of 15 European cities. Health Place 24:165-72.

19 

E Guillaume, C Pornet, O Dejardin, L Launay, R Lillini, M Vercelli, M Marí-Dell’Olmo, Fontelo A Fernández, C Borrell, AI Ribeiro, MFd Pina, A Mayer, C Delpierre, B Rachet, G Launoy, 2016. Development of a cross-cultural deprivation index in five European countries. J Epidemiol Commun Health 70:493-9.

20 

R Hoffmann, G Borsboom, M Saez, Dell’Olmo M Mari, B Burstrom, D Corman, C Costa, P Deboosere, MF Dominguez-Berjon, D Dzurova, A Gandarillas, M Gotsens, K Kovacs, J Mackenbach, P Martikainen, L Maynou, J Morrison, L Palencia, G Perez, H Pikhart, M Rodriguez-Sanz, P Santana, C Saurina, L Tarkiainen, C Borrell, 2014. Social differences in avoidable mortality between small areas of 15 European cities: an ecological study. Int J Health Geogr 13:8.

21 

CM Hood, KP Gennuso, GR Swain, BB Catlin, 2016. County health rankings: relationships between determinant factors and health outcomes. Am J Prev Med 50:129-35.

22 

M Huisman, AE Kunst, O Andersen, M Bopp, JK Borgan, C Borrell, G Costa, P Deboosere, G Desplanques, A Donkin, S Gadeyne, C Minder, E Regidor, T Spadea, T Valkonen, JP Mackenbach, 2004. Socioeconomic inequalities in mortality among elderly people in 11 European populations. J Epidemiol Commun Health 58:468-75.

23 

INE, 2001a. Inquérito aos centros de saúde. Instituto Nacional de Estatística, Lisboa, Portugal.

24 

INE, 2001b. Inquérito aos hospitais. Instituto Nacional de Estatística, Lisboa, Portugal.

25 

INE, 2016. População residente (N.º) por local de residência (NUTS - 2013), sexo e grupo etário; anual - INE, estimativas anuais da população residente. Instituto Nacional de Estatística, Lisboa, Portugal.

26 

J Kim, 2014. Association between social factors of health ageing and longevity: determinants of the longevity index (LI) in OECD Countries. Ageing Int 39:97-105.

27 

JI Kim, G Kim, 2014. Factors affecting the survival probability of becoming a centenarian for those aged 70, based on the human mortality database: income, health expenditure, telephone, and sanitation. BMC Geriatrics 14:113.

28 

C Lee, 2002. Environmental justice: building a unified vision of health and the environment. Environ Health Perspect 110:141-4.

29 

J Lv, W Wang, Y Li, 2011. Effects of environmental factors on the longevous people in China. Arch Gerontol Geriatr 53:200-5.

30 

JP Mackenbach, L Slobbe, CW Looman, Heide A van der, J Polder, J Garssen, 2011. Sharp upturn of life expectancy in the Netherlands: effect of more health care for the elderly? Eur J Epidemiol 26:903-14.

31 

JP Mackenbach, I Stirbu, AJ Roskam, MM Schaap, G Menvielle, M Leinsalu, AE Kunst, 2008. Socioeconomic inequalities in health in 22 European countries. New Engl J Med 358:2468-81.

32 

SU Magnolfi, E Petruzzi, P Pinzani, F Malentacchi, M Pazzagli, FM Antonini, 2007. Longevity index (LI%) and centenarity index (CI%): new indicators to evaluate the characteristics of aging process in the Italian population. Arch Gerontol Geriatr 44:271-6.

33 

M Mari-Dell’Olmo, M Gotsens, L Palencia, B Burstrom, D Corman, G Costa, P Deboosere, E Diez, F Dominguez-Berjon, D Dzurova, A Gandarillas, R Hoffmann, K Kovacs, P Martikainen, M Demaria, H Pikhart, M Rodriguez-Sanz, M Saez, P Santana, C Schwierz, L Tarkiainen, C Borrell, 2015. Socioeconomic inequalities in cause-specific mortality in 15 European cities. J Epidemiol Commun Health 69:432-41.

34 

S Martino, H Rue, 2009. Implementing approximate bayesian inference using integrated nested Laplace approximation: a manual for the inla program. Available from: https://www.math.ntnu.no/~hrue/GMRFsim/manual.pdf

35 

HG Nogueira, 2010. Deprivation amplification and health promoting resources in the context of a poor country. Soc Sci Med 70:1391-5.

36 

B Obrist, N Iteba, C Lengeler, A Makemba, C Mshana, R Nathan, S Alba, A Dillip, MW Hetzel, I Mayumana, A Schulze, H Mshinda, 2007. Access to health care in contexts of livelihood insecurity: a framework for analysis and action. PLoS Medicine 4:e308.

37 

JR Pearce, EA Richardson, RJ Mitchell, NK Shortt, 2010. Environmental justice and health: the implications of the socio-spatial distribution of multiple environmental deprivation for health inequalities in the United Kingdom. Trans Inst Br Geogr 35:522-39.

38 

JR Pearce, EA Richardson, RJ Mitchell, NK Shortt, 2011. Environmental justice and health: A study of multiple environmental deprivation and geographical inequalities in health in New Zealand. Soc Sci Med 73:410-20.

39 

F Peters, W Nusselder, N Reibling, C Wegner-Siegmundt, J Mackenbach, 2015. Quantifying the contribution of changes in healthcare expenditures and smoking to the reversal of the trend in life expectancy in the Netherlands. BMC Public Health 15:1-9.

40 

P Polzin, J Borges, A Coelho, 2014. An extended kernel density two-step floating catchment area method to analyze access to health care. Environ Plann B Plann Design 41:717-35.

41 

M Porta, 2001. A dictionary of epidemiology. Oxford University Press, Oxford, UK.

42 

M Poulain, GM Pes, C Grasland, C Carru, L Ferrucci, G Baggio, C Franceschi, L Deiana, 2004. Identification of a geographic area characterized by extreme longevity in the Sardinia island: the AKEA study. Exp Gerontol 39:1423-9.

43 

E Ramos, C Lopes, H Barros, 2004. Investigating the effect of nonparticipation using a population-based case-control study on myocardial infarction. Ann Epidemiol 14:437-41.

44 

AI Ribeiro, ET Krainski, R Autran, H Teixeira, MS Carvalho, MdF de Pina, 2016a. The influence of socioeconomic, biogeophysical and built environment on old-age survival in a Southern European city. Health Place 41:100-9.

45 

AI Ribeiro, ET Krainski, MS Carvalho, MdFd Pina, 2016b. Where do people live longer and shorter lives? An ecological study of old-age survival across 4404 small areas from 18 European countries. J Epidemiol Commun Health 70:551-6.

46 

AI Ribeiro, A Mayer, A Miranda, MF Pina, 2017. The Portuguese version of the European deprivation index: an instrument to study health inequalities. Acta Médica Portuguesa 30:17-25.

47 

AI Ribeiro, MdFd Pina, R Mitchell, 2015. Development of a measure of multiple physical environmental deprivation. After United Kingdom and New Zealand, Portugal. Eur J Pub Health 25:610-7.

48 

EA Richardson, R Mitchell, NK Shortt, J Pearce, TP Dawson, 2010. Developing summary measures of health-related multiple physical environmental deprivation for epidemiological research. Environment and Planning A 42:1650-68.

49 

J-M Robine, FR Herrmann, Y Arai, DC Willcox, Y Gondo, N Hirose, M Suzuki, Y Saito, 2012. Exploring the impact of climate on human longevity. Exp Gerontol 47:660-71.

50 

IA Rossi, V Rousson, F Paccaud, 2013. The contribution of rectangularization to the secular increase of life expectancy: an empirical study. Int J Epidemiol 42:250-8.

51 

H Rue, S Martino, F Lindgren, D Simpson, A Riebler, 2009. INLA: Functions which allow to perform full Bayesian analysis of latent Gaussian models using Integrated Nested Laplace Approximation. R Package Version.

52 

P Santana, 2015. A Geografia da Saúde da População. Evolução nos últimos 20 anos em Portugal Continental. Centro de Estudos em Geografia e Ordenamento do Território (CEGOT), University of Coimbra, Coimbra, Portugal.

53 

P Santana, C Costa, G Cardoso, A Loureiro, J Ferrão, 2015a. Suicide in Portugal: Spatial determinants in a context of economic crisis. Health Place 35:85-94.

54 

P Santana, C Costa, M Marí-Dell’Olmo, M Gotsens, C Borrell, 2015b. Mortality, material deprivation and urbanization: exploring the social patterns of a metropolitan area. Int J Equity Health 14:55.

55 

SM Shah, IM Carey, T Harris, S DeWilde, DG Cook, 2013. Mortality in older care home residents in England and Wales. Age Ageing 42:209-15.

56 

V Skalická, F van Lenthe, C Bambra, S Krokstad, J Mackenbach, 2009. Material, psychosocial, behavioural and biomedical factors in the explanation of relative socio-economic inequalities in mortality: evidence from the HUNT study. Int J Epidemiol 38:1272-84.

57 

T Takano, K Nakamura, M Watanabe, 2002. Urban residential environments and senior citizens’ longevity in megacity areas: the importance of walkable green spaces. J Epidemiol Commun Health 56:913-8.

58 

K Tatsiramos, 2006. Residential mobility and housing adjustment of older households in Europe. Available from: http://ftp.iza.org/dp2435.pdf

59 

TC Vogt, JW Vaupel, 2015. The importance of regional availability of health care for old age survival - Findings from German reunification. Popul Health Metr 13:26.

60 

J Wills, D Briggs, 1995. Developing indicators for environment and health. World Health Stat Quart 48:155-63.

61 

LM Woods, B Rachet, M Riga, N Stone, A Shah, MP Coleman, 2005. Geographical variation in life expectancy at birth in England and Wales is largely explained by deprivation. J Epidemiol Commun Health 59:115-20.

62 

W Yu, K Mengersen, X Wang, X Ye, Y Guo, X Pan, S Tong, 2012. Daily average temperature and mortality among the elderly: a meta-analysis and systematic review of epidemiological evidence. Int J Biometeorol 56:569-81.

Figure 1.

Spatial distribution of socioeconomic deprivation, access to healthcare and physical environmental deprivation in continental Portugal.

gh-12-2-581-g001.jpg
Figure 2.

Spatial distribution of old-age survival rates posterior means in continental Portugal.

gh-12-2-581-g002.jpg
Figure 3.

Old-age survival rates posterior means as a function of socioeconomic deprivation, access to healthcare and physical environmental deprivation in continental Portugal.

gh-12-2-581-g003.jpg
Table 1.

Univariable associations among old-age survival and socioeconomic deprivation, access to healthcare and physical environmental deprivation.

OR (95% CrI)
Men Women
Socioeconomic deprivation    
1* 1.051 (1.008-1.101) 1.146 (1.079-1.222)
2 1.042 (1.008-1.080) 1.087 (1.038-1.138)
3 1.040 (1.004-1.086) 1.083 (1.031-1.142)
4 1.004 (0.971-1.035) 1.029 (0.986-1.074)
5 0.974 (0.935-1.007) 0.994 (0.951-1.037)
6 0.991 (0.959-1.026) 0.986 (0.943-1.033)
7 0.991 (0.954-1.031) 0.937 (0.884-0.986)
8 0.983 (0.947-1.021) 0.925 (0.876-0.971)
9 0.969 (0.931-1.005) 0.940 (0.894-0.991)
10° 0.959 (0.915-1.001) 0.903 (0.850-0.955)
Access to healthcare    
1# 1.029 (0.993-1.070) 1.036 (0.981-1.094)
2 1.036 (1.005-1.074) 1.055 (1.010-1.107)
3 1.023 (0.995-1.056) 1.037 (0.994-1.088)
4 1.010 (0.982-1.040) 1.004 (0.957-1.046)
5 1.003 (0.975-1.034) 1.014 (0.971-1.065)
6 0.983 (0.948-1.011) 0.986 (0.938-1.030)
7 0.986 (0.957-1.014) 0.978 (0.934-1.019)
8 0.982 (0.949-1.012) 0.997 (0.952-1.057)
9 0.975 (0.942-1.004) 0.976 (0.934-1.023)
10§ 0.976 (0.937-1.012) 0.923 (0.862-0.982)
Physical environment    
1^ 0.979 (0.931-1.022) 0.960 (0.901-1.014)
0 0.992 (0.961-1.023) 0.965 (0.923-1.004)
1 0.974 (0.937-1.004) 0.967 (0.922-1.006)
2 0.992 (0.958-1.021) 1.012 (0.974-1.055)
3 1.026 (0.992-1.070) 1.042 (0.996-1.035)
4$ 1.039 (0.989-1.106) 1.058 (0.992-1.147)

[i] OR, odds ratio; CrI, credible interval.

[ii] *Least deprived

[iii] °most deprived

[iv] #higher

[v] §lower

[vi] ^least environmentally deprived

[vii] $most environmentally deprived. Statistically significant results are in italics.

Table 2.

Association among old-age survival and socioeconomic deprivation, access to healthcare and physical environmental deprivation (men).

Model 1 OR (95% CrI) (socioeconomic deprivation only) Model 2 OR (95% CrI) (plus healthcare) Model 3 OR (95% CrI) (plus healthcare and physical environment)
Socioeconomic deprivation      
1* 1.051 (1.008-1.101) 1.043 (1.001-1.092) 1.044 (1.001-1.094)
2 1.042 (1.008-1.080) 1.035 (1.001-1.073) 1.035 (1.001-1.073)
3 1.040 (1.004-1.086) 1.034 (0.999-1.079) 1.033 (0.999-1.077)
4 1.004 (0.971-1.035) 1.006 (0.975-1.037) 1.006 (0.976-1.037)
5 0.974 (0.935-1.007) 0.978 (0.939-1.009) 0.977 (0.939-1.008)
6 0.991 (0.959-1.026) 0.991 (0.960-1.023) 0.989 (0.957-1.020)
7 0.991 (0.954-1.031) 0.991 (0.956-1.029) 0.989 (0.954-1.026)
8 0.983 (0.947-1.021) 0.986 (0.952-1.002) 0.987 (0.952-1.022)
9 0.969 (0.931-1.005) 0.973 (0.935-1.008) 0.974 (0.936-1.008)
10° 0.959 (0.915-1.001) 0.967 (0.922-1.008) 0.970 (0.926-1.011)
Access to healthcare      
1#   1.020 (0.988-1.061) 1.021 (0.989-1.060)
2   1.025 (0.997-1.062) 1.024 (0.997-1.060)
3   1.016 (0.991-1.049) 1.016 (0.991-1.047)
4   1.005 (0.978-1.038) 1.005 (0.980-1.033)
5   1.003 (0.977-1.033) 1.002 (0.977-1.030)
6   0.986 (0.951-1.010) 0.987 (0.957-1.014)
7   0.989 (0.960-1.013) 0.988 (0.960-1.012)
8   0.987 (0.955-1.015) 0.987 (0.957-1.014)
9   0.983 (0.950-1.011) 0.984 (0.952-1.011)
10§   0.987 (0.949-1.022) 0.986 (0.950-1.021)
Physical environment      
-1^     0.995 (0.950-1.039)
0     1.001 (0.972-1.032)
1     0.974 (0.938-1.004)
2     0.991 (0.959-1.019)
3     1.017 (0.985-1.058)
4$     1.023 (0.978-1.082)
DIC 2268.58 2269.45 2269.15
WAIC 2263.53 2264.69 2263.71
CPO -1066.65 -1070.99 -1072.39
Reduction of spatial effect** 13.1 18.7 23.7

[i] OR, odds ratio; CrI, credible interval; DIC, deviance information criteria; WAIC, Watanabe-Akaike information criterion; CPO, conditional predictive ordinate.

[ii] *Least deprived

[iii] °most deprived

[iv] #higher

[v] §lower

[vi] ^least environmentally deprived

[vii] $most environmentally deprived

[viii] **percent reduction in the variance of the spatial effect (structured and unstructured). Statistically significant results are in italics.

Table 3.

Association among old-age survival and socioeconomic deprivation, access to healthcare, and physical environmental deprivation (women).

Model 1 OR (95% CrI) (socioeconomic deprivation only) Model 2 OR (95% CrI) (plus healthcare) Model 3 OR (95% CrI) (plus healthcare and physical environment)
Socioeconomic deprivation      
1* 1.146 (1.079-1.222) 1.135 (1.068-1.210) 1.132 (1.064-1.207)
2 1.087 (1.038-1.138) 1.080 (1.032-1.131) 1.080 (1.032-1.129)
3 1.083 (1.031-1.142) 1.074 (1.025-1.132) 1.072 (1.023-1.128)
4 1.029 (0.986-1.074) 1.032 (0.990-1.077) 1.032 (0.990-1.077)
5 0.994 (0.951-1.037) 0.994 (0.952-1.035) 0.994 (0.952-1.035)
6 0.986 (0.943-1.033) 0.981 (0.939-1.026) 0.978 (0.937-1.022)
7 0.937 (0.884-0.986) 0.940 (0.889-0.987) 0.941 (0.890-0.987)
8 0.925 (0.876-0.971) 0.931 (0.889-0.976) 0.933 (0.886-0.978)
9 0.940 (0.894-0.991) 0.942 (0.897-0.991) 0.943 (0.899-0.991)
10° 0.903 (0.850-0.955) 0.916 (0.862-0.969) 0.920 (0.866-0.973)
Access to healthcare      
1#   1.020 (0.971-1.073) 1.020 (0.973-1.072)
2   1.032 (0.994-1.083) 1.032 (0.994-1.081)
3   1.021 (0.984-1.070) 1.020 (0.984-1.066)
4   0.994 (0.947-1.032) 0.996 (0.950-1.032)
5   1.014 (0.977-1.067) 1.011 (0.975-1.061)
6   0.987 (0.941-1.025) 0.988 (0.944-1.025)
7   0.979 (0.933-1.015) 0.980 (0.935-1.015)
8   1.004 (0.964-1.062) 1.003 (0.964-1.058)
9   0.993 (0.954-1.038) 0.992 (0.954-1.036)
10§   0.957 (0.896-1.008) 0.961 (0.901-1.009)
Physical environment      
-1^     0.993 (0.945-1.041)
0     0.987 (0.952-1.019)
1     0.972 (0.928-1.006)
2     1.003 (0.971-1.040)
3     1.020 (0.985-1.067)
4$     1.025 (0.974-1.093)
DIC 2481.10 2481.43 2481.53
WAIC 2471.92 2473.54 2472.88
CPO -1252.19 -1255.65 -1257.34
Reduction of spatial effect** 31.4 33.5 35.6

[i] OR, odds ratio; CrI, credible interval; DIC, deviance information criteria; WAIC, Watanabe-Akaike information criterion; CPO, conditional predictive ordinate.

[ii] *Least deprived

[iii] °most deprived

[iv] #higher

[v] §lower

[vi] ^least environmentally deprived

[vii] $most environmentally deprived

[viii] **percent reduction in the variance of the spatial effect (structured and unstructured). Statistically significant results are in italics.

Abstract views:
151

Views:
PDF
58
HTML
11

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM


Copyright (c) 2017 Ana Isabel Ribeiro, Elias Teixeira Krainski, Marilia Sá Carvalho, Maria Fátima Pina

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
 
© PAGEPress 2008-2018     -     PAGEPress is a registered trademark property of PAGEPress srl, Italy.     -     VAT: IT02125780185