Spain

The programme was evaluated by Pericas and colleagues (2022) in a stratified RCT using a sample of 2625 in a total of 37 schools in Barcelona (Intervention Group: 1291, 17 schools; Control Group: 1334, 20 schools). The researchers used a linear multilevel regression model with repeated measures (beginning and end of the school year). Individual student scores on emotional competence were nested at school-level that in turn were stratified by type of school and neighbourhood socio-economic status. The newly designed Emotional Competence Assessment Questionnaire (ECAQ) was validated (Bartroli et al., 2022) and used to register (teacher registration) the level of emotional competencies of each child.

In boys, the mean change in the scores for Preschool year 3 (P3) was 24.51 points for those in the Intervention Group and 13.77 for those in the control group. In girls, the mean change in the scores for P3 was 24.40 points for those in the intervention group and 13.83 for those in the control group. Generally, a negative gradient was observed, as the change became smaller as the school year became higher, suggesting that the intervention may be more effective in the first year of pre-school.

San Pio and colleagues (2023) analysed the outcomes with nested linear regression models in a sample of 1586 children in 35 Barcelona schools. Considering sociodemographic variables and implementation outcomes, they compared 3 groups with the same data: 1) participated in 1, 2, 3, Emoció! during the three years of preschool: P3, P4, and P5, (= Complete Programme: CP); 2) participated in the programme for only one year (P5, Partial Programme - PP); and 3) did not participate in the programme (Comparison - C). The measured emotional competence level increased significantly after one year (4.1 in boys; 5.6 in girls; P < 0.05) and after three years of intervention (5.5 in boys; 8.0 in girls; P < 0.01), compared to comparison group. The level of emotional competence was the highest for the 3-year intervention group: obtaining an average ECAQ score of 131.1 (95% CI 126.9-135.2) for boys and 141 (95% CI 137.2-144.9) for girls. Completing the programme with high fidelity increased the level of emotional competence at the end of the school year by 20.5 points (95%CI: 15.9-25) for boys and 24.3 points (95%CI: 18.7-29.9) for girls, in comparison to 4.6 (95% CI: 1.5-7.7) and 5.9 (95% CI: 3.1-8.7) points for boys and girls, respectively, who received the programme with low fidelity.

Bartroli et al. (2023) designed a linear regression model for paired measures to study the ECAQ score among children diagnosed with an autism spectrum condition (ASC) within Pericas and colleagues’ sample. Levels of emotional competence were evaluated with the validated ECAQ (score range from 30 to 180) among those participants diagnosed with an ASC. 34 and 11 children were assigned to respectively the intervention and the control group. Children in the intervention group showed an increase from the initial score to the final score that was almost three times higher (16.1) than the increase observed in children in the comparison group (6.6). The difference between the two groups was statistically significant, with 8.9 points (95% confidence interval: 1.2–16.6) attributed to the intervention. This study uses a very small sample, analyses data at less than one-year follow-up and does not report on implementation fidelity.

While this intervention appears to be promising due to its population-level outcomes concerning substance use among youth in Iceland, the authors report that it remains unclear whether and how the intervention and its components impacted substance use in Iceland (Kristjansson et al., 2010 and 2016).

The programme aims at reducing individual risk factors and promoting individual protective factors but it remains unclear whether and how these outcomes are related to the intervention components. The core characteristics of the interventions (e.g. dosage, logic model etc.), besides their community aspects, are not described in the available studies. Subsequently, it is unclear which interventions or actions were offered and implemented. Furthermore, there is no systematic monitoring and subsequent analysis available on the changes in the community context (besides substance use). For instance, no direct linear link was documented between participation in sports activities and the decrease in the prevalence of substance use among youth. 

The Kristjansson et al. (2010) study has a quasi-experimental design, where the effects however are small. Other issues: 1) It is unclear how the control group was impacted by the overall Icelandic alcohol policy, in the currently available studies, 2) The fact that five communities were excluded from the intervention group could imply fidelity and dosage issues that are not accounted for and considerable dropout. Included were only communities that participated fully and many years in the intervention, 3) There is no analysis of baseline differences such as socio-economic status, 4) The measured outcomes do not relate to the intervention components or the logic model, 5) The analysis, as pointed out by the authors themselves, does not ascertain a causal relation between the intervention and the outcome.

The 2016 study (Kristjansson et al. 2016) could not be included in the assessment as as it is a repeated cross-sectional study of population-based school surveys, with seven waves of pooled data, and it is unclear whether it is an ITS and how many data assessments were conducted prior to the intervention. Thus, as mentioned by the authors, the study faces issues such as the inability to dismiss secular trends in adolescents’ substance use and the coexistence of the intervention with other, unmonitored interventions.

Other promising recently published studies (Asgeirsdottir et al. 2021; Kristjansson et al. 2021; Meyers et al. 2023), face similar problems, as they are repeated cross-sectional studies, and due to the study design it cannot be determined whether there is a causal effect or if only correlational relationships can be considered. Some of the main limitations of these studies are insufficient data points before and after the intervention, lack of detail on what is implemented and when, and unclear comparability across samples.

What to improve: future studies should provide a more in-depth description of the components and delivery modes and perform better monitoring of dosage and fidelity. Also, the changes in the targeted behaviours (other than substance use) could be assessed to understand the active ingredients in the logical model. Additionally, baseline equivalence and attrition should be included in the evaluation method to account for differential impact. These methods would help other countries to effectively reproduce the effects, test the logical model and adapt the design to new contexts.

In the Romero et al 2017 study, a screening process identified children with significant conduct problems both at home with their family and at school, and the programme was implemented in eight schools. It included a long-term follow-up of 56 children for seven years. Multivariate analysis showed significant differences in attitudes (higher in the control group) and in the intention of using tobacco or alcohol (also higher in the control group). When the proportion of adolescents that will “probably yes” use tobacco was analysed, 11% of the intervention group was willing to use, compared with 42% of the control group (chi-squared: 7.59, 1 df, p < .001). As to adolescents that will “probably yes” use alcohol, 11% of the intervention group chose this response,compared with 35% of the control group (chi-squared: 6.23, 1 df, p < .01). No differences were found in age at onset. There were significant differences in the frequency of actual tobacco usel, but no significant differences in relation to aggressive conduct, despite the fact that facing rage is one of the most emphasised contents of the component for children. Also no significant differences were found in emotional control skills or in empathy, which seems to suggest that the specific effects on interpersonal emotions are attenuated over time and, therefore, that these components require reinforcements during the intervention. No significant effects were found in cannabis use; probably due to its later onset than tobacco and alcohol use.

The Romero et al. 2019 study assessed the efficacy of Empecemos in 128 children with behavioural problems in 18 public schools in urban and semi-urban areas: 67 in the intervention condition and 59 controls. There was random allocation to IG or CG at school level: 9 schools each. The tutors of the children enrolled them in the third to fifth years of primary education completing a brief screening instrument (10 items). Inclusion criteria were higher score at a T of 70 in the “Externalizing” dimension of this instrument. The data suggest that Empecemos improves emotional, cognitive and social skills, and reduces conduct problems, especially teacher-reported conduct problems. The study did not assess substance use related outcomes but showed significant effects when comparing the evolution of the intervention group versus the control group in other relevant behavioural outcomes. 

The limited sample size (common  in  these  types  of  studies  with  high-risk  children) weakens the statistical strength of both analyses carried out. Key issues for the rating: The studies all focus on different components and target groups. Only one study is a RCT with less than 12 month follow-up (post-test after 1 month) that only examined child outcomes, plus the IG had significantly higher age compared to CG (Romero et al., 2019). 

The programme has been evaluated in three studies in Spain: (i) a quasi-experimental study (1984) in Mallorca conducted in 15 schools among 261 students, four months after the programme ended; (ii) an RCT (1988) in Baja, conducted in 12 schools among 376 students, 12 months after completing of the programme; and (iii) a quasi-experimental study (doctoral thesis, 1985) in San Just Desvern and Viladecans, conducted in two schools with randomly selected classes, among a small sample (n= 84 students in both groups), with the post-test conducted four months, one year, and two years after completing the programme.

The first study (QED) found that the intervention group’s consumption was below the mean of the control group. There was significant reduction in getting drunk. There were significant increases in knowledge about the types of drugs (higher in the intervention group than the control group), and about the consequences of substance abuse. Attitudes towards future drug use were descriptively analysed, the questionnaire was not sufficiently aimed at measuring comprehension of external influences on use. There was an increase in acceptance of social and institutional control mechanisms related to substance use, reduction in getting drunk during the last six months, and reduction in money spent in alcohol. The results indicated that high-risk students were not/less affected by the intervention.

The second study (RCT) found that the control group significantly increased alcohol and tobacco use, and the intervention group remained at the same level. There was an increase in more rational reaction to drugs, less aggressive reaction towards measures of control (law or adults) among intervention group. There were less favourable attitudes toward drugs among intervention group, however there was no effect on decision-making. Similarly the high-risk group was less/not affected by the intervention. Only non-consumers maintained their non-use attitude while users did not reduce their use (the control group participants did increase their use).

The last study (doctoral thesis/QED) found that the intervention group showed big preoccupation with the health risks before use and this seemed to influence actual use. Smoking decreased in post-test but increased in follow-up one year after. There was no significant decrease in drug use in the intervention group, but there was an increase in the control group (the programme had an influence). At 4 months there was a decrease in alcohol consumption, decrease in smoking and weekend smokers, but not the daily smokers. Two years after, there is a slight increase in new drinkers in intervention group, but significant increase in control group. There were no iatrogenic effects.

The programme has been evaluated in one quasi-experimental study and one four-year follow-up study in Spain, and one quasi-experimental design in Italy.

The Spanish quasi experimental study was conducted with an intervention group (n=2,567) and a control group (n=2328) enrolled in the last year of Primary Education in Spain (1999) at two time points. Intervention effects were measured by means of chi square, t-tests and tests of covariance. Last month and weekly tobacco use were not significantly affected by the intervention. Initiation of tobacco use among those that had initiated first use was significantly lower in the intervention group. Among those that had already tried alcohol the increase of use was significantly lower in the intervention group. A higher increase of alcohol use in the control group was established. A significantly higher number of participants reduced monthly wine use in the intervention group. There was a significant difference between first time alcohol use between IG and CG. These results should be interpreted cautiously considering that they were studied in a time-frame of only 6 months (November 1997 - January 1998). Effects on 'anti-social behaviour' were analysed but not analysed in relation to the substance use outcomes. 

The Spanish four-year (1995-1999) follow-up study (Gomez Frágüela 2003) involved one control group (n=485) and two intervention groups. The first intervention group (n=235) got the intervention from teachers, the other from professional prevention workers (n=309). A validated questionnaire was administered at four time points and analysed by means of ANOVA and pairwise multiple comparisons. The 15 and 27 months follow-up demonstrated some intervention effects. The 39 month follow up demonstrated similar monthly consumption frequency of tobacco, beer and spirits for all three groups but significantly lower general consumption of cannabis, tranquillizers and amphetamines in the intervention group. Consumption in the two intervention groups was significantly lower compared to the control group.

The Italian quasi-experimental design was conducted among 31 intervention group schools and 24 comparison group schools. The pre-test measurement was conducted prior to the start of the programme, and the post-test eight months after the first year, follow-up after the booster sessions in two subsequent years. The study showed significant effects at post-test in smoking initiation during the first year, weekly drunkenness initiation, and smoking initiation during third years. With regards to alcohol use, there was a lower normative expectation about adults’ drinking and fewer students reported weekly drunkenness. It appears that substance use related differences are less significant at two-year follow-up.

The programme has been rated as Model Plus by Blueprints for Healthy Youth Development based on a review of studies conducted world-wide

The programme has been evaluated in one quasi-experimental study in Spain and one pre-post study with 12 months follow-up without control group. The quasi-experimental study was conducted among families (n=259) served by social services or specialised units in 12 territories, less than 12 months after the intervention (parents with problems of addiction were excluded). The study showed small but significant effects on communication skills, conflict management and cohesion, and family bonding among parents moderated by implementation and participation. There were no outcome measures related to substance use (only parents were evaluated). There was no difference in cohesion, family bonding and family communication.

The longitudinal study was conducted between 12 and 16 months after the pre-test, and 10 – 14 months after the post-test. The study showed statistically significant differences; it showed that all the measures were positive, except for behavioural monitoring, indicating that most outcomes were maintained. Family positioning on drugs increased over time.

The programme has been evaluated in five quasi-experimental studies, two studies in the Netherlands and one across the Netherlands and Belgium evaluated the programme with delinquent adolescents, while one study from Netherlands and one from Spain evaluated the programme implemented as universal prevention in schools.

In two Dutch studies, delinquent male adolescents aged 12 to 21 years recruited from three/four similar high-security juvenile correctional facilities participated, intervention group participants came from one facility while the control group from the remaining. In one study, 7/20 outcomes (measuring moral development, social information processing and social skills) showed a significant positive intervention effect. In the other study, there a significant positive intervention effect on cognitive distortions. The study also measured recidivism: the more participants from intervention group reoffended (53% v 29% at 12 months, and 86% v 65% at 24 months), but the difference was not significant.

234 incarcerated youth from high-security juvenile correctional facilities participated in the study, conducted across the Netherlands and Belgium. Intervention and control group participants came from different facilities. The majority (69%) were males, and the mean age was approximately 15.5 years. Significant differences were found between the experimental and control groups in the development of social skills and moral value evaluation both of which had a small to moderate effect size as the intervention group remained stable while the control group’s scores decreased. No significant differences were found between the experimental and control groups in the development of cognitive distortions and moral judgment.

Another Dutch study takes a universal preventive approach and was implemented in schools with students with a mean age of 14 years. 9 schools were assigned to groups according to their ability to implement the programme. There was a significant positive effect on self-serving cognitive distortions and attitudes towards antisocial behaviour in the experimental group compared to the control group, but the effect on prevalence of antisocial behaviour and moral judgement was not significantly different between groups.

The Spanish study was also conducted in schools with 89 participants aged approximately 14-16 years, and evaluated an adaption called EQUIPAR, the Spanish version of EQUIP for Educators. The study had mixed effects on thinking errors and no significant effects on victimisation.

The programme has been evaluated in one quasi-experimental study across Finland, Denmark, Netherlands, UK, Portugal and Spain. The average age of young people was 13.3 years. Self-reported measures were completed, and all results reported here were post-test.

In Spain, fewer young people in the intervention condition began to smoke (statistically significant) and more participants in the control group began to smoke weekly (only marginally statistically significant p=0.08). There was no effect on intention to take up smoking in the next year.

In Finland, fewer intervention group participants began smoking but this was not a statistically significant effect. Fewer intervention group participants began to smoke weekly (statistically significant at 24 months after baseline, and marginally significant (p=0.06) at 30 months i.e. post-test). There was no effect on intention to take up smoking in the next year.

In Portugal, there were fewer participants from the intervention group who began to smoke, and who began to smoke weekly, compared to the control group. Fewer intervention participants reported intentions to smoke in the next year. (All of these differences were statistically significant.)

In the UK, there was no effect on smoking. However, there was a statistically significant favouring the intervention on intention to take up smoking in the next year.

Counter-productive effects were found in the Netherlands and Denmark, where more participants in the intervention group became smokers and more participants began to smoke weekly compared to the control group. Although the negative effect on weekly smoking was statistically significant in Denmark in the initial years of the programme, the only (negative) effect that was statistically significant at the end of programme was weekly smokers in the Netherlands. The effect on intention to take up smoking in the next year was not statistically significant in either country.

Overall analysis showed that there was a statistically significant effect on weekly smoking, but not on the number of smokers. The study also measured attitudes and self-efficacy using five measures, the effects on which were largely not statistically significant at the end of the programme (30 months from baseline) across the different countries. Overall, fewer intervention participants believed in the advantages of smoking and more intervention participants had social self-efficacy (refraining from smoking in social situations).

The programme has been evaluated in a cluster randomised controlled trial (RCT) involving children aged 12-14 years in several European countries: Austria, Belgium, Germany, Greece, Italy, Spain and Sweden (Caria et al., 2010; Faggiano et al., 2007, 2008, 2010; Giannotta et al., 2014; Vigna-Taglianti et al., 2009, 2014). There were also two cluster RCTs in the Czech Republic (Miovsky et al., 2012; Jandáč et al., 2021) involving children with a mean age of respectively 11.8 years and 15 years.

For the cross-country study at post-test, exposure to Unplugged was associated with a statistically significant lower prevalence of self-reported daily use of cigarettes, episodes of drunkenness and cannabis use in the past 30 days in the intervention condition compared with the control condition. Young people receiving the programme were less likely than those in the control condition to move from non-smoking or sporadic smoking to daily smoking. Similar patterns emerged in the use of other substances. An analysis by gender found that delayed progression and enhanced regression were higher in the intervention condition among boys, whereas no, minimal or reverse differences were observed among girls.

At 18-month follow-up (Faggiano et al., 2010; Vigna-Taglianti et al., 2014), the use of tobacco and frequency of drunkenness was lower among students in the intervention condition compared to those in the control condition. Students in the intervention condition showed higher tendencies to remain non-users of tobacco or to regress from occasional to no use. The number of students reporting no drunkenness in the past 30 days was higher among students in the intervention condition compared to those in the control condition. Intervention condition participants also reported fewer alcohol-related behaviour problems compared to controls. Further, participants who reported not drinking at baseline were more likely to retain this status at follow-up after participating in the intervention, and those who reported drinking only occasionally at baseline showed a slower progression towards frequent drinking by follow-up if they participated in the intervention. When considering cannabis use, the proportion of persistent non-users was higher among the intervention condition than the control condition. All of these differences were statistically significant.

The first Czech study (Miovsky et al., 2012) found a statistically significant effect favouring the intervention, with intervention participants less likely than those in the control condition to have smoked cigarettes in the last 30 days at 3-, 15- and 24-months post-intervention. At the other two time periods (1 and 12 months), differences between conditions in 30-day cigarette use were not statistically significant. There were no statistically significant differences between intervention and control conditions on lifetime cigarette prevalence rates.

The second Czech study (Jandáč et al., 2021) consisted of 70 schools randomly selected with stratification according to their affiliation with a region and size, assigned to one of three groups (the control group, the intervention group 1 and the intervention group 2 exposed to the Unplugged intervention and n-Prevention. The n-Prevention programme is a follow-up (12 months) programme and consists of four lessons providing a general background addressing social norms, social beliefs, refusal skills and gender-specific differences, neurological aspects and the effects of substance use.  Children from families where the mother reported using alcohol weekly or less frequently, however, reported a decrease in drunkenness in the last 30 days compared to the control group. However, the study found no statistically measurable effect on drinking among children who came from families where the mother uses alcohol more than weekly.  These results were observed at a 24-month follow-up, which implies that the Unplugged programme may not be sufficient for high-risk children. Moreover, it is unclear how randomisation took place, and what the drop-out rate and baseline equivalence was. Additionally, in this study, a universal programme was used as a targeted intervention.

Lecrique et al. (2019) conducted an RCT in France to assess the effectiveness of the intervention on drug use among adolescents. Measures were taken at baseline and 6 and 8 months follow-up. The results showed that at the 8-month follow-up, the probability of consumption of cigarettes (OR=0.36; p<0.01), being drunk (OR=0.23, p<0.01) or using cannabis (OR=0.31, p<0.05) in the last 30 days was higher in the control group compared to the intervention group. In the intervention group, there was an improvement in other outcomes including psychosocial skills, reduced peer perception of substances and increased knowledge of the effects of substances. However, the study is a non-peer-reviewed report, it is not clear how randomisation was carried out, and there is no information on the level of attrition.

The RCTs conducted in Slovakia (Orosová et al., 2020; Abrinkova et al., 2021; Curova et al., 2021; Orosova et al., 2022) were not included in the assessment as the quality of the randomisation was low. The outcome measurement took place shortly after implementation in a quite young target group (11).