Methodological Approaches in Ukraine

    reSCORE Ukraine

    reSCORE Ukraine 2024

    The reSCORE Ukraine is a joint initiative funded by the USAID and UNDP, implemented by SeeD, which serves as an annual assessment tool of societal resilience and recovery.

    Data from the Ukraine reSCORE in 2024 relies on face-to-face, structured and quantitative interviews with citizens in Ukraine, collected between June 8, 2024 and September 1, 2024. The data, collected from 7,758 respondents, is representative of all territories controlled by the Government of Ukraine at the time of surveying, and excludes Luhansk oblast, Donetsk oblast and the Autonomous Republic of Crimea. This sample includes a booster sample of Kirovohrad oblast (N = 298), as well as the cities of Kryvyi Rih, Dnipro, Kharkiv, Zaporizhzhia, and Odesa (total N = 1,496).

    The reSCORE 2024 also includes a panel dataset of 646 citizens, surveyed face-to-face in 2023 and in 2024, as well as 494 IDPs living in Kirovohrad, Kharkiv, Dnipropetrovsk, Zaporizhzhia, Odesa and Mykolaiv oblasts.

    The reSCORE also includes an additional booster sample of 500 veterans from across the country who are currently not in service but participated in combat after February 24th 2022. Further booster samples include 292 residents of Mykolaiv oblast, and 496 young people (aged 18-35) living in Chernihiv, Dnipropetrovsk, Kharkiv, Kherson, Mykolaiv, Poltava, Sumy, and Zaporizhzhia oblasts. 

    reSCORE Ukraine 2023

    Following Russia's full-scale invasion, SCORE Ukraine was recalibrated to focus on Resilience Recovery and Reconstruction. reSCORE Ukraine, which is a joint initiative funded by the USAID and UNDP, implemented by SeeD, will continue to serve as an annual assessment tool of societal resilience and recovery, that informs the policies and programming of national, regional and international partners. The reSCORE aims to identify pathways and respond to complex needs, geared at strengthening individual and collective coping mechanisms, and fostering a democratic, just, inclusive and cohesive Ukraine. 

    Data from the Ukraine reSCORE in 2023 relies on face-to-face, structured and quantitative interviews with citizens in Ukraine, collected between March 26th and June 12th 2023. The data, from 5,914 respondents, is based on stratified random sampling, and is representative of all territories controlled by the Government of Ukraine at the time of surveying, excluding Luhansk and Donetsk oblasts, and excluding the Autonomous Republic of Crimea.

    The reSCORE also includes a panel sample of 167 respondents, and a hromada component that contains the views of 504 hromada representatives, CSO representatives, sector-specific public servants, and OSBB or starosta representatives from 32 hromadas. There are also additional booster samples of 500 persons with disabilities and of 500 young people (aged 18 to 35) in Chernihiv, Dnipropetrovsk, Kharkiv, Mykolaiv, Poltava, Sumy and Zaporizhzhia oblasts, and a booster sample of 500 veterans from across the country who are currently not in service but participated in combat after February 24th 2022. As such, a total of 8,085 Ukrainians have participated in the fieldwork as respondents.

    The reSCORE 2023 measures a range of socio-economic and political issues in the country, including levels of social cohesion, satisfaction with public services, perception of human security and trust towards institutions, as well as civic behaviors, political orientations and information consumption of Ukrainians. This year's iteration also measures attitudes towards adversities stemming from Russia's full-scale invasion, including the experience of occupation. The reSCORE provides a golden opportunity for pre- and post-war comparison of societal dynamics, deep insights into the socio-economic and political consequences of the war, and how it affects social cohesion on vertical and horizontal levels in the country. The reSCORE is strategically positioned as a robust evidence source to help support government and donor coordinated efforts for recovery and reconstuction, in building a methodologically-robust index to measure and track Human Recovery Capital and in bridging humanitarian and development efforts with people-centred scientific analysis. 

     

    SHARP Ukraine

    SHARP Ukraine 2022
    The SCORE-inspired Holistic Assessment of Resilience of Population (SHARP) is an agile and responsive mobile surveying tool, deployed in three waves. SHARP Wave 1 was funded by the Partnership Fund for a Resilient Ukraine (PFRU), and implemented in partnership with SeeD, the USAID funded Democratic Governance East (DG East), USAID's Transformation Communications Activity (TCA) and the UNDP.

    The first phase of the SHARP study comprised two different sampling techniques, namely: purposive panel sampling and nationwide random sampling. Data was collected through structured and quantitative computer-assisted telephone interviewing (CATI). The fieldwork for this phase was conducted by the Kyiv International Institute of Sociology (KIIS) between September 23rd and October 5th, 2022 for the panel sample, and September 26th to November 5th, 2022, for the main random sample. 

    The SHARP study's nationwide random sample (N = 4,327) is spread across all oblasts and is representative of the adult population as of 2021, according to data from the State Statistics Service of Ukraine. To account for the volatility of displacement patterns following Russia’s escalation of the war against Ukraine on February 24, 2022, the sample was constructed using the question about the respondents’ place of residence before that date. The use of probabilistic sampling and a large sample size should ensure that the SHARP random sample reflects the perceptions of a larger population in Ukraine.  Only the areas controlled by Ukraine as of the time of data collection were included in the survey.

    The SHARP panel sample (N = 495) consists of SCORE respondents surveyed in 2021 who consented to be contacted and provided their phone numbers. While this sample is used in the SHARP report, it is not included in the presentation of scores on the interactive data platform on this website. For more on the panel analysis, please refer to the SHARP report under the Publications tab. 

    The second phase of the SHARP study consisted of nationwide random sampling (N = 4,995). The sample was structured around specific oblasts of programmatic priority, which were subsequently oversampled (Chernihivska, Chernivetska, Donetska, Dnipropetrovska, Kharkivska, Khersonska, Kyiv city, Kyivska, Ivano-Frankivska, Lvivska, Mykolaviska, Poltavska, Rivnenska, Sumska, Zakarpatska, Zaporizka oblasts). Data was collected between June 27th and August 20th 2023 by KIIS.

    Weights were applied to ensure representation at national level. Weights were constructed based on the oblast in which the respondent lived before February 24, 2022, alongside the available data of the State Statistics Service of Ukraine on the distribution of the population of each oblast by gender and age as of January 1, 2022. The sample was representative of gender and age. Luhanska oblast was not included in the survey due to the low level of reach of current residents of this oblast.

    The third phase of the SHARP study consisted of nationwide random sampling (N = 4,981). The sample  was structured around specific oblasts of programmatic priority, which were subsequently oversampled (Chernihivska, Chernivetska, Donetska, Dnipropetrovska, Kharkivska, Khersonska, Kyiv city, Kyivska, Ivano-Frankivska, Lvivska, Mykolaviska, Poltavska, Rivnenska, Sumska, Zakarpatska, Zaporizka oblasts). Data was collected between January 26th and March 14th 2024 by KIIS.

    Weights were applied to ensure representation at national level. Weights were constructed based on the oblast in which the respondent lived before February 24, 2022, alongside the available data of the State Statistics Service of Ukraine on the distribution of the population of each oblast by gender and age as of January 1, 2022. The sample was representative of gender and age. Luhanska oblast was not included in the survey due to the low level of reach of current residents of this oblast. 

     

    SCORE Ukraine 2016 to 2021

    SCORE Ukraine 2021
    Is a joint initiative funded by the USAID, UNDP and the EU, implemented by the Centre for Sustainable Peace and Democratic Development (SeeD). The data was collected from January to May 2021, and consisted of a nationally representative sample of 12,482 face to face interviews. Of these, 3,490 were from government-controlled areas of Luhansk and Donetsk oblasts, and 2,857 were from oblasts in the Azov and Black Sea areas (Zaporizhzhia, Kherson, Odesa, Mykolaiv oblasts). In addition to the nationally representative sample, 1,010 interviews were conducted near the line of contact in Luhansk and Donetsk oblasts, 3,600 interviews were conducted across 18 cities and urban communities, there were also 519 interviews of ATO/JFO veterans in 5 oblasts, 325 interviews with persons with disabilities from 3 oblasts, and 1,000 interviews with youth in 5 oblasts. From the non-government-controlled areas (NGCAs) of Luhansk and Donetsk oblasts, a sample of 638 interviews were collected used CATI methodology.

    SCORE Ukraine 2018
    USAID and SeeD acknowledged that it is crucial to conduct longitudinal and comparative research that can provide temporal analysis in order to monitor key indicators as well as to better understand of socio-political dynamics and trends. SCORE Ukraine Wave Two was calibrated in the summer of 2017 with stakeholder consultations and 8 regional focus groups conducted in Kherson, Kharkiv, Kramatorsk, Kyiv, Ivano-Frankivsk, Lviv, Sieverodonetsk and Odessa. Following the calibration process, the Wave Two questionnaire was designed and fieldwork was deployed in January 2018 by Growth from Knowledge (GfK) Survey Company. In total 9,018 face-to-face household interviews in 24 oblasts and Kyiv city were conducted via stratified random sampling. 6,102 respondents from Wave One also participated in Wave Two as part of the panel sample, which allowed for more robust and causal analysis. SCORE Ukraine 2018 General population study focused on the following areas of investigation:

    • Monitor prevailing societal and inter-group dynamics, and investigate civic assets, attitudes and orientations towards fostering constructive citizenship;
    • Understand the level of human security, satisfaction and access to services, and socio-economic challenges that can undermine social cohesion and development; and detect human security threats that different communities are experiencing;
    • Investigate how alternative policies would enhance or undermine social cohesion, civic participation and sense of security in Ukraine; and identify the drivers of support for reforms and support for reintegration to better understand citizens' anxieties and fears;
    • Provide a deeper investigation into zones of crisis, vulnerable groups, priority issues and specific societal tensions identified in Wave One.

    SCORE Ukraine 2016
    In 2015, SeeD was invited to implement the SCORE Index in Ukraine, by the USAID/OTI supported program “Ukraine Confidence Building Initiative” (UCBI). To address the issues underpinning community tensions and cohesion, SeeD in partnership with UCBI implemented the Wave One of the SCORE Ukraine between August 2015 – December 2016, with a nation-wide sample of over 7,700 respondents, over 300 interviews per oblast plus Kyiv city, as well as a booster sample of 1,600 Internally Displaced People (IDPs). SCORE Ukraine Wave One (General Population 2016) focused on investigating citizens' views on public policy reforms and the peace process, their sense of identity, access to services and civic engagement as well as the challenges faced by IDPs. Wave One succeeded in developing evidence-based and participatory policy briefs and communication messages, and mobilized significant interest among a wide range of key stakeholders (e.g. senior policy makers, journalists, civil society).  

     

    SCORE Ukraine Schools Study for Adolescents 2017 to 2020

    SCORE Ukraine Schools Study for Adolescents is conducted in close partnership with UNICEF and Ministry of Education in Ukraine. The research team thoroughly reviewed all ethical considerations to ensure the protection of children’s rights during the study, and the measures complied with UNICEF’s and national ethical considerations. Approval for the survey was obtained from the Commission on Psychology and Pedagogy of the Scientific-Methodical Council of the Ministry of Education and Science of Ukraine.

    The indicators were selected and designed based on extensive literature reviews and consultations with experts on adolescent development. These ranged from adolescents’ positive experiences in the family and school context (e.g. family connectedness, parental involvement, teacher and peer support, dropout tendency), experiences of adversity (e.g. physical abuse, bullying, exposure to domestic violence and conflict) and different developmental outcomes (e.g. aggression, anxiety, and depression). The study also included assessment of life skills and civic assets (e.g. creativity, problem solving, active citizenship).

    Before administering the questionnaire, regional field managers from the Ukrainian Institute for Social Research after Oleksandr Yaremenko (UISR) National network received a full-day training. Students were then informed about the objectives of the study, how the data would be used and that participation was on a voluntary basis. Each student received a questionnaire and an individual envelope in which they sealed their completed questionnaire.

    SCORE Ukraine Schools 2020 
    In 2019, the study was recalibrated in partnership with by UNICEF and the Wave Three was launched. The data were collected during the first semester of the 2019-2020 school-year. The sample consisted of 8,643 adolescents aged between 13-19 years old, as it was decided that the reliability of the data from 12 year olds was limited. Adolescent respondents were students from 215 education institutions in Ukraine who resided in the government-controlled areas of Donetsk and Luhansk, Dnipropetrovsk, Zaporizhia, Kharkiv, Mykolaiv, Lviv, and Kyiv oblasts, and Kyiv city.

    SCORE Ukraine Schools 2019 
    In 2018, SCORE Ukraine Schools Wave Two, commissioned by UNICEF, was expanded both its scope and its reach. The data were collected during the first semester of the 2018-2019 school-year. The sample consisted of 7,846 adolescents aged between 12-19 years old. Adolescent respondents were students from 200 education institutions in Ukraine who resided in the government-controlled areas of Donetsk and Luhansk, Dnipropetrovsk, Zaporizhia, Kharkiv, Mykolaiv, Lviv, and Kyiv oblasts.

    SCORE Ukraine Schools 2017
    The Wave One of SCORE Ukraine Schools Study for Adolescents was launched under SCORE for Eastern Ukraine, also known as UN SCORE for Eastern Ukraine (USE 2017), which was implemented in partnership with SeeD, UNDP, UNICEF and IOM. Wave One surveyed 3,300 adolescents aged 13-17 from 48 education facilities in Donetsk and Luhansk oblasts (government controlled areas). The findings and reports of the Wave one can be found under the Eastern Ukraine page, here.
     

    For a more detailed explanation of SCORE methodology, please read the short methodology paper here, explore the publications page here or read the general methodology page here. For information on how to use the platform, you can watch the short video on our Facebook page here or read the How to Read SCORE manual here. 

    The SCORE Vocabulary

    • Outcomes of interest: Identified desirable and high priority normative objectives that relate to assessing and fostering social cohesion in a given context (i.e. intergroup harmony, mitigating violent tendencies, fostering a human rights ethos). These outcomes are listed under the dimensions section of the SCORE Platform but should not be confused with categorical/thematic groupings. The outcomes of interest that make up the key components of social cohesion for Moldova are illustrated in the figure above. The other dimensions listed on the SCORE platform are categories that group different indicators thematically.

    • Indicators: The components of the conceptual model are translated into metrics and indicators that are quantifiable and measurable via public opinion polls. Each indicator that is measuring a particular phenomenon (e.g. economic security, discrimination towards out groups, belief in human rights, support for certain policy options, post-traumatic stress disorder, etc.) is usually assessed with minimum 3 questionnaire items, scaled following reliability tests, to ensure that the SCORE can robustly capture different dynamics underlying the given indicator.

    • Drivers/Predictors: Indicators that have a strong positive or negative impact on the outcome of interest are called drivers or predictors. They provide strategic entry points that hold the most likelihood of impact on the desired outcome of interest. These can be seen under the path analysis section.

    • Heatmaps: A score is calculated for each indicator. The scores range from 0 to 10, where 0 means that the phenomenon the indicator is measuring is not observed in the context at all, and 10 means that it is observed strongly and prevalently. Heatmaps demonstrate the regional differences of these scores in order to identify areas of concern and tailor interventions more precisely. For example, the Personal Security indicator is measured through the following questions:

      • To what extent do you feel safe from violence in your daily life?

      • To what extent do you feel confident that the police or other institutions can protect you from violence?

      • To what extent would you feel safe walking alone in the street at night?

    A score of 0 for personal security would mean that no one in a given region feels secure at a personal level, while 10 would signify that every person feels absolutely secure. As such, a heatmap of personal security for Ukraine illustrates the scores for each oblast, allowing for an understanding of the regional variance in terms of personal security across the country.

    • Path analysis (predictive models): Based on advanced statistical analysis including regression, network analysis and structural equation modelling, predictive models investigate the relationship between different indicators and the outcomes of interest. Predictive models reveal those indicators that may have a reinforcing or mitigating influence on other indicators as well as the outcome. While the first wave of the SCORE can be used for identifying directional correlations and benchmarking, second and third waves of the SCORE, where temporal comparisons are possible, can help identify trends and causal relationships. The colour of the connecting lines represents the nature of the relationship - blue symbolizes a positive correlation, and red symbolizes a negative correlation. The thickness of the lines represents the t of the correlation – the thicker the line, the stronger the relationship.