Disclaimer: This is not a formal, rigorous academic study. It is an attempt by a small charity to understand how young people might be benefitting from our activities, beyond the qualitative feedback we receive in-club and from parents.

We began asking young people to complete a resilience feedback form in February 2021. This is voluntary and anonymous to our volunteer mentors. The evaluation form was developed in partnership with University of Bristol student as part of a community engagement placement.

Since then we have collected data on 242 project completions (n=76 young people). Of these data sets, 14 contain >5 data points (project evaluations). The evaluation is not intended to assign an absolute score for a young person’s resilience.

Each snapshot has 3 key pieces of information (4 if you include the timestamp). We haven’t included timestamps as they could be used to identify young people (attendance on specific evenings equating to a particular club session).

The three pieces of information are:

1. project complexity,
2. confidence in completing the project, and
3. support from volunteers in completing the project

Our projects are (roughly) ordered from easy to more complex. This happens across three difficult ‘levels’, White, Yellow, Blue. This general colour grading allows volunteers to quickly recognise the relative complexity of the project the young person is working on, and suggest other projects to undertake.

The null hypothesis is that if our activities are of no benefit, then young people should be less confident / require more volunteer support as projects get more complex.

Fortunately, none of the data sets show this!

There are some interesting profiles that can be seen within the data.

‘Aha moment’

This chat shows an ‘aha’ moment on completion of our ‘Painting with Scratch’ project. This is classed as a starter project, but is actually quite complex. There is an important element of abstraction involved, with a single sprite (the pen) being controlled by several other sprites (the buttons).

The subsequent dip is the transition from White (starter) projects, to Yellow (intermediate) projects. Sentence Creator, and Secret Messages, are both quite complex list manipulation projects in Scratch. However, confidence is maintained and increases with Balloon Pop (introduces cloning) and password generator (more lists and string contamination).

Throughout this journey, the mentor support has been sustained.

Transitioning

This chart shows a young person’s journey from Scratch to Python over nearly 10 months. The project titles are hard to read but the last one is the transition to Python with ‘Hello World’.

The decreasing support over the first few projects suggests that they were making good progress and not requiring additional support. Their confidence remains high throughout. Password Generator, and Clone Wars are conceptually challenging projects, reflecting the increased volunteer support given.

The Memory Game (the second confidence uptick) is often a challenge for young people as there’s an abstraction between the main sprite and the buttons you press depending on the sequence of random values chosen. Once that is overcome, it’s relatively easy to complete and is a great springboard to more complex projects. 

The flat line over the last few projects suggests that the young person’s confidence is matched to the increasingly challenging projects. The uptick on completing their first python project ‘Hello World’, shows a significant increase in confidence on transition to python. This is likely due to their experiences and successes in Scratch previously.

More profiles

As we continue to collect data, we’ll continue to post interesting profiles to demonstration different trajectories through our projects.