Met Office technology user survey

Penny H C Dinh & Liz Hourahine

Every year, the online coding forum Stack Overflow run a survey of their developer community, taking responses for metrics such as healthy lifestyles, education, and the most popular technologies used within the community. From our perspective, the most interesting centre point of their presented results was a network of the most popular technologies. Correlation between the use of specific tools is calculated and displayed in a network visualisation. In this network, nodes represent different technologies, and edges connecting between nodes represent correlation between usage of technologies: the closer the nodes are together, the stronger correlation there is between their usage [1].

 

Technology use in the Met Office

 

In many use cases, there’s a lot to be said for capturing behavioural information. We decided to run a tailored version of this survey on Met Office employees to see the technology usage and behaviour within the corporate structure. We could think of Met Office HQ as its own ecosystem, with many layers and employees with a wide variety of different purposes. To be able to visualise our technology use based on these structural layers may give some insight.

We were also interested in whether tech is clustering based on purpose. If technologies predominantly used by one job role tend to cluster together, this may indicate that certain groups of technologies are being used because they are all appropriate for certain purposes. However, if the nodes cluster together primarily based on team or people’s working location, it may display that usage is more indicative of a cultural consensus rather than fitness for purposes.

Analysis

We distributed our survey across Met Office employees and obtained 95 responses overall, 68 of which were from technology department, 26 from Science, and 1 from Other.

In terms of data cleaning, some of our central metrics posed a slight challenge. For department, team and job role of the participant we collected free text responses, both because there were too many options to list and because this meant we could have information of people’s own opinions of what they do. This resulted in more general than specific categories however, as people inevitably chose different amounts of specificity. This meant some loss of information for final analysis.

To make the network visualisation we used bokeh functions calling on the networkx library. This is a tool that can be used to separately draw nodes and edges, with nodes representing the technologies used by participants. The layout defined is what dictates how the nodes and edges position on the graph, in this case the intention being that the position demonstrates any clusters that appear within the edge weightings. In this case ‘spring_layout’ was used, which is dictated by the Fruchterman-Reingold force-directed algorithm. This is a layout and tool that can be used to create the famous Facebook friends network visualisations [2].

Force-directed graph drawing algorithms assign attractive forces to pull node pairs together, whilst assigning repulsive forces that tend to push unconnected nodes apart. This also means that edges become more similar lengths and do not cross over wherever possible. Any relationships between nodes are more clearly visualised and clustered, whilst also being more aesthetically pleasing. These algorithms are very easy to use and the tools available mean no graph theory knowledge is required.

Bokeh Plot

This visualisation shows clustering seems to be related to job role, therefore the purpose of the tool rather than cultural decisions. There were still examples of tools being used in isolation rather than within a broader network. HPC (high powered computing) support seem to use tools in isolation rather than having a network of complementary usage, and scientists using Fortran in isolation where perhaps further tech knowledge would aid purpose.

Do tech users find the technology used at work suitable for their job?

This is supported by the question of whether staff think the technology they use is suitable for what they do. Overall participants were satisfied, but there are potential areas for improvement within certain jobs. Approximately a third of scientists do not think their technology is suitable, supporting evidence for their lack of a network cluster. In highly tech based roles such as software developers and software engineers, perhaps improvement in this area is important.

Education

Highest qualification

We also analysed other metrics included in the original survey. In terms of education, the Science department all hold above undergraduate qualifications, whereas an undergraduate degree is the most popular response for participants from Technology; a lot of Technology participants’ highest education level is below undergraduate level, demonstrating that the entry requirements in terms of academic qualification for Science are higher than that for Technology.

What is your highest level of academic qualification?-3.png
Degree Specialism-3.png

The most popular degree specialisms in both Technology and Science are computer science, engineering, natural sciences, and mathematics or statistics. Interestingly, some participants in Technology also hold degrees in arts and humanity subjects, whilst no participants in Science hold these degrees. Perhaps this indicates that arts and humanity courses might have equipped these participants with skills transferable to technology jobs, but not transferable to science jobs, or that tech skills can be developed outside of formal education.

 

 

 

 

 

Other forms of education

Data from the survey shows that some platforms provided and facilitated by the Met Office such as on the job training, internal Met Office documentation or Yammer, have been well utilised as forms of independent learning. Some of the less utilised platforms are extracurricular courses in person, and tutoring or mentoring sessions with a colleague. Perhaps this is because these platforms are not as useful or efficient, however it is also possible that these platforms are efficient but not well promoted to tech users in the Met Office. Further investigation is needed in order to find out whether the problem lies in quality of under-utilised platforms or the need for promotion.

Healthy and unhealthy habits

We asked participants to rate their regular frequency of the following activities:

  • Unhealthy Habits:

    • Spending time on a computer (hours per day)

    • Skipping a meal (times per week)

  • Healthy Habits:

    • Spending time outside (hours per day)

    • Exercising (times per week)

For time on a computer and time outside we coded participants’ scores as the median within the range they chose; for example, if a participant chose the range ‘5 to 8 hours’, we recorded the answer as 6.5.

For the other 2 variables, we coded participants’ answers:

  • ‘I don’t typically exercise’ or ‘Never’ as 0

  • ’1 - 2 times per week’ as 1

  • ‘3 - 4 times per week’ as 3

  • ‘Daily or almost everyday’ as 4.

Kernel density estimation graphs which illustrate trends in distribution of responses and scatter plots showing pairwise correlations are presented below:

habitautocorrelations-2.png

As shown in the kernel density estimation graphs for each activity, an encouraging figure is that very few participants scored extremely highly in unhealthy habits. The only concerning variable is the amount of time spent outside, the majority of participants spending less than one hour outside every day. Given the benefits of being outside on people’s wellbeing and productivity [3], the Met Office would potentially benefit from encouraging its employees to spend more time outside; perhaps implementing more outdoor seating areas would improve this.

An interesting finding is that there is almost no correlation between healthy and unhealthy habits. This means that participants who engaged in one of the two healthy habits did not necessarily engage in the other, and participants who engaged in one of the two unhealthy habits did not necessarily engage in the other. In addition, participants showing a healthy habit did not mean the absence of unhealthy habits, and vice versa.

Of course it’s hard to draw more conclusions without greater sample size (which within only one company would have to be close to totality), however perhaps if this could be run in future years and more momentum built with uptake, this would result in the ability to gain more representative results. This would also give us the opportunity to refine the questions asked based on what was learnt this time round, and further the analysis that we do with the tool knowledge already gained.

Bibliography

  1. "Stack Overflow Developer Survey 2018." Stack Overflow. https://insights.stackoverflow.com/survey/2018/#technology-how-technologies-are-connected.

  2. "How to Visualize Your Facebook Network." Linkurious. July 21, 2017. https://linkurio.us/blog/how-to-visualize-your-facebook-network/.

  3. Loria, Kevin. “Being Outside Can Improve Memory, Fight Depression, and Lower Blood Pressure - Here Are 12 Science-Backed Reasons to Spend More Time Outdoors.” Business Insider, Business Insider, 22 Apr. 2018, http://uk.businessinsider.com/why-spending-more-time-outside-is-healthy-2017-7/#walking-in-nature-could-improve-your-short-term-memory-1.

Penny Dinhissue-2-extra