This WP aims to develop a methodology and the necessary IT and techniques for revealing the impact of algorithmic biases in personalisation-based platforms to non-experts (e.g. youths), and for co-developing “fairer” algorithms in close collaboration with specialists and non-expert users.

In Year 1, based on co-developed scenarios from WP 1 and WP 4, we will develop a testbed for simulating hypothetical (collective) user behaviour (based on observations in WP 3) using agent-based modelling techniques and treating exemplary real-world platforms (e.g. search engines, recommender systems) as “black boxes” whose algorithmic properties can only be empirically evaluated a posteriori. Initially, fairness properties will be defined using calculative notions of individual and collective utility, but we expect that these will need to be refined into novel metrics through validation in the co-design workshops (e.g. we expect that human users are much more interested in “satisficing” rather than “optimal” behaviour of the system). At the end of this phase we will combine the observations from each of the four WPs to obtained definitions of human-centric fairness properties and will have developed usable fairness testing tools.

In Year 2 the objective is to move from improving transparency to enabling non-expert stakeholders to take an active role in designing interventions. Hackathon events will bring together expert programmers and users who participated in WP 1, WP 3 or WP 4 workshops to work on producing “fairer” algorithms. This will offer opportunities to distill requirements for explaining limits to algorithmic fairness and principles for poly-centric governance of Web-based social interaction-based platforms. Implementation work at this stage will be based on post-processing user inputs and systems outputs through “add-ons” to existing platforms. Specifically, we will explore the use of different amounts of user profile information and query detail as well as different levels of abstraction (in terms of taxonomies of tags and labels), and how these affect results, as well as re-ranking results based on user-defined metrics not explicitly submitted to the original system. The usability of interfaces for non-expert users will be important so that they can run their own experiments and examine results on these platforms while varying the configuration of the add-on functionality. The litmus test for success of this approach will be a double-blind evaluation of the modified system in terms of fairness compared to the original version of the platform as judged by users oblivious to the modifications.

We do not expect that this process will produce one-size-fits-all “fairer” algorithms. Rather, we expect that different users will favour different biases. We will therefore work toward building a platform that allows users to know, understand and judge the behaviour characteristics of the systems they are interacting with and potentially amend their interactions appropriately.

In addition to connecting with the other WPs by mapping the capabilities and limitations of technological solutions, the key outputs of this WP will be:

1. A collection of add-on tools for evaluating ‘bias’ in information filtering/recommending systems.
2. Policy brief on novel technological features that enhance global ‘fairness’ of social media algorithms.

Academic publications on user centred design of ‘fair’ algorithms.