If you\u2019re a resident of Australia, you will be familiar with \u201cRobodebt\u201d. Robodebt was a scheme, put in place by the previous government, to identify and recover overpayments of welfare. It was later scrapped after growing controversy and thousands of incorrect debt recovery actions. The impact on people caught up in the scheme was profound, with associated claims of self-harm and suicide. Parts of the scheme were later deemed unlawful by a federal court.<\/p>\n
A Royal Commission (public enquiry) into the scheme was announced in August 2022, which put it back into the news. Listening to the ensuing avalanche of old tropes and scapegoats, you\u2019d have been forgiven for thinking you\u2019d gone back to the 1990\u2019s. \u00a0We thought the days of blaming I.T. for poorly conceived and executed solutions were over, but clearly not\u00a0 (the discussion in an episode of ABC\u2019s The Drum was a personal favourite[1])<\/span><\/a>.<\/p>\n Robodebt will remain a great case study for years to come \u2013 its generosity knows no bounds. With so many threads that you could pull apart, it is a great illustration for some themes that cry out for inspection through an ethical lens[2]<\/span><\/a>.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] Robodebt was fundamentally a data project. At its simplest, it worked like this:<\/p>\n What you focus on is shaped by your values and your values shape your focus. The focus of this project was clearly on transactional outcomes and not relationships. A transactional focus, with no emphasis on relationships, erodes trust in both directions. \u00a0Worse,\u00a0the scheme appears from the outside to be premised on two assumptions:<\/p>\n Thinking through these assumptions at the start should have triggered a serious look at:<\/p>\n Ethical goal setting isn\u2019t antithetical to innovation or efficiency. \u00a0It looks beyond the projects goal to the beneficence and capability improvement of the solution, key to sustainable innovation. The goal of this project to minimise ineligible spending is not in itself controversial (though you could argue that there are richer veins to mine). The primary problem was its design and execution, driven by un-checked assumptions about the problem. Secondly, it put a Band-Aid on the problem, clawing back overpayments after the fact, rather than improving the process.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” min_height=”321.6px” global_colors_info=”{}”] The signifiers of human existence are messy. Despite the potential benefits of new technologies, we must be alert to the dangers of low-hanging fruit. That is to say, data that is chosen primarily for its accessibility (versus accuracy and representational qualities).<\/p>\n There are many challenges with linking data across systems, including discrepancies with timing and granularity. \u00a0Possibly to get around such challenges, Robodebt went for the low-hanging data. Rather than use actual income data from the ATO for the period in question, it used a calculated average: a mechanism called \u201cincome averaging\u201d to calculate an \u201capportioned fortnightly income\u201d\u00a0 (see pages 4 and 5 of Deanna Amato v The Commonwealth of Australia court order<\/a>). \u00a0This average was used to determine whether they had made an overpayment for the period and instigate debt recovery actions[4]<\/span><\/a>, and, like any average, it was a poor substitute for the real number.<\/p>\n It must be assumed that actual income information for the period in question was not easily available, so they used what they could get and made it fit. Ethically a scheme with such major real-world impacts should use real-world (validated) information.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] Beyond the design decisions that made Robodebt problematic, the biggest failure was the apparent lack of a plan for addressing failure itself. Without being fatalistic, it behoves us to approach every project with humility and assume that even our sophisticated risk management tools will not identify every risk. Planning to fail acknowledges that things can, and likely will, go wrong. This is the difference between a correctable mistake and abject disaster in the event.<\/p>\n You get to your destination faster if the path in front is already marked and paved. Not to be confused with a risk management plan, a plan for failure paves the path for response to failure by pre-defining the project’s tolerances, responsibilities and communications. To detect and address unintended outcomes, regular monitoring and verification of results (especially if you\u2019re using estimated values coupled with an automated process) is essential. So is a common understanding of what constitutes a \u201csignificant\u201d issue and how it will be recognised as well as showing the community, as major stakeholders, how they can engage in the process. For example – \u00a0affected citizens spent hours on the Centrelink helpline \u2013 there should have been a hotline and\/or website for reporting problems with the scheme given its potential impact and the vulnerability of much of the community.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] There is a story we sometime use in our workshops about a U.S. study conducted by Professor Bar-Yam of the New England Complex Systems Institute.<\/a> This study used sentiment analysis on geo-located Tweets to create a sentiment map of Manhattan. The \u201csaddest spot in Manhattan\u201d, according to this study, turned out to be\u00a0an elite public school in the city. The finding was picked up and reported by \u201clocal\u201d papers, including the New York Times. Staff and students at the school were distressed and confused by this, and worried it would affect the school\u2019s reputation. When the Professor took a closer look at the data he<\/p>\n \u201crealized that he had incorrectly interpreted a data map\u201d and that \u201cCloser analysis revealed that the posts had actually come from a single Twitter account \u2018from a region just south of the school\u2019<\/em> \u201d, according to a NYT follow-up article<\/a>.<\/p>\n It is unlikely that this is the type of interest Prof. Bar-Yam or the Institute wished to generate from his study (the Lure of Low hanging data again). When creating new \u201cdatafied\u201d assets it is important to remember that this is a real-world proxy. Like Robodebt, this case study offers lessons at many levels. For now, we want to focus on the need for designing data systems that build in respect for and feedback from the people represented in the data.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] As Jer Thorpe writes in his own discussion<\/a> of this case (emphasis added):<\/p>\n It\u2019s a world that flows in one direction: data comes from us, but it rarely returns to us. The systems that we\u2019ve created are designed to be unidirectional: data is gathered from people, it\u2019s processed by an assembly line of algorithmic machinery, and spit out to an audience of different people [\u2026]. This new data reality is from us, but it isn\u2019t for us<\/strong>.<\/p>\n<\/blockquote>\n How can we build new data systems that start as two-way streets, and consider the individuals from whom the data comes as first-class citizens?<\/p>\n By inviting the faculty and students to review the pre-published data as subject matter experts, we would shift from a perception of them as mere data sources to full data partners. This opens the opportunity to catch data errors before publication and gain insights that may have otherwise been missed. Most importantly this two-way street empowers the \u201csubjects\u201d to have a voice on the data\u2019s legitimate and beneficial use.<\/p>\n We need to be more deliberate in our efforts to ensure that people and communities are appropriately represented in the data we work with. Imagining the pathway from person to proxy as a two-way street would go a long way to building more representative – and thereby better quality – data. Returning to the community earlier in the process would elicit richer understandings about any data collected. Involving them in the analysis of the data in some way also offers an opportunity to enhance their data literacies, empowering them to speak about data that represents them. At the very least, better documentation of context (for instance in the descriptive metadata associated with a data set) could disambiguate the data and support community feedback.<\/p>\n Working to create such two-way streets in our data work would make sure that the community from whom data was taken also benefit from the process. A good example of this approach is the MobileMe study[5]<\/span><\/a>. We looked at mobile phone activity of students to inform school policies on phone use at a point in time when the mobile was an emerging technology. We gave this data back to the students, and trained them in research methods, so that they could tell us what they thought the data revealed (with the added benefit of building their data literacy skills). The importance of a two-way street is all the more acute when dealing with young people, who have precious little power and influence over policy decisions that affect them.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] <\/strong><\/a><\/span>Many of us agree that if you market high power weapons to civilians in an irresponsible manner you share some blame for a negative outcome. By the same token, if you\u2019re in the business of publishing or sharing data, we don\u2019t believe that your responsibility ends when the data leaves your servers.<\/p>\n Publicly available data is regularly used to generate what we call \u201cclickable factoids\u201d for attention-grabbing headlines and to promote clickthrough\u2019s. The High School Twitter story is a typical example of this. Of course, like weapons, we can\u2019t control what someone does with our data when it leaves our servers, but we can control who we share it with and how we frame its ethical use. It is our responsibility to think about how our data could be used and how this could impact specific groups. It is important to know our ethical red lines and communicate them to the downstream users of our data products. For example, data publishers could have a clause about \u201cfrivolous use\u201d in their terms of use, and a definition of what they consider \u201cbeneficial use\u201d (note that Twitter does talk in a limited way about data use).<\/p>\n By the way, thanks for the metaphor Cathy O\u2019Neil \u201cWeapons of Math Destruction\u201d<\/a> if you haven\u2019t read it, stop reading this article and go get a copy now.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] In Australia, all employees who serve alcohol must have a Responsible Serving of Alcohol certificate. What would an RSA for data look like?<\/p>\n Data has a social life that influences the way it is made and used. How it is shaped by researchers and analysts will reveal specific aspects of the critical social networks the data inhabits. In the words of Brown and Duguid[6]<\/span><\/a><\/p>\n \u201c\u2026to participate in that shaping and not merely to be shaped requires understanding such social organization, not just counting on (or counting up) information.\u201d\u00a0<\/span><\/p>\n<\/blockquote>\n Prof. Bar-Yam would have surfaced and ranked many groups in his analysis, based on sentiment scoring. The press chose to name one data point (a high school) as the \u201csaddest place\u201d in their re-telling of the study (the “clickable factoid”). In this case the data point turned out to be incorrect. What responsibility would Bar-Yam and the Institute, alongside the press, have had if the data had in fact been \u201ccorrect\u201d? By extension, what was Twitter’s responsibility as the original supplier of the data?<\/p>\n The responsible use of data involves doing more than simply scratching the surface on the data you are working with. Data\u2019s value\u2014and its power \u2013 is more than skin deep. It means not just paying attention to regulated uses of the data, but appreciating that data\u2019s meaning \u2013 and power \u2013 is shaped by its social life.\u00a0Responsible use also involves thinking beyond what you can do, and instead focusing on what you OUGHT to do.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] There are many dimensions to the ethical use of data. More often than not, discussions of data ethics pivot around data privacy and security, and specific applications such as Artificial Intelligence (AI) and automation. Yes, there are important ethical considerations in each of these areas, but, to be a responsible server of data, we should be thinking of a multitude of ethical considerations from the moment the data comes into our orbit. Doing so will make us all better data practitioners, generating more useful and reliable data products, ethically.<\/p>\n We need to find ways to engage productively, both individually and collectively, with risk and adversity. We need to ensure oversight so outcomes can be assured of serving all sectors of our communities — particularly the most vulnerable. We need a culture of care.<\/p>\n This is the first in a number of articles we intend to publish over the coming months on the subject of data ethics. It dips a toe in the water but leaves many issues untouched. We hope to dig into these in some detail in future discussions.<\/p>\n What do you<\/em> think it takes to be a responsible server of data?<\/p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_divider _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_divider][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] <\/p>\n We bring a mixed disciplinary approach to our work on the ethical use of data. We have worked in Information Technology and Data Science disciplines for 25+ years. Both focussing on different aspects of ethical practice in Data Science and AI for the past decade. The strong socio-technical components of Theresa\u2019s research in ethical data practices and Ruth\u2019s experience in data science and experiential learning design round out our collaborative practice. We both have experience in the development of reference and actionable frameworks at national or international levels and are currently involved in the development of international standards for Data Usage and Trustworthiness (ISO JTC1\/SC32\/WG6 & JTC1\/WG13).<\/p>\n Please leave a comment or reach out to us on LinkedIn:<\/p>\n <\/p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=”1_2,1_2″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”1_2″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_image src=”https:\/\/www.hoc.one\/wp-content\/uploads\/2022\/10\/Ruth-round-head-1.png” title_text=”Ruth round head” align=”center” _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] Ruth Marshall<\/u><\/strong> \u2013 Director, Ethical and Privacy-Preserving Analytics here at Hocone Pty Ltd<\/a><\/p>\n linkedin.com\/in\/ruthemarshall<\/a><\/p>[\/et_pb_text][\/et_pb_column][et_pb_column type=”1_2″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_image src=”https:\/\/www.hoc.one\/wp-content\/uploads\/2022\/10\/Theresa-round-head.png” title_text=”Theresa round head” align=”center” _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] Dr. Theresa Anderson<\/u><\/strong> \u2013 Director and Social Informaticist at Connecting Stones<\/p>\n linkedin.com\/in\/theresadanderson<\/a><\/p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_divider _builder_version=”4.18.0″ _module_preset=”default” min_height=”18.8px” custom_margin=”||14px|||” custom_padding=”34px|||||” global_colors_info=”{}”][\/et_pb_divider][et_pb_text _builder_version=”4.19.5″ _module_preset=”default” min_height=”72.2px” hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″] [1]<\/span><\/a> ABC\u2019s The Drum August 25th 2022 (39:49)<\/a>. Blaming the developers seems especially clumsy following Shane Wright\u2019s analysis just beforehand \u201c\u2026this was always about dollars and the people involved in the welfare system were ignored completely\u2026\u201d<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] [2]<\/span><\/a> This is an outsiders perspective. We have no inside information on Robdebt and other high profile cases discussed in this article.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] [3]<\/span><\/a> \u201cWhile Australia saw a record level increase in public trust in institutions during the pandemic, this \u2018trust bubble\u2019 has since burst, with societal trust in business dropping by 7.9% and trust in government declining by 14.8% from 2020-21.\u201d CSIRO Future Report:\u00a0 Our Future World – CSIRO<\/a><\/span><\/a><\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] [4]<\/span><\/a> This part of the scheme was later deemed illegal by a federal court.<\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”] [5]<\/span><\/a> an ARC-funded project conducted in partnership with the NSW Commission for Children and Young People (2006-2011), using child-centred participatory research practices, the project brought together young people, educators, researchers and industry players into productive dialogue to explore the benefits and dangers of mobile phones in schools. \u00a0See the following link for a little more discussion: http:\/\/informationr.net\/ir\/18-1\/paper565.html#.Y0TmUHZBzD4<\/a><\/p>[\/et_pb_text][et_pb_text _builder_version=”4.18.0″ _module_preset=”default” global_colors_info=”{}”]Data Ethics Starts at the Beginning<\/h2>\n
\n
\n
\n
\n
\n
\n
Beware the Temptation of \u201cLow Hanging Data\u201d<\/h2>\n
Plan to Fail<\/h2>\n
Don\u2019t forget the humans behind the data<\/h2>\n
Create a two-way street<\/h2>\n
\n
\u201cI just sell the weapons\u201d<\/span><\/h2>\n
The \u201cresponsible serving of data\u201d<\/h2>\n
\n
Data Ethics is more than\u2026<\/h2>\n
About Us<\/h2>\n