The professional body for IT has warned that human review of AI decision making should not go without legal protections.
BCS, The Chartered Institute for IT said that, as AI doesn’t always involve personal data to make decisions about us, true protection of our right to revisit must consider wider regulation of the technology.
A government consultation on personal data suggests human appeal against some automated decisions by AI could be unnecessary.
Launched by the Department for Digital, Culture, Media and Sport (DCMS), the consultation, Data: A New Direction, is looking to update the UK’s version of the GDPR after leaving the EU.
The DCMS is seeking further evidence before forming firm proposals with regards to reform of the UK’s existing data legislation, including considering the removal of Article 22 of the GDPR – which focuses specifically on the right to review fully automated decisions.
Explaining in more detail, Dr Sam De Silva, Chair of BCS’ Law Specialist Group and a partner at law firm CMS, said: “Article 22 is not an easy provision to interpret and there is danger in interpreting it in isolation like many have done.
“We still do need clarity on the rights someone has in the scenario where there is fully automated decision making which could have significant impact on that individual.
“We would also welcome clarity on whether Article 22(1) should be interpreted as a blanket prohibition of all automated data processing that fits the criteria or a more limited right to challenge a decision resulting from such processing.
“As the professional body for IT, BCS is not convinced that either retaining Article 22 in its current form or removing it achieves such clarity.
“We also need to consider that protection of human review of fully automated decisions is currently in a piece of legislation dealing with personal data. If no personal data is involved the protection does not apply, but the decision could still have a life-changing impact on us.”
BCS said it supports the consultation and will be gathering views from across its membership. The consultation says that automated decision-making is “likely to increase greatly” across industries in the coming years.
The need to maintain a capability to provide human review may, in future, not be practicable or proportionate, and it is important to assess when this safeguard is needed and how it works in practice, BCS said.
The organisation acknowledged that there may be “legitimate need for certain ‘high risk’ AI-derived decisions to require a human review, even if this restricts the scope of use of such systems or makes them slower.”
De Silva continued: “Based on the input, the decision could be that you’re not eligible for a vaccine. But any protections in the GDPR would not apply as there is no personal data.
“So, if we think the protection is important enough it should not go into the GDPR. It begs the question – do we need to regulate AI generally – and not through the “back door” via GDPR?
“It is welcomed that government is consulting carefully before making any changes to people’s right to appeal decisions about them by algorithms and automated systems – but the technology is still in its infancy.”
DCMS said it is seeking further evidence as part of the consultation before forming firm proposals on Article 22 and the right to request a review of fully automated decisions.
Source: DIGIT