Question

What are "black box algorithms"?

Hi APM team,

I had a question on this term that appears in the syllabus. Here's what it's about:

In IT jargon, "black box" is when you can't see the programming code, you can only see the input and output of the system, e.g. black box testing.

A "black box algorithm" is a term from AI. This term describes an algorithm that is using a big data set with complicated calculations and machine learning, so users don't understand what is happening 'under the hood,' they only see what comes out of it. So there is lack of transparency in the process.

The link to APM is the ethics of using AI, not the technical background of this topic. For example, is a company discriminating against certain segments of the population when they use AI in the recruitment and selection process? How about a bank when they use AI to score you for loan approval? There might be bias in the decision making supported by these AI systems created by black box algorithms that is not understood by the users of these systems.

So, If you are asked to evaluate the use of AI in a question, you can use black box algorithms as example of a potential problem when you discuss the drawbacks. Link your ideas to the scenario (e.g. be specific about what stakeholders could be affected and how) and you'll earn both technical and business acumen marks.