At times the so-called AI or the algorithm that decides what is and what isn’t can be a bit of a silly annoyance. Other times, it can make the difference between achieving an objective (such as withdrawing your money or verifying your identify) and not achieving it. And at the end of the day, there’s very little or no accountability. These days, if you’re digitally shortchanged, it’s harder and harder to break through the barriers to find an actual human being that might be able to help.
Therein lies my main complaint, which is actually a human decision thing. When a company gets to a certain size, it implements more and more levels of automation that are designed to (a) save money by requiring fewer employees, and (b) keep the customer and the attendant accountability at a distance. When you have millions of customers, losing a few or a few thousand to frustration due to bad (or intentional) programming is just an entry on a ledger somewhere, like pencils. It’s easily offset by the employee savings.
I had a very concrete example of this phenomenon when CitiBank decided to close my mother's bank account claiming it was out of paperwork compliance. They were claiming they did not have my power-of-attorney on file even though they must have; my name had been explicitly added to the account titleas POA! Even flying to California to give the actual human banker my notarized documents AGAIN could not stop the "machine" from pushing forward once the account had been flagged as to be closed. I sat there in the branch and watched a powerless "personal banker" try to talk on the phone with an accountable human being somewhere in "headquarters" and even she was unable to get the "system" to reverse a clear mistake on their part.
The truly evil thing is that this account was closed by THEM and then notated as "Closed at the request of the customer."
Well, yes, in a sense that's exactly my point. Building and authorizing tech that can operate and influence human behavior in significant ways without identifiable human oversight is the problem. Of course, we know from history that hierarchical "machines" can perpetuate evil even with humans running each step--can, in effect, become soulless bureaucracies where each human "cog" in the machine plays their part without any single actor having accountability. It's been argued that corporations are no different than totalitarian governments in that regard. The only solution in all cases is...robust democratic oversight. Imperfect but better than nothing.
Great article, Sandhya.
At times the so-called AI or the algorithm that decides what is and what isn’t can be a bit of a silly annoyance. Other times, it can make the difference between achieving an objective (such as withdrawing your money or verifying your identify) and not achieving it. And at the end of the day, there’s very little or no accountability. These days, if you’re digitally shortchanged, it’s harder and harder to break through the barriers to find an actual human being that might be able to help.
Therein lies my main complaint, which is actually a human decision thing. When a company gets to a certain size, it implements more and more levels of automation that are designed to (a) save money by requiring fewer employees, and (b) keep the customer and the attendant accountability at a distance. When you have millions of customers, losing a few or a few thousand to frustration due to bad (or intentional) programming is just an entry on a ledger somewhere, like pencils. It’s easily offset by the employee savings.
Good food for thought. Thank again!
I had a very concrete example of this phenomenon when CitiBank decided to close my mother's bank account claiming it was out of paperwork compliance. They were claiming they did not have my power-of-attorney on file even though they must have; my name had been explicitly added to the account titleas POA! Even flying to California to give the actual human banker my notarized documents AGAIN could not stop the "machine" from pushing forward once the account had been flagged as to be closed. I sat there in the branch and watched a powerless "personal banker" try to talk on the phone with an accountable human being somewhere in "headquarters" and even she was unable to get the "system" to reverse a clear mistake on their part.
The truly evil thing is that this account was closed by THEM and then notated as "Closed at the request of the customer."
I know. There are no words.
Wow. I hear you. I've had similar experiences with banks. They seem to be the vanguard of not giving a damn.
Would you not need to have a soul to be accountable? Serious question; I'm not well read.
Another thought is that one can imagine a machine being built with limitations such that it can't be operated without democratic human oversight.
Of course, my entire piece and this entire line of thinking are probably at least 25 years too late to be of any use.
Well, yes, in a sense that's exactly my point. Building and authorizing tech that can operate and influence human behavior in significant ways without identifiable human oversight is the problem. Of course, we know from history that hierarchical "machines" can perpetuate evil even with humans running each step--can, in effect, become soulless bureaucracies where each human "cog" in the machine plays their part without any single actor having accountability. It's been argued that corporations are no different than totalitarian governments in that regard. The only solution in all cases is...robust democratic oversight. Imperfect but better than nothing.