If the premise is accurate this means you could take a person who fit this profile, one that seemed to lack a conscience, and operated largely using their id into a position to emulate what an AI might do. Rather than a computer emulating a human, what O’Neal seems to be arguing is that you’d have a human emulating an AI. Or, in this case, President Trump becomes a model for how you might create an AI that could run government.
For President Trump, O’Neil argues the end result we are now seeing is the outcome of having him move from an initial training process based on the election, which was focused on dynamic competitive information on his opponents to a very different feed now that he is President and that his changing behavior is based on those new information sources. It also showcases a system where the reward structure appears to be largely based on attention and suggests that such a structure would be problematic.
You’d then have a real-life example of how informational or programing errors could manifest in bad decisions and operational problems. From this you could then develop models to either assure information accuracy tied to proper metrics so you wouldn’t end up with a Terminator Judgment Day outcome.
Avoiding a Judgment Day scenario
O’Neil suggests the way to fix the system is to fix the quality of information being fed into it, I’d also argue you’d need to fix the reward mechanism. But, I do think there is merit in using people with certain behavioral elements to emulate AIs as we seek to hand over control to them and let them make decisions in simulations. This would allow us to iterate and improve training, reward and data models prior to applying them to machines and significantly slowing down the proliferation of problems resulting from mistakes. This would all be to assure that when we did create something like Skynet, (fortunately the real SkyNet is a delivery service), it wouldn’t result in a Judgement Day scenario.
Something to think about this weekend.
Sign up for CIO Asia eNewsletters.