Context Matters

This is a response to discussions concerning AI generated disaster images.

If we want to reduce the power of bad actors in the era of generative AI, many things we've accepted as implicit are going to need to become explicit, and things we've allowed others to do for us are going to become our responsibility again.

There are three essential components to this effort:

Provenance

Content we encounter today has no "chain-of-custody"; there is no obvious way to differentiate between a photo originally taken by our grandmother, and an image originally generated by the DPRK.

And with the rise of consumer AI, it is now cheaper to produce realistic fake content than it is to go out and capture real images. In our engagement-driven attention economy, that means the economic incentives now favor the most duplicitous actors and the most destructive content.

To correct this, we must demand that content be verifiable. In the same way that we now expect websites to use secure connections (and we avoid, for our own protection, websites that lack this security) we must expect content to include a cryptographic chain-of-custody that shows who created, modified, or amplified it... and refuse to view or interact with content that lacks this record.

Reputation

Provenance is only half the story; it doesn't do any good to be able to prove cryptographically that John Doe wrote an article... if we don't know anything about John Doe. Reputation is the essential compliment to provenance, allowing us to couple that chain-of-custody with a clear understanding of each custodian's expected behavior.

I'm not just talking about big media companies here... I'm talking about having a way to model that fact that Uncle Gerald regularly amplifies content that either lacks strong provenance and/or is linked to content farms. Or even how our own actions contribute to the spread of misinformation. Reputation systems must be able to hold each of us accountable for the content we amplify in our own digital communities.

Autonomy

The personal nature of reputation systems means we're going to want to build and manage these reputation systems independently. There can't be "one reputation system to rule them all", since we may rightly hold divergent views on the merits of different perspectives or approaches to reputation.

My conviction is that managing reputation becomes feasible insofar as we enable each person to do so independently.