In the Land of the Blind, the One-Eyed Man Is King: Knowledge Brokerage in the Age of Learning Algorithms
Card Grid View — Paper by Waardenburg et al. (2022)
1. Case Study: Dutch Police CAS
- Crime Anticipation System (CAS)
- Predictive policing algorithm developed in-house
- Dutch police chose not to buy US "PredPol" system
- Used machine learning to predict crime hotspots
- Knowledge brokers
- Intelligence officers acted as intermediaries
- Between data scientists and police managers
- Tasked with translating algorithmic predictions
2. Black Box Problem of ML
- Opaque nature of learning algorithms
- Difficulty understanding how connections between data are made
- How predictions are generated is not transparent
- Three parts of AI (von Krogh)
- Task inputs (data fed into system)
- Task processes (how algorithm processes — the "black box")
- Task outputs (predictions produced)
- Key problem
- "Noise" is NOT one of the three parts
- Impassable knowledge boundary between brokers and ML
3. Three Brokerage Roles
- 1. Messenger (early stage)
- Simply relay predictions without interpretation
- Police managers dismissed raw predictions
- 2. Interpreter (middle stage)
- Translate predictions into actionable insights
- Begin adding context and judgment
- 3. Curator (final stage)
- Substitute AI predictions with own judgment
- Become "kings" — the only ones who understand
- Police managers now dependent on their expertise
4. Knowledge Boundaries
- Why boundaries are impassable
- ML algorithms reason differently than humans
- Pattern recognition vs causal understanding
- Even brokers cannot fully explain predictions
- Consequence
- Brokers eventually stop trying to understand
- They substitute algorithmic outputs with own expertise
- Contextual bilingualism is key but limited
- Translation practices
- 5 practices to translate to user community
- Some boundary-spanning tools actually solidified the boundary
5. Why Brokers Become Kings
- Central puzzle
- Algorithmic brokers gain power over time
- Their interpretation becomes authoritative
- Risk
- Brokers become too influential
- Subjective human judgment replaces algorithmic output
- Managers cannot verify or challenge brokers
- "Land of the blind" metaphor
- Everyone else is blind to algorithm's workings
- Broker (one-eyed) becomes indispensable
- Power shifts to those who control interpretation
6. Brokerage vs Traditional Knowledge Work
- Key difference
- Traditional: broker translates between human communities
- Algorithmic: broker translates between machine and humans
- The machine side is fundamentally opaque
- Knowledge broker vs boundary spanner
- Different concepts — broker actively translates
- Boundary spanner connects across existing boundaries
- Central challenge
- Learning algorithms cannot fully explain their reasoning
- Brokerage becomes one-way interpretation, not true translation
7. Theoretical Contributions
- Main contribution
- Shows how algorithmic brokerage differs from traditional
- Opens up the "black box" of knowledge work with ML
- Brokerage evolution
- Messenger → Interpreter → Curator
- Each stage increases broker influence
- Implications
- Organizations must manage broker power carefully
- Need transparency in how predictions are translated
- Addressing opacity requires more than technical fixes