a game of AI charades:
Learning Machine LearningA game of charades based on machine error.
played by Anna Smeragliuolo and Mantas Lilis
![](https://freight.cargo.site/t/original/i/a6931737cf3bdbd5eb8d96cb703014289b14936d5fbc2355f0273f33635bd455/LML-actionb.gif)
This is a project about computer perception.
About understanding that machines do not operate without error, and in order for them to bend to our wills, we must also bend a little bit to theirs.
About understanding that machines do not operate without error, and in order for them to bend to our wills, we must also bend a little bit to theirs.
here’s the game:
The machine generates the caption for a player to act it out. ![](https://freight.cargo.site/w/800/q/67/i/159e7e8b1b1fc63a5de72193e71b4ab4a5065d3eecc7febcf18d2cf23fc2e272/LML-caption.png)
![](https://freight.cargo.site/w/800/q/67/i/159e7e8b1b1fc63a5de72193e71b4ab4a5065d3eecc7febcf18d2cf23fc2e272/LML-caption.png)
![](https://freight.cargo.site/w/800/q/67/i/026b8fb678954f36e6e2945687d1373e58681f1a536976ebedb479c5a917daf2/LML-caption-banana.png)
After a success, the machine will generate a new caption.![](https://freight.cargo.site/w/800/q/67/i/128a0b75e6c68d4642101c45753ffc26c295e71a2a5c86d4ae87e0a4109752fb/LML-caption-reset.png)
![](https://freight.cargo.site/w/800/q/67/i/128a0b75e6c68d4642101c45753ffc26c295e71a2a5c86d4ae87e0a4109752fb/LML-caption-reset.png)
here’s the process:
To test the limits of im2txt, we took a computer for a walk outside to see how it would perform.
We noticed some curious errors and misidentifications. What was it about input that caused it to mislabel the scene?![](https://freight.cargo.site/t/original/i/0e9770aa8f99e06279262631836df0760751e57af0ed293392290596ebfb104a/LNL-Caption-Drinks.gif)
We tried to trick the algorithm into repeating its own misidentifications, and we realized it felt like playing a game of charades.
Sometimes the errors made sense according to human perception - legs as scissors, thick paint as cake, stripes as zebras - other times not so much.
![]()
![](https://freight.cargo.site/t/original/i/1cae12a28417b739c560be9cc7cb0b3554fae81cea17c9d395e63c2d78434120/almost.png)
![](https://freight.cargo.site/t/original/i/3d7db1855367cc18c6615a947f7652d51b0318b5c25d49305a5743e4eb4a2b0d/bwPhoto.gif)