Using AI to play Typing of the Dead 🧟

Some time ago I shared a Python script that I’d written that could complete the first level of Super Mario Land 🎮.

Since then I’ve being thinking of other games that I could try to automate playthrough’s of. One game that I’ve never played (until recently) is Typing of the Dead Overkill, which is basically House of the Dead but instead of shooting enemies manually, you type a word to kill them, improving your typing skills whilst playing – who needs Mavis Beacon ⌨️!

As this is fairly simple in nature it made me think that I could probably do something as follows to automate playing the game (using Python of course).

• Take a screenshot of the game (using PyAutoGUI)
• Pass this screenshot to the Azure AI Vision service (using the OCR capabilities) to extract the text
• Automate typing the text, again using PyAutoGUI
• Rinse and repeat

I grabbed a copy of the game from Steam and set about putting my master plan into action!

I managed to create a proof of concept for this (script available here), however it simply wasn’t performant enough so I kept dying ☹️.

I think moving the OCR processing from Azure to my local device would make this a workable solution and is definitely something I’ll look at in the future when I have time.

Before I threw in the towel however, I thought I’d try a low-tech approach, which to many astonishment worked really well and effectively can complete the game without any manual user input!

This approach does the following……runs a loop that types each character on the keyboard (a-z) in order and then repeats, it’s not pretty but it does the job! I put in a 10 second pause so that the script can be launched before the game (I have two monitors and had the script running on my second monitor).

import pyautogui
import time

time.sleep(10)
i = 1
while i == 1:
    pyautogui.write("qazwsxedcrfvtgbyhnujmikopl")

Here is a short video of the script in action: