My friends and I got into playing a rhythm game called DJMAX Respect V after a trip together earlier this year, and have been competing against each other on scores since. However, the in-game leaderboards don’t have a friends option so you can easily see what your friends scores are.
We were originally just screenshotting scores and posting them manually to Discord, but there’s a much nerdier way to do this so I made a small companion app that we could all run which would automatically grab the scores and post them to Discord for us.
Both the companion app and the server were written in C#. The companion app uses WPF, and the server app is uses Linux compatible .NET.
DJMAX Discord Companion
The companion is the app that everyone runs on their PC when they play if they want to submit scores to Discord. It’s very simple to use, you simply press PrintScreen when on the song select screen and it’ll read out which song is selected, button mode, difficulty, your score, and send them all to the server component to process.
DJMAX Respect V runs on Unity, but its always online and connects to a private API to get user info and scores, and it runs an anticheat to stop people from uploading faked scores. Rather than mess with the anticheat to access game memory directly and potentially get my account banned, I opted for an OCR solution that wouldn’t upset the anticheat (especially since I’m giving this to friends to use too.)
The OCR runs in three parts: one to detect the current song, one to read button mode and difficulty, and one to read the score.
The first part, song detection, works by taking a screenshot of the name and preview image of the currently selected song, shrinking it down to a specific size, turning it into a 2 bit greyscale image and using the scaled-down-and-greyscaled image as a bitmask that we can compare other songs against. Every song in the game needs to be screenshotted and entered into a database for the song detection to perform a search against when the user wants to submit a score, but this is a one time thing and only needs to be updated when new songs come out. The song database is a json file and gets updated automatically when opening the app, so when new DLC for the game released we don’t need to download new versions of the app just to submit scores.
I don’t use an OCR text engine for the song titles as some titles can have quite noisy backgrounds that mess with the engine’s ability to read the text cleanly. Additionally, there are songs in the game where the title has non-Latin characters or special characters in the name which the OCR engine I used struggled to read with consistent results.
Button mode (4/5/6/8 button play modes) is detected by checking the hue of a specific area of the full screenshot. Each button mode has a different color associated with it, in the screenshot below I’m playing on 6 button which is orange. I check the hue against the known values for each button mode and so long as we’re within acceptable values use the mode that has the best matching color. I also use this method to check if the player achieved a Max Combo in the song, and to count the number of stars that appear below the difficulty selection (these give a general idea of the actual difficulty of the song) so that we can send this to the server too, as these can change between updates.
For difficulty, I crop out each individual difficulty button (normal, hard, maximum, SC) and calculate an overall brightness value for each image. As DJMAX displays the current difficulty (maximum, in the screenshot above) in knockout text it means that the selected difficulty should always have the brightest overall value.
Finally, for score I use the Tesseract OCR engine as this one needs to read actual text. Unlike the song names though, the background for the score is a mostly black background, and the characters in the text will always be the 0-9 characters with a maximum score of 1,000,000. I crop out the score, increase the contrast so that the text is even more pronounced from the background, and invert the image if required so that the text is black-on-white and easier for Tesseract to read. It gets processed by the OCR engine and gives us a score which we can send on alongside all the other data we read.
There’s some error handling in there too, but the companion app doesn’t really need to do much more than this for what my friends and I use it for.
Discord Bot
Once we get the players results from OCR we need a place to send them. For this I made a very simple Discord bot that listens for incoming scores, processes the results and adds everything into a database before posting updates into a Discord channel.
Like the companion app, it’s written in C#, but it uses .NET Core to ensure I can deploy it on my little Linux server I’ve got for things like this. The scores are all saved to a local SQLite database.
The scores get posted to our private Discord where we can all compare our results and try to one up each other, or complain about how hard songs are.
There’s no fancy commands for asking the bot to do things since everything is posted into Discord anyway, so if we want to check song results then we use the search that’s built into Discord itself. If there’s no results, there’s no scores posted for that song. Why reinvent the wheel.