I want to create a custom interface for Starcraft 2 to do an HCI investigation, and I am currently using the client's SDK to obtain information and interactions from the client's game.
While the SDK is ideal for automating customer interactions or replacing the user completely through AI, what I haven't found is a good way to create an alternative human interface for the game.
I would like my custom client to show multiple simultaneous views of the camera. Ultimately, I would like to access something like a mesh of each camera viewpoint on each tic, or even just the z-buffer and an image.
I was contemplating hacking together a conversion channel for SCII assets in UE4 (my target client environment) through Blender automation, and rendering myself. I think the biggest difficulty here is the implementation of details such as model animations, sounds, particle effects, stickers, etc.
Something like the scraping process or the GPU memory to get this information could be easier …