All of the robots in RoboJackets are built to solve a competition problem statement: knock a robot out of the ring, play soccer, navigate off-road, destroy another robot, or race around a track. When our members are not solving these crazy problem statements (and of course balancing life and school work), many like to kick back, relax, and play some video games. It seems that robots can’t help but still find their way into our relaxation time, even video games. This is certainly the case for a new personal project for a few of our members: Aimi.
Automation in video games is certainly not a new concept. There have been complaints about aiming assistant computer bots (or “aimbots” as they are often called) for the popular first-person shooter (FPS) video games over the years. An aimbot helps put the in-game weapon sight of the player on enemies faster or more accurately as opposed to manually by the player using a mouse or joystick.These aimbots operate exclusively in software running on the player’s computer or on a console (with current generation console security this is often seen to a lesser degree) and act as an input device to move the player’s sight.
Aimbots are often a source of frustration in FPS games, with losing to a player using an aimbot on the enemy team producing a feeling of annoyance due to the lack of an even playing field and a “cheapening” of the competitive experience. The bots are often quite effective, but their utilization typically leads to punishment by game developers as an attempt to disincentivize their use. Whether we like it or not, software can be better than humans in certain situations, and aimbots are just another example.
Being vastly superior to humans at identifying enemies in a video game via software was not the initial intention with Aimi, though. Cory Stine, a recent Georgia Tech Mechanical Engineering Masters graduate (of course as a RoboJackets member) and initial creator of the Aimi concept, said, “I was thinking about aimbots, and I thought, ‘What if a [robot] arm aimed for me?’“
Unlike most aimbots that operate in software, Aimi will physically move the input device, simulating a human arm and hand to move the computer mouse. It will have all of the software of a typical aimbot, including filtering the computer view and identifying a target. Additionally, there are the tasks of planning and executing trajectories to line up the shot, problems unique to a physical robot. Along with the software, the project will also require mechanical design as the basis of interfacing with the input device and electrical design for motor control along with using sensors for tracking position to make Aimi move (hopefully) accurately. With the added layers of processing required and time delays involved with moving a physical device, the development team expects a much slower response movement time compared to “traditional” aimbots or even a human. However, the fun and interesting aspect, as identified by the team, is making the process physical, not necessarily winning (while hopefully also not getting banned for using an aimbot).
The Aimi project is still in its infancy, as the team has just begun building prototypes to solve many of the problems that will need to be dealt with in the final robot. There has been progress on automating the playing of Osu!, a rhythm game heavily focused on accurate input movements and timing. In its current stage of development, the goal is to capture the screen of the PC running Osu (“Game PC”) with a HDMI capture card being sent to another PC for processing (“Processing PC”). The Processing PC then sends back to the Game PC mouse positions and clicking commands using an Arduino, simulating a mouse. In “Osu!standard”, there are three input elements: “hit circles”, “sliders”, and “spinners”. Each input element requires a different method of interaction, with hit circles (red circles in Figure 2) requiring a click (be it a keyboard key or mouse click), sliders requiring a click and drag action in a specific path (red circles with yellow paths in Figure 2), and spinners requiring a circular click and drag action while rotating several times (not pictured in Figure 2).
After the initial mouse simulation of Osu! is complete, work will begin on designing the robot. Some initial ideas include a SCARA robot, robotic arm, or even an omnidirectional driving robot. The team is in the early stages of physical robot design, and they will take into consideration speed, cost, and manufacturability, just as we do for RoboJackets robots!
Aimi is a side project for a few of our members balanced with summer internships, classes, or even full-time jobs. Because of this, progress will be expected to be slow and carry on to 2021 as time permits, with the eventual goal of Aimi being able to aim in the new FPS game Valorant. Though this isn’t officially a RoboJackets project, the members involved have been using the skills learned in the organization, on both the technical and project management side. In an often typical RoboJackets fashion, the project will be open source on GitHub: https://github.com/thayes46/aimi.