Input recording & re-recording

This time we’re diving into the world of tool assisted speedrunning – also known as ‘TASing’. Simply put, the process of creating a speedrun using third-party tools. In this post, we’ll build the core recording & playback logic, add the ability to re-record and finally, replay the recording on a live server.

Wait, what’s a speedrun? Essentially, it’s a playthrough of a video game where you try to reach the end as quickly as possible. In the context of most online Source games, a speedrun is performed on a single map – usually in the surf, bunny-hop or climbing style. Servers hosted by the community usually include timing and ranking functionality which allow players to keep track of their personal records and compare them with others.

So where does the tool-assisted part come in? While a regular speedrun is performed within the players own ability; a tool-assisted speedrun (TAS) is a different story entirely. TAS creators can utilise tools including slow-motion and individual frame advancing to push the game to its limits. It’s worth noting that these kind of speedruns aren’t solely distributed as videos, instead, the input from each frame is saved to a file which can be replayed or re-recorded over at a later date.

Re-recording? Here’s the scenario: you’re on the final stage, everything has gone perfectly up until now but you’ve just made a mistake and now the whole recording is ruined! If only there was a way to re-do that last part… that’s re-recording. To be specific, our implementation will allow us to play the recording back and take over at any point by simply pressing a button. Once finished, we’ll have the choice to either keep the re-recorded frames or discard them and try again.

Before we start, there are some potential issues that need to be addressed. Our input recorder will be entirely client-side. This means we can record locally and then play it back on a remote server using the same settings. However, unlike most server-side recorders, there is the possibility of our recording desynchronizing during playback. We’re also somewhat limited in terms of rewinding or jumping to specific parts but hey, it’s part of the challenge.

For this tutorial, I’ll be playing surf_kitsune on Counter-Strike: Source. You’re free to follow along with a different game since most of the code is fairly generic. I’ve uploaded template projects for both Counter-Strike: Source and Counter-Strike: Global Offensive to save you some time. Inside, you’ll find all the required classes and structures, a basic CreateMove virtual hook and a DirectX 9 ImGui implementation since we’ll need a basic user interface.

First things first, let’s think about the core implementation. I’ve previously covered a way to retrieve and manipulate player input with the CreateMove function. We can use this for both recording and playback – simply copying the values in the CUserCmd while recording and setting them during playback. Don’t worry about re-recording just yet, we’ll get there eventually.

In order to create a reproducible recording, we only need a limited selection of what the CUserCmd has to offer. Let’s start off by making a structure called ‘Frame’ which will contain these values.

struct Frame {
  float viewangles[2];
  float forwardmove;
  float sidemove;
  float upmove;
  int buttons;
  unsigned char impulse;
  short mousedx;
  short mousedy;
};

During recording we’ll create one of these, populate it with the data from the current CUserCmd and then store it in the ‘current recording’ container. On the other hand, when playing a recording back we would do the inverse, copying the previously stored data into the current CUserCmd. For convenience, let’s create a constructor that reads from a CUserCmd and a public method that writes called ‘Replay’. These will be used later in CreateMove.

Frame(CUserCmd* cmd) {
  this->viewangles[0] = cmd->viewangles.X;
  this->viewangles[1] = cmd->viewangles.Y;
  this->forwardmove = cmd->forwardmove;
  this->sidemove = cmd->sidemove;
  this->upmove = cmd->upmove;
  this->buttons = cmd->buttons;
  this->impulse = cmd->impulse;
  this->mousedx = cmd->mousedx;
  this->mousedy = cmd->mousedy;
}

void Replay(CUserCmd* cmd) {
  cmd->viewangles.X = this->viewangles[0];
  cmd->viewangles.Y = this->viewangles[1];
  cmd->forwardmove = this->forwardmove;
  cmd->sidemove = this->sidemove;
  cmd->upmove = this->upmove;
  cmd->buttons = this->buttons;
  cmd->impulse = this->impulse;
  cmd->mousedx = this->mousedx;
  cmd->mousedy = this->mousedy;
}

Perfect! Now we can move on to the Recorder class. This is also fairly simple; for now, we’ll only need a “is recording” variable and a dynamic size container for storing recorded frames. We’ll also add some simple accessors.

typedef std::vector<Frame> FrameContainer;

class Recorder {
  private:
    bool is_recording_active = false;
    FrameContainer recording_frames;
  public:
    void StartRecording() {
      this->is_recording_active = true;
    }

    void StopRecording() {
      this->is_recording_active = false;
    }

    bool IsRecordingActive() const {
      return this->is_recording_active;
    }

    FrameContainer& GetActiveRecording() {
      return this->recording_frames;
    }
};

Believe it or not, that’s everything for the recorder class. Let’s create the Playback class now, start with a “is playback active” variable, an unsigned integer for the current frame and finally, a reference to the recording.

class Playback {
  private:
    bool is_playback_active = false;
    size_t current_frame = 0;
    FrameContainer& active_demo = FrameContainer();
  public:
    void StartPlayback(FrameContainer& frames) {
      this->is_playback_active = true;
      this->active_demo = frames;
    };

    void StopPlayback() {
      this->is_playback_active = false;
      this->current_frame = 0;
    };

    bool IsPlaybackActive() const {
      return this->is_playback_active;
    }

    size_t GetCurrentFrame() const {
      return this->current_frame;
    };

    void SetCurrentFrame(size_t frame) {
      this->current_frame = frame;
    };

    FrameContainer& GetActiveDemo() const {
      return this->active_demo;
    }
};

There’s a little more going on here compared to the Recording class – still, nothing complicated. Playback can be started by calling StartPlayback with a reference to an existing recording. Now we’re ready to use our classes in CreateMove.

It’s worth noting that the CreateMove we’re hooking gets called from ExtraMouseSample. We only want to run our code once per user command so simply check if the provided command number is equal to zero.

if (!command->command_number)
  return false;

Okay, that’s all the preparation out of the way. Start by checking if either recording or playback is currently active. Recording is simple, simply push a new frame by passing the current CUserCmd to the Frame constructor we made earlier.

bool is_playback_active = playback.IsPlaybackActive();
bool is_recording_active = recorder.IsRecordingActive();

FrameContainer& recording = recorder.GetActiveRecording();

if (is_recording_active)
  recording.push_back({ command });

Playback requires a few more lines but it’s by no means complicated. Just use the current frame number to get the corresponding Frame structure from the active recording. Once you have that, simply call the public Replay function. Make sure to increment the frame counter if there’s still more to play back, otherwise, call StopPlayback.

if (is_playback_active) {
  const size_t current_playback_frame = playback.GetCurrentFrame();

  try {
    recording.at(current_playback_frame).Replay(command);
    engine->SetViewAngles(command->viewangles);

    if (current_playback_frame + 1 == recording.size()) {
      playback.StopPlayback();
    } else {
      playback.SetCurrentFrame(current_playback_frame + 1);
    }
  } catch (std::out_of_range) {
    playback.StopPlayback();
  }
}

That’s all for basic recording and playback. Before we implement re-recording, we should make sure everything up to this point is working as intended. All that’s missing for now is the user interface code. If you’re following along with my example projects, switch to Menu/Interface.cpp and add the following to BasehookInterface::OnEndScene.

if (recorder.IsRecordingActive()) {
  if (ImGui::Button("Stop Recording"))
    recorder.StopRecording();
} else {
  if (ImGui::Button("Start Recording"))
    recorder.StartRecording();

  if (!recorder.GetActiveRecording().empty()) {
    if (ImGui::Button("Clear Recorded Frames"))
      recorder.GetActiveRecording.clear();

    if (playback.IsPlaybackActive() && ImGui::Button("Stop Playback"))
      playback.StopPlayback();
      
    if (!playback.IsPlaybackActive() && ImGui::Button("Start Playback"))
      playback.StartPlayback(recorder.GetActiveRecording());
  }
}

Nothing fancy, just a few buttons depending on whether recording, playback or neither are currently active. With that, we’re ready to compile and test in-game. By default, the example projects I’ve provided use the Insert key to toggle menu visibility.

So far, so good. Of course, without re-recording this isn’t particularly useful.

Let’s get right to that, we’ll need to make some changes in a few places, starting with the Recorder class.

bool is_recording_active = false;
bool is_rerecording_active = false;

size_t rerecording_start_frame;

FrameContainer recording_frames;
FrameContainer rerecording_frames;

I’ve added three new variables to the class. Firstly, is_rerecording_active – well, this one is quite self-explanatory. Secondly, rerecording_start_frame – we’ll set this to the frame we were playing back when re-recording started. I’ll go into more detail about the importance of this shortly. Finally, rerecording_frames – another FrameContainer which will store all the temporary re-recorded frames before we decide to keep or discard them.

In addition to some new member variables, we’ll also need some new accessor functions.

void StartRerecording(size_t start_frame) {
  this->is_rerecording_active = true;
  this->rerecording_start_frame = start_frame;
}

void StopRerecording(bool merge = false) {
  if (merge) {
    this->recording_frames.erase(this->recording_frames.begin() + (this->rerecording_start_frame + 1), this->recording_frames.end());
    this->recording_frames.reserve(this->recording_frames.size() + this->rerecording_frames.size());
    this->recording_frames.insert(this->recording_frames.end(), this->rerecording_frames.begin(), this->rerecording_frames.end());
  }

  this->is_rerecording_active = false;
  this->rerecording_start_frame = 0;
  this->rerecording_frames.clear();
}

bool IsRerecordingActive() const {
  return this->is_rerecording_active;
}

FrameContainer& GetActiveRerecording() {
  return this->rerecording_frames;
}

Most of these are standard stuff but I sure hope StopRerecording caught your attention. This function is responsible for merging the re-recorded frames into the active recording. This is achieved by removing all existing frames after we started re-recording and replacing them with the contents of rerecording_frames.

No need to touch the Playback class but we’ve still got some work to do with CreateMove.

bool is_rerecording_active = recorder.IsRerecordingActive();
FrameContainer& rerecording = recorder.GetActiveRerecording();

if (is_recording_active)
  recording.push_back({ command });
else if (is_rerecording_active)
  rerecording.push_back({ command });

Now then, we need a quick way to start re-recording – in my opinion, the most intuitive method is to simply press a button while playback is active. Since we have access to the CUserCmd before changes are made, we can simply check if the buttons variable is not equivalent to zero. If so, we start re-recording from that frame.

if (command->buttons != 0) {
  recorder.StartRerecording(current_playback_frame);
  playback.StopPlayback();

  return false;
}

For the finishing touches, we’ll add “Save Re-recording” and “Clear Re-recording” buttons to the interface.

if (recorder.IsRecordingActive()) {
  [...]
} else if (recorder.IsRerecordingActive()) {
  if (ImGui::Button("Save Re-recording"))
    recorder.StopRerecording(true);

  if (ImGui::Button("Clear Re-recording"))
    recorder.StopRerecording(false);
} else {
  [...]
} 

That’s it for the coding part, now we’re ready to make our first tool-assisted speedrun!

After you’ve found a suitable server, take note of all the important settings that might affect your recording. For example, the air acceleration, gravity, tick-rate values and maybe most importantly, the starting position. One way to get a consistent starting position (on servers with a timer plugin) is to restart the timer and then type getpos in console.

Next, disconnect and start a local server – make sure you’re using the same settings as the remote server you selected. After you’ve confirmed the settings match those of the remote server, teleport to the starting position using the command you received from getpos. You’re just about ready now – feel free to adjust the host timescale value to your liking.

Begin recording from here, then play through the map like normal. If you need to correct a mistake or simply try a segment again, stop recording and return to the starting position. To start re-recording, simply press any button during playback and you’ll take control. You can then use the “Save Re-recording” or “Clear Re-recording” buttons in the menu to either keep or clear these frames. Repeat this process until you’re satisfied with the final recording.

And with that, you’re all done! Keep in mind that poor network conditions can affect playback. You may also find that some maps are unsuitable due to nondeterministic behaviour. That said, there’s still a lot of room for additional features – for instance, importing and exporting recordings to disk, visualizing the recording path, useful overlay features, etc.

Good luck and have fun!