After a long day of downloading event packs, creating BGM loops, converting graphics from INFINITAS, dual-booting Windows 10 on the game PC and fixing encoding issues, I had finally set up beatoraja to my liking. There was just one thing left to do: figure out where to put the on screen camera for recordings.
Greetings and Happy New Year! I'm taking a quick break from the ongoing IIDX score sharing series to prove that I can still put something up here without it taking half a year. This is an abridged version of how I got scene switching to work in beatoraja, since I had a few people ask me about it today. Hopefully it's of interest to someone.
First things first, I needed to set up a development environment. After cloning the beatoraja repository from GitHub, I just had to decide which IDE I was going to use. I ended up going with IntelliJ IDEA since it seemed to have all the basics, the community version was free, and I already use another JetBrains IDE, PhpStorm, so I'd know where to find the common stuff.
After opening the project I had to set up an SDK which was simple enough. I also had to manually mark the source directory to get syntax highlighting and auto-completion to work in the editor. You can do this by right-clicking the src directory in the Project tool window and clicking "Mark Directory as > Sources Root".
Before getting stuck into the source code, I wanted to do a full compile to see if the game would actually run without any issues. I found the main class in MainLoader.java but also ran into a load of errors in the process. At first it looked like some libraries were missing, but it turned out that the library search path just wasn't configured yet.
All the required libraries were conveniently included inside the lib repository directory. That cleared up all the broken import related errors, but I still had to set the compiler output directory before it would let me build the project.
With that out of the way, the game compiled and the configuration window opened as expected.
FileNotFoundException in the console was just a missing configuration file, nothing important.
Then it was time to configure the game instead. Not much to talk about here, this mostly consisted of copying across a few charts to test with, then clicking through all the LITONE6 options until it looked close enough to what I run on my game PC.
Now for the scene switching part. My initial idea was to swap between two scenes. An 'idle' scene which would be active during non-gameplay times such as music selection and the result screen, and a 'play' scene which would only be active while the music is playing. As for where to do the actual switching, I noticed a few lines in the console while the game was running.
INFO: STATE_READYに移行
Jan 01, 2020 10:16:20 PM bms.player.beatoraja.play.BMSPlayer render
INFO: STATE_PLAYに移行
Jan 01, 2020 10:16:32 PM bms.player.beatoraja.play.BMSPlayer render
INFO: STATE_FINISHEDに移行
Jan 01, 2020 10:16:32 PM bms.player.beatoraja.play.KeyInputProccessor$JudgeThread run
A quick search for the string "STATE_PLAYに移行" pointed me to a line in BMSPlayer.java. Specifically, this was where the current state changes from STATE_READY to STATE_PLAY.
After confirming with a breakpoint, this looked like it would be the perfect place to switch to the 'play' scene. In the recent beatmania IIDX arcade games (since CANNON BALLERS), this is when the in-game cameras would activate.
Now I just had to tackle the interacting with OBS part. As mentioned ages ago, I run OBS on a separate PC, so using the API isn't an option. Luckily, it turns out there's a plugin for this sort of thing: obs-websocket. The readme even links to a Java client!
I didn't really know how libraries worked in Java so this took a little while, but I eventually managed to get it included.
All the other libraries were .jar files in the lib directory so that seemed like the right thing to do. I cloned websocket-obs-java locally and opened it in IDEA. I let it import Maven projects when it asked. I still don't really know what Maven is, but IDEA apparently supports it. After clicking View > Tool Windows > Maven, I noticed a 'package' item in the Lifecycle tree.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ obs-remote-java ---
[INFO] Building jar: A:\Projects\websocket-obs-java\target\obs-remote-java-1.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.951 s
[INFO] Finished at: 2020-01-01T22:58:46Z
[INFO] ------------------------------------------------------------------------
Nice, it built a .jar file. That's it, right? Of course not.
websocket-obs-java relies on a few external libraries like a JSON parser and a WebSocket client. These weren't included in the built .jar file, but they could be. Special thanks to the first hit for 'maven include dependencies in jar' on Google here.
After pasting the plugin block into pom.xml and installing maven-assembly-plugin, I was ready to run 'package' again.
Much better. I renamed the version that contained dependencies to obs-remote-java-1.0.jar and copied it into the lib directory. IDEA didn't pick it up until I removed the lib search path in Project Settings and added it again.
Oh boy, time to actually write some Java. I made a new package in the src directory uncreatively named aixxe, then a new class called StreamController. I also made an enum called Scene. Here are the contents of both files.
package aixxe;
public enum Scene {
MusicSelect {
public String toString() { return "BMS: Music Select"; }
},
Decide {
public String toString() { return "BMS: Decide"; }
},
PlayLoading {
public String toString() { return "BMS: Play (Loading)"; }
},
Play {
public String toString() { return "BMS: Play"; }
},
PlayFailed {
public String toString() { return "BMS: Play (Failed)"; }
},
PlayFadeOut {
public String toString() { return "BMS: Play (Fade Out)"; }
},
Result {
public String toString() { return "BMS: Result"; }
}
}
The strings here represent the scenes that should exist in OBS. I've expanded on the original two for additional flexibility, but it's easy enough to change the strings around so multiple 'game' scenes map to the same scene in OBS if you don't need them.
package aixxe;
import net.twasi.obsremotejava.OBSRemoteController;
public class StreamController extends Thread {
private boolean ready = false;
private OBSRemoteController controller = null;
private void connect() {
controller = new OBSRemoteController("ws://localhost:4444", false);
controller.registerConnectCallback(response -> ready = true);
controller.registerDisconnectCallback(response -> ready = false);
}
public void run() {
while (!this.isInterrupted()) {
if (!ready) {
connect();
}
try {
sleep(100);
} catch (InterruptedException e) {
break;
}
}
}
public void Switch(Scene scene) {
if (ready) {
controller.setCurrentScene(scene.toString(), response -> {});
}
}
}
It's far from perfect, but this class is effectively just a thread that tries to stay connected to the OBS WebSocket server. The only real method here is the Switch one, which sends the actual SetCurrentScene message.
Now to instantiate it. I stuck it in MainController since it seemed to be accessible everywhere it needed to be.
// src/bms/player/beatoraja/MainController.java
import aixxe.Scene;
import aixxe.StreamController;
public class MainController extends ApplicationAdapter {
// ...
public final StreamController streamController = new StreamController();
public MainController(Path f, Config config, PlayerConfig player, PlayMode auto, boolean songUpdated) {
// ...
streamController.start();
}
}
Heading back to BMSPlayer.java now to trigger the scene switch.
Logger.getGlobal().info("STATE_PLAYに移行");
main.streamController.Switch(Scene.Play);
Perfect! Now that the core concept was working, I looked around for places to trigger the other scene switches. I won't go into detail since they were all essentially the same as this first one, just different Scene values in various places.
Now it was time to get the camera implemented in the play scene. Here's what typical gameplay looks like.
The space below the video on the right is where I want to put the camera, just like IIDX AC. First things first, I removed the custom image by setting the 'BGA縮小画像(BGA Shrink Image)' play skin option to 'カスタマイズ(Customize)'.
There aren't any images in the customize folder by default, so the entire area was now empty. I then created a mask image using the screenshot above. The area to be replaced with the webcam was filled black, while the rest of the image was filled white. The part overlapping the judge table was set to 25% black, so that it would only be partially visible in OBS.
Here's the final mask image. It'll need to be adjusted if you play on the P2 side, use non-reversed score graph position or use a different judge table setting, but yeah, it's nothing difficult.
I added the Image Mask/Blend filter to the capture source, then placed a placeholder image below it. It looked great.
Now that everything was working, it was time to package it all up and copy it over to the game PC. I made sure to change the OBS WebSocket address from localhost to my server PC first. (Note to self: read this from configuration instead.)
After some looking around, I found that build.xml is responsible for creating a runnable jar file. Right-clicking this in IDEA and selecting 'Add as Ant Build File' brought up a new tool window, with one of the options being create_run_jar.
This ran without issue and produced a beatoraja.jar in the build directory. However, it wasn't quite there yet.
Exception in thread "Thread-7" java.lang.NoClassDefFoundError: net/twasi/obsremotejava/OBSRemoteController
at aixxe.StreamController.connect(Unknown Source)
at aixxe.StreamController.run(Unknown Source)
Opening up beatoraja.jar in 7-Zip revealed that obs-remote-java-1.0.jar was indeed missing. A closer look at build.xml revealed that the libraries included in the jar were explicitly defined. I added the appropriate line and tried again.
<zipfileset excludes="META-INF/*.SF" src="./lib/obs-remote-java-1.0.jar" />
With that, I was finished! Here's an excerpt from my last Twitch stream demonstrating the camera automatically fading in and out at the beginning and end of a chart. I'm only using two scenes at the moment so there's plenty more room for creativity.
To use it, replace your existing beatoraja.jar with the custom one, then open the game and close it when you get to the music select screen. This should write the default configuration values to the end of your config.json file.
"enableObsWebSocket": false,
"obsWebSocketAddress": "localhost",
"obsWebSocketPort": "4444",
"obsWebSocketScenes": {
"MusicSelect": "BMS: Music Select",
"Decide": "BMS: Decide",
"PlayLoading": "BMS: Play (Loading)",
"Play": "BMS: Play",
"PlayFailed": "BMS: Play (Failed)",
"PlayFadeOut": "BMS: Play (Fade Out)",
"Result": "BMS: Result",
"CourseResult": "BMS: Course Result",
"Config": "BMS: Config",
"SkinConfig": "BMS: Skin Config"
}
Flip enableObsWebSocket to true, then change the scene names as needed. If you run OBS on the same PC you won't need to change the address or port values. Oh yeah, also make sure you have obs-websocket installed.
Keep in mind that custom builds of beatoraja like this won't work with MochaIR. If that's not a deal breaker for you, enjoy!