The code side of things has seen great progress in the last two weeks: now the GVMachine controllers can access mouse axes so I was able to set up mallet based controls for the pinball table:
Linking the mouse input to the mallet and tweaking the values for a good look&feel took about an hour, most of which I spent playtesting. Again, no code was written for the pinball table specifically, only generalized building blocks which were assembled on the map.
The next task we are facing is performance capture: recording the property changes of given actors and replaying them when needed. It will work something like a video editor where you place video clips one after the other to produce a movie. Only here the clips contain actor(s) which do stuff in the scene, independently from other actors and from the camera.
For example I could play for a while on this pinball table and record what the ball and the flippers do. Then put them on “auto pilot” so they keep repeating that captured performance every time I start the map. As a next step I play as an ant running around avoiding the steel ball, collecting bread crumbles stuck behind drop targets (or something). The actions of the ant are recorded and then replayed too. As the third step I take control of the camera: fly around, jump to predefined locations, zoom, pan and generally do what a cameraman does to produce an effective footage. Again, everything is recorded and replayed, so at the end I have a full story, rendered in realtime.
Practically anything can be changed at anytime: art assets can be replaced, lighting adjusted, timings tweaked, performances re-captured, etc. When everything is sound then the final video can be rendered using Unreal’s inbuilt tools which produce footage at arbitrary resolutions and frame rates.