Using C++ as a scripting language, part 8

https://fwsgonzo.medium.com/using-c-as-a-scripting-language-part-8-d366fd98676

by fwsgonzo • 3 years ago

134 46 3 years ago

fwsgonzo.medium.com

mike_hock• 3 years ago

On the performance side, pretty much the entire game logic would have to be implemented on the "script" side for this to be worth it compared to just embedding Lua and calling it a day, right?

I can see the benefit of typechecking the "script" code, though. The vulnerable parts are the "syscall" wrappers but they don't change all the time.

jamesu• 3 years ago

Is performance really going to be a deciding factor for using this? I'd guess it depends what you really want in your "scripting" language. For me I'd want some sort of runtime with an easily readable language that can tolerate bad code, so in that case I would prefer something like lua.

Must admit though, things like this really make one think... what do people even want in a "scripting" language these days?

masklinn• 3 years ago

Probably depends how much of the game is implemented via "scripting", some games just put scripting at the edges (UI, data definition) while others do everything up from the game engine via "scripting" e.g. https://forums.unrealengine.com/t/ive-created-an-infinite-vo...

The whole project s 100% blueprint with no C++ being used. I think this project is a great example for the power of blueprint.

And that's a post from 2016. These days you can probably create entire games via visual scripting (https://unity.com/features/unity-visual-scripting) as long as they're not too bespoke or demanding.

fwsgonzo2• 3 years ago

I don't know about the entire script. The game and game engine has tens of thousands of lines of code that is not script. But, I feel like it opens creative doors to be able to make (lets just say) a magnitude more calls into the virtual machine compared to other options. That's not to say that I am done with this research I am doing. The idea is to be able enter and leave the script as fast as possible, while also call back into the game engine as fast as possible. Once this is achieved, game logic that once had to rely on finite options can have script callbacks. Also the game will be naturally moddable from the start.

Here is an outtake from one of my game scripts: https://gist.github.com/fwsGonzo/d9116c46e5b7f8ed4743ab15f89...

While it may not look like it, flow.cpp calls back into the game engine in a surprising amount of places. It's also something that regularly gets called 10k times per simulation step if there is a cave under the water, for example.

For me right now, I feel like I have reached a sweet spot where I would no longer consider going back. Also, it's a fun project! Sometimes I am wondering if I am enjoying making the game engine more than the game. Oh well.

Might as well tack on to this that you can also choose your language. I have emulated Go, Rust and Nim pretty successfully. Enough to know that they would be a possible choice for emulation. Go is perhaps weird in that you want to avoid the C API (cgo) and just figure out the assembly instead, but that could also lock you to a Go version.

fwsgonzo2• 3 years ago

Let me just add that perhaps we can sandbox python too? LPython is on the frontpage now and if I read it right, it will compile to C and C++ when targeting AOT.

See: https://lpython.org/blog/2023/07/lpython-novel-fast-retarget...

bsenftner• 3 years ago

You ought to meet and discuss your project with Arnaud Hervas, the original author of the Shake compositing software. Shake was a VFX industry compositor that used GCC C++ as the "scripting language" that non-developers wrote. It was not exposed as C, but macro modified to look simpler. Arnaud did several industry firsts, including using the GPU to render compositing layers back when all that type of work was in software. His company before Apple bought them was called Nothing Real, https://en.wikipedia.org/wiki/Nothing_Real He's a nice guy, so don't be intimidated.

dgan• 3 years ago

part _8_ .. oh that doesn't start well

kwijibo123456• 3 years ago

If you want to see real C++ scripting: the (slightly insane) people of CERN use a 99% ANSI compliant, self-written C++ interpreter (!) for online data analytics.

https://root.cern/manual/first_steps_with_root/

andromeduck• 3 years ago

That is so incredibly cool!!!

noobermin• 3 years ago

Unless things have changed, root is quite horrible actually.

moelf• 3 years ago

yeah... basically... in fact they're leaning more into JIT + string programming now and seeing big JIT latency etc. (imo is kinda ~approaching Julia from C++ side)

amadsen2• 3 years ago

Root is absolutely, mind-blowingly, amazing. It gets a bad rap because it forces you to use primitives that were designed back in the early nineties. If you're "just" trying to analyze some data, your experience will indeed be "horrible" compared to what's offered by Python, R, Matlab, or Julia. But beyond that... Root adds fully working reflection to C++. Root gives you dynamic library loading and reloading - you can fix a bug or add a new feature, recompile parts of your program and keep working without restarting it. Root has a feature complete C++ interpreter, with scripting and a REPL loop. You can work with it completely interactively. After prototyping you can save your code as a script. After identifying performance critical parts of your code, you can compile them and get the full power of bare metal C++, without changing anything about the code. Yes this is technically possible with e.g. python + numba as well, but not as straight-forward. Root is fully interoperable with Python and R - you can mix scripts and REPLs between the languages and pass objects between them. Root can serialize any object, without requiring any custom code whatsoever (some serious dark magic needed for this). In fact you can pause your entire program and save it to disk or send it over the network to keep running somewhere else. Root has its own file format for efficiently storing massive amounts of data in arbitrarily complex structures. It can stream it over the network too, with probabilistic read ahead and caching for maximum efficiency. Root comes with libraries for physics/math/stats that rival those of the largest commercial and open source offerings. Each one of these is a massive technical achievement and Root has had most of them for decades now. Oh, and it has largely maintained backwards compatibility through all this time as well.

Of course, very few people outside of CERN need all of this. Even within CERN, many projects don't. But for those who do, there are very few - if any - alternatives.

WanderPanda3• 3 years ago

I recently plugged my extremely, abusively templated C++ library into Cling (the C++ interpreter mentioned) and everything worked like a charm. I can even use it in Jupyter notebooks. The speed feels more on the order of Python, though. But if the heavy lifting is done by precompiled libraries, I can see Cling having a pretty nice workflow for experimenting with and interfacing C++ code

binary132• 3 years ago

If I’m not mistaken, Cling is an LLVM-based JIT compiler, so it really should be performing close to the same level as an ahead-of-time Clang-compiled binary executable. Perhaps what you are noticing is JIT latency (from compiling on the fly.)

AtlasBarfed2• 3 years ago

"The speed feels more on the order of Python, though."

What part of "interpreted" did you not understand?

pjmlp• 3 years ago

The CERN beer garden helps with the craziness, specially during Summertime. :)

gavinray• 3 years ago

As an FYI for folks, "Cling" is being integrated upstream into LLVM under the name "clang-repl":

https://clang.llvm.org/docs/ClangRepl.html

pyuser583• 3 years ago

The most eye-catching title I’ve seen.

Jeaye2• 3 years ago

On the topic of using C++ for scripting, and related to the discussion of CERN's ROOT/Cling, I am developing a Clojure dialect on C++/LLVM called jank: https://jank-lang.org/

jank is a true Clojure, meaning you get interactive, REPL-based development (thanks to Cling) and a whole stdlib of persistent, immutable data structures (thanks to immer) and functions to transform them. But it's also C++, so you can write inline C++ within your jank source, and interpolate jank expressions within that. You can link with existing native code using LLVM and you can embed jank into your existing native projects to use for scripting.

jank is pre-alpha, right now, and I've only been showing it to Clojure devs so far, but there's a huge audience of C++ devs which may be interested in introducing Clojure to their native code.

widdershins• 3 years ago

Count me among them. I write C++ all day and I'd love to introduce Lisp/Clojure to orchestrate my more performant code. This is a very interesting project!

naruhodo2• 3 years ago

I don't know too much about either Jank or Ferret, but would you care to compare the two?

Jeaye2• 3 years ago

Happy to. Ferret and jank have similar goals, but some key differences:

1. Ferret targets embedded platforms primarily; jank is more general and less constrained

2. Ferret doesn't have a JIT-based runtime, meaning no nREPL while you build your programs; jank provides the whole Clojure interactive programming experience, meaning you can start with a REPL and an empty file and end up with a whole server/GUI application/game without stopping to recompile/run again

3. Ferret is written in Clojure and requires the JVM to run, whereas jank's host is native, meaning its runtime and compiler and both native (C++)

My overall sentiment for Ferret is that it's great and I'm excited that it exists. I have my eye on it so that we might be able to learn from some of the decisions made over there. In particular, its interop and memory management systems are of interest.

Worth noting, when you look at all of the existing native Clojures (Ferret, Carp, jank, etc), only one of them aims to be Clojure source compatible while providing a true REPL-driven experience: jank.

corysama• 3 years ago

Visual Studio upgraded Edit And Continue to support live code updates. https://learn.microsoft.com/en-us/visualstudio/debugger/hot-...

If that’s not enough, there’s https://liveplusplus.tech/index.html

hgs3• 3 years ago

Worth mentioning that Quake 3 used C as a scripting language back in 1999. The engine used a modified LCC compiler [1] to generate byte code for execution in its own virtual machine. The same could be achieved today by embedding a WASM VM and compiling your C/Zig/Rust whatever to web assembly.

As an aside, I find the idea of writing game code in unmanaged C++ perplexing. Even the original Quake, which targeted DOS-era hardware, relied on a scripting language running in a plain vanilla interpreter (no JIT here). In the [current year] I would think advances in computing power, parallelism, and JIT technology would be more than adequate to accommodate an interpreted scripting language.

[1] https://en.wikipedia.org/wiki/LCC_(compiler)#Projects_incorp...

__s• 3 years ago

Using an interpreter on old hardware was often to save memory

https://lwn.net/Articles/412024

ramesh31• 3 years ago

As an aside, I find the idea of writing game code in unmanaged C++ perplexing.

https://github.com/id-Software/DOOM-3

kevingadd• 3 years ago

"advances in computing power, parallelism, and JIT technology would be more than adequate to accommodate an interpreted scripting language."

While this is true, game consoles + iOS/iPadOS prohibit JIT and in some cases prohibit script interpreters, so people resort to stuff like 'we wrote a compiler for C# that turns it all into C++ source files' in order to pass review and collect Fortnite Money*.

* I'm using Fortnite Money here to represent 'the kind of money you can make if you're willing to sell lootboxes to teens on the App Store'

Tozen• 3 years ago

Add Vlang to that list, as another that can compile to web assembly. In addition to having a C2V translator (and some other languages have similar). A number of safer and easier/convenient options are out there for those that care to look.

bullen• 3 years ago

I'm making a Action MMO engine where the game will be coded in C and hot-reloaded with this API: http://edit.rupy.se?host=move.rupy.se&path=/file/game.cpp&sp...

So far the API is really small and the turnaroud is 100 millisec.

_448• 3 years ago

ChaiScript is another option: https://github.com/ChaiScript/ChaiScript

nathants• 3 years ago

this is interesting.

i’m working on a high performance game in c++17 with clion and wickedengine on linux.

compiles are 20 seconds, which is fine. 1 second would be better.

blender exports are about 20 seconds too. 1 second would be better.

Using C++ as a scripting language, part 8

Related Stories