Golang Package: plugin

https://tip.golang.org/pkg/plugin/

by pythonist • 10 years ago

107 30 10 years ago

tip.golang.org

orta• 10 years ago

This is awesome, a plugin structure like this is one of the fundamentals of macOS programming (they're called NSBundles in those cases.)

junke• 10 years ago

A plugin is only initialized once, and cannot be closed.

Why not?

IshKebab2• 10 years ago

I guess it's because if you load a symbol there's no clean way to invalidate it, especially in a thread-safe way. Something like that anyway.

jerf• 10 years ago

I think it's safe to say there is just plain no way to invalidate it. Even if it were hacked together, it would be unusably dangerous for even trivial cases.

Module loading is something that can be added to a language later, but I suspect safe module unloading is something that has to go in from day one to ever work right. Even in languages that support it (Erlang) it's got a tricky list of caveats to keep track of that strike me as fairly essential to the problem, rather than accidental (in the Fred Brooks sense of the terms).

pjmlp3• 10 years ago

Because unloading, while possible is a pain to get right.

Hence why Java and .NET restrict it to Class Loaders/AppDomains.

Someone• 10 years ago

For those who wonder why: some of the things you have to make sure of are if you want to implement this correctly:

- no thread is running code in the plugin.

- no thread has a return address on the stack somewhere in the plugin.

- no variable stores a pointer pointing into the plugin (This includes objects whose destructor is implemented in the plugin)

- no signal handler points into the plugin.

- no kernel thread is reading the plug-ins memory.

In addition there are race conditions to take care of (e.g. thread T unloads a plugin that thread U is loading)

With a 64 bit address space, I guess you can get around most if not all problems by mapping the memory that was mapped to the plugin as "can't read, can't write, can't execute", so that you crash hard if somebody tries to access it, but if you do that, you can just as well keep the code around.

Go is garbage-collected, so it likely already has a lot of the logic to do this, but even then, combining the functionality to get a robust implementation of plugin unloading is a lot of work.

gtrubetskoy• 10 years ago

Curious what it would take (if at all possible?) to load such a .so from C. Because that would open the door for writing Go libs for all your favorite interpreted languages, web servers, etc.

fjl• 10 years ago

You can already create shared libraries that expose a C API using -buildmode=shared.

dvirsky6• 10 years ago

That's already possible, I've written an API that encapsulates Go libraries as Redis modules.

Goopplesoft• 10 years ago

Link?

robbles• 10 years ago

here's a pretty cool example: https://blog.heroku.com/see_python_see_python_go_go_python_g...

speps• 10 years ago

I see an API fail coming. It seems plugin.Load takes the filename WITH extension where it shouldn't. Okay, right now it seems it's only supported on Linux but it ends up in Windows, you'd have to patch code to remove the .so. Mono handles it pretty well, the extension is optional.

regecks• 10 years ago

Not necessarily. https://tip.golang.org/src/plugin/plugin_dlopen.go

" 5 // +build linux,cgo"

The Windows version isn't there yet - it could encapsulate that logic.

IshKebab2• 10 years ago

I don't think so. I think it is better that the file naming is handled in the application. It is less restrictive, e.g. you can use custom extensions this way.

Consider another instance where file names are automatically mangled: GCC linking with libraries with the switch -lfoo. That has honestly caused me no end of problems. It would have been better if they required an explicit -llibfoo.so from the start.

Go already has several easy methods for OS-specific code if you want.

dvirsky6• 10 years ago

Not sure it should be be part of this package, but it's trivial to write a function that exposes something like "SharedLibraryFile(libname string) string". Depending on the OS it just formats the default shared object name.

For extra runtime speed you could build OS specific versions of that function using build flags, so for each target OS it has one implementation and you have no switch/case or anything in runtime.

dsl• 10 years ago

Assuming or magically guessing file extensions leads to all sorts of confusion attacks that could result in arbitrary code execution.

I would go so far as to say an absolute path should be required.

SoapSeller• 10 years ago

Didn't one of the main selling points of Go is one big static executable with no external dependencies?

What's next? Generics?

dvirsky6• 10 years ago

Although this can be abused, the use case is not external dependencies IMO, but rather extending systems. Think of a web server etc. Each plugin is still a static binary that is self contained, but you can load capabilities into the server in runtime.

This is not like dynamic linking hell you can get yourself into when writing C.

clappski• 10 years ago

But isn't that exactly how dynamic linking works? By loading a DLL at runtime? The only issues I've ever had with dynamic linking are at _runtime_, where XXX.dll can't be found.

wtbob• 10 years ago

Each plugin is still a static binary that is self contained, but you can load capabilities into the server in runtime.

In my Go code, I've just used subprocesses for that. Stdin & stdout, plus a simple serialisation format, and I'm done.

Not suitable for high performance stuff, of course, but good enough for my needs.

pjmlp3• 10 years ago

This is not like dynamic linking hell you can get yourself into when writing C.

No, but they will eventually need to handle plugin versions.

StavrosK• 10 years ago

Are you saying that, in cases where that's not possible, you'd rather not have a program at all instead of a program with external dependencies?

Because that doesn't sound very reasonable to me, even after ignoring that plugins aren't really dependencies anyway.

nine_k• 10 years ago

Generics would be great, but they are much trickier. It took Java many years to implement them, while having fewer runtime constraints.

stirner• 10 years ago

Standard library packages are not the same thing as the language and are not mentioned in the Go specification. The standard library is just a collection of packages that implement common functionalities that might otherwise lead to multiple competing implementations (like hyper, tiny_http, rotor-http, and solicit in Rust). Including a plugin mechanism is not an endorsement of that style of architecture, but simply an acknowledgement that there are programs that will need plugins.

pjmlp3• 10 years ago

Nice to see this coming, one step closer to have Oberon System 3 with Gadgets rewritten in Go. :)

guessmyname• 10 years ago

I agree with this comment from Reddit four days ago:

The inability to unload a plugin defeats the use I would have for this, unfortunately. A long running service with plugins may want to load new, updated versions of an existing plugin. Not being able to unload the old version, ends up creating stale references which will keep piling up for the lifetime of the process. As I understand it, this is essentially a memory leak. [...]
https://www.reddit.com/r/golang/comments/53adu8/a/d7rmmco

There is also this project by Hashicorp: https://github.com/hashicorp/go-plugin

zupa-hu• 10 years ago

Amazing, so this is essentially a high performance alternative to Go scripting. One can already include the go build toolchain in a binary, right? Having JIT is only small steps away.

Disagree with the problem of unloading plugins. From the sizes of Go binaries and memory sizes, one could easily load 10k+ plugins before the process needed a restart. Would be nice to have GC, but having plugins is such an increased benefit. To me this sounds like, Git is crap because you can't unload stored objects.

Golang Package: plugin

Related Stories