Znafon

"Nothing came of it, but I took the code and shoved it into my back pocket for a rainy day.

My idea was to take this code and spruce it up for Uber’s use case."

"My first reaction was to publish the code on Github."

I’m very surprised by this, isn’t the code property of Box, or Uber? The author does not mention their authorisation before releasing it under MIT license.

The AGPL says "including scripts to control those activities" but if I understand what you are saying properly it means that the text of the AGPL is overreaching and some parts of it wouldn't be applicable?

Would the details depend on the country? Like it might be one thing under US law, and another in another country?

The AGPL defines "Corresponding Source" as "all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities".

Then in section 13 it says that "your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge".

Isn't "scripts to control those activities quite vague? Under those terms wouldn't AWS Proton, IBM Schematics, Env0, Digger, Spacelift, etc. be required to open-source a very large part of their product as well?

It's not just big tech that hates AGPL. If Terraform was under AGPL a lot of companies might have to release their source code as AGPL too because of its virality. Where it stops is not exactly clear from the license.

You could modify it easily, for example the user could register an additional function to the mapping. That would not be as easy to do with a module and getattr().

You can also list the registered function with math_exprs.keys() and that might not be as easy with a module either.

Oh yeah, that’s right! Thanks for the correction

I'm not sure

    print(firstname, lastname)

    print((firstname, lastname))

    print((surname,))

Variadic functions are rather classic, I think Go, Rust, C and JavaScript also have them.

I think it is consistent, it works a bit like filtering an element from a mathematical set.

Given a set of sheeps, let x be the five-legged sheep is inconsistent because we know neither the existence or uniqueness of shuch sheep, so it raises an exception.

Given a set of sheeps, let x be the subset of five legged sheeps is the empty set because there is no such sheep.

but this may also just be because I internalised Python's behavior.

Some language have a specific value to denote the first thing, for example:

   ["a", "b", "c"][4]

It is used when the number of argument can vary, like:

    def sum(*args: int) -> int:
        if len(args) == 0:
            return 0
        return args[0] + sum(*args[1:])

spacectl, vcs-agent and terraform-provider-spacelift are only useful with your proprietary product. prometheus-exporter is a very small exporter to monitor your proprietary product and has no utility beyond that.

I'm not sure what spcontext does because it has no documentation but I'm sure I could read the 10 commits to find out. The only none trivial project in the list is celplate which actually looks nice, but even there most of its complexity is in cel-spec.

Whenever there's an opportunity to sponsor a project on which we heavily rely, we do that.

Come on, on Spacelift GitHub profile there is one project sponsored, and only since August 24!

we don't maintain any major open source projects as a company

So you never had to handle project management of large open-source supports, community support, doing code reviews every day and handling feature requests from the community?

Yes, HashiCorp has been slow to respond to PRs and did not always commmunicate clearly, but so does many large open-source project like PostgreSQL, Python, Linux, etc. Managing a large open-source project takes more time than clicking on the merge button on GitHub. Users don't magically come fix all the bugs in the software, and develop complex new features.

HashiCorp did contribute a lot, and there is still a lot to learn from their projects, there is Raft, the autopilot, various library that are used a lot in the Go ecosystem, including go-plugin, a programming language with its specific type system and plenty more. After all, the change of license is making noises because we their tools defines part fo the DevOps world and we all use them a lot.

I wish HashiCorp would have kept using the MIT license, but using the BUSL is still miles ahead better than companies that are completely closed source.

From the outside perspective it looks like you support Free Software as in Free Beer more than in Free Speech. You are supportive when others are actually maintaining the projects for you, and you just want them to merge your contributions quickly. It is a very efficient way to externalize some of your costs, but you only step forward to actually to help manage them when your bottom line is threatened.

That's ok, it looks like you love money more than open-source ideas. That's perfectly fine but don't pretend otherwise.

Some others companies that signed the pledge have actually been maintaining open-source projects and have a leg to stand on. Spacelift loves having the high moral ground and the extra publicity.

Release the core of your product as open-source, like HashiCorp did for 9 years, and it will change my mind.

So the software from your company is fully closed-source, isn't it?

The employes can do open-source on Friday but no part of Spacelift is actually open-source?

This is a very non-charitable read of the history here.

Also, none of the companies that pledge FTE support to OpenTF are open-source.

The PEP goes into details about the motivations and why the single thread model has limitations: https://peps.python.org/pep-0703/#motivation.

Yes, some applications will remain hard / impossible, but I'd argue Python is not a good fit for apps requiring high performance / high scalability anyway

People are already trying to use Python for parallel tasks. Forcing people to use another language does not help them much.

They were always usable using the C-API but not exposed to the Python library (and until 3.12 you were blocked by the shared GIL so there was little point anyway).

Now that the work from Eric Snow has been merged, you can use https://pypi.org/project/interpreters-3-12/ to create one from Python code.

I would be fine with have it never be removed, and I think we should try sub-interpreter first.

There is a lot of work going on with sub-interpreters and the per-interpreter GIL is shipped in Python 3.12.

The results are very impressive: https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/, as good as one could have hoped I think.

It seems to me like the work on sub-interpreters will continue in parallel ;) to the work on free-threading.

Sub-interpreters and no-GIL have different use-cases though.

I think you can make a non thread-safe list by just making the same object without the lock described at https://peps.python.org/pep-0703/#container-thread-safety if you really want the maximum performance of single threads.

Maybe it could be part of a non_threadsafe_containers module on Pypi.

Yes this is a thing was trivially protected by the GIL. There is the same thing with mutating the same map concurrently in Go that will panic for example.

PEP 703 goes over this in the "Container Thread-Safety" (I think container here refers to the fact that the object has references to other objects, this is the things that already are special-cased in CPython to be managed specifically by the garbage collector):

This PEP proposes using per-object locks to provide many of the same protections that the GIL provides. For example, every list, dictionary, and set will have an associated lightweight lock. All operations that modify the object must hold the object’s lock. Most operations that read from the object should acquire the object’s lock as well; the few read operations that can proceed without holding a lock are described below.

More information at https://peps.python.org/pep-0703/#container-thread-safety

Would the work on sub-interpreters be interested for that then (https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/) ?

If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless.

The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal.

The PEP goes into the motivation behind this work, and using multiple process does not magically solves all the issues:

Multiprocessing, with communication via shared memory or UNIX sockets, adds much complexity and in effect rules out interacting with CUDA from different workers, severely restricting the design space.

I reimplemented parts of HMMER, a standard method for multiple-sequence alignment. I chose this method because it stresses both single-thread performance (scoring) and multi-threaded performance (searching a database of sequences). The GIL became the bottleneck when using only eight threads. This is a method where the current popular implementations rely on 64 or even 128 threads per process. I tried moving to subprocesses but was blocked by the prohibitive IPC costs.

NumPy does release the GIL in its inner loops (which do the heavy lifting), but that is not nearly enough. NumPy doesn’t offer a solution to utilize all CPU cores of a single machine well, and instead leaves that to Dask and other multiprocessing solutions. Those aren’t very efficient and are also more clumsy to use. That clumsiness comes mainly in the extra abstractions and layers the users need to concern themselves with when using, e.g., dask.array which wraps numpy.ndarray. It also shows up in oversubscription issues that the user must explicitly be aware of and manage via either environment variables or a third package, threadpoolctl. The main reason is that NumPy calls into BLAS for linear algebra - and those calls it has no control over, they do use all cores by default via either pthreads or OpenMP.

and it discusses the alternatives at https://peps.python.org/pep-0703/#alternatives.

The performance patches have already been merged and there is ongoing working in the Faster CPython project.

Sentry has not been using BSL since the beginning.

What's wrong with monetization? You understand that OSS's significant problem is a lack of funding, where authors don't want or don't know how to monetize their product? Sentry looks like a good model for OSS, and it's proof that you can make a living from OSS.

Sentry is using the BSL, the very same license that HashiCorp has switched to but that change has not been received well.

Isn't that what email and mailing lists are for?

Yes, you can do this: https://pip.pypa.io/en/stable/topics/vcs-support/

This is quite an oversimplification. Some modules have no active maintainers and take time from the small team that work on CPython. Some modules are deprecated and removed to make it possible to allocate time on other improvements, and they went back on some removals when users came forward showing that they were still needed.

There is some discussion about the rationale at https://peps.python.org/pep-0594/#rationale

I’m referring to the easy looking, tricky to solve problems. I’ve heard them referred to by that term because they have been used at Moscow University to discriminate against Jewish students but I don’t know if they have other names.

There is some discussions and examples of such problems at https://arxiv.org/pdf/1110.1556.pdf and http://3038.org/press/shen.pdf.

That’s a very elegant solution. Hard looking problems that had simple, trick solutions were called Jewish Problems at the entrance of some universities.

This would let the examiner fail a student for subjective reasons instead of academic success. The examiner would then be able to say « I failed them because of their skills, not because of their religions, look how easy the solution is ».

Perhaps this is what we are reproducing as an industry with all our convoluted interview processes, and it may be a decorum to choose the candidates we want instead of using objective criteria.

Indeed, somehow this went completely over my head.

I would not have though to use this formula. The sum of the integers grows as the square of n so if n is anything but small you will overflow and get an erroneous answer.

This interview question has everything bad imo: no practical uses, test knowledge of math formulas (which every math/CS major would have but none of the self learning folks). It’s very easy to stress and fail when given such a test, and even already knowing I would have tried something else because of the overflow.

Python typing supports duck typing: https://peps.python.org/pep-0544/

and heterogeneously typed ducts: https://peps.python.org/pep-0589/